Hello , Please guide me with the steps involved in inferencing this model to generate csv of the chart provided

[Aaryaveerkrishna23]

[Aaryaveerkrishna23]

I was facing issues like :
1)FileNotFoundError: [Errno 2] No such file or directory: '${./base_decoder/model_base_decoder.py}/ChartVLM/base/instruction_adapter/vectorizer.pkl'
2)ERROR: Cannot install torch==2.1.0 and triton==2.0.0 because these package versions have conflicting dependencies.

[renqiux0302]

Hi @Aaryaveerkrishna23,

1. You should modify the command (line 4 in inference.py) to the path where you save the ckpt (i.e. ../ChartVLM/base). You can download and organize the ckpt followed by Pre-trained Checkpoints of ChartVLM mentioned in README.md.

2. It is highly recommended that you install torch ==1.13.0. You can also check the environment installation according to envs_ours.txt

[Aaryaveerkrishna23]

Hi @Aaryaveerkrishna23,

1. You should modify the command (line 4 in inference.py) to the path where you save the ckpt (i.e. ../ChartVLM/base). You can download and organize the ckpt followed by Pre-trained Checkpoints of ChartVLM mentioned in README.md.
2. It is highly recommended that you install torch ==1.13.0. You can also check the environment installation according to envs_ours.txt

Hello renuix I did download and organized the checkpoints as suggested by you , I got this error "IsADirectoryError: [Errno 21] Is a directory: './ChartVLM-base/auxiliary_decoder/base'

[renqiux0302]

Hi @Aaryaveerkrishna23,
There are two possible reasons:

1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.

2. Maybe you miss the base folder under auxiliary_decoder, please check it.
   

Best regards

[Aaryaveerkrishna23]

Hi @Aaryaveerkrishna23, There are two possible reasons:

1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.
2. Maybe you miss the base folder under auxiliary_decoder, please check it.
   

Best regards

Thanks for responding "I was getting this error , which I have pasted below also I have installed the base folder .
my model path is """model = './ChartVLM-base' #${PATH_TO_PRETRAINED_MODEL} """"My doubt was which file exactly my model path is pointing to ?

Traceback (most recent call last):
File "/work/Arya/ChartVLM/ChartVLM/inference.py", line 10, in
output = infer_ChartVLM(image, text, model)
File "/home/jovyan/work/Arya/ChartVLM/ChartVLM/tools/ChartVLM.py", line 20, in infer_ChartVLM
num = infer_adapter(text, model)
File "/work/Arya/ChartVLM/ChartVLM/adapter/model_adapter.py", line 31, in infer_adapter
vectorizer = pickle.load(file)
_pickle.UnpicklingError: in
valid load key, 'v'.

[renqiux0302]

In inference.py, the path points to the top-level directory of ChartVLM (i.e. ./ChartVLM-base).

We have modify the individual path to every module in adapter/model_adapter.py, base_decoder/model_base_decoder.py and auxiliary_decoder/model_auxiliary_decoder.py respectively.

[zhanzhaoli]

Hi @Aaryaveerkrishna23, There are two possible reasons:

1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.
2. Maybe you miss the base folder under auxiliary_decoder, please check it.
   

Best regards

Thanks for responding "I was getting this error , which I have pasted below also I have installed the base folder . my model path is """model = './ChartVLM-base' #${PATH_TO_PRETRAINED_MODEL} """"My doubt was which file exactly my model path is pointing to ?

Traceback (most recent call last): File "/work/Arya/ChartVLM/ChartVLM/inference.py", line 10, in output = infer_ChartVLM(image, text, model) File "/home/jovyan/work/Arya/ChartVLM/ChartVLM/tools/ChartVLM.py", line 20, in infer_ChartVLM num = infer_adapter(text, model) File "/work/Arya/ChartVLM/ChartVLM/adapter/model_adapter.py", line 31, in infer_adapter vectorizer = pickle.load(file) _pickle.UnpicklingError: in valid load key, 'v'.

I had the same issue. It turned out that git-lfs was not installed prior to downloading the pre-trained weights from Huggingface. Without git-lfs, only the pointer files were downloaded, not the actual pickle files, which caused the _pickle.UnpicklingError.

For future reference, I got it solved by following the top answer in this post. You may also refer to this guide if you don't have sudo access to install git-lfs.