Tuned Model's processor can't tokenize my text input properly.

[miko8422]

Reminder

• I have read the above rules and searched the existing issues.

System Info

OK, Let's cut through, I'm a dummy coder, I thought it would be much easier for me to use webui instead of bash code, I just quick configured my webui interface and tuned my model.

I tuned my model with completely default setting, lora, on a image-text-to-text task. After I tuned the model, it chats properly, and I've successfully uploaded the model to huggingface, which is much easier for me to use consider later publication.

And the problem happend, during the model export, I also used the default setting you can find in llamafactory's webui, with max shard size 5GB, none quantization, export quantization dataset default set to “/data/c4_demo.json”. I exported the model in local path and push to huggingface at the same time. And successfully uploaded it. But when I try to do a simple inference, I had some problem here.

`from transformers import Qwen2VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from PIL import Image

model = Qwen2VLForConditionalGeneration.from_pretrained(
"CAS-SIAT-XinHai/mllmChem10K", # Qwen/Qwen2-VL-7B-Instruct
device_map="auto")

min_pixels=2242828
max_pixels=2242828
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct", min_pixels=min_pixels, max_pixels=max_pixels)`

I had to use the original model to properly tokenzie my text input, the image were handled properly, but not the text. it showed empty when I tried to print it out. but when I use the base model's processor, it works fine.

I wonder what caused this problem, and is it because I didn't upload my model properly to huggingface side?

Reproduction

  File "/data/home/Neroism/WorkSpace/ModelEvalTest/Demo/try_qwenvl7B.py", line 45, in <module>
    output_ids = model.generate(**inputs, max_new_tokens=128)
  File "/data/home/miniforge3/envs/LlamaFactory/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/data/home/miniforge3/envs/LlamaFactory/lib/python3.10/site-packages/transformers/generation/utils.py", line 2326, in generate
    result = self._sample(
  File "/data/home/miniforge3/envs/LlamaFactory/lib/python3.10/site-packages/transformers/generation/utils.py", line 3279, in _sample
    model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
  File "/data/home/miniforge3/envs/LlamaFactory/lib/python3.10/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py", line 1792, in prepare_inputs_for_generation
    model_inputs = super().prepare_inputs_for_generation(
  File "/data/home/miniforge3/envs/LlamaFactory/lib/python3.10/site-packages/transformers/generation/utils.py", line 419, in prepare_inputs_for_generation
    or cache_position[-1] >= input_ids.shape[1]  # Exception 3
IndexError: index -1 is out of bounds for dimension 0 with size 0

Others

No response

[hiyouga]

Have you tried copying the original processor files to the new dictionary?