why three special token is ignored in label

[Pride-Huang]

The label just consists of image token, with the special token <|image start|> ignored. Why compute sft loss like this?

[Masaaki-75]

This might partly answer the question: https://github.com/baaivision/Emu3/blob/main/emu3/mllm/processing_emu3.py#L178-L183

During inference, the boi_token (which is "<|image start|>" in string form), the resolution information, and the img_token (which is "<|image token|>") are directly provided as a starter for the generation. (BTW, the mismatch between the string form and the variable naming is confusing and annoying, lol)

That said, i am also curious why the authors limited the supervision to the first visual token id and the last visual token id, while ignoring eol_token, eof_token, and eoi_token.