Skip to content

Position embedding diff when loading model in different method #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
yangjiabupt opened this issue May 12, 2025 · 1 comment · May be fixed by huggingface/transformers#38151
Open

Position embedding diff when loading model in different method #271

yangjiabupt opened this issue May 12, 2025 · 1 comment · May be fixed by huggingface/transformers#38151

Comments

@yangjiabupt
Copy link

model = Qwen2_5OmniForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-Omni-7B", torch_dtype="auto", device_map="auto")

When enable torch_dtype="auto"

the audio tower dtype is float32

Image

disable torch_dtype="auto"

the audio tower dtype is also float32

but the value diff

Image

I think the first got wrong
BakerBunker added a commit to BakerBunker/transformers that referenced this issue May 15, 2025
@BakerBunker
Fix Qwen2.5 Omni SinusoidsPositionEmbedding precision
f16be1e 
fixes QwenLM/Qwen2.5-Omni#271
@BakerBunker BakerBunker linked a pull request May 15, 2025 that will close this issue
Open
5 tasks
@BakerBunker
Copy link
Collaborator

Will be fixed at huggingface/transformers#38151

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Qwen2.5 Omni SinusoidsPositionEmbedding precision BakerBunker/transformers
2 participants
@BakerBunker @yangjiabupt