Why temperature=0,top_p=1,seed=42 is still not enough to fix the llm's output!?

[beyondguo]

We are using Qwen2.5-14B-Instruct with vLLM. However, we found the following things can make the output different, even we set temperature=0,top_p=1,seed=42:

• vllm serve is different with vllm offline inference, using the same chat_template
• vllm serve with different number of cards
• different vllm version
• using H100 or H200 can make a difference

That is strange. Can someone tell me why? and how can I fix the output, when changing inference enveriments?