Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ascend-npu,执行模型评估脚本时,出现IndexError: list index out of range #7574

Closed
1 task done
doit-5618 opened this issue Apr 2, 2025 · 2 comments
Closed
1 task done
Labels
npu This problem is related to NPU devices solved This problem has been already solved

Comments

@doit-5618
Copy link

Reminder

  • I have read the above rules and searched the existing issues.

System Info

(llamafactory2) root@unicom:/mnt/llm-technology/LLaMA-Factory# llamafactory-cli env
[2025-04-02 17:23:39,239] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to npu (auto detect)

  • llamafactory version: 0.9.3.dev0
  • Platform: Linux-5.15.0-131-generic-aarch64-with-glibc2.35
  • Python version: 3.10.16
  • PyTorch version: 2.4.0 (NPU)
  • Transformers version: 4.49.0
  • Datasets version: 3.3.2
  • Accelerate version: 1.4.0
  • PEFT version: 0.12.0
  • TRL version: 0.9.6
  • NPU type: Ascend910B2
  • CANN version: 8.0.0
  • DeepSpeed version: 0.16.4
  • Git commit: 555b71a

Reproduction

我参考https://ascend.github.io/docs/sources/llamafactory/example.html#benchmark,使用下面的脚本进行模型评估。
llamafactory-cli eval \
    --model_name_or_path /mnt/llm-technology/LLaMA-Factory/saves/sft-41-full \
    --template fewshot \
    --task mmlu \
    --split validation \
    --lang en \
    --n_shot 5 \
    --batch_size 1
报错:llamafactory-cli: error: ambiguous option: --split could match --split_special_tokens, --split-special-tokens
随后我删除--split,报错IndexError: list index out of range 。
可以执行训练、推理。

Others

No response

@doit-5618 doit-5618 added bug Something isn't working pending This problem is yet to be addressed labels Apr 2, 2025
@github-actions github-actions bot added the npu This problem is related to NPU devices label Apr 2, 2025
@Feihustc
Copy link

Feihustc commented Apr 9, 2025

me too

@doit-5618
Copy link
Author

me too

改为:--task ceval_validation

@hiyouga hiyouga added solved This problem has been already solved and removed bug Something isn't working pending This problem is yet to be addressed labels Apr 9, 2025
@hiyouga hiyouga closed this as completed Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
npu This problem is related to NPU devices solved This problem has been already solved
Projects
None yet
Development

No branches or pull requests

3 participants