generating manifest file for training titanet-large

[ukemamaster]

Hi,
I am trying to train the titanet-large model with a huge dataset (10M+ samples) and it is taking more than 20 hours (still running) to generate the manifest.json file for trainig data. i run:
python NeMo/scripts/speaker_tasks/filelist_to_manifest.py --filelist path/to/train.txt --id -2 --out path/to/manifest.json --split
Is there any possibility to make it faster? or to skip this step and be able to train the model with the files list .txt?

[ukemamaster]

Increasing the max_workers in lines = process_map(get_duration, lines, chunksize=100, max_workers=128) solved the problem.