Introduce parallelism for batch conversion: --parallel
/ -P
#628
+313
−119
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces
--parallel
/-P
option to set the number of concurrency of parallel conversion for multiple files.# Convert Markdown files in `./marp-slides` parallel (up to 5 workers: default) marp --parallel 5 ./marp-slides
By applying the robust async process (async initialization is running only once, and all of successors will wait that initialization) to browser operations, the parallel processing could be achieved in a simple queue and workers.
Marp CLI will use up to 5 workers while converting multiple files in the queue by default. If used Marp CLI for converting multiple Markdowns into PDF/PPTX/image(s), this setting will make drastically faster conversion than previous. CLI user can tweak the number of concurrency by using
--parallel
or-P
option.Resolves #509.
Performance improvement
marp-parallel
folder is including 30 empty Markdown files, and Marp CLI tries to convert every Markdown files into PDF.time marp --pdf --no-parallel ./marp-parallel
time marp --pdf --parallel=5 ./marp-parallel
time marp --pdf --parallel=10 ./marp-parallel
Increasing the number of concurrency from 5 to 10, the conversion will become much faster, but the CPU usage may stick to 100% while conversion in my environment. For the balance between the speed and CPU, I decided the default concurrecny as 5.