Skip to content

Commit 05d8cfe

Browse files
committed
update doc
Signed-off-by: Yuki Huang <yukih@nvidia.com>
1 parent ddfc7b5 commit 05d8cfe

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

docs/design-docs/fsdp2-parallel-plan.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This guide outlines the parallelization strategy for FSDP2 training in NeMo-RL.
44

55
## Fallback Priority
66

7-
Three parallelization approaches are supported, which is falling back in this priority.
7+
Three parallelization approaches are supported, with the following fallback priority.
88

99
**Custom Parallel Plan**
1010

@@ -14,13 +14,13 @@ For implementation details and usage guidelines, please refer to [Custom Paralle
1414

1515
**Optimized Parallel Plan**
1616

17-
Optimized parallel plans are available for specific model architectures, potentially offering superior performance compared to the Hugging Face tensor parallel implementation.
17+
Optimized parallel plans are available for specific model architectures and may offer superior performance compared to the Hugging Face tensor parallel implementation.
1818

19-
This approach is employed when no custom parallel plan is specified and the model class supports optimized parallelization.
19+
This approach is used when no custom parallel plan is specified and the model class supports optimized parallelization.
2020

2121
**Hugging Face Tensor Parallel Plan**
2222

23-
Hugging Face provides tensor parallel of most models through `._tp_plan`.
23+
Hugging Face provides tensor parallelism for most models through `._tp_plan`.
2424

2525
It serves as the default when neither custom nor optimized parallel plans are available.
2626

0 commit comments

Comments
 (0)