This repository has been archived by the owner on Oct 16, 2023. It is now read-only.
model parallel in multi node #3
Comments
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Hi, i want to know some information about model parallel. Does the current code support model parallelism under multiple nodes? such as i have two nodes, and each node have n gpus. model is split into n gpus use model parallel in each node. the two nodes use ddp.
The text was updated successfully, but these errors were encountered: