You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In my training process, I only need to train a portion of the parameters, and I know that the framework uses requires_grad=False to accomplish parameter freezing, but it seems that the full parameters are still added in the optimizer, so I'm wondering if there is a way to go about adding just the training parameters? Or, to take it a step further, I'd like to know how much of an impact it would have on the memory footprint if I added all of the parameters?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
In my training process, I only need to train a portion of the parameters, and I know that the framework uses requires_grad=False to accomplish parameter freezing, but it seems that the full parameters are still added in the optimizer, so I'm wondering if there is a way to go about adding just the training parameters? Or, to take it a step further, I'd like to know how much of an impact it would have on the memory footprint if I added all of the parameters?
Beta Was this translation helpful? Give feedback.
All reactions