New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is it possible to reduce GPU memory usage during inference? #19

Open

tshmak opened this issue Nov 23, 2023 · 0 comments

tshmak commented Nov 23, 2023

It appears it is taking >30GB of memory for inference. What are the parameters I can set which can reduce the demand?

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment