llama.cpp and 7G Llama-2-Chat model: resource requirements, if possible #5172
Unanswered
christian-2
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am a new to this project and would like to try inference with llama.cpp and a 7G Llama-2-Chat model: is this combination currently supported and what are the resource requirements? I have e.g. a VM with 8 vCPUs and 16 GB RAM or (if need be) a bare-medal server with 56 CPUs and 500 GB RAM available. The (underlying) hardware is fairly recent in both cases. I guess the bare-metal second could serve in principle, but could the VM as well?
Beta Was this translation helpful? Give feedback.
All reactions