From 84cc78ea42008208a521fe9a1fd6bfcc5ab67223 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Wed, 20 Aug 2025 12:23:12 +0200 Subject: [PATCH 1/6] add a page on local apps --- docs/hub/local-apps.md | 93 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/hub/local-apps.md diff --git a/docs/hub/local-apps.md b/docs/hub/local-apps.md new file mode 100644 index 000000000..0fe559fda --- /dev/null +++ b/docs/hub/local-apps.md @@ -0,0 +1,93 @@ +# Use AI Models Locally + +You can run AI models from the Hub locally on your machine. This means that you can benefit from these advantages: + +- **Privacy**: You won't be sending your data to a remote server. +- **Speed**: Your hardware is the limiting factor, not the server or connection speed. +- **Control**: You can configure models to your liking. +- **Cost**: You can run models locally without paying for an API provider. + +## How to Use Local Apps + +Local apps are applications that can run Hugging Face models directly on your machine. To get started: + +1. **Enable local apps** in your [Local Apps settings](https://huggingface.co/settings/local-apps). + +![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/settings.png) + +1. **Choose a supported model** from the Hub by searching for it. + +![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/search_llamacpp.png) + +3. **Select the local app** from the "Use this model" dropdown on the model page. + +![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/button.png) + +4. **Copy and run** the provided command in your terminal. + +![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/command.png) + +## Supported Local Apps + +The best way to check if a local app is supported is to go to the Local Apps settings and see if the app is listed. Here is a quick overview of some of the most popular local apps: + + + +To use these local apps, copy the snippets from the model card as above. + + + +### Llama.cpp + +Llama.cpp is a high-performance C/C++ library for running LLMs locally with optimized inference across different hardware. If you are running a CPU, this is the best option. + +**Advantages:** +- Extremely fast performance for CPU-based models +- Low resource usage +- Multiple interface options (CLI, server, Python library) +- Hardware-optimized for CPU and GPU + +To use Llama.cpp, navigate to the model card and click "Use this model" and copy the command. + +```sh +# Load and run the model: +./llama-server -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M +``` + +### LM Studio + +LM Studio is a desktop application that provides an easy way to download, run, and experiment with local LLMs. + +**Advantages:** +- Intuitive graphical interface +- Built-in model browser +- Developer tools and APIs +- Free for personal and commercial use + +Navigate to the model card and click "Use this model". LM Studio will open and you can start chatting through the interface. + +### Jan + +Jan is an open-source ChatGPT alternative that runs entirely offline with a user-friendly interface. + +**Advantages:** +- Complete privacy (all data stays local) +- User-friendly GUI +- Chat with documents and files +- OpenAI-compatible API server + +To use Jan, navigate to the model card and click "Use this model". Jan will open and you can start chatting through the interface. + +### Ollama + +Ollama is an application that lets you run large language models locally on your computer with a simple command-line interface. + +**Advantages:** +- Easy installation and setup +- Direct integration with Hugging Face Hub + +To use Ollama, navigate to the model card and click "Use this model" and copy the command. + +```sh +ollama run hf.co/unsloth/gpt-oss-20b-GGUF:Q4_K_M +``` \ No newline at end of file From 848df568d8c7180f2d3f05ba7856d7e141e44fad Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Wed, 20 Aug 2025 12:23:19 +0200 Subject: [PATCH 2/6] add to menu --- docs/hub/_toctree.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index d98591767..fcdf94084 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -134,6 +134,8 @@ title: Frequently Asked Questions - local: model-release-checklist title: Model Release Checklist + - local: local-apps + title: Local Apps - local: models-advanced title: Advanced Topics sections: From bb01c7ed3140d0a4f9e1109625f4984e716b04ff Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 25 Aug 2025 14:25:46 +0200 Subject: [PATCH 3/6] move FAQ back to bottoms of models menu --- docs/hub/_toctree.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index fcdf94084..11770ef67 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -130,12 +130,12 @@ title: Model Inference - local: models-download-stats title: Models Download Stats - - local: models-faq - title: Frequently Asked Questions - local: model-release-checklist title: Model Release Checklist - local: local-apps title: Local Apps + - local: models-faq + title: Frequently Asked Questions - local: models-advanced title: Advanced Topics sections: From 6531d948ad6d510613420a7e40a8d4706c30a045 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 25 Aug 2025 14:31:45 +0200 Subject: [PATCH 4/6] respond to feedback Co-authored-by: Pedro Cuenca --- docs/hub/local-apps.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/hub/local-apps.md b/docs/hub/local-apps.md index 0fe559fda..f658bb0ae 100644 --- a/docs/hub/local-apps.md +++ b/docs/hub/local-apps.md @@ -33,19 +33,21 @@ The best way to check if a local app is supported is to go to the Local Apps set -To use these local apps, copy the snippets from the model card as above. +👨‍💻 To use these local apps, copy the snippets from the model card as above. + +👷 If you're building a local app, you can learn about integrating with the Hub in [this guide](https://huggingface.co/docs/hub/en/models-adding-libraries). ### Llama.cpp -Llama.cpp is a high-performance C/C++ library for running LLMs locally with optimized inference across different hardware. If you are running a CPU, this is the best option. +Llama.cpp is a high-performance C/C++ library for running LLMs locally with optimized inference across lots of different hardware, including CPUs, CUDA and Metal. **Advantages:** -- Extremely fast performance for CPU-based models +- Extremely fast performance for CPU-based models on multiple CPU families - Low resource usage - Multiple interface options (CLI, server, Python library) -- Hardware-optimized for CPU and GPU +- Hardware-optimized for CPUs and GPUs To use Llama.cpp, navigate to the model card and click "Use this model" and copy the command. @@ -71,10 +73,10 @@ Navigate to the model card and click "Use this model". LM Studio will open and y Jan is an open-source ChatGPT alternative that runs entirely offline with a user-friendly interface. **Advantages:** -- Complete privacy (all data stays local) - User-friendly GUI - Chat with documents and files - OpenAI-compatible API server +- OpenAI-compatible API server, so you can run models and use them from other apps To use Jan, navigate to the model card and click "Use this model". Jan will open and you can start chatting through the interface. From 3489a93474c67a5a7afdcd627cf8f99df2fe7fe9 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 25 Aug 2025 14:39:31 +0200 Subject: [PATCH 5/6] reorder apps --- docs/hub/local-apps.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/hub/local-apps.md b/docs/hub/local-apps.md index f658bb0ae..2aa7e21ac 100644 --- a/docs/hub/local-apps.md +++ b/docs/hub/local-apps.md @@ -56,17 +56,20 @@ To use Llama.cpp, navigate to the model card and click "Use this model" and copy ./llama-server -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M ``` -### LM Studio -LM Studio is a desktop application that provides an easy way to download, run, and experiment with local LLMs. +### Ollama + +Ollama is an application that lets you run large language models locally on your computer with a simple command-line interface. **Advantages:** -- Intuitive graphical interface -- Built-in model browser -- Developer tools and APIs -- Free for personal and commercial use +- Easy installation and setup +- Direct integration with Hugging Face Hub -Navigate to the model card and click "Use this model". LM Studio will open and you can start chatting through the interface. +To use Ollama, navigate to the model card and click "Use this model" and copy the command. + +```sh +ollama run hf.co/unsloth/gpt-oss-20b-GGUF:Q4_K_M +``` ### Jan @@ -75,21 +78,18 @@ Jan is an open-source ChatGPT alternative that runs entirely offline with a user **Advantages:** - User-friendly GUI - Chat with documents and files -- OpenAI-compatible API server - OpenAI-compatible API server, so you can run models and use them from other apps To use Jan, navigate to the model card and click "Use this model". Jan will open and you can start chatting through the interface. -### Ollama +### LM Studio -Ollama is an application that lets you run large language models locally on your computer with a simple command-line interface. +LM Studio is a desktop application that provides an easy way to download, run, and experiment with local LLMs. **Advantages:** -- Easy installation and setup -- Direct integration with Hugging Face Hub - -To use Ollama, navigate to the model card and click "Use this model" and copy the command. +- Intuitive graphical interface +- Built-in model browser +- Developer tools and APIs +- Free for personal and commercial use -```sh -ollama run hf.co/unsloth/gpt-oss-20b-GGUF:Q4_K_M -``` \ No newline at end of file +Navigate to the model card and click "Use this model". LM Studio will open and you can start chatting through the interface. From 37689da5d83262c77c1d0d4c6fb0d866f56c3a3f Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Tue, 26 Aug 2025 14:27:30 +0200 Subject: [PATCH 6/6] add filter by app Co-authored-by: Pedro Cuenca --- docs/hub/local-apps.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/local-apps.md b/docs/hub/local-apps.md index 2aa7e21ac..81834863a 100644 --- a/docs/hub/local-apps.md +++ b/docs/hub/local-apps.md @@ -15,7 +15,7 @@ Local apps are applications that can run Hugging Face models directly on your ma ![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/settings.png) -1. **Choose a supported model** from the Hub by searching for it. +1. **Choose a supported model** from the Hub by searching for it. You can filter by `app` in the `Other` section of the navigation bar: ![Local Apps](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local-apps/search_llamacpp.png)