@@ -20,15 +20,20 @@ Use OpenVINO™ Test Drive to:
20
20
21
21
* **Chat with LLMs ** and evaluate model performance on your computer or edge device;
22
22
* **Experiment with different text prompts ** to generate images, using Stable
23
- Diffusion and Stable DiffusionXL models (coming soon) ;
23
+ Diffusion and Stable DiffusionXL models;
24
24
* **Transcribe speech from video **, using Whisper models, including generation
25
- of timestamps (coming soon) ;
25
+ of timestamps;
26
26
* **Run inference of models ** trained by Intel® Geti™ and **visualize the results **.
27
27
28
28
29
29
Installation (Windows)
30
30
###############################################################################################
31
31
32
+ .. important ::
33
+
34
+ For Intel® NPU, use the latest available version of
35
+ `Intel® NPU Driver <https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html >`__.
36
+
32
37
1. Download the latest archive from the
33
38
`release repository <https://storage.openvinotoolkit.org/repositories/openvino_testdrive/packages >`__.
34
39
To verify the integrity of the downloaded package, use the SHA-256 file attached.
@@ -45,7 +50,7 @@ Quick start
45
50
When starting the application, you can import an LLM model from Hugging Face Hub
46
51
or upload an Intel® Geti™ model from a local drive.
47
52
48
- Inference of models from Hugging Face
53
+ Text generation and LLM performance evaluation
49
54
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
50
55
51
56
1. Find a model on `Hugging Face <https://huggingface.co/ >`__ and import it.
@@ -54,7 +59,8 @@ Inference of models from Hugging Face
54
59
:align: center
55
60
:alt: how to import a model to test drive
56
61
57
- 2. Chat with LLMs via the `Playground ` tab.
62
+ 2. Chat with LLMs via the `Playground ` tab. You can export an LLM by clicking
63
+ the `Export model ` button.
58
64
59
65
.. image :: ../../../assets/images/TestDrive_llm_model_chat.gif
60
66
:align: center
@@ -67,6 +73,86 @@ Inference of models from Hugging Face
67
73
:align: center
68
74
:alt: verifying llm performance in test drive
69
75
76
+
77
+ Retrieval-Augmented Generation with LLMs
78
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
79
+
80
+ 1. Upload files and create knowledge base for RAG (Retrieval-Augmented Generation),
81
+ using `Knowledge base ` tab.
82
+
83
+ .. image :: ../../../assets/images/TestDrive_rag_base.gif
84
+ :align: center
85
+ :alt: creating a knowledge base for RAG in test drive
86
+
87
+ The knowledge base can be used for text generation with LLM models.
88
+
89
+ .. image :: ../../../assets/images/TestDrive_rag_1.gif
90
+ :align: center
91
+ :alt: using a knowledge base for text generation with LLMs
92
+
93
+ 2. You can also upload a document directly, using the `Playground`` tab.
94
+
95
+ .. image :: ../../../assets/images/TestDrive_rag_2.gif
96
+ :align: center
97
+ :alt: uploading a document to the knowledge base
98
+
99
+
100
+ Image analysis with Visual Language Models (VLMs)
101
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
102
+
103
+ 1. Import a VLM for image analysis.
104
+
105
+ .. image :: ../../../assets/images/TestDrive_vlm_1.gif
106
+ :align: center
107
+ :alt: importing a visual language model for image analysis
108
+
109
+ 2. Select the VLM from the `My models ` section, upload an image and analyze it.
110
+
111
+ .. image :: ../../../assets/images/TestDrive_vlm_2.gif
112
+ :align: center
113
+ :alt: importing a visual language model for image analysis
114
+
115
+
116
+ Video transcription with Whisper models
117
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
118
+
119
+ 1. Import a Whisper model for video transcription.
120
+
121
+ .. image :: ../../../assets/images/TestDrive_st_import.gif
122
+ :align: center
123
+ :alt: importing a Whisper model for video transcription
124
+
125
+ 2. Select the speech-to-text LLM from the `My models ` section, and upload a video for transcription.
126
+
127
+ .. image :: ../../../assets/images/TestDrive_ts_video.gif
128
+ :align: center
129
+ :alt: importing a visual language model for image analysis
130
+
131
+ You can search for words in the transcript or download it.
132
+
133
+ 3. Use the `Performance metrics ` tab to get performance metrics of the LLM on your computer
134
+ or an edge device.
135
+
136
+
137
+ Image generation with LLMs
138
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
139
+
140
+ 1. Import an image generation LLM from a predefined set of popular models or from
141
+ `Hugging Face <https://huggingface.co/ >`__, using `Import model ` -> `Hugging Face `.
142
+
143
+ 2. Select the LLM from the `My models ` section and start the chat to generate an image.
144
+ You can export the model by clicking the `Export model ` button.
145
+
146
+ .. image :: ../../../assets/images/TestDrive_image_generation.gif
147
+ :align: center
148
+ :alt: image generation with a chosen LLM
149
+
150
+ You can download the generated image.
151
+
152
+ 3. Use the `Performance metrics ` tab to get performance metrics of the LLM on your computer
153
+ or an edge device.
154
+
155
+
70
156
Inference of models trained with Intel® Geti™
71
157
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
72
158
0 commit comments