Feature Request: Enable upload of files to chat with

[fsagini]

Would be great to have the user click on an upload button next to the input box and upload a file, which then gets embedded and can be used as context in the chat

[VatsaDev]

Thats a great feature, use it alot in claude. But isn't it not nearly as useful as it is with claude, because of How claude has a 100K context length, and it's good at document QA, while Llama has no training on that, and has a context length of 4096, which will fill up quick with documents.

[julien-blanchon]

This PR is a draft for this feature: #449, you can find a gif video in the corresponding issue #441.

This is even going bit further than the Claude implementation as this PR leverage Mathpix to convert the pdf to a markdown file with the latex equation and table in it. Claude just extract pure text from the document and don't process equation and table. However Mathpix is a paid API and most user don't need such powerfull transform.

If many peaple are looking for this feature might update the PR or open a new one with a more simple implementation, more like the Claude one.

If you want you can also improve my PR

[oxaronick]

It might be worth looking at the open-webui feature. It allows you to upload a PDF to a conversation and does RAG using the document for any messages in that conversation.

I would prefer this to having a PDF converted to markdown within the chat. My use case is bascially uploading PDFs of several hundred pages and having the LLM act as a librarian for that data.

Alternatively, if chat-ui could call an external URL to get the augmented prompt, I'd also be happy with that. However, I don't know if there is any sort of standard API for prompt augmentation. Seems like it's included within most apps like chat-ui.

[shrwnsan]

Just saw this earlier, "upload file" button in chat

🎉

[bruno-oliveira]

@nsarrazin @julien-blanchon Sorry to bring this back up.
Is there a possibility via the .env.local configuration file to enable file upload in the Chat UI?
So this would be just to parse the contents and add them in the prompt, similar to how other LLMs do it I believe.

I tried to look into the code and the repo but couldn't find anything actionable.

[AlexBlack2202]

i have same question