Agents

[nsarrazin]

Still a WIP

Requires huggingface/huggingface.js#249 as a linked package in local dev mode to work

Roadblocks:

• Need to hook-up agents updates to websearch collapsable modal
• Need to check the streaming tokens to extract the final answer as it comes. (right now we only parse the JSON when it's done streaming but we would like to stream as it comes in)
• SDXL pipeline requires base + refiner. We don't currently have an endpoint for the refiner, but it's being worked on internally. base model works fine though.
• need to check prompts, final answer doesn't reference files for some reason.

[gururise]

WOW! This is amazing. I hope the Agents will be modular/extensible. For example, be able to call replicate SDXL API to use SDXL.
Would love to be able to use Agents with OpenAI or other LLM's (see PR #452)

[nsarrazin]

Tests are failing because we need the new agents release for them to pass.

Road to merging:

1. Merge Agents in chat ui huggingface.js#249
2. Do a release on huggingface.js
3. Add the new release to the deps of chat-ui
4. Merge this PR
5. Deploy to prod

Notes:

• We still don't have a pipeline for SDXL that's base + refiner, or optimized for speed, not sure if we want it before merging
• We should let the team know before doing comms so we can ensure the sdxl endpoint doesn't die

Let me know what you think @gary149

[nsarrazin]

Closing this for now, will come back to this later 😁

[hungryalgo]

This is such an amazing and huge feature, when are you @nsarrazin planning to come back to this?
I am guess that you are waiting for the SDXL base + refiner endpoint to be ready?

Will you integrate some of the file drop UI from #442?

[julien-blanchon]

@hungryalgo, I think the dropzone UI was already integrated with #531 and live here https://github.com/huggingface/chat-ui/blob/3cbea3450ddaf06d09a7790c9068bc08340dd425/src/lib/components/chat/FileDropzone.svelte