/generate for image generation

[nsarrazin]

Simpler than agents, just uses /generate [prompt] to generate images.

Contains:

• Support for file uploads to mongodb from the backend
• Serving files from mongodb
• Letting the front-end know about files in message
• Modular tools env variable (tools defined in .env using TOOLS var, see readme)

[nsarrazin]

I played with settings a bit, I think

TOOLS = `[
  {
    "name" : "textToImage",
    "model" : "stabilityai/stable-diffusion-xl-base-1.0",
    "parameters" : {
      "num_inference_steps" : 25,
      "guidance_scale": 8
    }
  },

gives the best balance between quality & performance, since we're not using the refiner. We could increase the number of steps but the trade off would be extra load on the endpoint/slower response time.

[nsarrazin]

For visibility on the feature we can use one of the examples prompt in the UI:

    "promptExamples": [
      {
        "title": "Draw a llama with SDXL 📷",
        "prompt": "/generate funky colorful llama wearing sunglasses, cartoon"
      }, {
        "title": "Code a snake game",
        "prompt": "Code a basic snake game in python, give explanations for each step."
      }, {
        "title": "Assist in a task",
        "prompt": "How do I make a delicious lemon cheesecake?"
      }
    ],

[julien-blanchon]

Cool stuff ! How it's going on ?

[nsarrazin]

Closing this for now, I think we can implement it easily if we want when #531 is done.

[nipierre]

Hi @nsarrazin, any news on this feature as #531 merged ? :)