Guide for End-to-End Connection with Gemini in PromptFlow

[anusonawane]

Hi PromptFlow team,

I am trying to integrate Gemini with PromptFlow but couldn't find a detailed guide on setting up an end-to-end connection. Could you provide a step-by-step guide or point me to relevant documentation?

Specifically, I need help with:

• Setting up authentication for Gemini
• Configuring PromptFlow to use Gemini as an LLM
• Example use cases or workflows

Any guidance would be greatly appreciated!

I am also trying to integrate Gemini with PromptFlow using CustomConnection, but I am encountering an error when running pf.flow.test
ResolveToolError: Tool load failed in 'classify_with_llm': (InvalidConnectionType) Connection type CustomConnection is not supported for LLM.
Here is how I have configured my flow.dag.yaml:
- name: classify_with_llm type: CustomConnection source: type: code path: classify_with_llm.jinja2 inputs: model: "gemini-1.5-flash" max_tokens: 128 temperature: 0.2 url: "${inputs.url}" text_content: "${summarize_text_content.output}" examples: "${prepare_examples.output}" api: chat connection: gemini_connection aggregation: false

And my connections/gemini_connection.yaml:

$schema: https://azuremlschemas.azureedge.net/promptflow/latest/CustomConnection.schema.json name: gemini_connection type: custom configs: endpoint: "https://generativelanguage.googleapis.com" secrets: api_key: "api_key"

Does PromptFlow support CustomConnection for LLMs like Gemini?
If not, what is the correct way to configure Gemini as an LLM connection in PromptFlow?

Thanks in advance.

[vsantele]

Hi,

You cannot use a CustomConnection with the built-in LLM tool. Only an AzureOpenAiConnection, OpenAiConnection, or ServerlessConnection. Source

The easiest way to use Gemini models is probably with the experimental openai endpoint from google ai

To do that, create a new OpenAiConnection with the property base_url set to https://generativelanguage.googleapis.com/v1beta/openai/

Warning: I didn't test this solution. Maybe some features will not be available. And I'm not part of the promptflow team.

[FabianSchurig]

#3972 (comment)
#3972 (comment)

There were two issues, very similar. I answered on the other one.

It is working with my custom Semantic Kernel LLM Chat Tool.

#3972

[github-actions]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!