from autogen_ext.models.ollama import OllamaChatCompletionClient init #6155
Comments
|
First of all, this is not a bug. haha When using a model name that is not a valid OpenAI model, you need to provide the If you're trying to use a custom model like "gpt-4o-mini", you need to explicitly provide the model info like this: model_client = OllamaChatCompletionClient(
model="chevalblanc/gpt-40-mini:latest",
host="http://192.168.1.155:11434/api/chat",
model_info = {
"vision": False,
"function_calling": False,
"json_output": False,
"family": "unknown"
"structured_output": False,
}
)You can adjust the capabilities (vision, function_calling, etc.) based on what your model actually supports. If you're unsure about the model's capabilities, starting with all set to False and then adjusting as needed is a reasonable approach. For reference, here are the models currently defined in the model_info dictionary: Hope this helps! If you could solved it, with that direction, please close that issue. |
|
非常感谢！thanks！ import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_core import CancellationToken
from autogen_ext.tools.mcp import StdioServerParams, mcp_server_tools
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.ollama import OllamaChatCompletionClient
from dotenv import load_dotenv
import autogen
import os
# 加载环境变量
load_dotenv()
llm_config={"config_list": [
{
"model": "deepseek-chat",
"api_key": os.getenv("DEEPSEEK_API_KEY"),
"base_url": os.getenv("DEEPSEEK_BASE_URL"),
"tags":["花钱的"]
}]}
async def main():
# Setup the MCP fetch server parameters
fetch_mcp_server = StdioServerParams(command="node", args=["E:/ruanjian/Trae/work/fetch-mcp-server/fetch-mcp-main/dist/index.js"])
# Get the fetch tool from the MCP server
tools = await mcp_server_tools(fetch_mcp_server)
#model_client = OpenAIChatCompletionClient(model="gpt-4o")
model_client = OllamaChatCompletionClient(
model="qwen2.5:72b",
host="http://127.0.0.1:11435/v1/",
model_info = { #不是openai的模型需要配置这个
"vision": False,
"function_calling": True,
"json_output": True,
"family": "unknown",
"structured_output": True,
}
)
print(model_client)
fetch_agent = AssistantAgent(
name="content_fetcher",
system_message="你是一个网页内容获取助手。使用fetch工具获取网页内容。",
model_client=model_client,
tools=tools
)
# Create rewriter Agent (unchanged)
rewriter_agent = AssistantAgent(
name="content_rewriter",
system_message="""你是一个内容改写专家。将提供给你的网页内容改写为科技资讯风格的文章。
科技资讯风格特点：
1. 标题简洁醒目
2. 开头直接点明主题
3. 内容客观准确但生动有趣
4. 使用专业术语但解释清晰
5. 段落简短，重点突出
当你完成改写后，回复TERMINATE。""",
model_client=model_client
)
# Set up termination condition and team (unchanged)
termination = TextMentionTermination("TERMINATE")
team = RoundRobinGroupChat([fetch_agent, rewriter_agent], termination_condition=termination)
# Run the workflow (unchanged)
result = await team.run(
task="获取https://www.aivi.fyi/llms/introduce-Claude-3.7-Sonnet的内容，然后将其改写为科技资讯风格的文章",
cancellation_token=CancellationToken()
)
print("\n最终改写结果：\n")
print(result.messages[-1].content)
return result
# This is the correct way to run async code in a Python script
if __name__ == "__main__":
asyncio.run(main())code end I can execute the above code normally using gpt4, but switching to something else like qwen2.5:72b will result in the following error error start error end |
|
Can you debug? What is the request got sent out to the Ollama server? Looks like you are getting 404 -- is the model running? |
|
|
|
|
|
For OllamaChatCompletionClient, you can just do this: model_client = OllamaChatCompletionClient(
model="qwen2.5:72b",
)I think there is some issue with your URL. Also, I edited your comment to add around code blocks. |
|
fetch-mcp-server.zip |
|
|
|
|
|
Let's check step by step for find real issue in there.
and, first of all, OllamaChatCompletionClient in AutoGen, maybe using ollama SDK. because,
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_core import CancellationToken
from autogen_ext.tools.mcp import StdioServerParams, mcp_server_tools
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.ollama import OllamaChatCompletionClient
from autogen_core.models import UserMessage
client = OllamaChatCompletionClient(
model="gemma2:2b",
host="http://localhost:11435",
api_key="ollama",
model_info={
"vision": False,
"function_calling": True,
"json_output": True,
"family": "unknown",
},
)
messages = [
UserMessage(content="hello", source="user"),
]
print(asyncio.run(client.create(messages=messages)))here is my testcase When I using same as you, I could find same error as that. client = OllamaChatCompletionClient(
model="llama3.1:latest",
host="127.0.0.1:11435/v1",
api_key="ollama",
)
messages = [
UserMessage(content="hello", source="user"),
]
print(asyncio.run(client.create(messages=messages)))Error |
|
And, at the AutoGen... Case client = OllamaChatCompletionClient(
model="llama3.1:latest",
host="121.212.1212:1212/",
api_key="ollama",
)
messages = [
UserMessage(content="hello", source="user"),
]
# print(asyncio.run(client.create(messages=messages)))
print(client)
print("DONE IT")Result, With out any error. |
|
import asyncio client = OllamaChatCompletionClient( messages = [ print(asyncio.run(client.create(messages=messages))) |
|
|
|
Umm why, at the code model name is |
|
7lama3:8b I wrote the wrong name during the first execution, just ignore it |
|
At present, it is not a problem to execute the test separately without calling MCP |
|
Cool, how about with MCP? However I do not know about MCP a lot. |
|
MCP cannot be used. Regarding this issue, some models may encounter errors when calling MCP. Who should I consult and what should I do? |
|
import asyncio async def main() -> None: asyncio.run(main())
Can you please help me check this issue? It won't work if we switch to Olama, and I found that the parameter formats of OpenAIChatCompletionClient and OllamaChatCompletionClient are also different, which may be the reason for the MCP error |
client1 = OllamaChatCompletionClient(
model="llama3.1:latest",
host="127.0.0.1:11435",
api_key="ollama",
)
client2 = OpenAIChatCompletionClient(
model="llama3.1:latest",
base_url="http://127.0.0.1:11435/v1",
api_key="ollama",
model_info={
"vision": False,
"function_calling": True,
"json_output": True,
"family": "unknown",
},
)
messages = [
UserMessage(content="hello", source="user"),
]
print("OLLAMA SDK : ", asyncio.run(client1.create(messages=messages)))
print("OPENAI SDK : ", asyncio.run(client2.create(messages=messages)))
print(client2)
print("DONE IT")Here is code snippet for OpenAI SDK aware version of ollama |
|
If you could solving your whole isssue. |
|
What I mainly want to ask is why calling MCP works with GPT4O, but it doesn't work with the Olama model and reports errors. This still hasn't been resolved. So I feel like there may still be bugs |
|
How about other function call (tool) is work? without MCP? |
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_core import CancellationToken
from autogen_ext.tools.mcp import StdioServerParams, mcp_server_tools
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.ollama import OllamaChatCompletionClient
import os
# 加载环境变量
async def main():
# Setup the MCP fetch server parameters
fetch_mcp_server = StdioServerParams(command="node", args=["./fetch-mcp-main/dist/index.js"])
# Get the fetch tool from the MCP server
tools = await mcp_server_tools(fetch_mcp_server)
#model_client = OpenAIChatCompletionClient(model="gpt-4o")
model_client = OllamaChatCompletionClient(
model="llama3.1:latest",
host="127.0.0.1:11435",
model_info = { #不是openai的模型需要配置这个
"vision": False,
"function_calling": True,
"json_output": True,
"family": "unknown",
"structured_output": True,
}
)
print(model_client)
fetch_agent = AssistantAgent(
name="content_fetcher",
system_message="你是一个网页内容获取助手。使用fetch工具获取网页内容。",
model_client=model_client,
tools=tools
)
# Create rewriter Agent (unchanged)
rewriter_agent = AssistantAgent(
name="content_rewriter",
system_message="""你是一个内容改写专家。将提供给你的网页内容改写为科技资讯风格的文章。
科技资讯风格特点：
1. 标题简洁醒目
2. 开头直接点明主题
3. 内容客观准确但生动有趣
4. 使用专业术语但解释清晰
5. 段落简短，重点突出
当你完成改写后，回复TERMINATE。""",
model_client=model_client
)
# Set up termination condition and team (unchanged)
termination = TextMentionTermination("TERMINATE")
team = RoundRobinGroupChat([fetch_agent, rewriter_agent], termination_condition=termination)
# Run the workflow (unchanged)
result = await team.run(
task="获取https://www.aivi.fyi/llms/introduce-Claude-3.7-Sonnet的内容，然后将其改写为科技资讯风格的文章",
cancellation_token=CancellationToken()
)
print("\n最终改写结果：\n")
print(result.messages[-1].content)
return result
# This is the correct way to run async code in a Python script
if __name__ == "__main__":
asyncio.run(main())Is it work in my pc. haha... |
|
Hi, my attempt is like this。Firstly, I ran the code you gave me.I also did not report the previous error and successfully initialized MCP. But there was a prompt, followed by an error message stating that the redirection can be done for port requests. As I am a server forwarding ports, it failed. Currently, it seems that you do not support this and the overall situation. Next, I would like to try changing some models and using this code to see why I couldn't do it before |
|
|
|
However, when I ran the same code for the second and third times, I got different results
|
What happened?
but use OllamaChatCompletionClient fail
Which packages was the bug in?
Python Extensions (autogen-ext)
AutoGen library version.
Python dev (main branch)
Other library version.
No response
Model used
No response
Model provider
None
Other model provider
No response
Python version
None
.NET version
None
Operating system
None
The text was updated successfully, but these errors were encountered: