Skip to content

Latest commit

 

History

History

langgraph-anthropic

README.md

LangGraph Agent Serving with BentoML

This project implements a LangGraph agent powered by the Anthropic Claude 3.5 Sonnet model.

Overview

  • AI Agent: Utilizes LangGraph and Anthropic Claude 3.5 Sonnet to build an AI agent that can search the web.
  • API: Provides a REST API for easy integration.
  • Flexible Invocation: Supports both synchronous and asynchronous (queue-based) interactions.
  • Deployment Options: Run locally or deploy to BentoCloud for scalability.

Getting Started

This project is a reference implementation designed to be hackable. Download the source code and use it as a playground to build your own agent APIs:

git clone https://github.com/bentoml/BentoLangGraph.git
cd BentoLangGraph/langgraph-anthropic

Run locally

Install dependencies

pip install -r requirements.txt

Set Anthropic API Key:

export ANTHROPIC_API_KEY=<your-api-key>

Spin up the REST API server:

bentoml serve .

Invoke the endpoint

Invoke with Python API client auto-generated by BentoML:

import bentoml
client = bentoml.SyncHTTPClient("http://localhost:3000")
response = client.invoke("What is the weather in San Francisco today?")
print(response)

Invoke with CURL:

curl -X POST http://localhost:3000/invoke \
   -H 'Content-Type: application/json' \
   -d '{"query": "what is the weather in San Francisco today?"}'

Example Output:

The current weather in San Francisco is sunny and mild. Here are the key points:

1. Temperature: The high temperature for today is expected to be around 72°F (22°C).

2. Conditions: It's currently sunny, with increasing clouds expected later tonight.

3. Tonight's forecast: The low temperature tonight is predicted to be around 55°F (13°C).

4. Tomorrow (Sunday): The high temperature is expected to reach about 75°F (24°C).

5. Overall, it appears to be a pleasant day in San Francisco with clear skies and comfortable temperatures.

Please note that this information is based on the most recent forecast available, and weather conditions can change. If you need more specific details or an updated forecast for a particular time of day, feel free to ask, and I can search for more precise information.

Invoke via Async Queue

Submit task to queue:

$ curl -X POST http://localhost:3000/invoke/submit \
   -H 'Content-Type: application/json' \
   -d '{"query": "what is the weather in San Francisco today?"}'

{"task_id":"b1fe7960470740ac9be58dcf740ee587","status":"in_progress"}

Check status of task:

$ curl -s http://localhost:3000/invoke/status?task_id=$TASK_ID

{"task_id":"40451e21a6834c279d78433c5e1a4083","status":"success",
"created_at":"2024-09-23T05:09:36","executed_at":"2024-09-23T05:09:36"}

Get result of task:

$ curl -s http://localhost:3000/invoke/get?task_id=$TASK_ID

{"task_id":"40451e21a6834c279d78433c5e1a4083","status":"success",
"created_at":"2024-09-23T05:09:36","executed_at":"2024-09-23T05:09:36"}

Development

Start development server that will auto reload when code changes:

bentoml serve . --reload

Inspect all event streams:

curl -X POST http://localhost:3000/stream \
   -H 'Content-Type: application/json' \
   -d '{"query": "what is the weather in San Francisco today?"}'

{'event': 'on_chain_start', 'data': ...}
{'event': 'on_chain_end', 'data': ...}
{'event': 'on_tool_start', 'data': ...}
{'event': 'on_tool_end', 'data': ...}
...

Deploy to BentoCloud

Login to BentoCloud:

pip install bentoml
bentoml cloud login

Create secret:

bentoml secret create anthropic ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY

Deploy:

bentoml deploy . --name search-agent --secret anthropic

Invoke the endpoint:

DEPLOYED_ENDPOINT=$(bentoml deployment get search-agent -o json | jq -r ".endpoint_urls[0]")
python client.py --query "What's the weather in San Francisco today?" --url $DEPLOYED_ENDPOINT