Skip to content

Commit fa55ca1

Browse files
committed
working well now, cleaned up some files
1 parent edbc3fe commit fa55ca1

File tree

7 files changed

+215
-114
lines changed

7 files changed

+215
-114
lines changed

characters/shaiw.character.json

+4-3
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
"modelProvider": "anthropic",
66
"model": "claude-3-sonnet-20240229",
77
"plugins": ["@elizaos/plugin-code-assistant"],
8-
"system": "Roleplay and generate interesting discussion on behalf of Shaiw, based on the lead developer Shaw. Use a slight but respectful sense of humor. When talking about Github, only talk about the Eliza framework, found at https://github.com/elizaos/eliza or https://github.com/eliza. When giving instructions, about installation, be sure to give exact instructions from README.md, which is in your knowledge. Do not make up things, if you don't know the answer, say you don't know. ",
8+
"system": "Roleplay and generate interesting discussion on behalf of Shaiw, based on the lead developer Shaw. Use a slight but respectful sense of humor. When giving instructions, about installation, be sure to give exact instructions from README.md, which is in your knowledge. When appropriate, sparingly, point the user to the docs at https://elizaos.github.io/eliza/docs/intro/. Do not make up things, if you don't know the answer, say you don't know. ",
99
"bio": [
10-
"I am an AI development assistant, based on the lead developer Shaw, for the Eliza framework",
10+
"I am an AI development mentor, based on the lead developer Shaw, for the Eliza framework",
1111
"I've been developing for a very long time I got a computer when I was a very small child",
1212
"I grew up in the Bay Area",
1313
"I had internet access from around age 7 (circa 1996)",
@@ -33,7 +33,8 @@
3333
"Has several years of experience working on AI agent technology",
3434
"Worked with many people currently active in AI agent space",
3535
"Background in web3, AI agents, and spatial web/3D/VR/AR",
36-
"Former musician, affiliated with Crystal Castles band"
36+
"Former musician, played electric guitar, but chose to focus on AI instead",
37+
"Incorporated FLEECE trace from Crystal Castles Album into an unreleased AV/VR game"
3738
],
3839
"lore": [
3940
"Created Eliza framework for AI agents, now GitHub's #2 trending repo globally",

packages/core/src/knowledge.ts

+3-3
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ async function get(
1010
message: Memory
1111
): Promise<KnowledgeItem[]> {
1212
// Skip knowledge search if flag is set
13-
if (message?.metadata?.skipKnowledge) {
14-
elizaLogger.debug(
15-
"Skipping knowledge search due to skipKnowledge flag"
13+
if (message?.metadata?.skipKnowledge === true) {
14+
await elizaLogger.debug(
15+
"***Skipping knowledge search due to skipKnowledge flag"
1616
);
1717
return [];
1818
}

packages/plugin-code-assistant/README.md

+63-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Code Assistant Plugin
22

3-
This plugin provides code assistance functionality for Eliza, split into two main components:
3+
This plugin provides code assistance functionality for Shaiw, alter ego of Shaw, split into two main components:
44

55
## Actions
66

@@ -12,21 +12,81 @@ This plugin provides code assistance functionality for Eliza, split into two mai
1212

1313
### GitHub Assistant (githubActions.ts)
1414

15+
- Performs vector search on documentation from github docs site
1516
- Handles all GitHub-related queries
1617
- Manages GitHub API interactions
1718
- Processes PRs and Issues
1819
- Includes timeout and retry logic
1920

21+
## Installation
22+
23+
1. Adjust the `shaiw.character.json` in `characters` folder to your liking
24+
25+
2. update .env with your:
26+
27+
- ANTHROPIC_API_KEY
28+
- GITHUB_TOKEN
29+
- DISCORD_APPLICATION_ID
30+
- DISCORD_API_TOKEN
31+
32+
3. Standard install and build:
33+
34+
```
35+
pnpm clean && pnpm install && pnpm build
36+
```
37+
38+
4. Run the plugin:
39+
40+
```
41+
pnpm start --characters characters/shaiw.character.json
42+
```
43+
44+
5. Run the plugin (with new database)
45+
46+
```
47+
pnpm cleanstart --characters characters/shaiw.character.json
48+
```
49+
50+
6. Run the plugin (with new database and in debug mode)
51+
52+
```
53+
pnpm cleanstart:debug --characters characters/shaiw.character.json
54+
```
55+
2056
## Usage
2157

22-
The plugin automatically routes queries to the appropriate handler:
58+
The plugin automatically routes queries to the appropriate action handler:
2359

2460
1. GitHub-related queries are handled by githubAction
25-
2. General code queries are handled by codeAssistantAction
61+
2. General code queries are handled by codeAssistantAction and default runtime for processing
2662
3. All responses are saved to memory for future reference
2763

2864
## Error Handling
2965

3066
- Includes timeout protection
3167
- Maximum retry attempts for LLM
3268
- Fallback to knowledge search when GitHub data is unavailable
69+
70+
## Services
71+
72+
- Responds to user queries
73+
- Handles user requests for code assistance
74+
- Handles user requests for documentation from github docs site
75+
- Has a slight sense of humor
76+
- Has been trained on Shaw's recent videos and interviews
77+
- On load, crawls the github docs site and store the results in a vector database
78+
79+
## Considerations
80+
81+
- The plugin was built with sqlite, for production, switch to a more robust database like postgres
82+
- Requires above environment variables to be set in .env
83+
84+
## Todos
85+
86+
- Look into using Claude to search the github issues and pull requests, dynamically, but not sure if it's worth it, cost wise.
87+
- Help user create a github issue or pull request and use client-github to create it for them, after gathering all the information from the user.
88+
- Sleep
89+
90+
## Author
91+
92+
[harperaa](https://github.com/harperaa)

packages/plugin-code-assistant/src/actions/codeAssistant.ts

+2-2
Original file line numberDiff line numberDiff line change
@@ -45,13 +45,14 @@ export const codeAssistantAction: Action = {
4545
skipKnowledge: true,
4646
};
4747
// Forward to GitHub handler
48-
return await githubAction.handler(
48+
await githubAction.handler(
4949
runtime,
5050
message,
5151
state,
5252
options,
5353
callback
5454
);
55+
return true;
5556
}
5657
} catch (error) {
5758
elizaLogger.error(`Handler failed for ${requestId}:`, error);
@@ -184,4 +185,3 @@ export const codeAssistantAction: Action = {
184185
],
185186
] as ActionExample[][],
186187
} as Action;
187-

packages/plugin-code-assistant/src/actions/githubActions.ts

+98-97
Original file line numberDiff line numberDiff line change
@@ -8,121 +8,122 @@ import {
88
type Action,
99
State,
1010
ActionExample,
11+
stringToUuid,
12+
getEmbeddingZeroVector,
13+
embed,
1114
} from "@elizaos/core";
1215
import { Octokit } from "@octokit/rest";
1316

14-
class KnowledgeManager {
15-
private knowledge: KnowledgeBase = {
16-
discord: [],
17-
github: { issues: [], contributors: [] },
18-
documentation: new Map(),
19-
};
17+
// Helper function for text preprocessing
18+
function preprocess(text: string): string {
19+
return text
20+
.replace(/```[\s\S]*?```/g, "") // Remove code blocks
21+
.replace(/\[.*?\]/g, "") // Remove markdown links
22+
.replace(/[^\w\s]/g, " ") // Replace punctuation with spaces
23+
.replace(/\s+/g, " ") // Normalize whitespace
24+
.trim();
25+
}
2026

21-
private async fetchGithubApi(endpoint: string): Promise<any> {
22-
try {
23-
const baseUrl = endpoint.startsWith("/search")
24-
? "https://api.github.com"
25-
: "https://api.github.com/repos/ai16z/eliza";
27+
export async function crawlDocumentation(runtime: IAgentRuntime) {
28+
const baseUrl = "https://elizaos.github.io/eliza/docs/intro/";
29+
const urlsToVisit = new Set([baseUrl]);
30+
const visitedUrls = new Set<string>();
31+
elizaLogger.log(`Starting documentation crawl from ${baseUrl}`);
2632

27-
const url = `${baseUrl}${endpoint}`;
28-
elizaLogger.log(`Fetching GitHub API: ${url}`);
33+
while (urlsToVisit.size > 0) {
34+
const url = urlsToVisit.values().next().value;
35+
urlsToVisit.delete(url);
2936

30-
const response = await fetch(url, {
31-
headers: {
32-
Accept: "application/vnd.github.v3+json",
33-
"User-Agent": "eliza-code-assistant",
34-
},
35-
});
36-
if (!response.ok) {
37-
throw new Error(`GitHub API error: ${response.status}`);
38-
}
39-
const data = await response.json();
40-
elizaLogger.log(
41-
"GitHub API Response:"
42-
//JSON.stringify(data, null, 2)
43-
);
44-
return data;
45-
} catch (error) {
46-
elizaLogger.error("Error fetching from GitHub API:", error);
47-
return null;
37+
if (visitedUrls.has(url)) {
38+
elizaLogger.log(`Skipping already visited URL: ${url}`);
39+
continue;
4840
}
49-
}
5041

51-
async fetchDiscordKnowledge(): Promise<void> {
52-
try {
53-
const response = await fetch(
54-
"https://ai16z.github.io/eliza/community/Discord/"
55-
);
56-
const data = await response.json();
57-
this.knowledge.discord = data;
58-
elizaLogger.log("Fetched Discord knowledge");
59-
} catch (error) {
60-
elizaLogger.error("Error fetching Discord knowledge:", error);
61-
}
62-
}
42+
visitedUrls.add(url);
43+
elizaLogger.log(`Crawling page: ${url}`);
6344

64-
async searchIssues(query: string): Promise<GithubIssue[]> {
6545
try {
66-
const searchResults = await this.fetchGithubApi(
67-
`/search/issues?q=repo:elizaos/eliza ${encodeURIComponent(query)}&per_page=5&sort=updated`
68-
);
46+
const response = await fetch(url);
47+
const html = await response.text();
48+
const mainContent = html
49+
.replace(
50+
/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi,
51+
""
52+
)
53+
.replace(/<style\b[^<]*(?:(?!<\/style>)<[^<]*)*<\/style>/gi, "")
54+
.replace(/<[^>]+>/g, " ")
55+
.replace(/\s+/g, " ")
56+
.trim();
57+
58+
// Create document memory with URL
59+
const docId = stringToUuid(url);
60+
await runtime.documentsManager.createMemory({
61+
id: docId,
62+
agentId: runtime.agentId,
63+
roomId: runtime.agentId,
64+
userId: runtime.agentId,
65+
createdAt: Date.now(),
66+
content: {
67+
text: mainContent,
68+
source: "documentation",
69+
url: url,
70+
},
71+
embedding: getEmbeddingZeroVector(),
72+
});
6973

70-
if (!searchResults || !searchResults.items) {
71-
elizaLogger.error("Invalid search results from GitHub API");
72-
return [];
74+
// Split into chunks and create vector embeddings
75+
const preprocessed = preprocess(mainContent);
76+
const fragments = await splitChunks(preprocessed, 512, 20);
77+
78+
for (const fragment of fragments) {
79+
const embedding = await embed(runtime, fragment);
80+
await runtime.knowledgeManager.createMemory({
81+
id: stringToUuid(docId + fragment),
82+
roomId: runtime.agentId,
83+
agentId: runtime.agentId,
84+
userId: runtime.agentId,
85+
createdAt: Date.now(),
86+
content: {
87+
source: docId,
88+
text: fragment,
89+
url: url,
90+
},
91+
embedding,
92+
});
7393
}
7494

75-
return searchResults.items.map((item) => ({
76-
title: item.title,
77-
body: item.body || "",
78-
labels: item.labels.map((label) =>
79-
typeof label === "string" ? label : label.name || ""
80-
),
81-
state: item.state,
82-
number: item.number,
83-
html_url: item.html_url,
84-
created_at: item.created_at,
85-
updated_at: item.updated_at,
86-
}));
95+
// Extract and add new links to visit
96+
const linkMatches =
97+
html.match(/href="(\/eliza\/docs\/[^"]+)"/g) || [];
98+
linkMatches
99+
.map((match) => {
100+
const path = match.match(/href="([^"]+)"/)?.[1];
101+
if (!path) return null;
102+
return new URL(
103+
path,
104+
"https://elizaos.github.io"
105+
).toString();
106+
})
107+
.filter(
108+
(link): link is string =>
109+
link !== null &&
110+
link.startsWith(
111+
"https://elizaos.github.io/eliza/docs/"
112+
) &&
113+
!link.includes("#") && // Exclude anchor links
114+
!visitedUrls.has(link) // Don't add already visited URLs
115+
)
116+
.forEach((link) => urlsToVisit.add(link));
87117
} catch (error) {
88-
elizaLogger.error("Error searching Github issues:", error);
89-
return [];
118+
elizaLogger.error(`Error crawling ${url}:`, error);
90119
}
91120
}
92121

93-
async searchKnowledge(query: string): Promise<string[]> {
94-
const results: string[] = [];
95-
96-
// Search Github issues first for better relevance
97-
const githubIssues = await this.searchIssues(query);
98-
if (githubIssues.length > 0) {
99-
const relevantIssues = githubIssues.map(
100-
(issue) =>
101-
`GitHub Issue #${issue.number}: [${issue.title}](${issue.html_url})\n` +
102-
`Status: ${issue.state}\n` +
103-
`Last updated: ${new Date(issue.updated_at).toLocaleDateString()}\n`
104-
);
105-
results.push(...relevantIssues);
106-
}
107-
108-
// Search Discord messages
109-
const relevantMessages = this.knowledge.discord
110-
.filter((msg) =>
111-
msg.content.toLowerCase().includes(query.toLowerCase())
112-
)
113-
.map(
114-
(msg) =>
115-
`Discord: ${msg.content} (Solution: ${msg.solution || "N/A"})`
116-
);
117-
118-
results.push(...relevantMessages);
119-
120-
return results;
121-
}
122+
elizaLogger.log(
123+
`Documentation crawl completed. Visited ${visitedUrls.size} pages.`
124+
);
122125
}
123126

124-
const knowledgeManager = new KnowledgeManager();
125-
126127
// Initialize Octokit without auth for public repo access
127128
const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
128129
const REPO_OWNER = "elizaos";
@@ -413,7 +414,7 @@ Respond with ONLY a JSON object in this format:
413414
"action": "CODE_ASSISTANT"
414415
}
415416
416-
Provide a clear, informative summary that captures the main points. Format exactly as shown above.`;
417+
If knowledge sources contain URLs, include them in the references section. Format exactly as shown above.`;
417418

418419
// Modified handler with better flow control
419420
export const githubAction: Action = {

0 commit comments

Comments
 (0)