Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add pagination support and account status management across database adapters (CONFLICTED) #3359

Closed
wants to merge 71 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
60ce697
feat: Add pagination support and account status management across dat…
tercel Feb 7, 2025
fe4f85e
feat: Enhance manage API routes for account management
tercel Feb 8, 2025
5413122
feat: Add JWT authentication and token-based access control
tercel Feb 8, 2025
326c885
feat: Add client-side stop mechanism for Twitter client components
tercel Feb 8, 2025
ffb3980
feat: Add Swagger documentation and enhance API authentication for cl…
tercel Feb 10, 2025
29a9f5f
feat: Enhance pagination and query support across database adapters
tercel Feb 11, 2025
f0a4358
feat: Add dotenv configuration for Swagger generation
tercel Feb 11, 2025
bf2cb9e
fix: Improve dotenv configuration for Swagger generation
tercel Feb 11, 2025
740137b
fix: Simplify character object handling in account update route
tercel Feb 11, 2025
8090baa
fix: Improve character ID generation and agent start process
tercel Feb 11, 2025
e18f658
fix: Improve error logging and formatting in logger
tercel Feb 11, 2025
caf6226
fix: Improve error handling and logging in manage API routes
tercel Feb 11, 2025
7f9d743
fix: Refactor account update route with improved error handling and I…
tercel Feb 11, 2025
050e066
fix: Improve account creation logic for unique username and ID genera…
tercel Feb 11, 2025
743a092
fix: Improve plugin importing and character merging logic
tercel Feb 11, 2025
601ce41
feat: Add plugin list endpoint to manage API
tercel Feb 12, 2025
89dd57b
feat: Implement plugin discovery and loading mechanism
tercel Feb 12, 2025
51897aa
feat: Enhance plugin and memory management in DirectClient
tercel Feb 13, 2025
4ff8558
chore: Add PM2 ecosystem configuration file
tercel Feb 13, 2025
859b9fc
feat: Add PostgreSQL setup and text search optimization
tercel Feb 13, 2025
adae76b
feat: Add agent restart script with dynamic service naming and port m…
tercel Feb 13, 2025
c70760e
fix: Remove default character initialization in agent startup
tercel Feb 14, 2025
993f807
refactor: Update plugin discovery mechanism to use package.json depen…
tercel Feb 14, 2025
57ab3c0
fix: Configure SSL for PostgreSQL database connection
tercel Feb 14, 2025
10bfa13
fix: Improve agent restart script reliability and startup process
tercel Feb 14, 2025
f63dc5b
fix: Improve agent startup logging and account status management
tercel Feb 14, 2025
c271a34
refactor: Update 3D generation plugin export and import plugin mechanism
tercel Feb 17, 2025
2951e6c
feat: Add package metadata to plugin imports and character saving
tercel Feb 17, 2025
7f687ff
feat: Add client discovery and metadata retrieval mechanism
tercel Feb 17, 2025
467f046
feat: Enhance plugin name extraction for consistent naming
tercel Feb 17, 2025
67f3f79
feat: Dynamically set Swagger documentation host based on request
tercel Feb 17, 2025
796f2ea
feat: Improve README file discovery with case-insensitive search
tercel Feb 18, 2025
c8fe9c8
feat: Add README files for multiple ElizaOS clients and plugins
tercel Feb 18, 2025
e06b6e8
docs: Add comprehensive README for Deva client package
tercel Feb 18, 2025
2630f76
feat: Add README parsing for environment variables in clients and plu…
tercel Feb 18, 2025
43a621c
docs: Standardize README environment variable formatting across clien…
tercel Feb 18, 2025
b52bf52
docs: Add comprehensive README for ElizaOS Core Package
tercel Feb 19, 2025
54fb047
feat: Add log streaming endpoint with SSE and client management
tercel Feb 19, 2025
433a231
refactor: Improve log streaming message format for SSE endpoint
tercel Feb 19, 2025
19c43c0
feat: Add system metrics endpoint for monitoring process and system r…
tercel Feb 20, 2025
a5ee4c7
refactor: Optimize memory usage monitoring and logging
tercel Feb 20, 2025
8480a83
refactor: Remove unnecessary console log in token verification middle…
tercel Feb 20, 2025
54c9ed2
refactor: Enhance restart-agent.sh with flexible start/stop/restart a…
tercel Feb 20, 2025
89e2f7a
fix: Improve log stream heartbeat and timeout timestamp handling
tercel Feb 20, 2025
20a53c7
feat: Add disk space metrics to system metrics endpoint
tercel Feb 20, 2025
9369a02
docs: Add descriptive comments for disk space metrics type
tercel Feb 20, 2025
30d5559
chore: Update dependencies and version, adjust CPU usage calculation
tercel Feb 20, 2025
1f7d352
feat: Enhance CPU usage metrics with system-wide and process-specific…
tercel Feb 20, 2025
72c55e8
feat: Enhance provider configuration and API key management
tercel Feb 21, 2025
0a1dc6a
feat: Add template generation endpoint with JSON parsing and response…
tercel Feb 24, 2025
717731e
feat: Add character template generator configuration
tercel Feb 25, 2025
cdc7043
fix: Standardize API error responses with JSON format
tercel Feb 25, 2025
484ed59
feat: Enhance character loading with tplgen agent and source tracking
tercel Feb 25, 2025
6a93247
fix: Automatically activate or pause accounts based on agent status
tercel Feb 25, 2025
11be4b3
feat: Add source tracking to tplgen character template
tercel Feb 25, 2025
c6cad48
feat: Add token retrieval for model provider in runtime configuration
tercel Feb 25, 2025
befa3e7
feat: Improve runtime model provider handling and JSON parsing flexib…
tercel Feb 25, 2025
766f4bc
feat: Improve memory ID generation and response message metadata
tercel Feb 25, 2025
8de4d8f
refactor: Improve memory ID generation with descriptive prefixes
tercel Feb 25, 2025
692ce39
feat: Increase body parser size limits for large JSON payloads
tercel Feb 26, 2025
a212b0b
feat: Optimize CPU metrics tracking and calculation
tercel Feb 26, 2025
4a6f16e
feat: Implement log buffering for SSE client connections
tercel Feb 26, 2025
3f88f1c
refactor: remove parsing bool string
tercel Feb 27, 2025
1ef1c09
refactor: Enhance action matching and character saving logic
tercel Feb 27, 2025
d33cb9f
feat: Add account update method for character activation
tercel Feb 27, 2025
d66df6b
refactor: Improve logging and error handling in Solana and Near plugins
tercel Feb 27, 2025
0330aa0
feat: Add string escape character cleaning utility and improve parsing
tercel Feb 28, 2025
677b3d7
refactor: Disable most plugin configurations by default
tercel Feb 28, 2025
63c8416
refactor: Remove commented-out plugin configurations
tercel Feb 28, 2025
74fac96
feat: Add conversation length and action control to character configu…
tercel Feb 28, 2025
4eb8fd8
feat: Enhance logging for direct client and Solana token transfers
tercel Feb 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -968,3 +968,9 @@ BUNDLE_EXECUTOR_ADDRESS= # Address of the bundle executor contract
DESK_EXCHANGE_PRIVATE_KEY= # Required for trading and cancelling orders
DESK_EXCHANGE_NETWORK= # "mainnet" or "testnet

# JWT
JWT_ENABLED=
JWT_SECRET_KEY=
JWT_EXPIRED=
JWT_USERNAME=
JWT_PASSWORD=
3 changes: 0 additions & 3 deletions agent/.gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
*.ts
!index.ts
!character.ts
.env
*.env
.env*
Expand Down
1 change: 0 additions & 1 deletion agent/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,6 @@
"@elizaos/plugin-twitter": "workspace:*",
"@elizaos/plugin-primus": "workspace:*",
"@elizaos/plugin-cronoszkevm": "workspace:*",
"@elizaos/plugin-cronos": "workspace:*",
"@elizaos/plugin-3d-generation": "workspace:*",
"@elizaos/plugin-fuel": "workspace:*",
"@elizaos/plugin-avalanche": "workspace:*",
Expand Down
87 changes: 87 additions & 0 deletions agent/src/client.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
import path from "path";
import fs from "fs";
import { fileURLToPath } from 'url';
import { elizaLogger } from "@elizaos/core";
import { getAndParseReadme } from "./document";
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

interface ClientInfo {
package: string;
name: string,
env?: Record<string, string|number|boolean>;
document?: string;
}

async function getClientInfo(clientDir: string): Promise<ClientInfo | null> {
try {
// get package.json
const packageJsonPath = path.join(clientDir, 'package.json');
elizaLogger.debug(`Reading package.json from: ${packageJsonPath}`);
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf-8'));

// try to read README.md with case-insensitive search
const {document, env} = await getAndParseReadme(clientDir);

return {
package: packageJson.name,
name: packageJson.name.startsWith('@elizaos/client-') ? packageJson.name.slice(16) : packageJson.name,
env,
document,
};
} catch (error) {
elizaLogger.error(`Error loading client from ${clientDir}:`, error.message);
return null;
}
}

export async function getClients() {
try {
// Read package.json from the agent directory
const packageJsonPath = path.resolve(__dirname, '../package.json');
elizaLogger.log(`Reading package.json from: ${packageJsonPath}`);

const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf-8'));

// Get all client dependencies (starting with @elizaos/client-)
const clientPackages = Object.keys(packageJson.dependencies || {})
.filter(dep => dep.startsWith('@elizaos/client-'))
.sort();

elizaLogger.info(`Found ${clientPackages.length} client packages in package.json`);

// Get client info for each package
const clientsInfo = await Promise.all(
clientPackages.map(async (packageName) => {
try {
// Import client module to get client info
elizaLogger.log(`load client: ${packageName}`);
// Try to read README.md from node_modules
const nodeModulesDir = path.resolve(__dirname, '../node_modules');
const clientDir = path.join(nodeModulesDir, packageName);

const {document, env} = await getAndParseReadme(clientDir);

return {
package: packageName,
name: packageName.startsWith('@elizaos/client-') ? packageName.slice(16) : packageName,
env,
document,
};
} catch (error) {
elizaLogger.error(`Error loading client ${packageName}:`, error.message);
return null;
}
})
);

// Filter out clients that failed to load
const validClients = clientsInfo.filter(Boolean);
elizaLogger.info(`Successfully loaded ${validClients.length} clients`);

return validClients;
} catch (error) {
elizaLogger.error('Error getting clients:', error);
throw error;
}
}
51 changes: 51 additions & 0 deletions agent/src/document.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import path from "path";
import fs from "fs";
import { elizaLogger } from "@elizaos/core";

export async function getReadme(dir: string) {
// try to read README.md with case-insensitive search
try {
const files = fs.readdirSync(dir);
const readmeFile = files.find(file => file.toLowerCase() === 'readme.md');
if (readmeFile) {
return fs.readFileSync(path.join(dir, readmeFile), 'utf-8');
}
} catch (error) {
elizaLogger.debug(`No README.md found in ${dir}`);
return null;
}
}

export async function getAndParseReadme(dir: string) {
const document = await getReadme(dir);
const env: Record<string, string|number|boolean> = {};
if (!document) {
return {document: null, env: {}};
}
const envMatch = document.match(/```env\n([\s\S]*?)\n```/);
if (envMatch) {
const envLines = envMatch[1].split('\n').map(line => line.trim())
.filter(line => line.length > 0 && !line.startsWith('#') && !line.startsWith(' #'));
for (const line of envLines) {
// remove comment
const lineWithoutComment = line.split('#')[0].trim();
const [key, ...valueParts] = lineWithoutComment.split('=');
let value:any = valueParts.join('=').trim(); // handle value with =

// handle number
if (/^\d+$/.test(value)) {
value = Number.parseInt(value, 10);
}
// else if (value.toLowerCase() === "true") {
// value = true;
// } else if (value.toLowerCase() === "false") {
// value = false;
// }

if (key) {
env[key.trim()] = value;
}
}
}
return {document, env};
}
Loading