Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Privately turn on test run in CI/CD and file tickets on whatever breaks #13

Closed
snobbee opened this issue Nov 21, 2024 · 5 comments
Closed
Assignees
Labels
testing Improvements or additions to tests

Comments

@snobbee
Copy link
Collaborator

snobbee commented Nov 21, 2024

Privately turn on test run in CI/CD and file tickets on whatever breaks

@snobbee snobbee added the testing Improvements or additions to tests label Nov 21, 2024
@pgoos
Copy link
Collaborator

pgoos commented Nov 21, 2024

Log from CI: https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1

Run cd packages/core && pnpm test

> @ai1[6](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:7)z/eliza@0.1.3 test /home/runner/work/eliza/eliza/packages/core
> vitest run


 RUN  v2.1.5 /home/runner/work/eliza/eliza/packages/core

stdout | src/tests/goals.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/relationships.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/database.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/database.test.ts (8 tests) 9ms
 ✓ src/tests/relationships.test.ts (6 tests) 13ms
 ❯ src/tests/goals.test.ts (0 test)
stdout | src/tests/videoGeneration.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/messages.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/providers.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/videoGeneration.test.ts (3 tests) 11ms
 ✓ src/tests/providers.test.ts (5 tests) [7](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:8)ms
 ✓ src/tests/messages.test.ts (6 tests) 6ms
stdout | src/tests/evaluators.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/token.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/posts.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/posts.test.ts (3 tests) 5ms
 ✓ src/tests/evaluators.test.ts (4 tests) 5ms
stdout | src/tests/models.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/defaultCharacters.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/models.test.ts (7 tests) 6ms
 ✓ src/tests/defaultCharacters.test.ts (11 tests) [8](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:9)ms
(node:9652) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
stdout | src/tests/cache.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/env.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/env.test.ts > Environment Setup > should verify .env.test file exists
Current working directory: /home/runner/work/eliza/eliza/packages/core
__dirname: /home/runner/work/eliza/eliza/packages/core/src/tests
Path /home/runner/work/eliza/eliza/packages/core/.env.test exists: true
Path /home/runner/work/eliza/eliza/packages/core/packages/core/.env.test exists: false
Path /home/runner/work/eliza/eliza/packages/core/.env.test exists: true
Path /home/runner/work/eliza/eliza/packages/core/src/.env.test exists: false
Path /home/runner/work/eliza/eliza/packages/core/src/tests/.env.test exists: false

 ✓ src/tests/env.test.ts (1 test) 4ms
 ❯ src/tests/cache.test.ts (0 test)
stdout | src/tests/token.test.ts > TokenProvider Tests > should fetch token security data
 ["◎ sqlite-vec extensions loaded successfully."] 

 ["◎ sqlite-vec extensions loaded successfully."] 

 ✓ SUCCESS
   Agent ID 
   ec6bb8be-2524-0c27-b174-2e0fa328[9](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:10)[10](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:11)2 


 ❯ src/tests/token.test.ts (1 test | 1 failed) 25ms
   × TokenProvider Tests > should fetch token security data 24ms
     → Endpoint URL must start with `http:` or `https:`.

⎯⎯⎯⎯⎯⎯ Failed Suites 2 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/tests/cache.test.ts [ src/tests/cache.test.ts ]
ReferenceError: jest is not defined
 ❯ src/tests/cache.test.ts:9:5
      7|     let cache: CacheManager<MemoryCacheAdapter>;
      8| 
      9|     jest.useFakeTimers();
       |     ^
     10| 
     [11](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:12)|     beforeEach(() => {

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/3]⎯

 FAIL  src/tests/goals.test.ts [ src/tests/goals.test.ts ]
ReferenceError: CacheManager is not defined
 ❯ src/tests/goals.test.ts:34:23
     32| export const mockRuntime: IAgentRuntime = {
     33|     databaseAdapter: mockDatabaseAdapter as any,
     34|     cacheManager: new CacheManager(new MemoryCacheAdapter()),
       |                       ^
     35|     agentId: "qweqew-qweqwe-qweqwe-qweqwe-qweeqw",
     36|     serverUrl: "",

 Test Files  3 failed | 10 passed ([13](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:14))
      Tests  1 failed | 54 passed (55)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/3]⎯

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/tests/token.test.ts > TokenProvider Tests > should fetch token security data
TypeError: Endpoint URL must start with `http:` or `https:`.
 ❯ assertEndpointUrl ../../node_modules/@solana/web3.js/src/connection.ts:362:11
 ❯ new Connection ../../node_modules/@solana/web3.js/src/connection.ts:3234:25
 ❯ src/tests/token.test.ts:21:13
     19| 
     20|         const walletProvider = new WalletProvider(
     21|             new Connection(runtime.getSetting("RPC_URL")),
       |             ^
     22|             new PublicKey(runtime.getSetting("WALLET_PUBLIC_KEY"))
     23|         );

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/3]⎯

   Start at  [14](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:15):50:21
   Duration  1.79s (transform 418ms, setup 202ms, collect 1.21s, tests 99ms, environment 3ms, prepare 1.26s)


Error: ReferenceError: jest is not defined
 ❯ src/tests/cache.test.ts:9:5



Error: ReferenceError: CacheManager is not defined
 ❯ src/tests/goals.test.ts:34:23



Error: TypeError: Endpoint URL must start with `http:` or `https:`.
 ❯ assertEndpointUrl ../../node_modules/@solana/web3.js/src/connection.ts:362:11
 ❯ new Connection ../../node_modules/@solana/web3.js/src/connection.ts:3234:25
 ❯ src/tests/token.test.ts:[21](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:22):13


 ELIFECYCLE  Test failed. See above for more details.
Error: Process completed with exit code 1.

@pgoos
Copy link
Collaborator

pgoos commented Nov 21, 2024

Seems like 3 tests are broken. I'll file tickets for them

@snobbee
Copy link
Collaborator Author

snobbee commented Nov 21, 2024

@pgoos you can find the broken tests result in this comment here elizaOS#463 (comment)

@pgoos
Copy link
Collaborator

pgoos commented Nov 25, 2024

New PR created: elizaOS#590 - all tests are passing now.

@jkbrooks
Copy link

jkbrooks commented Dec 2, 2024

Okay so @pgoos should we consider this ticket done? If so, please close if not please explain why

@jkbrooks jkbrooks closed this as completed Dec 3, 2024
github-actions bot pushed a commit that referenced this issue Jan 8, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Elizaai 16 unit tests
github-actions bot pushed a commit that referenced this issue Jan 13, 2025
feat: add GoPlus Security Plugin to enhance security for agent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Improvements or additions to tests
Projects
None yet
Development

No branches or pull requests

3 participants