--test-timeout behavior bug

[brianwestphal]

Version

22.14.0

Platform

Darwin 24.3.0 Darwin Kernel Version 24.3.0: Thu Jan  2 20:24:16 PST 2025; root:xnu-11215.81.4~3/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

The --test-timeout flag passed to node seems to affect the module level rather than the individual tests, which doesn't make a lot of sense.

For example, with a test file like:

import { it } from "node:test";

const sleep = (durationMSec) => new Promise((resolve) => setTimeout(resolve, durationMSec));

it("bla1", async () => {
  await sleep(7000);
});
it("bla2", async () => {
  await sleep(7000);
});

run with node --test --test-timeout=10000 vs node --test --test-timeout=20000

Each test takes about 7000ms to run, so they should each pass and the module should pass, with either test-timeout value. However, in fact, the 10000 test-timeout value fails because the module as a whole takes longer than 10000ms.

How often does it reproduce? Is there a required condition?

Every time with the above conditions

What is the expected behavior? Why is that the expected behavior?

The --test-timeout flag should affect the individual tests and not the modules. Otherwise, you'd need to know how many tests you were planning to write ahead of time, which doesn't make sense.

What do you see instead?

Additional information

No response

[jakecastelli]

Thanks for the report, we had a discussion about it here #53773 (comment) a while ago, I was trying to fix this but lost track, I should pick this work up again.

cc. @cjihrig

[cjihrig]

Yes, this feature has been broken since it landed: #50431 (comment).

[jakecastelli]

Just finished some digging, will try to put up a PR tomorrow.

Also I read the doc and some of the previous convocations again, wanted to ask - is it too late to change the behaviour of --test-timeout? Should we introduce a new flag e.g. --test-timeout-per-test instead?

My understanding of the --test-timeout is set on the <root> so it is the timeout for the entire test execution. For example if there were 5 test files and the --test-timeout is set to 1 which is 1ms, so obviously it'd fail immediately, what we can observe is that all other 4 test files are being cancelled. Any use case for it? Maybe not very practical unless someone wants to make sure their tests finish within a certain time, but this change could definitely result a semver-major PRs that contain breaking changes and should be released in the next major version. (would probably just fine though).

What do you think? @cjihrig

[cjihrig]

The current behavior is absolutely a bug, so we should treat the fix as semver patch.