node-api: widen the measured slice, harden the harness, record the runs to 65% - #6
Open
Alan Wisper (wisper-dev) wants to merge 6 commits into
Open
node-api: widen the measured slice, harden the harness, record the runs to 65%#6Alan Wisper (wisper-dev) wants to merge 6 commits into
Alan Wisper (wisper-dev) wants to merge 6 commits into
Conversation
Node's core suite spawns children that outlive the test process and hold its stdout open, so killing the entry process alone left the read hanging. The process now runs in its own group and a timeout reaps the tree; a descendant that opened its own session escapes the kill, so the pipes are dropped after a two-second grace.
`--elide` usually points at the `dist/current` symlink, and the image build context must carry the real files; the staged copy now dereferences instead of building around a broken link.
The workload sets its own case timeout; the test asserted the previous default.
Adds the missing subsystem groups (child_process, cluster, dgram, domain, http, net, os, readline, tty, zlib and more) so the suite measures the public Node surface rather than a sparse slice. Each test now runs with a heap bound: the suite contains cases whose only brake is stream backpressure (test-zlib-brotli-16GB.js decompresses 16 GB), and unbounded such a case exhausts the host instead of failing. A static expectations file carries the skips with a reason each: the cluster group is out of project scope for now, and the node:vfs tests cover an experimental Node subsystem Elide does not implement.
Records the measured progression on the widened slice: 35.8%, 43.2%, 50.4%, 60.1% and 65.0% (1,387 of 2,133 counting the denominator without skips). The latest run carries its full report body; the earlier ones keep the raw results, counts and change summary, which is what the trend, the run database and the next run's diff read. The ratchet is retaken against the latest run and holds 857 known failures.
Replaces the July handoff doc: the failure map by subsystem with the hang subset broken out, what the static expectations skip and why, the ranked work items that follow, and run instructions that match the current selection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
node-api: widen the measured slice, harden the harness, record the runsBrings the
node-apisuite to an honest measurement and records the progression35.8% → 43.2% → 50.4% → 60.1% → 65.0%. The latest run is 1,387 passes out of
2,133 counting the denominator without skips, with no regressions against the
ratchet.
Harness
that outlive the test process and hold its stdout open, so killing the entry
process alone left the output read hanging and the run stalled. The process now
gets its own group and a timeout reaps the tree. A descendant that opened its
own session still escapes the group kill, so the pipes are dropped after a
two-second grace.
--elideusuallypoints at the
dist/currentsymlink, and the image build context must carrythe real files; the staged copy now dereferences instead of building around a
broken link.
the previous default.
Test selection
child_process,cluster,dgram,domain,http,https,net,os,perf_hooks,querystring,readline,tty,v8,zliband more, so the suite measures the public Nodesurface rather than a sparse slice.
-XX:MaxHeapSize=4g. The suite contains cases whoseonly brake is stream backpressure (
test-zlib-brotli-16GB.jsdecompresses16 GB); unbounded, such a case exhausts the host instead of failing.
expectations/node-api.tomlcarries the skips witha reason each: the
clustergroup is out of project scope for now (one glob, sothe decision stays visible and reversible), and the
node:vfstests cover anexperimental Node subsystem Elide does not implement.
Ratchet and reports
map is the working document for the next batch of fixes. The four earlier ones
keep only the raw results, counts and change summary: that is what the trend,
the run database (
ingestAll) and the next run's diff actually read, while therendered bodies are superseded as soon as a newer run lands.
BRIEFING-node-api.mdis rewritten around the current measurement: the failuremap by subsystem with the hang subset broken out, what the static expectations
skip and why, the ranked work items that follow, and run instructions matching
the current selection.
Checks
bun --cwd harness test— 171/171 pass.bun run typecheck— clean.