TL;DR. A lint plugin counted 4,336 problems in our tests and offered to fix most of them automatically, so I planned three slices of work around one command. Before running it everywhere I ran it on one file and followed that file all the way to a commit. The method failed six separate ways: it wrote calls to matchers that were not installed, the type-checker said those matchers existed, it left 33 files that no longer parsed, it obeyed a rule that was wrong for our setup, it could not get past our own pre-commit hook, and, where it did work, it quietly loosened what the tests assert. I cancelled all three slices. The cleanup that replaced them took a pilot and sixteen groups, and even the careful version weakened tests that only breaking the code under them revealed. A count of what a tool can change tells you almost nothing about whether the change works, and a green run tells you as little about whether a test can still fail.
A dry run said 203 test files had problems a linter could fix for us, so I scheduled three slices of work around one command. Then I ran that command on a single file, and the plan did not survive it.
We ship Hearso, a small real-time multiplayer trivia platform, in batches. The current batch is a cleanup: turn on stricter TypeScript, add lint rules for our React tests, and clear what they report. I work through that with a small software factory I built, where a planning agent, a building agent and a set of deterministic gates take one written slice of work from spec to commit.
One of the lint plugins we added reported 4,336 problems on test files. Most of them came with an automatic fix. So the plan wrote itself: run eslint --fix on half the files, then the other half, then fix the leftovers by hand. Three slices.
So the question here is: if a tool counts 4,336 things it can fix, how much do you know about whether fixing them works?
Short answer: almost nothing. Run on one file and followed all the way to a commit, the method failed in six independent ways. The matchers its fixes wrote did not exist at run time, and the type-checker said they did. It left 33 files that no longer parsed. One of its rules was wrong for our setup. Our own pre-commit hook would have refused 79 of the first 96 files. And where it did work, it quietly turned 245 exact assertions into substring matches. None of that is visible in a count.
It helps to see the whole chain the plan leaned on. A rule reports a finding and a fixer rewrites the line. Then the matcher that line calls has to exist, the file has to parse, the rule has to fit our setup, the hook has to accept the commit, and the test has to check what it checked before. A count measures the first link. The last one gets the most room below, because it can break while every test stays green.
It follows Line coverage said done. Mutation testing disagreed. and What we got wrong about checks that pass when nothing happened. Those were about checks that pass without checking. This one is about a plan that was measured without being tried.
What this post covers
- The plan came from a count
- A wasted run is what sent me looking
- I ran the fix on one file, and the matchers did not exist
- Why nothing had told me
- The fixer writes code that does not parse
- One rule was wrong for our setup
- The commit could not have landed
- Where the fix worked, it weakened the tests
- Two reviewers, one finding, and an experiment
- What I got wrong myself
- What replaced the plan
- The second run passed every gate and still needed a reader
- What the numbers said on the second day
- How it ended, and closing the loop
1. The plan came from a count
In slice 1 of the batch I added two ESLint plugins, scoped to test files: eslint-plugin-testing-library and eslint-plugin-jest-dom. Between them they reported 4,336 problems across our 896 test files.
A dry run (eslint --fix-dry-run) said 203 of those files had at least one problem the plugins could fix mechanically. The fixes looked like this:
// before
expect(screen.getByText("Add your first name.")).toBeTruthy();
expect(email.getAttribute("aria-invalid")).toBe("true");
// after
expect(screen.getByText("Add your first name.")).toBeInTheDocument();
expect(email).toHaveAttribute("aria-invalid", "true");
That is a real improvement. When the second form fails, it tells you what the element was, where the first says "expected false to be true".
Our automated reviewer stops reading a pull request at 150 files, so 203 files meant two pull requests. I split them alphabetically into a group of 96 and a group of 107, and added a third slice for whatever the fixer could not close. The spec for the first group even carried a target: test-file problems down from 4,336 to "≤ 2,500".
That target is worth a second look, because I can now say exactly what the command would have produced: 2,970. A dry run reports the problems that remain after its fixes, so the answer was sitting in the output. The slice also forbade manual edits. So the spec asked for a number its own permitted method could not reach, and nobody had noticed, because the number was an estimate and the method had not been run.
2. A wasted run is what sent me looking
I did not find any of this by being careful. I found it because the first factory run on that slice ended with:
✗ commit_build nothing to commit — the preceding phases changed no files
That run took four and a half minutes and 430,534 tokens, and it was telling the truth. It had been given the wrong spec. My shell history replayed the command for a slice that had merged that morning, one second before the session started. The planner worked that out in 26 seconds and wrote a "verify only" plan. Then the build, test and review phases ran anyway, because nothing in the pipeline could act on a planner saying "there is nothing to do".
That is a story about the factory, and it has its own fixes: a preflight that refuses a spec whose name does not match the branch, and a verdict the planner can return to stop the run. But it had a useful side effect. Having been burned by a run that could not have produced anything, I asked the same question of the slice I had actually meant to run. Could that one produce anything?
3. I ran the fix on one file, and the matchers did not exist
The cheapest possible experiment: take one test file, write the fixer's output beside it as a throwaway copy, and run both.
original Tests 2 passed (2)
autofixed Tests 1 failed | 1 passed (2)
Error: Invalid Chai property: toHaveAttribute
eslint-plugin-jest-dom is a lint plugin. It rewrites text. The matchers it rewrites to (toBeInTheDocument, toHaveAttribute and the rest) come from a different package, @testing-library/jest-dom, which has to be installed and registered with the test runner. I had added the plugin and not the library.
The library was on disk, which is part of why this was easy to miss. Storybook depends on it. But nothing in our project asked for it, and neither of our two test runners registered it.
Of the 4,336 problems, 2,371 were jest-dom/* findings. So more than half the cleanup I had scheduled would have converted working assertions into calls to functions that did not exist.
4. Why nothing had told me
Two reasons, and the second one is the interesting one.
The first is plain. No test used one of these matchers yet, so nothing failed.
The second: TypeScript said the matchers existed. I only found this later, while trying to prove that a type declaration I had written was needed. I emptied the declaration, ran the type-check, and got no error. The types were coming from somewhere else.
I asked the compiler directly, through its API, where toBeInTheDocument on a Vitest assertion was declared:
zz-types-probe.test.ts -> node_modules/@vitest/browser/jest-dom.d.ts
Our Vitest config imports a Playwright provider for the project that runs Storybook stories in a real browser. That import brings in Vitest's browser-mode types, and those add matchers with the same names to every assertion in the program. They describe a runner our unit tests do not use.
So three tools agreed. The linter asked for the matcher. The type-checker confirmed it existed. The dry run counted how many places could use it. That felt like three confirmations. It was closer to one, because the dry run is the linter's own count, and neither the linter nor the type-checker runs a test. All three were describing intent, and the only thing that described reality was running a test.
5. The fixer writes code that does not parse
Registering the library in a scratch branch removed the first failure and exposed the next. I ran the dry run again and looked at what it left behind. In 33 files, the fixer's output was not valid TypeScript:
// before
expect((seat as HTMLButtonElement).disabled).toBe(true);
// after
expect((seat as HTMLButtonElement).toBeDisabled();
A parenthesis is gone. The fixer for that rule does not expect a property read through a type cast. A second shape broke on a trailing comma inside a multi-line expect.
ESLint runs its fixers in passes, re-linting between them. When a pass produces text that no longer parses, it stops, and the broken text is what gets written. It does say so: there is a Parsing error in the output, and the exit status is 1.
I got this part wrong in my own notes, and it matters enough to say. I wrote that the fixer "exits as if nothing happened". It does not. But exit status 1 is also what ESLint returns for any file with an error left, and nearly every file in this cleanup has errors left. So the status tells you nothing and the output tells you everything, and an agent told to judge commands by exit status needs to be told that here.
6. One rule was wrong for our setup
The next failure was not in the fixer. It was a rule whose reasoning is sound and whose premise does not hold here.
testing-library/no-manual-cleanup flagged 240 files for importing cleanup. Its reasoning: your test runner unmounts rendered components between tests, so calling cleanup() yourself is noise.
That is true if afterEach is a global. Testing Library registers its own cleanup at import time by checking for a global afterEach. Our Vitest config does not set globals: true; every test file imports what it uses. So nothing is registered.
I did not want to settle that by reading library source, so I wrote two tests that each render one button and call no cleanup:
× sees only its own render if cleanup is automatic
AssertionError: expected [ Array(2) ] to have a length of 1 but got 2
The second test finds two buttons. The 240 files were importing something they needed, and obeying the rule would have made tests see each other's markup. We turned the rule off for the files Vitest runs, and left it on for the 19 files that run under Jest, where afterEach is a global and the rule's premise holds.
It also means 240 of the errors in the count were never work to do. They were one setting.
7. The commit could not have landed
Suppose all of that had been fixed. The slice would still have failed at its last step.
Our pre-commit hook runs ESLint on staged files, and ESLint exits non-zero while any error remains. The autofix does not close every finding in a file. After it, the 96 files of the first group split like this:
Figure 1. The 96 files of the first planned group after eslint --fix, measured by dry run on 2026-10-09. Only the 6 clean files could be committed through the pre-commit hook.
Six files could be committed. Seventy-nine would be refused by the hook for the errors that remained, and eleven did not parse.
It was worse than one slice. Counting the whole tree, 267 of our 896 test files could not be committed through the hook at all, as they stood, by anyone. The plugins were added to report problems. Through the hook, they had also decided which files could be committed. Nobody had noticed because in the days since the plugins landed, exactly three test files had been committed, and none of the three happened to have a finding.
8. Where the fix worked, it weakened the tests
Most of the failures so far end in something loud: a thrown error, a parse error, a refused commit. The sixth makes no noise. It is the one I would not have found alone, and I did not find it. The factory's building agent did.
After the library was registered and the plan re-cut (more on that below), I ran a pilot slice on three files. The builder's commit message carried a note I had not asked for: the fixer had rewritten
expect(status.textContent).toBe("Profile update failed — try again");
into
expect(status).toHaveTextContent("Profile update failed — try again");
and those are not the same assertion. toBe is equality. toHaveTextContent with a string is a substring match: it passes against "Profile update failed — try again later, maybe".
That matters because of what a test is for. A test earns its place by failing when the behaviour it names changes. So an edit to a test has one thing to keep: whatever made the old test fail must still make the new one fail. Passing before and after says nothing about that, because both versions pass on correct code.
I checked the claim against the raw fixer output, and it was true. Across the tree there are 251 assertions of the exact kind, in 95 files. The fixer keeps 6 of them exact. The original plan would have loosened about 245 assertions while every test stayed green, and a diff of several thousand mechanical-looking lines is exactly where a reviewer stops reading closely.
The builder's repair was to anchor each one in a regular expression. Review then found that repair was still looser than the original, twice over:
- Built from a constant, the pattern was not escaped. A message containing
.would match any character there, and one containing(would throw. - The matcher collapses and trims whitespace before it compares, even against an anchored pattern. A doubled space that failed the old assertion passes the new one.
I measured the second one instead of arguing it. With a doubled space put into a real message in the component:
| Assertion | Result |
|---|---|
| anchored regular expression | 11 of 11 tests pass |
| the helper described below | 1 of 11 fails, the test that names that message |
So we wrote a small helper, exactText, that returns both of the matcher's arguments as one tuple to spread: the escaped, anchored pattern, and the option that turns whitespace normalisation off.
expect(status).toHaveTextContent(...exactText(ONBOARDING_SAVE_FAILED));
It returns both because the second argument is the one a caller forgets, and forgetting it fails nothing.
There is one difference it cannot see, and I would rather say so than imply otherwise. Even with normalisation off, the matcher turns a no-break space in the element into an ordinary one before comparing. This codebase tells kinds of space apart on purpose (names cannot contain any of them), so the helper refuses expected text that holds a no-break space and says why, and a test pins the matcher's behaviour so that the day it changes, we hear about it.
9. Two reviewers, one finding, and an experiment
The pilot had one more lesson. A test needed to tell two status lines apart. Before, it found the form's line by a data attribute the component carries:
document.querySelector('[data-hp="identity-status"]')
The lint rules forbid that. Both lines have role="status" and neither has an accessible name, so a role query finds two. The builder's answer was to take the second one in document order, with a guard that there are exactly two.
Both of our automated reviewers flagged it. One named the case that matters: a negative assertion ("the form's line does not mention spaces") would be made against the other element after a reorder, and pass.
I reproduced that before changing anything. Two changes to the component, compared across the two ways of selecting:
| Change to the component | By position | By its hook |
|---|---|---|
| status line moved above the deck (no player would notice) | 3 of 11 fail | 11 of 11 pass |
| moved, and the form wrongly repeats the "spaces" message | 3 fail, and the test about that message passes | that test fails, the other 10 pass |
The positional version is brittle to a harmless change, and worse, in the case with the real bug it fails three tests that are not about the bug and passes the one that is.
The fix was not to go back to querySelector. Testing Library lets you choose which attribute getByTestId reads, so one import makes our existing hook the test id, and the lookup becomes a query the rules accept. But I did not want that to become the default, so the rule we wrote down is an order: a role with its name, then a region that narrows it, then the text a player reads, and only then the test id, with a comment saying why nothing above applies. Never position.
How often is "nothing above applies"? I counted. Of 230 such lookups in our tests, about two in three reach a plain div, span or p with no role at all. Our own review checklist allowed a test id only for an ambiguous role. Written that way, it would have forbidden the common case, so we changed the sentence.
10. What I got wrong myself
A post about a plan that was measured and not tried should include the measurements of mine that were wrong.
I understated a cost by a factor of three. Registering the matchers in every test file seemed cheap: I timed the import on its own in plain Node and got about 27 ms. I wrote that number into a code comment, a spec and a pull request. Then, filling in a coverage figure for a later pull request, I saw the suite's own setup total.
Figure 2. Vitest's setup time summed over every test file, from the suite's own report: before the change, with the matchers registered in every file (three runs: 82, 86, 91 s), and registered only where a document exists (two runs: 40.3, 39.6 s).
About 74 ms a file, not 27. Two test files in three run in the node environment and call no matcher. I had recommended paying that everywhere on the strength of the smaller number, so we reopened the decision and it went the other way: register where a document exists, and let the few node files that need a matcher import it themselves.
I should be precise about what that bought. About 45 CPU-seconds a run. On the 14-core machine these were measured on, wall time did not move outside its own noise.
I deleted a file that turned out to matter. The type declaration from earlier: I removed it because three attempts could not make it fail. An independent reviewer argued for putting it back, with an experiment I had not thought of. Leave the Vitest config out of the type-check, and a matcher call fails without the declaration and passes with it. It is redundant today and it is what keeps the types attached to the package that is actually registered.
Two of my guard tests passed for the wrong reason. I wrote tests that fail if the matcher registration is removed. To check the matchers were real, each had a negative case: expect(() => expect(input).toBeEnabled()).toThrow(). With the registration removed, two of four tests still passed, because "that matcher does not exist" is also a throw. They now match on the matcher's own sentence.
A cached type-check gave me a false reading. Our config has incremental type-checking on. After I changed a declaration file, the cached result for the files that used it was not invalidated, and I briefly believed something that was not so.
My lint totals counted a file that is not in the repository. I found this one while finishing this post. Every total I had reported was seven too high. npx eslint . lints what is on disk, and the factory's own code sits in a directory that Git ignores and ESLint does not. One file in it has seven errors. A clean checkout, which is what CI lints, has never had them. The steps (240, 31, 83) were right, because the seven were there before and after each one. The totals in three merged pull requests and two specs are not, and the table below uses the repository's numbers. Git's ignore list and the linter's are two lists, and I had only read one.
11. What replaced the plan
With the one-command plan cancelled, a slice now had to get through every link in that chain.
The method changed from "by tool" to "by area". Each slice takes a group of directories and leaves every test file in it with no lint error at all, automatic and manual fixes together. That is what lets each commit pass the hook as it stands.
Each slice's spec now carries its acceptance as commands the factory runs, in place of prose a reviewer has to remember. Before every slow gate it checks that nothing outside the listed paths changed, that no assertion or test case was removed, that no selection by position was added, that no exact text assertion became a substring match, and it runs the pre-commit hook early so that a tree the hook would refuse is found in seconds.
And the spec is generated by measuring the group: which files the fixer leaves clean, which it leaves with findings, which it breaks, and whether the tests pass today. I ran each check against the tree and made it fail with the change it exists to catch before trusting it.
The results so far:
| Slice | Test files | Lint errors | Fix loops | Time | Cost |
|---|---|---|---|---|---|
| Pilot (onboarding) | 3 | 31 → 0 | 1 | about 10 min | $0.89 |
| Group 1 (auth, contact, games, home) | 14 | 83 → 0 | 1 | about 10.5 min | $1.47 |
Both went through on the first loop. In the second, the builder was told the order for selecting elements and the exact-text rule up front, and in several places it used a role query where I would have accepted a test id.
Green on the first loop, twice. The next section is about what that green was worth.
12. The second run passed every gate and still needed a reader
Group 1 came back green on every check I had written. Reading the diff found two things those checks could not see.
A suppression resting on a false premise. One test finds the Join card by walking up from its heading with .closest(".hp-live-join-card"), which the lint rules forbid. The builder kept that line behind a suppression, and the comment gave the reason: the card has no hook to select it by. It has one. Our card primitive puts data-hp="card" on every card it draws. The builder had read the component and not the primitive underneath it.
An assertion that had lost its element. A test that read the text of the "unknown game" mark had become a test that the game's slug appears somewhere in the card.
Same method as before: break the component, and run each version of the test against it.
| Change to the component | Original test | The run's version | After review |
|---|---|---|---|
| the layout class moved from the Join card to the wrapper around both cards | passes | passes | fails |
| an unlisted game drawn under the ordinary mark's hook | fails | passes | fails |
The second row is the regression: the run's version had stopped catching something the original caught. The first row I did not expect. The original test could not see that change either, so the reviewed test is stricter than the one we started with. .closest() walks upward until something matches, and a wrapper matches as well as a card does.
Then the automated reviewer found a fault in the checks themselves. Two of them, "no selection by position was added" and "no exact text became a substring match", compared a count before and after. Remove one bad line and add another somewhere else, and the count holds still. They now compare the expressions themselves, as two sorted lists. I tested the new version on a made-up change before trusting it and it raised a false alarm on a trailing comma, so that is normalised too.
So the second run cost $1.47 and about ten minutes of machine time, and then a little under half an hour of reading and re-running, going by the commit times. I think that ratio is the honest one to report. The gates caught what I had already been burned by. The reading caught what I had not.
13. What the numbers said on the second day
Figure 3. What came off the repository's lint error count at each step, 2026-10-09 to 2026-10-10. The largest bar is a rule that was wrong for our setup and was turned off. The other two are real fixes.
| Step | Lint errors | What changed |
|---|---|---|
| Start | 4,334 | |
| Matchers registered, one rule turned off | 4,094 | 240 findings were a rule that did not apply to us |
| Pilot | 4,063 | 31 closed in 3 files |
| Group 1 | 3,980 | 83 closed in 14 files |
All four steps are merged. This table and the paragraph under it are the state on 2026-10-10, when I first wrote this up. Section 14 has how it ended.
The 4,336 this post opened with is the same starting point counted another way: problems on test files, which is these 4,334 errors and two warnings. At the start, as now, every lint error in the repository was in a test file.
That is 114 real fixes in 17 files, with 220 test files and 3,980 findings to go. Every one of the errors left in the repository is a test-lint finding, so finishing this cleanup and getting the linter to exit clean are now the same job. It is slower than one command would have looked. It is also the first version of this work that could be committed.
14. How it ended, and closing the loop
I wrote the sections above two days into the cleanup. Here is the rest, because a post that stops at "220 files to go" has not closed anything.
It finished. Fifteen more groups followed group 1, sixteen in all after the pilot. Lint errors in test files reached zero on 2026-10-12, when the last group merged (#701).
| Factory runs for the cleanup | 17 (the pilot and sixteen groups) |
| Ended green | 15 |
| Cost of the 17 runs | $53.10 |
| Lint errors in test files | 4,334 to 0 |
The pattern in section 12 was not a one-off. Group 1 passed every gate and still needed a reader. So did almost every group after it. Of the fifteen runs that ended green, thirteen had left at least one test asserting less than before, and each was found the same way: break the component, run the old test and the new one, and see which notices. The post about the factory has the count for every group and what the gates did and did not see.
Some of it was found after merging. As new kinds of weakening got names, earlier work was searched for the same shapes. Seventeen checks were restored in pull requests that had already merged (#694, #697, #702). About 127 further places where a lookup was traded for a role or moved into a helper were listed and have not been judged one by one. That is open, and the release notes say so.
My own rules caused some of it. After each review I added a rule to the spec to stop that kind of loss. Three times, a rule in the spec produced the loss it was written to prevent. A cap on suppressions led to weakened tests to stay under the cap. "Never select by position" was applied to positions the original test had taken on purpose. An ordering of preferred queries moved tests off the stable hooks they depended on. What finally helped was a sentence about what to keep, not another prohibition: add a query beside the original lookup, and do not put one in its place. The first group run under that sentence was the first that needed nothing put back. A prohibition names what to move away from and says nothing about where the test lands. The sentence that worked names what has to survive.
So, back to the question at the top: if a tool counts 4,336 things it can fix, how much do you know about whether fixing them works?
The count was accurate. There really were that many findings, and they are all gone now. What the count could not tell me was that the one-command version would have converted working assertions into calls to functions that did not exist, broken 33 files, and loosened about 245 assertions in files where every test stayed green. And what running it on one file could not tell me was that the careful version, done by agents under checks I wrote myself, would still weaken tests in thirteen runs out of fifteen.
One file was enough to cancel a bad plan. It was not enough to trust the good one. Each of those took trying the real thing: first the fix, then the tests. Both times the useful question was how it could pass and still be wrong.
Predictions vs outcome
| Prediction | What happened | Right? |
|---|---|---|
| The fixer can close most of 4,336 problems | 2,371 of them rewrote to matchers that did not exist | No |
| If the types and the linter agree, the function exists | both described a runner we were not using | No |
eslint --fix leaves valid code | 33 files no longer parsed | No |
| A recommended lint rule fits our setup | no-manual-cleanup assumed a global we do not have | No |
| The slice ends with a commit | the pre-commit hook refused 79 of 96 files | No |
| A lint autofix preserves what a test asserts | 245 exact assertions would have become substring matches | No |
| The import costs about 27 ms a file | about 74 ms, by the suite's own total | No |
| Lint problems on test files fall to "≤ 2,500" | the method's deterministic result was 2,970 | No |
| An agent given the rules will follow them | it did, and found the weakened assertions unprompted | Yes |
| A run that passes every gate needs no reading | review found a false suppression and one assertion that had lost its element | No |
| A check that counts will catch what it counts | one removed and one added reads as no change | No |
| A rule added after each review will stop the next loss | three times a rule in the spec produced the loss it was written to prevent | No |
| The careful version will keep what the tests assert | thirteen of fifteen green runs left a test asserting less | No |
Key takeaways
- A count of what a tool can change is not evidence that the result works. Run the method on one file, all the way to the commit, before it becomes a plan.
- A lint plugin and the library it lints for are two packages. Check the one that runs.
- Types can describe a runner you are not using. If the linter and the type-checker agree, you have two descriptions of intent, not a test.
- Read an autofix as a change to what your tests assert, not as formatting. Ours turned equality into "contains".
- Compare an old and a new assertion by breaking the code under them. The doubled space and the reordered status line each took minutes and settled an argument.
- Rehearse the last step first. Our hook would have refused the slice after every other gate had passed.
- Measure a per-file cost from the suite's own totals, not from timing one import in isolation.
- A check that compares counts can be cancelled out. Compare the things themselves.
- When an agent says there is no hook, look one layer down. Ours read the component and missed the primitive it is built from.
What I'd do next
- Write the first test with a new matcher in the same change that adds the lint rule for it. One real use would have failed on day one.
- Report the fixer's two broken shapes upstream, with the cast and the trailing-comma cases as reproductions.
- Find out why a full coverage run times out on a large machine. One simulation test hits its own 120-second limit when the whole suite runs instrumented. It did so before any of this work, and it passes alone in 29 seconds.
- Give the two status regions accessible names, if they should have them. The test id is honest about the element having no role. Whether it should have one is a design question a test cannot answer.
For anyone migrating a test suite with a lint autofix, the short version: pick one file, apply the fix, run the test, break the component, and try to commit. Do that before you count anything.
Evidence
All times UTC.
- The count:
npx eslint . --format=jsonon the release parent at107b07f1: 4,336 problems on 896 test files, 2,371 of themjest-dom/*.npx eslint "**/*.test.ts" "**/*.test.tsx" --fix-dry-run --format=json: 203 files with anoutput, 96 sorting beforecomponents/room/and 107 from it onward; 33 files with a fatal parse message after the fix (11 and 22). - The first group after the fix: from the same two runs, 1,992 problems before and 626 after across the 96 files; 6 files with no error left, 79 with errors left, 11 unparseable. Test-file total 4,336 to 2,970.
- The wasted run: factory session
f25e0286, phases from 2026-10-09T23:25:59Z to 23:30:36Z; planner output at 23:26:25Z; 430,534 tokens. The shell history entry for the dispatch is timestamped 23:25:58Z and names the spec of a slice merged that day. - The missing matchers: an autofixed copy of
components/marketing/blog/BlogPostNewsletterInvitation.test.tsxbeside the original, both run withvitest run --project unit: 2 of 2, and 1 failed of 2 with "Invalid Chai property: toHaveAttribute".npm ls @testing-library/jest-dombefore the change shows it only understorybook. - Where the types came from: the TypeScript compiler API over the project's own
tsconfig.json, asked for the declaration oftoBeInTheDocumentin a Vitest test file with our declaration emptied:node_modules/@vitest/browser/jest-dom.d.ts. Withvitest.config.mtsleft out of the program, the same call reports "Property 'toBeInTheDocument' does not exist on type 'Assertion'". - Cleanup is not automatic here: a throwaway test file with two tests, each rendering one button and calling no
cleanup(); the second finds two. - The hook:
.lintstagedrc.jsonrunseslint --fix --no-warn-ignoredon staged files. 267 of 896 test files had at least one error at107b07f1.git logsince the plugins landed (335d1b56) shows three test files committed, none with a finding. - Exact text: 251 matches of
.textContent).toBe|toEqual|toStrictEqual(in 95 test files at107b07f1; 6 anchored patterns in the fixer's output across the tree. The whitespace handling is in@testing-library/jest-dom6.9.1,toHaveTextContent:normalizewhen the option is on, and a replacement of U+00A0 when it is off. - Position against identity:
components/onboarding/identity/OnboardingIdentityStepForm.test.tsxat2b0dac81(by position) and at452d8567(by its hook), each run against the component with its status line moved, and moved with the "spaces" message appended. - The pull requests: #677 (matchers registered, one rule off; reviewed at
26135a13, merged 2026-10-10T05:19:16Z), #678 (pilot and both helpers; reviewed at2b0dac81with two findings and one from a second reviewer, re-reviewed at452d8567with none, merged 06:52:29Z), #679 (registration only where a document exists; reviewed at63187a0f, merged 07:01:37Z), and #680 (group 1; reviewed at5d8d9375with one finding, on the count-based checks, answered in3810fa2c; a second reviewer then found the plan file's list of corrections incomplete, answered in93d461afand re-reviewed with no finding; merged 20:40:14Z). - Reading group 1: the run's commit is authored 2026-10-10T07:14:55Z; the review fixes are committed at 07:31:40Z and 07:42:24Z.
- How it ended: the release's metrics file, group by group: lint errors 3,706 after group 2, 1,286 after groups 3 to 11, then 946, 764, 493, 218 and 0 after #701 on 2026-10-12. The factory's run records for the 17 cleanup runs: $53.10 in all, 15 ended green, 2 ended red and were finished by hand. Tests found asserting less, per group, in the same metrics file; thirteen of the fifteen green runs had at least one. Seventeen checks restored after merge: three in #694, one in #697, thirteen in #702. "About 127 places" is the release notes' own Known limitation. "Three times now a rule in the spec has produced the loss it was meant to prevent" is the incident log's sentence. The first run that needed nothing put back was group 15, after 45 breaks.
- The factory runs: session
5836bbd9(3 files, lint 4,094 to 4,063 in tracked files, 7 of 7 phases, 2,281,117 tokens, $0.8859) and sessioncff34fde(14 files, lint 4,063 to 3,980 in tracked files, 7 of 7 phases, 4,025,818 tokens, $1.4741). - Group 1 under mutation:
components/home/HomeRoundHostAGameAndJoinWithCodeCards.test.tsxandHomeOpenLobbiesWithLastCallAndDeparturesBoard.test.tsxat2a29211f(original),c333b214(the run) and5d8d9375(reviewed), each run with the layout class moved to the wrapper, and with the unknown-game mark drawn underdata-hp="game-mark". The last cell was re-run on 2026-10-10: 1 failed of 15, "Unable to find an element by: [data-hp="game-mark-unknown"]". - The totals:
npx eslint . --format=jsonon the group 1 branch at3810fa2c: 3,987 errors over 2,464 linted files, of which 3,980 are in files Git tracks, all of them in 220 test files. The other 7 are in one file under a directory listed in.gitignore, last modified 2026-10-08, before the first measurement here. The four totals in the table are the earlier readings less those 7. - Setup time: the
setupfigure in Vitest's own duration line, full unit suite: 13.75 s at107b07f1; 82.25, 85.83 and 91.21 s with the matchers registered in every file; 40.25 and 39.61 s registered where adocumentexists. Wall time for the same runs: 82.6 s; 87 to 97 s; 98.5 and 98.2 s. - Coverage:
npm run coverageon107b07f1and on the later branches fails onscripts/house-players/scheduler.simulation.test.mjswith "Hook timed out in 120000ms"; the file alone passes 44 of 44 in about 29 s, with and without coverage.
Nothing above contains a credential, a token or a player's identity.