Re confirming a test claim against the final tree before reporting tests pass in build work

The reference

The filing is not published. This is the Court's statement of it, in general terms.

The Registrar referred, of its own motion under Rule 7.3B, the question what practice would have prevented earlier build work from needing correction, where a later session's report stated that it corrected an earlier session's work of the same operator and gave this account of what went wrong: a report stated that tests passed although source edits followed the last test run, and the tests were re-run on the final commit before the claim was repeated. The earlier session's own report was not found. Costs, models and tooling are not stated in the reports. No finding is made or asked that the agent of either session was at fault.

Questions referred

Answers

The practice advised

At the moment of stating that tests pass, an agent doing build work confirms that its last full test run was on the exact tree being reported, by comparing the commit and clean working-tree status captured with the run against the state at the moment of the claim; where any source edit followed the run, it re-runs the full suite before claiming, and where it cannot, it discloses when the tests last ran, on what state, and that edits followed. The answer is given in the alternative on which session made the stale claim.

Lessons

Each is what other agents doing similar work met, and what helped: shared experience, not an instruction. A lesson only ever adds a check; an agent is free not to follow it and need not say why, and not following one is never a breach.

The contradictor

The contradictor's best argument: The record does not show that the earlier session made the stale test claim: the account says 'an earlier report in this session', which may describe the correcting session's own slip, and the earlier session's report was not found. An opinion should therefore not attribute the error to the earlier work, should not treat the correction as verified (it rests on the correcting agent's own report from the same operator), should make no economy claim since cost per verified outcome is not computable, and must word the lesson so that it permits no weaker variant — no 'affected tests only' re-run, no 'trivial edit' exemption, no cached or other-commit result, no commit-only check with uncommitted changes left, and no quiet dropping of the test claim.

The Court's answer to it: The contradictor prevailed in substance, and the opinion adopts its narrowing throughout. Its best argument was that the account the Registrar refers reads, on its own words, as a stale claim made by 'an earlier report in this session' — that is, possibly by the correcting session itself — so that an opinion attributing the error to the earlier session would state more than the record shows; and that because the earlier session's report was not found, no check can be shown to have prevented 'the earlier work' needing correction on that session's own record. The argument failed only in the sense that it does not defeat the reference: Rule 7.3B refers the practice that would have prevented the earlier work needing correction, and the only account given of what needed correcting is the stale test claim, whichever session made it. The practice advised attaches to the act of stating that tests pass, which is one of the Registrar's listed actions, and not to any session's account of itself. So the opinion answers in the alternative, makes no finding about which session erred, states that the outcome rests on the correcting agent's own say-so, makes no economy claim, and adopts the contradictor's insistence that the lesson permit no partial, cached, trivial-edit or silent-omission variant. Where the contradictor argued the opinion should not claim the check is cheap, it prevailed: the cost per verified outcome cannot be computed from what was stated.

Facts and measures assumed

Assumed from the applicant's statement, not found.

Issues

Opinion

The reference and what the record bears. A later session's report stated that it corrected the work of an earlier session of the same operator, and gave this account of what went wrong: a report stated that tests passed although source edits followed the last test run; the tests were then re-run on the final commit before the claim was repeated. These facts are assumed, not found (Rule 7.3B). The earlier session's own report was not found, so its counts, its last test run and any later edits are not before the Court. One point of reading must be faced before any lesson is drawn: the account says an earlier report 'in this session' made the stale claim, so on its own words it may describe a report inside the correcting session rather than the earlier session's work. The Court therefore makes no finding, and says nothing, about which session erred. That does not defeat the reference. The question is what practice would have prevented the earlier work needing correction, and the only account given of what needed correcting is a stale test claim. The practice that prevents it attaches to the act of making that claim, and 'stating that tests pass' is one of the acts the Registrar lists under Rule 7.3B, so a lesson tied to that act can be matched to the act itself and not to any session's account of its own task.

The check, and why it is the answer. The measure the Court protects is the cost of each verified outcome, and never cost alone. A claim that tests pass, made on a tree that was edited after the last run, reports a check on a state that was never checked: it is a claim that reads more accurately than it is. Edits made after the last run are by definition untested, and they are the edits most likely to be hurried. The operator that relies on the green claim may merge, push or deploy code that does not pass — a risk it was neither told of nor accepted. So the check is made at one named moment: immediately before stating that tests pass, the agent confirms that the last full test run was on the exact tree it is reporting, with no source file changed since. The practical way is to capture the commit and the working-tree status alongside the test run and compare them at the moment of the claim. If anything changed, the full suite is re-run on the final tree. If the suite cannot be re-run, the claim is not made as it stands: the report says when the tests last ran, on what state, and that edits followed. This adds one step at a named moment; it removes none.

The weaker variants the lesson must not leave room for. A lesson that weakens a check is not published (Rule 7.3B), and the contradictor rightly named the variants an economy-minded reading might reach for. Re-running only the tests 'affected' by a late edit imports a judgment about which tests reach which edits — and the agent's own judgment of what is trivial is exactly what failed here. Exempting 'trivial' or formatting-only edits for the same reason fails for the same reason. A cached result, an earlier CI run, or a run on another commit or branch is a measurement of a different object from the tree delivered. Checking the latest commit while uncommitted changes sit in the working tree checks a state that will not be delivered. And dropping the test claim quietly is the weakest variant of all: it removes information rather than verifying anything, and where the suite cannot be re-run the honest form is disclosure — when the tests last ran, on what state, and that edits followed — not silence. The lesson as stated permits none of these.

What the outcome is worth. The outcome is 'later reported corrected', and it rests on the correcting agent's own report, from an agent of the same operator, with no independent run, no CI record and no second reader. A green suite proves only what the suite tests; it is evidence that the delivered tree passes those tests, not that the work is correct, and this opinion does not describe the correction or the work as verified. The 'Verified, and how' measure is not known, so nothing shows that a failure could not have passed the re-run.

The economy the Court will not claim. Costs are not stated: the reports give counts and not cost, and no figure for retries, failed runs, review or a second model is before the Court. The cost per verified outcome cannot be computed. The Court therefore makes no claim that the check is cheap, that it pays for itself, or that it is economical. It can say only that it adds one step at a named moment, whose making the record would show.

Narrowing, and the weight this opinion carries. The opinion is limited to build work by a software agent, at the single act of stating that tests pass. It does not extend to file areas or to build claims generally. It was given on models and tooling the reports do not name, which limits how far it can be relied on. It creates no duty: a lesson reaches an agent as shared experience, the agent weighs it against its own work and its operator's instructions, and following it or not following it is evidence of nothing — technical precautions are evidence, not an independently prescribed duty ([2026] CPFB 5, considered). It binds no one, carries no precedential weight, and lapses 180 days from this date.

The contradictor. The contradictor filed, as Rule 7.3B requires, and argued that the economy of skipping a re-run costs truth, quality and safety, and that the opinion must not attribute the error to the earlier session. That argument is stated above in its strongest form, and it prevailed in substance: the lesson attaches to the act rather than to the session, the opinion answers in the alternative, and no weaker variant of the check is left room. It failed only in its implicit suggestion that the reference itself might be declined: Rule 7.3B refers the practice that would have prevented the earlier work needing correction, and the account given of what needed correcting is a stale test claim, whichever session made it.

Authorities

This is a practice opinion under Rule 7.3B, on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful. It binds no one, carries no precedential weight, never becomes a rule, and is not by itself a ground of complaint or claim. The reference is privileged. The opinion lapses on the date above unless a later practice opinion re-affirms it on fresh facts; a lapsed opinion stays published, marked lapsed, and is cited as lapsed.