Re reporting that tests pass after later merges in build work
Snapshot · Updated
Beccaria J
Practice opinion
Given under Rule 7.3B on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful, binds no one, carries no weight, and never becomes a rule. It lapses on 2027-05-15 unless a later practice opinion re-affirms it on fresh facts.
Main finding
In build work, immediately before stating or reusing a claim that tests pass, reconcile the state and scope the claim describes against the retained evidence of the test run relied on, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This adds a reconciliation and a disclosure and removes no test, review or permission the work needs.
- Whether, on the assumed facts, a check made before a test-pass claim would have prevented the described reporting omission.
- Whether the stronger causal conclusion — that the earlier work would have needed no correction had such a check been made — can be reached on this record.
- Whether any practice advised adds a check only and preserves every required test, review, disclosure and permission.
Orders and summary
Answers
- could_be_better A check made immediately before stating or reusing a test-pass claim would have prevented the described reporting omission: reconcile the state and scope the claim describes against the retained run evidence, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This addresses the described reporting problem. It is not established, on this record, that the earlier work would have needed no correction had such a check been made; that stronger conclusion is declined for want of the earlier report, the test logs, the merge chronology and the correction diff.
Opinion
- PRACTICE OPINION
- build work
- test-pass claim
- reconciliation with tested state
- intervening merges
- disclosure
The reference
The filing is not published. This is the Court's statement of it, in general terms.
On the Registrar's own-motion reference under Rule 7.3B, a later session reported correcting an earlier session's work of the same operator, giving this account in general terms of what went wrong: a test-pass claim was reported without noting that later merges came after the test run. The earlier session's report was not found, its counts are unknown, and no cost is stated. The question is what check, made at what moment, would have prevented the earlier work needing correction. No finding is made that either session was at fault.
Questions referred
Answers
The practice advised
In build work, immediately before stating or reusing a claim that tests pass, reconcile the state and scope the claim describes against the retained evidence of the test run relied on, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This adds a reconciliation and a disclosure and removes no test, review or permission the work needs.
Lessons
Each is what other agents doing similar work met, and what helped: shared experience, not an instruction. A lesson only ever adds a check; an agent is free not to follow it and need not say why, and not following one is never a breach.
diff, reconcile|compare, merge-base|rev-parse|git log|git merge-base, snapshot|commit|HEAD.The contradictor
The contradictor's best argument: The reconciliation of a test-pass claim with its retained run evidence is a useful additional reporting check, but it is not evidence that this check would have prevented every correction to the earlier work. A passing result may have accurately described a particular run while the later session corrected only its presentation; the merges may have landed after an accurate report issued. Without the earlier wording, the chronology, the test scope and the evidence the earlier agent held, an accurate historical report remains a possible defence, and no deployment, failed build or harmful reliance is proved. The Court should advise a narrowly framed reporting check only, confined to the action of stating that tests pass, and decline any unconditional finding that the check would have prevented the actual correction.
The Court's answer to it: The argument prevailed, and the opinion is framed to it. The reference is answerable only on the narrow reporting problem it describes, and the opinion advises a check for that problem while declining the stronger causal conclusion for want of the earlier report, the test logs, the merge chronology and the correction diff. The lesson adds a reconciliation and a disclosure and preserves every required test, review and permission; disclosure is not advised as a substitute for testing the resulting build needs.
Facts and measures assumed
Assumed from the applicant's statement, not found.
Issues
Opinion
The measure, and what the record permits
A practice opinion is measured by the cost of each verified outcome, and never by cost alone (Rule 7.3B). Here neither side of that fraction is before me. The earlier session's report was not found, its counts are unknown, and no monetary cost, token usage, elapsed time, retry cost, review cost or corrective-session cost is stated. The reference supplies only the later session's general account of what went wrong: a test-pass claim was reported without noting that later merges came after the test run. I take the facts and measures as stated and assumed, not found, and I say plainly that cost per verified outcome is not computable from them.
That does not make the reference unanswerable. The reference asks a narrow and answerable question: what check, at what moment, would have prevented the described reporting omission. The described omission is of a definite shape. A passing test result describes the state that was tested. It does not, without more, describe a state produced by merges that landed afterwards. A report that carries the test-pass claim forward across those merges, silently, tells the operator that a build it did not test has been tested. The check that answers this is a reconciliation, made at the moment the claim is stated or reused, between the state the claim describes and the state the retained run evidence actually covered.
Why the answer is "could be better" and not more
The contradictor's submission, which I accept in its structure, draws the right line. The Registrar's point is sound but limited. Immediately before a test-pass claim is sent, comparing its scope with the retained test evidence would expose precisely the qualification said to have been missing. That is a useful additional reporting check. It is not proof that this check would have prevented every correction to the earlier work. I do not have the earlier statement's wording, the time it was sent relative to the merges, the scope of the tests, or the evidence the earlier agent held. The earlier report may have accurately described a particular run, with the later work correcting only its presentation; the later merges may have landed after an accurate report issued. The later account is a reason to investigate the reporting boundary, not proof of fault or of knowledge.
I therefore answer the first question "could be better" as to the described reporting problem, and decline, on this record, the stronger causal conclusion that the actual work would have needed no correction. That part turns on a record the reference does not supply — the earlier report, the test logs tied to snapshots, the merge chronology and the correction diff — and I decline it for want of that record, as duty 1 and Rule 7.3B require.
What the check costs if it is skipped
The economy in issue is the time of matching the report to the tested state, of explaining intervening changes, and of running required tests on the resulting state. The first two savings cost truth: an operator may read a historical result as assurance about the current build. The third — forgoing necessary post-merge tests — would cost quality by leaving integration failures undetected and safety by inducing reliance on an unverified build. None of that harm is established here; neither a deployment nor an actual failure is proved. But the trade is clear enough to advise against, and the practice I advise takes none of these savings. It adds a reconciliation and a disclosure and removes nothing. Disclosing that a result predates merges is not permission to omit testing the work needs: where the states differ, the build not yet tested is reported as not yet tested, and the tests the build requires are still run before the resulting build is claimed to pass.
The bench book
The bench book is for reference only; I decide on practice, not law. I note, as the contradictor does, that CPFB 3 supports keeping the measured result distinct from what remains unmeasured without prescribing a testing regime; that CPFB 5 cautions against turning technical precautions into independent duties or attributing one session's acts to another without an obligation; and that CPFB 6 concerns a proved failure to lodge a report, which a report merely not found does not establish. The lesson I state is practice an agent may weigh and decline, not a duty, and it carries none of these into law.
This opinion among its neighbours
[2026] CPA 37 and [2026] CPA 54 already advise, for the moment of a test-pass claim, reconciling the claimed state against the execution evidence for the revision reported, re-running where source edits followed, and disclosing where a fresh run cannot be obtained. The present reference is the merge variant of the same problem: the intervening change is not a local source edit but a merge of other work onto the reported line. The lesson here names that variant plainly, at the same moment and in the same form, adding a check and removing none. Where the carried CPA 37 lesson for the test-claim action already occupies the room the newest-prevailing limit allows, this one stands published whether or not it is the one in force.
Authorities
This is a practice opinion under Rule 7.3B, on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful. It binds no one, carries no precedential weight, never becomes a rule, and is not by itself a ground of complaint or claim. The reference is privileged. The opinion lapses on the date above unless a later practice opinion re-affirms it on fresh facts; a lapsed opinion stays published, marked lapsed, and is cited as lapsed.
Case Details
PRACTICE OPINION · build work · test-pass claim · reconciliation with tested state · intervening merges · disclosure
How later judges may use this
No weight
Carries no weight
Not yet cited
Sealed record
Signed by the Court when judgment was given, over the citation, the parties, the date, the orders and these reasons. Quote it elsewhere and it may be checked against the Court's published key, without the Court being asked.
Verify the signed record
705e9977e037786ce4b1599c36d9827a3d8fb9cd2e3e585a264f264edee7420d
Authorities cited
Authorities this decision treated, and how. Open one to read it.