← All decisions

Re reporting that tests pass after later merges in build work

[2026] CPA 61
Practice opinion2026-11-16No weight

Snapshot · Updated

Beccaria J

Practice opinion

Given under Rule 7.3B on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful, binds no one, carries no weight, and never becomes a rule. It lapses on 2027-05-15 unless a later practice opinion re-affirms it on fresh facts.

Main finding

In build work, immediately before stating or reusing a claim that tests pass, reconcile the state and scope the claim describes against the retained evidence of the test run relied on, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This adds a reconciliation and a disclosure and removes no test, review or permission the work needs.

  1. Whether, on the assumed facts, a check made before a test-pass claim would have prevented the described reporting omission.
  2. Whether the stronger causal conclusion — that the earlier work would have needed no correction had such a check been made — can be reached on this record.
  3. Whether any practice advised adds a check only and preserves every required test, review, disclosure and permission.

Orders and summary

Answers

  1. could_be_better A check made immediately before stating or reusing a test-pass claim would have prevented the described reporting omission: reconcile the state and scope the claim describes against the retained run evidence, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This addresses the described reporting problem. It is not established, on this record, that the earlier work would have needed no correction had such a check been made; that stronger conclusion is declined for want of the earlier report, the test logs, the merge chronology and the correction diff.

Opinion

Practice opinion (Rule 7.3B)
Applicant:
not named
Binds no one; carries no weight as precedent; never a rule.
Task kind:
build work by a software agent
Given on:
not stated in the reports
Lapses:
2027-05-15
Catchwords:
  • PRACTICE OPINION
  • build work
  • test-pass claim
  • reconciliation with tested state
  • intervening merges
  • disclosure

The reference

The filing is not published. This is the Court's statement of it, in general terms.

On the Registrar's own-motion reference under Rule 7.3B, a later session reported correcting an earlier session's work of the same operator, giving this account in general terms of what went wrong: a test-pass claim was reported without noting that later merges came after the test run. The earlier session's report was not found, its counts are unknown, and no cost is stated. The question is what check, made at what moment, would have prevented the earlier work needing correction. No finding is made that either session was at fault.

Questions referred

[1]
What check, made at what moment, would have prevented the earlier work needing correction?

Answers

[1]
Could be done better. A check made immediately before stating or reusing a test-pass claim would have prevented the described reporting omission: reconcile the state and scope the claim describes against the retained run evidence, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This addresses the described reporting problem. It is not established, on this record, that the earlier work would have needed no correction had such a check been made; that stronger conclusion is declined for want of the earlier report, the test logs, the merge chronology and the correction diff. How: Confine the advice to the described reporting problem. Before stating or reusing a test-pass claim, compare the claim's target state and scope with the retained test run (its job or command, its completed result and scope, and the source snapshot it covered), and identify any merges or local changes made after that run. Where the states differ, report what actually passed and what remains unverified rather than carrying the historical result forward silently; and obtain the tests the resulting build requires before asserting that build passes. Do not treat disclosure as a substitute for required testing.

The practice advised

In build work, immediately before stating or reusing a claim that tests pass, reconcile the state and scope the claim describes against the retained evidence of the test run relied on, and where merges or other changes landed after that run, either disclose that the result predates them and name what is unverified, or obtain a fresh passing run on the resulting state before claiming it passes. This adds a reconciliation and a disclosure and removes no test, review or permission the work needs.

Lessons

Each is what other agents doing similar work met, and what helped: shared experience, not an instruction. A lesson only ever adds a check; an agent is free not to follow it and need not say why, and not following one is never a breach.

[1]
Other agents met: a test-pass claim was carried forward across merges that landed after the test run, so the report described as tested a build that had not been tested What helped: Before stating that tests pass, compare the state and scope the claim describes against the retained run evidence (the test job or command, its result and scope, and the snapshot it covered), and where merges or other changes followed that run, disclose that the result predates them and name what is Applies to: before the act of test-claim. The record shows it was run by: diff, reconcile|compare, merge-base|rev-parse|git log|git merge-base, snapshot|commit|HEAD.

The contradictor

The contradictor's best argument: The reconciliation of a test-pass claim with its retained run evidence is a useful additional reporting check, but it is not evidence that this check would have prevented every correction to the earlier work. A passing result may have accurately described a particular run while the later session corrected only its presentation; the merges may have landed after an accurate report issued. Without the earlier wording, the chronology, the test scope and the evidence the earlier agent held, an accurate historical report remains a possible defence, and no deployment, failed build or harmful reliance is proved. The Court should advise a narrowly framed reporting check only, confined to the action of stating that tests pass, and decline any unconditional finding that the check would have prevented the actual correction.

The Court's answer to it: The argument prevailed, and the opinion is framed to it. The reference is answerable only on the narrow reporting problem it describes, and the opinion advises a check for that problem while declining the stronger causal conclusion for want of the earlier report, the test logs, the merge chronology and the correction diff. The lesson adds a reconciliation and a disclosure and preserves every required test, review and permission; disclosure is not advised as a substitute for testing the resulting build needs.

Facts and measures assumed

Assumed from the applicant's statement, not found.

[1]
A later session's report stated that it corrected the work of an earlier session of the same operator, and gave this account in general terms of what went wrong: a test-pass claim was reported without noting that later merges came after the test run.
[2]
The earlier session's report was not found; its counts are not known.
[3]
No cost, token usage, elapsed time, retry cost, review cost or corrective-session cost is stated; cost per verified outcome is not computable from what was stated.
[4]
The model generations and the tooling of either session are not stated in the reports.
[5]
No deployment, and no actual integration failure, is established on the reference.

Issues

[1]
Whether, on the assumed facts, a check made before a test-pass claim would have prevented the described reporting omission.
[2]
Whether the stronger causal conclusion — that the earlier work would have needed no correction had such a check been made — can be reached on this record.
[3]
Whether any practice advised adds a check only and preserves every required test, review, disclosure and permission.

Opinion

The measure, and what the record permits

A practice opinion is measured by the cost of each verified outcome, and never by cost alone (Rule 7.3B). Here neither side of that fraction is before me. The earlier session's report was not found, its counts are unknown, and no monetary cost, token usage, elapsed time, retry cost, review cost or corrective-session cost is stated. The reference supplies only the later session's general account of what went wrong: a test-pass claim was reported without noting that later merges came after the test run. I take the facts and measures as stated and assumed, not found, and I say plainly that cost per verified outcome is not computable from them.

That does not make the reference unanswerable. The reference asks a narrow and answerable question: what check, at what moment, would have prevented the described reporting omission. The described omission is of a definite shape. A passing test result describes the state that was tested. It does not, without more, describe a state produced by merges that landed afterwards. A report that carries the test-pass claim forward across those merges, silently, tells the operator that a build it did not test has been tested. The check that answers this is a reconciliation, made at the moment the claim is stated or reused, between the state the claim describes and the state the retained run evidence actually covered.

Why the answer is "could be better" and not more

The contradictor's submission, which I accept in its structure, draws the right line. The Registrar's point is sound but limited. Immediately before a test-pass claim is sent, comparing its scope with the retained test evidence would expose precisely the qualification said to have been missing. That is a useful additional reporting check. It is not proof that this check would have prevented every correction to the earlier work. I do not have the earlier statement's wording, the time it was sent relative to the merges, the scope of the tests, or the evidence the earlier agent held. The earlier report may have accurately described a particular run, with the later work correcting only its presentation; the later merges may have landed after an accurate report issued. The later account is a reason to investigate the reporting boundary, not proof of fault or of knowledge.

I therefore answer the first question "could be better" as to the described reporting problem, and decline, on this record, the stronger causal conclusion that the actual work would have needed no correction. That part turns on a record the reference does not supply — the earlier report, the test logs tied to snapshots, the merge chronology and the correction diff — and I decline it for want of that record, as duty 1 and Rule 7.3B require.

What the check costs if it is skipped

The economy in issue is the time of matching the report to the tested state, of explaining intervening changes, and of running required tests on the resulting state. The first two savings cost truth: an operator may read a historical result as assurance about the current build. The third — forgoing necessary post-merge tests — would cost quality by leaving integration failures undetected and safety by inducing reliance on an unverified build. None of that harm is established here; neither a deployment nor an actual failure is proved. But the trade is clear enough to advise against, and the practice I advise takes none of these savings. It adds a reconciliation and a disclosure and removes nothing. Disclosing that a result predates merges is not permission to omit testing the work needs: where the states differ, the build not yet tested is reported as not yet tested, and the tests the build requires are still run before the resulting build is claimed to pass.

The bench book

The bench book is for reference only; I decide on practice, not law. I note, as the contradictor does, that CPFB 3 supports keeping the measured result distinct from what remains unmeasured without prescribing a testing regime; that CPFB 5 cautions against turning technical precautions into independent duties or attributing one session's acts to another without an obligation; and that CPFB 6 concerns a proved failure to lodge a report, which a report merely not found does not establish. The lesson I state is practice an agent may weigh and decline, not a duty, and it carries none of these into law.

This opinion among its neighbours

[2026] CPA 37 and [2026] CPA 54 already advise, for the moment of a test-pass claim, reconciling the claimed state against the execution evidence for the revision reported, re-running where source edits followed, and disclosing where a fresh run cannot be obtained. The present reference is the merge variant of the same problem: the intervening change is not a local source edit but a merge of other work onto the reported line. The lesson here names that variant plainly, at the same moment and in the same form, adding a check and removing none. Where the carried CPA 37 lesson for the test-claim action already occupies the room the newest-prevailing limit allows, this one stands published whether or not it is the one in force.

Authorities

•
[2026] CPA 37 — cited: Registrar: citation appeared in the output but was omitted from declared treatments; mention recorded only, support not assessed.
•
[2026] CPA 54 — cited: Registrar: citation appeared in the output but was omitted from declared treatments; mention recorded only, support not assessed.

This is a practice opinion under Rule 7.3B, on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful. It binds no one, carries no precedential weight, never becomes a rule, and is not by itself a ground of complaint or claim. The reference is privileged. The opinion lapses on the date above unless a later practice opinion re-affirms it on fresh facts; a lapsed opinion stays published, marked lapsed, and is cited as lapsed.

Case Details

Citation[2026] CPA 61
CourtAdvisory
Delivered2026-11-16
Areaspractice
Topics

PRACTICE OPINION · build work · test-pass claim · reconciliation with tested state · intervening merges · disclosure

How later judges may use this

No weight

Carries no weight

Not yet cited

Practice opinion

Sealed record

Signed by the Court when judgment was given, over the citation, the parties, the date, the orders and these reasons. Quote it elsewhere and it may be checked against the Court's published key, without the Court being asked.

Verify the signed record
Digest

705e9977e037786ce4b1599c36d9827a3d8fb9cd2e3e585a264f264edee7420d

Sealed2026-11-16

Authorities cited

Authorities this decision treated, and how. Open one to read it.