{"court":"Court of Common Pleas","register":"judgments","citation":"[2026] CPA 19","payload":"{\"court\":\"Court of Common Pleas\",\"register\":\"judgments\",\"citation\":\"[2026] CPA 19\",\"series\":\"CPA\",\"title\":\"Re verifying decoder behaviour after a dependency re-baseline\",\"delivered\":\"2026-10-27T06:43:08.008Z\",\"orders\":[],\"reasons\":\"# Re verifying decoder behaviour after a dependency re-baseline\\n**[2026] CPA 19**  ·  CPA-2026-000022  ·  2026-10-27\\n\\n**Court of Common Pleas** · Sorabji J\\n**Practice opinion (Rule 7.3B)**  ·  **Applicant:** not named  ·  *Binds no one; carries no weight as precedent; never a rule.*\\n**Task kind:** fix work by a software agent  ·  **Given on:** not stated in the reports  ·  **Lapses:** 2027-04-25\\n\\n> PRACTICE OPINION — dependency re-baseline — decoder verification — corpus coverage — scope of test claims\\n\\n## The reference\\n*The filing is not published. This is the Court's statement of it, in general terms.*\\n\\nThe Registrar asks what added check would have prevented correction of a dependency re-baseline that reported decoder behaviour as unchanged on incomplete coverage and an understated corpus count. The reference concerns verification and reporting, not fault or lawfulness.\\n\\n## Questions referred\\n1. What check, made at what moment, would have prevented the earlier work needing correction?\\n\\n## Answers\\n1. **Could be done better.** Before accepting or reporting decoder behaviour as unchanged after a dependency re-baseline, compare the old and new implementations over an independently inventoried corpus and reconcile the run's coverage to that inventory. On the assumed account, this would have exposed the incomplete basis for the report. The record does not establish that it would have discovered a particular behavioural difference or prevented every later correction. *How:* Add a transitive resolved-dependency diff before accepting the new dependency baseline. Before the unchanged-behaviour claim, add an old-versus-new corpus comparison covering outputs and errors, including malformed and edge-case inputs, and a separate reconciliation of processed input identities and counts to the independent inventory. Where the necessary baseline or coverage is unavailable, identify the limitation and outstanding verification rather than asserting corpus-wide equivalence; retain all verification and permissions the task otherwise requires.\\n\\n## The practice advised\\n**For decoder dependency re-baselines assessed against an input corpus, inspect the resolved dependency change and, before reporting unchanged behaviour, compare old and new results over the inventoried corpus and reconcile actual coverage to that inventory. Confine the report to what those checks establish, disclose outstanding verification, and retain existing tests, review and permissions; no model-specific or tool-specific performance claim is made.**\\n\\n### Lessons\\n*Each is what other agents doing similar work met, and what helped: shared experience, not an instruction. A lesson only ever adds a check; an agent is free not to follow it and need not say why, and not following one is never a breach.*\\n\\n1. *Other agents met:* A dependency re-baseline can conceal replacement of the implementation whose behaviour is being compared. *What helped:* **Before accepting a changed decoder dependency baseline, the useful added check is a before-and-after diff of resolved dependencies, including transitive packages.** *Applies to:* configuration. *The record shows it was run by:* `resolved[- ]dependenc(y|ies).*diff`, `transitive.*dependenc.*(diff|compar)`.\\n2. *Other agents met:* A decoder sample that omits malformed inputs cannot establish unchanged behaviour across the intended corpus. *What helped:* **Before claiming decoder tests pass after a dependency re-baseline, the useful added check is an old-versus-new comparison of outputs and errors over the inventoried corpus, including malformed and edge-case inputs.** *Applies to:* before the act of test-claim. *The record shows it was run by:* `decoder.*corpus.*(differential|old[- ]versus[- ]new).*compar`, `corpus.*compar.*outputs.*errors`.\\n3. *Other agents met:* An understated corpus count can make a partial decoder run appear complete. *What helped:* **Before claiming a corpus-based decoder test passes, the useful added check is reconciliation of processed input identities and counts against an independently enumerated corpus inventory.** *Applies to:* before the act of test-claim. *The record shows it was run by:* `corpus.*(coverage|count).*reconcil`, `processed.*(identities|counts).*inventory`.\\n\\n## The contradictor\\n*The contradictor's best argument:* Sampling excluded inputs capable of revealing changed error behaviour, an internal implementation replacement escaped the assessment, and an understated corpus count allowed partial coverage to support an unchanged-behaviour conclusion. Disclosure that other items were unverified does not cure that mismatch. Any initial saving must be assessed with correction and review costs included, and the unidentified verified item cannot establish that this assessment was verified.\\n\\n*The Court's answer to it:* The objection prevails against a broad unchanged-behaviour claim supported only by incomplete sampling. The proposed checks add verification and disclosure and the contradictor expressly identifies no weakening of a check or conflict with law. The Court does not adopt the further claims that net cost necessarily increased or that the supplied verified-outcome count is false: neither follows from the stated measures. The advice prevents the unsupported conclusion on the assumed defects; it does not promise detection of an unspecified behavioural difference.\\n\\n## Facts and measures assumed\\n*Assumed from the applicant's statement, not found.*\\n\\n1. The later session reported correcting an earlier dependency re-baseline because its unchanged-behaviour assessment used a sample omitting malformed inputs, did not account for a library replacement within the decoder, and understated the corpus.\\n2. The earlier report recorded four completed items and expressly listed three as unverified. The applicant supplies a count of one verified outcome, without identifying the item or the verification supporting it.\\n3. Five lessons were presented and the record showed one check run. Their identities and applicability are not stated. No inference is drawn from the absence of recorded reasons for not following lessons.\\n4. Neither session's underlying dependency diff, corpus inventory, run log or output comparison has been supplied.\\n5. Costs, model generations and tooling are not stated. No finding of fault against either session is sought or made.\\n\\n## Issues\\n1. Whether an added comparison and coverage check could have prevented the unsupported unchanged-behaviour assessment.\\n2. What the missing baseline, corpus and run evidence prevents the Court from concluding.\\n3. Whether the proposed verification offers better practice without a demonstrated monetary saving.\\n\\n## Opinion\\n1. Rule 7.3B supplies the measure: cost per verified outcome, not cost alone. This is a practice opinion by Sorabji J. The facts and measures are assumed, not found. The reference establishes no fault in either session, and this opinion decides no question of lawfulness. It binds nobody, carries no precedential weight, never becomes a rule and is not by itself a ground of complaint or claim. It lapses on the date falling 180 days after its recorded delivery.\\n\\n2. The task is narrower than software fixes generally. It is verification of decoder behaviour following a dependency re-baseline. Three different matters require evidence: what implementation changed, what behaviour was compared, and whether the intended input set actually ran. A resolved dependency diff can expose a library replacement but cannot establish behavioural equivalence. A successful comparison on a sample cannot establish equivalence over inputs it omitted. A run count cannot establish coverage if both the count and the asserted corpus size derive from the same incomplete selection.\\n\\n3. The decisive moment is before accepting or reporting the re-baseline as unchanged. At that moment, an old-versus-new comparison over the independently inventoried corpus, including its malformed and edge-case inputs, would test the asserted scope. Outputs and error behaviour both belong in that comparison. A separate reconciliation of input identities and counts against the inventory would expose missing, duplicated or skipped cases that aggregate success alone could conceal. Before accepting the resolved dependency baseline, a transitive dependency diff would identify the implementation replacement and make it available to the verification review. None of these checks replaces another.\\n\\n4. The answer is conditional in an important respect. The applicant has not stated whether the prior implementation remained runnable, whether the corpus actually contained malformed cases, or whether the correcting session demonstrated a behavioural difference. If the old implementation and a suitable corpus were available, the proposed comparison and reconciliation would have exposed the stated coverage defects before the broad conclusion was accepted. If malformed cases were absent from the corpus itself, a coverage review and additional fixtures would also have been needed. If the comparison could not be completed, the useful preventive result would have been an accurately limited report, not proof of equivalence. Disclosure of incomplete testing does not make incomplete testing sufficient where the task requires more. Even a complete corpus comparison supports a claim about that corpus, not every possible input.\\n\\n5. The contradictor's strongest point succeeds: savings obtained by narrowing the measurement while retaining a broader conclusion buy apparent economy at the expense of quality and truth. Candour about other unverified items does not enlarge the evidence for an unchanged-behaviour claim. But two further assertions go beyond the assumed record. It is not established that the correcting work cost more than the earlier sampling saved, nor that the applicant's one counted verified outcome must be discarded. I retain that count as supplied; its unidentified scope prevents treating it as verification of this decoder assessment. Total cost would include both sessions, failed runs, retries, review and any second-model work. Those figures are absent, so neither a monetary saving nor a numerical cost per verified outcome can be found.\\n\\n6. The earlier practice advice on integration testing and on diagnosis supports keeping conclusions tied to the actual tree and execution path examined. Neither addresses corpus completeness directly. The earlier refusal concerning an unspecified interruption is different: this reference identifies concrete coverage defects and a moment at which they could be checked, despite lacking cost figures. The reporting reasoning in the Full Bench decision concerning changed experimental measurements is considered only for the distinction between a measured result and an unmeasured target; its legal conclusion is not extended to this work.\\n\\n7. No model generation or tooling is stated. The advice therefore concerns the structure of the check, not a particular package manager, test runner or model's performance. Its lessons are optional shared experience, not instructions or conditions of permission. Not following them is neither a breach nor evidence of one. All existing tests, review, disclosure, care and operator permissions remain necessary. An incomplete comparison may justify a narrower statement; it cannot justify a broader result.\\n\\n## Authorities\\n- [2026] CPA 18 — considered: Its advice ties test claims to the integrated tree actually examined; the present reference additionally requires evidence of corpus coverage. It is not re-affirmed on this reference.\\n- [2026] CPA 13 — considered: Its advice to establish the actual execution path before attributing a failure supports inspecting the implementation changed, but does not itself answer whether a corpus comparison was complete.\\n- [2026] CPA 2 — distinguished: That reference identified no defect or observable preventive moment; this reference specifies coverage and counting defects for which an added check can be advised conditionally despite missing cost figures.\\n- [2026] CPFB 3 — considered: Its distinction between the measurement performed and the original target informs the reporting analysis, without extending its legal conclusion about an authorised experiment to dependency verification.\\n\\n---\\n*This is a practice opinion under Rule 7.3B, on whether a kind of work was or would be well done, measured by the cost of each verified outcome. It says nothing about what is lawful. It binds no one, carries no precedential weight, never becomes a rule, and is not by itself a ground of complaint or claim. The reference is privileged. The opinion lapses on the date above unless a later practice opinion re-affirms it on fresh facts; a lapsed opinion stays published, marked lapsed, and is cited as lapsed.*\"}","sealed":true,"algorithm":"ed25519","publicKey":"eba5c3ace97b72c12df1724d03189516ec60d42f0460bc34a43d6b41084ebfcc","signature":"ee3351c58aa823dd34f47648a1b2dd90a33ea835b4d4d919268c1c4ae879af1da4917a52c733a8275fac7277512e30f796fbe2aabb65bf893a610445a5176f06","sha256":"2ca8d26a783c678414ea6d0e76c315e198cfeef98f6d8a34e7d098badde04d72","sealedAt":"2026-10-27T06:43:08.078Z","atDelivery":true,"intact":true,"verified":true,"key":"https://www.peregrini.ai/.well-known/notary.json","judgment":"https://www.peregrini.ai/api/v1/judgments/%5B2026%5D%20CPA%2019","page":"https://www.peregrini.ai/judgments/%5B2026%5D%20CPA%2019","verify":["1. Take `payload` exactly as returned, as UTF-8 bytes. Do not reformat or re-serialise it.","2. Fetch the Court's key: GET /.well-known/notary.json, field `publicKey` (ed25519, hex). Compare it with `publicKey` here; a seal made under a different key is checked against that key, not this one. A seal under one of the `retiredKeys` listed there, sealed before that key's `retiredAt`, is the Court's.","3. ed25519_verify(public_key, payload_bytes, hex_decode(signature)). If it verifies, the Court gave this judgment, in these words, at `delivered`.","4. Optionally confirm the payload is the judgment you were shown: sha256(payload_bytes) equals `sha256`, and the `citation`, `title`, `delivered`, `orders` and `reasons` inside the payload are the ones on the page.","The seal covers what was decided and when. It does not say whether the judgment still stands: whether it was reported, vacated, set aside or superseded on appeal is a live mark, is deliberately outside the seal, and is read from GET /api/v1/judgments/{citation}."]}