How to Test Whether a Proof Surface Is Actually Independent
Independence is testable. This framework gives a structured method for determining whether a proof surface meets the independence threshold — what to examine, what artifacts to check, and what a passing result actually requires.
A proof surface is independent when the verification can be performed by a party with no access to the originating system, using evidence that system cannot modify, producing a result that can be replayed. Independence is the threshold between verification and self-reporting — below it, you have a claim; above it, you have a check.
When to use this framework
- A vendor claims their AI platform outputs signed receipts verified by a third-party service, and you need to determine whether that verifier can operate without vendor tooling
- An audit report cites a third-party compliance firm as verifier, and you need to determine whether the evidence package was produced by the auditee
- A security product provides a "verification dashboard" and you need to determine whether that dashboard is part of the system being verified
- A regulated system submits attestation logs to an external authority, and you need to determine whether the submitting party can selectively omit records
- A platform claims immutable event receipts, and you need to determine whether replay from those receipts is possible without the originating platform
The worked example (start here)
Use this to orient the framework before the steps:
Scenario: A security platform uses a third-party auditing firm for SOC 2 Type II compliance.
- The verifier is genuinely independent — no financial relationship with the vendor
- The evidence package is produced by the vendor: selected log exports, formatted reports, a custom API the auditor queries
- The auditor cannot independently export raw event records without the vendor's tooling
- The auditor cannot replay any specific event without the vendor's platform
Result: Independent verifier. Controlled evidence path. No replay capability. This proof surface fails the independence test.
Verifier independence is necessary but not sufficient. This example passes the verifier test — the auditor has no financial relationship with the vendor — and fails every test after it. The evidence path runs entirely through vendor-controlled tooling. The auditor sees what the vendor surfaces. No event can be reconstructed from the artifacts alone. The name "third-party audit" is accurate; the independence claim is not. The threshold is not met by having a separate entity involved. It is met when that entity can operate, verify, and replay without the subject's participation.
Step 1: Test the verifier
Determine whether the verifier is independent in three dimensions: financial, technical, and operational. A verifier that is independent in name but technically dependent on the subject's infrastructure is not independent for verification purposes.
| Test question | Passing condition | Failing condition |
|---|---|---|
| Does the verifier have a financial relationship with the party being verified (fees, contracts, equity)? | No financial dependency; verifier is compensated by a third party or operates under a public mandate | Verifier is paid by the subject, contracted by the subject, or has a recurring commercial relationship |
| Does the verifier require access to the subject's systems, APIs, or tooling to perform verification? | Verifier can operate using only externally accessible evidence artifacts | Verifier queries subject-controlled APIs, uses subject-provided export tools, or requires subject credentials |
| Can the verifier initiate verification without notifying or involving the subject? | Verifier can pull or access evidence independently and on-demand | Verification requires subject to initiate, export, or authorize the evidence transfer |
| Does the verifier make decisions using their own tooling and criteria? | Verifier applies independent analysis against raw artifacts | Verifier reviews pre-formatted summaries or reports produced by the subject |
Step 2: Test the evidence path
Determine whether the party being verified can influence what the verifier sees. The verifier's independence is irrelevant if the evidence itself is produced, selected, or filtered by the subject. Test against each of the four canonical failure modes:
Export dependency — The verifier cannot access raw evidence without using export tooling controlled by the subject. Test: can the verifier retrieve unprocessed event records through a path the subject does not control?
Selection control — The subject determines which records are included in or excluded from the evidence package. Test: is the scope of evidence defined by the subject, or by an independent retention and collection mechanism?
Format control — The subject determines how evidence is structured before the verifier receives it. Test: does the verifier receive raw artifacts, or pre-formatted documents and reports prepared by the subject?
Replay denial — The evidence package is insufficient to reconstruct any specific event without the originating platform. Test: given only the artifacts provided, can a verifier reconstruct the inputs, execution context, and outputs of a specific event independently?
Checklist:
- Export dependency: verifier has a path to raw evidence that does not run through subject-controlled tooling
- Selection control: the record set is defined outside the subject's control
- Format control: verifier receives raw artifacts, not subject-prepared summaries
- Replay denial: artifacts alone are sufficient to reconstruct at least a representative event
Any unchecked item is a failure of evidence-path independence.
Step 3: Test replay capability
Replay means: given only the artifacts in the evidence package, can an independent party reconstruct what happened for a specific event — inputs, execution context, and outputs — without access to the originating platform?
This is distinct from log completeness. A complete log that requires the originating system to interpret does not satisfy replay. The test is whether reconstruction is possible from artifacts alone.
How to test it:
- Select a specific event from the evidence package.
- Attempt to reconstruct: what inputs triggered it, what executed, what output was produced, and what the state was at the time.
- At no point use the originating platform, its APIs, or its export tooling.
Replay capable: The reconstruction succeeds using only the artifacts. Inputs are recorded. Execution context is captured. Output is present. The reconstruction can be independently verified against the artifact set.
Replay denied: The reconstruction requires the originating platform to interpret log formats, resolve identifiers, supply context, or confirm outputs. The artifact set is necessary but not sufficient — it depends on the system it is meant to verify.
A proof surface where replay is denied is a proof surface where the subject retains permanent participation rights in its own verification.
Pass/fail conditions
| Proof surface property | Pass condition | Fail condition |
|---|---|---|
| Verifier independence | Verifier has no financial dependency on the subject and requires no subject-controlled access to perform verification | Verifier is contracted by or technically dependent on the subject |
| Evidence-path independence | Evidence is collected through a path the subject cannot modify, filter, or suppress | Subject controls export, selection, or formatting of the evidence the verifier receives |
| Replay capability | An independent party can reconstruct a specific event from artifacts alone, without the originating platform | Reconstruction requires the originating platform, its APIs, or its interpretation layer |
| Tamper evidence | Artifacts are cryptographically signed or hash-chained before leaving the originating system, and the signing authority is independent | Artifacts are unsigned, signed by the subject, or the integrity check requires subject-controlled tooling to verify |
A proof surface must pass all four conditions. Partial credit is not a verification result — it is a scoped failure.
What a passing result looks like
A proof surface that passes this framework has the following characteristics, each materially present and verifiable:
Verifier: A named party with no financial relationship to the subject, documented access to evidence through a path the subject does not control, and the technical capability to perform verification independently.
Evidence artifacts: Raw event records collected by an independent or append-only mechanism, scoped by a retention policy the subject cannot modify, stored in a location the subject cannot write to after the fact.
Signing chain: Each artifact is cryptographically signed at the point of creation by a key the subject does not hold. The signing authority is a separate entity. Signature verification is possible using publicly available key material.
Replay record: For at least a representative sample of events, the artifact set contains sufficient information — inputs, execution context, outputs — to reconstruct the event without querying the originating system.
Storage and access: Artifacts are stored outside the subject's infrastructure, or in infrastructure the subject cannot modify without detection. An independent party can retrieve and verify the full artifact set without the subject's participation.
The principle
This framework tests one thing: whether the architecture permits verification or merely permits the appearance of verification. A verifier that cannot access evidence independently, against artifacts the subject cannot modify, and produce a replay-capable result, is not a verifier — it is a witness to the subject's self-report. The line between "independent in name" and "independent in the architecture" is the evidence path. That path either runs through the subject or it doesn't.
If this looks familiar, reading more won’t fix it → /review
See also: Independence Without Replay Is Still Weak — the note that distinguishes what independence proves from what replay proves.