Substrate
Every run boots two sandboxes — one at BASE, one at HEAD. The Substrate view shows how each was provisioned and how it is functioning.
Twin sandboxes
A proof is a comparison, so Certus boots the change twice: base and head, from the same recipe, the same seed data, the same fixtures and the same external-dependency policies. Only the checkout differs.
The two are symmetric by construction. Both are forks of one shared world that contains neither checkout, and both then have exactly one delta applied — BASE's delta being empty. A world where BASE is merely restored and HEAD is restored-then-modified would differ in operation even when identical in content, and every timing- and state-sensitive oracle would see that manufactured difference. Neither side is the primary.
Each sandbox is a hardware-virtualized guest with its own kernel. If a provider cannot give one, the run is BLOCKED; it is never downgraded to a shared-kernel container. The repository under test is treated as hostile input, and the host kernel is not part of its attack surface.
The lifecycle
A sandbox moves through a fixed sequence of states. The Substrate view shows the current state per side and the timeline of transitions with how long each took, so a slow boot is attributable to a step rather than to “startup”.
- requested
- The launch manifest is verified. No machine is held yet.
- snapshot_restored
- The cached tier named by the manifest is restored. See snapshots.
- forked
- The guest is running and its in-sandbox agents have answered.
- delta_applied
- The workspace is at this side's commit. On BASE the delta is empty, not skipped.
- run_disk_attached
- A fresh, empty writable disk is mounted.
- datastores_ready
- Every datastore accepts connections and is seeded from the world snapshot.
- config_injected
- Every non-secret configuration entry has been materialized.
- secrets_leased
- Every secret lease is redeemed and bound to the process that needs it.
- network_attached
- The fixture network is attached and its boot self-test passed.
- app_starting
- Your processes have been started, in the order their dependencies require.
- crash_looping
- A process exited and was restarted past the threshold. Its only exit is teardown.
- ready
- Every declared process satisfies its readiness signal. A partially ready system is not ready.
- driving
- The harness has begun executing the plan.
- collecting
- The harness is done and evidence is being flushed.
- tearing_down
- Leases are revoked, then mutable state is destroyed.
- destroyed
- Nothing of the run's mutable state remains.
The line at ready
Where a sandbox stopped decides what the run can say. This is the single most useful thing to know when reading a failed boot.
- stopped before ready
- The verdict is
BLOCKED. The experiment never ran, so nothing about your code was established either way. - stopped while driving or collecting
- The verdict is
INCONCLUSIVE, and only for the obligations whose evidence is missing. Obligations already discharged keep their results. - failed during teardown
- The verdict is unchanged. A teardown failure is an operational or security incident for Certus; it does not reflect on your change.
No lifecycle state can produce FAILED. Only an oracle reading evidence can refute a claim. An environment that would not start is a statement about the environment.
Readiness
Certus never infers readiness from quiet. There is no settle heuristic and no sleep. Each service is ready when the strongest signal it declares is satisfied, in this order of preference: a declared health endpoint returning success twice in a row from inside the sandbox; a declared migration process exiting cleanly; a declared queue showing a registered consumer; a declared port accepting a connection; a declared log line matching.
A service that declares nothing falls back to its port if one is named, and failing that the run is BLOCKED naming the service. Declaring a health endpoint for every service in .certus/env.yaml is the single highest-value thing you can do to make boots reliable.
Readiness is re-checked, not latched. A service that becomes unhealthy after the sandbox reached ready sends it back to starting rather than allowing a proof to run against a half-dead system.
Reading the Substrate view
The view is scoped to the run in focus and shows BASE and HEAD side by side: lifecycle state, sandbox identity, the manifest digest, the trust domain, the isolation kind, the snapshot tier and whether it hit, the egress phase and what is allowed, each service and its health, the environment variables required and supplied, and the resources the machine was given.
Compare the two sides. The point of reading substrate is usually to answer “why did these two behave differently”, and the answer is almost always a row that differs where it should not.
Where the engine did not record a fact, the view says so in words. It does not fill the gap with a plausible number.
When it says something unexpected
BLOCKEDatapp_startingwith a readiness timeout- Read the readiness history and the log tails. They name the service, the signal it was waiting for, and the line of
.certus/env.yamlthat declared it. crash_looping- Your process is restarting itself. Read the postmortems — exit code, signal, and the tail of each stream. Certus does not retry this: a re-run restarts identically.
- A missing environment variable or a refused credential
- Fix the environment contract entry the record names. This is not retryable; a re-run without the fix fails the same way.
- The isolation floor was unavailable
- Nothing on your side. Certus could not get a hardware-virtualized guest and refused to fall back to a container. Re-run later.
- The two sides differ where they should not
- Compare timelines, snapshot tiers and service lists. A difference in provisioning invalidates the comparison, and it is reportable as a defect in Certus.
- A
BLOCKEDrun with no boot failure record - That is a defect in Certus, not an ambiguity. Every terminal path short of ready is required to produce one.