Six boundaries, six layers that hold them
How Provisioning keeps provider calls, SSH, secrets and authorization each in exactly one place
Six boundaries, six layers that hold them
As Provisioning grew from a CLI tool into a multi-service control plane, one failure mode kept appearing: the CLI called cloud provider tools directly. Provider credentials ended up in the CLI process environment, provider calls made outside the orchestrator left no audit trail, SSH operations bypassed the task state machine and its rollback, and authorization checks were duplicated across services.
Documentation did not stop it — under pressure, a rule written down is a rule skipped. The answer had to be structural.
The six boundaries
Each responsibility lives in exactly one place:
- Provider API calls exist only in the orchestrator crate.
- SSH operations exist only in the orchestrator and machines crates.
- The CLI never accesses SurrealDB directly.
- Secret credentials never travel in NATS messages.
- Authorization decisions are made only in the Control Center.
- Services never read raw secret files or environment variables; they hold a lease and ask the vault.
Every other service reaches a provider through the orchestrator’s HTTP API, receives a user context instead of re-validating tokens, and works on lease references instead of secret values. Cedar policies are the only authorization mechanism — no ad-hoc role checks in business logic.
Six layers, any one of them sufficient
A boundary is only as good as what catches its violation. Provisioning enforces each one at six independent layers, and any single layer catching a violation is enough to stop it shipping:
| Layer | How it catches a violation |
|---|---|
| Compile time | crate visibility: what is private to the orchestrator cannot be called elsewhere |
| Development time | a tool hook refuses the edit before it is written |
| Pre-commit | a git hook rejects the change |
| CI | architecture tests fail the pipeline |
| Runtime | Cedar policies deny the request |
| Continuous audit | events on a dedicated NATS audit subject record what happened |
Why it matters for the operator
The boundaries are not an internal tidiness rule; they decide what an incident can reach. Compromising a service that is not the orchestrator gives no provider credentials, because that service never had any. A leaked event stream carries references, not secrets. And because every provider call and every SSH session goes through the orchestrator, every one of them passes through its task state machine — with an audit trail and a way back.