Milestones¶

Development progress across all phases of Civitas. Two parts: Part 1 — Shipped (completed work, historical record — do not edit) and Part 2 — Backlog (the active todo list — everything not yet done).

Status legend¶

Symbol	Status
✅	Completed
🔄	In Progress
⏳	Planned
⏸️	Deferred
💡	Idea — to be specced
🗂️	Tracked backlog (index of deferred work)

Part 1 — Shipped¶

Everything in this part is done.

Overview¶

Phase	Milestone	Completed
1	Core Runtime	Mar 2026
2	Ecosystem — Transports	Mar 2026
2	Ecosystem — Observability	Apr 2026
2	Ecosystem — EvalLoop (local)	Apr 2026
2	Ecosystem — Remote Eval Exporters	Apr 2026
3	Developer Experience — CLI & Dashboard	Mar 2026
3	Developer Experience — MCP Integration	Apr 2026
3	Developer Experience — GenServer	Apr 2026
—	Infrastructure & Release	Apr 2026
4	Dynamic Agent Spawning	Apr 2026
4	Security Hardening	May 2026
4	Codebase Security & Enterprise Posture	Apr 2026
4	Capability-Aware Registry	May 2026
4	HTTP Gateway	Apr 2026
4	Gateway API Surface	Apr 2026
4	Postgres StateStore + Migration	May 2026
—	v0.4.0 Release Fixes	Jul 2026
—	v0.5.0 — Released	Jul 2026
—	v0.6.0 — Gateway Completion	Jul 2026
—	v0.7.0 / v0.7.1 / v0.7.2 / v0.7.3 / v0.7.4 — Spawn Maturation, Gateway Auth & Bus-Native Streaming	Jul 2026
—	v0.8.0 — Supervision Core Hardening	Jul 2026
—	v0.8.1 — Verification Perimeter	Jul 2026
—	v0.8.2 — Hygiene	Jul 2026
—	v0.9.0 — Supervision Endgame	Jul 2026
—	v0.9.1 — Post-endgame Polish	Jul 2026
—	v0.9.2 — Examples Completeness	Jul 2026
—	v0.9.2.1 — Bugfix Release	Jul 2026
—	v0.9.3 — Telemetry: OTEL Linkage, Prometheus, Grafana, Native Storage, Query Layer, TUI	Jul 2026

Phase 1 — Core Runtime¶

Status: ✅ Completed — March 2026

#	Deliverable	Priority	Status
M1.1	`AgentProcess` base class, mailbox, `handle()` lifecycle	🔴 High	✅
M1.2	`Supervisor` with `ONE_FOR_ONE`, `ONE_FOR_ALL`, `REST_FOR_ONE` strategies	🔴 High	✅
M1.3	Backoff policies (`CONSTANT`, `LINEAR`, `EXPONENTIAL`), restart windows, crash timestamps	🔴 High	✅
M1.4	`Serializer` with msgpack + schema versioning; `DeserializationError` contract	🔴 High	✅
M1.5	`InProcessTransport` + `MessageBus` routing; request-reply with ephemeral topics	🔴 High	✅
M1.6	`StateStore` protocol; SQLite plugin; state persistence across restarts	🟡 Medium	✅
M1.7	Plugin system; LLM providers (Anthropic, OpenAI, Gemini, Mistral, LiteLLM)	🔴 High	✅

M1.8 (Medicus self-healing hero demo) is not done — tracked in Part 2 — Backlog.

Phase 2 — Ecosystem¶

M2.1 — ZMQ Multi-Process Transport¶

Status: ✅ Completed — March 2026

Deliverable	Status
`ZMQTransport` with XSUB/XPUB proxy	✅
`ZMQProxy` daemon thread	✅
PUB/SUB bridging across OS processes	✅
Request-reply over ephemeral topics	✅
`Worker` process class for multi-process deployment	✅

M2.2 — NATS Distributed Transport¶

Status: ✅ Completed — March 2026

Deliverable	Status
`NATSTransport` with JetStream support	✅
At-least-once delivery via durable consumers	✅
Multi-machine deployment support	✅
Worker multi-transport handoff	✅

M2.3 — OTEL Observability¶

Status: ✅ Completed — April 2026

Deliverable	Status
`Tracer` with automatic span generation per message	✅
`SpanQueue` with overflow protection	✅
`OTELAgent` batch exporter with configurable flush interval	✅
`ConsoleBackend` and `FanOutBackend`	✅
OTLP gRPC exporter plugin	✅
Trace propagation across agents (trace_id, parent_span_id)	✅

M2.5 — EvalLoop (Local)¶

Status: ✅ Completed — April 2026

Corrective observability loop: a supervised EvalAgent process monitors agent behaviour and injects correction signals back into running agents. Local in-process evaluation only — remote eval engine integrations are M2.6. See design spec.

Deliverable	Status
`civitas/evalloop.py` — `EvalEvent`, `CorrectionSignal`, `EvalAgent` base class	✅
`AgentProcess.emit_eval(event_type, payload, eval_agent)` — emit observable events	✅
`AgentProcess.on_correction(message)` — override hook for nudge/redirect signals	✅
`civitas.eval.halt` message type — cleanly stops target agent (on_stop still runs)	✅
Rate limiting — sliding window per target agent (`max_corrections_per_window`, `window_seconds`)	✅
`EvalExporter` protocol — interface defined, not implemented (M2.6)	✅
Topology YAML — `type: eval_agent` shorthand in `Runtime.from_config()`	✅
20 unit + integration tests	✅
`EvalAgent` exported from `civitas` top-level package	✅

Implementation checklist¶

M2.6 — Remote Eval Exporters¶

Status: ✅ Completed — v0.4 | Priority: 🔴 High

Plugin adapters connecting Civitas's EvalEvent stream to external eval engines. All platforms consume the same EvalEvent schema; each exporter translates to the platform's expected format. OTEL GenAI Semantic Conventions are the alignment layer — EvalEvent fields map directly to standard OTEL attributes. See design spec.

Deliverable	Status
`EvalExporter` protocol implementation + registration on `EvalAgent`	✅
`civitas[arize]` — Arize Phoenix exporter (OTEL GenAI spans via OTLP)	✅
`civitas[fiddler]` — Fiddler exporter (export to Fiddler AI; two-way guardrail receive deferred to M4.2)	✅
`civitas[langfuse]` — Langfuse exporter (open-source, self-hostable)	✅
`civitas[braintrust]` — Braintrust exporter	✅
`civitas[langsmith]` — LangSmith exporter	✅
`emit_eval()` forwards to all registered exporters in addition to local EvalAgent	✅
Topology YAML — declare exporters per eval_agent node	✅
≥ 5 unit tests per exporter (mocked SDK calls)	✅

Phase 3 — Developer Experience¶

M3.1–M3.3 — CLI and Dashboard¶

Status: ✅ Completed — March 2026

Deliverable	Status
`civitas init` project scaffolding	✅
`civitas run` supervisor + worker modes	✅
`civitas topology validate / show / diff`	✅
`civitas deploy docker-compose` generation	✅
`civitas state list / clear`	✅
`civitas dashboard` live terminal dashboard	✅

M3.4 — MCP Integration¶

Status: ✅ Completed — April 2026 | Corrected — July 2026

Correction (July 2026): This section originally described MCPClient and MCPTool as civitas-core deliverables. Per boundary.md's ownership split, the MCP tools gateway (Fabrica) is a civitas-contrib concern. The actual implementation moved there; civitas core kept only the wire-layer types and the lazy-import integration point. Table below reflects current reality, not the original plan.

MCP protocol plumbing — the wire layer between Civitas agents and MCP tool servers. Agents call tools by direct address (mcp://server/tool); civitas core owns the config types and the AgentProcess.connect_mcp() integration point; the actual client/transport implementation lives in Fabrica (civitas-contrib).

Scope: civitas core keeps config types (no mcp SDK dependency at import time) and the connect_mcp() lazy-import hook. Connection handling, tool wrapping, connection pooling, circuit breakers, unified tool namespacing, and semantic retrieval are not in core scope — they belong to Fabrica. See design spec (describes the original plan; superseded on the client/tool split, see correction above).

Dependency chain: M3.4 (types + integration point) → Fabrica (MCPClient, MCPTool, pooling, retrieval)

Deliverable	Status	Lives in
`civitas.mcp.types` — `MCPServerConfig`, `MCPToolSchema`, `MCPToolError` (no `mcp` SDK import)	✅	civitas core
`AgentProcess.connect_mcp(config)` — lazily imports `fabrica.mcp.client.MCPClient`, registers tools into `self.tools`; idempotent	✅	civitas core
`ToolRegistry.deregister_prefix(prefix)`	✅	civitas core
Topology YAML `mcp.servers` block — parsed into `Runtime._mcp_configs`, auto-connect at agent startup	✅	civitas core
`MCPClient` — connect (stdio + SSE), `list_tools`, `call_tool`	✅	civitas-contrib (fabrica) — not this repo
`MCPTool(ToolProvider)` — `mcp://server_name/tool_name` name scheme, `civitas.mcp.call` OTEL span	✅	civitas-contrib (fabrica) — not this repo
`civitas[mcp]` optional extra	❌ removed	use `pip install fabrica-context` instead
`CivitasMCPServer(GenServer)` — expose an agent tree as an MCP server	⏸️	deferred to Fabrica (scope boundary decision), not started anywhere
Unit tests (types + YAML parsing, core-side only)	✅	civitas core (`tests/unit/test_mcp.py`)

Explicitly out of scope for civitas core: - MCPClient / MCPTool implementation — Fabrica (civitas-contrib) - Connection pooling / persistent sessions — Fabrica (MCPToolSource) - Circuit breakers per server — Fabrica - Semantic or keyword tool retrieval (find_tools) — Fabrica - Unified cross-agent tool namespace — M4.4 ToolStore - Per-agent credential isolation — M4.2 Security Hardening

M3.5 — GenServer¶

Status: ✅ Completed — April 2026

OTP-style generic server primitive for separating stateful API/RPC service processes from AI agent processes on the message bus. See design spec.

Deliverable	Status
`GenServer` base class with `handle_call` / `handle_cast` / `handle_info` dispatch	✅
`call()` — synchronous request-reply with timeout	✅
`cast()` — async fire-and-forget	✅
`send_after()` — delayed self-message (tick / timer support)	✅
`init()` — startup initialisation hook	✅
Supervision-compatible (works as a child of any `Supervisor`)	✅
Topology YAML support (`type: gen_server`)	✅
19 unit tests	✅
`examples/rate_limiter.py` — token-bucket rate limiter demo	✅

Implementation checklist¶

Ordered tasks — each step is independently mergeable.

Core module — civitas/genserver.py
- GenServer(AgentProcess) class — no LLM or tool plugin injection
- handle() dispatcher: route by reply_to → handle_call; __cast__ marker → handle_cast; else → handle_info
- handle_call / handle_cast / handle_info stubs with correct signatures
- async def init() hook invoked once at process start
- send_after(delay_ms, payload) — schedules handle_info to self
- Track send_after tasks; cancel all on stop()
- Enforce handle_call returns a dict (reject None to prevent caller hangs)
call() / cast() aliases
- AgentProcess.call(name, payload, timeout) — alias over existing ask()
- AgentProcess.cast(name, payload) — send() with __cast__ marker
- Runtime.call() / Runtime.cast() — external entry points
Topology YAML support
- Loader accepts type: gen_server (module/class resolution identical to type: agent)
- civitas topology validate passes for gen_server nodes
- civitas topology show renders gen_server with distinct icon/label
- civitas topology diff treats gen_server nodes correctly
Observability
- Emit civitas.genserver.call span for handle_call
- Emit civitas.genserver.cast span for handle_cast
- Emit civitas.genserver.info span for handle_info
- Trace propagation preserved across call() boundaries
Tests (≥ 15 cases in tests/test_genserver.py)
- handle_call returns reply via reply_to
- handle_cast runs, no reply emitted
- handle_info invoked for non-call non-cast messages
- call() timeout raises within configured bound
- send_after fires handle_info after delay
- send_after tasks cancelled cleanly on stop()
- init() runs before first message handled
- GenServer as child of ONE_FOR_ONE, ONE_FOR_ALL, REST_FOR_ONE supervisors
- Restart triggers init() again (state resets unless StateStore configured)
- StateStore-backed state survives restart
- self.llm not present on GenServer instance
- self.tools not present on GenServer instance
- handle_call returning non-dict raises
- GenServer ↔ AgentProcess sibling communication round-trip
- Topology YAML round-trip: load → run → topology show matches
Example + documentation
- examples/rate_limiter/ — end-to-end RateLimiter(GenServer) with consumer agent
- User guide page referencing docs/design/genserver.md
- API reference entry for civitas.genserver
- mkdocs.yml nav updated
Release
- CHANGELOG.md entry under ## [0.3.0]
- Cross-reference M3.4 (MCP) and M2.5 (EvalLoop) for coordinated v0.3 cut

Infrastructure & Release¶

Status: ✅ Completed — April 2026

Deliverable	Status	Completed
Agency → Civitas rename (115 files)	✅	Apr 2026
Pre-commit hooks (ruff, mypy, file hygiene)	✅	Apr 2026
GitHub Actions CI (Python 3.12 / 3.13 / 3.14)	✅	Apr 2026
PyPI publishing via OIDC trusted publishing	✅	Apr 2026
GitHub Pages documentation site	✅	Apr 2026
Test coverage raised from 85% → 90%+	✅	Apr 2026
Framework adapters: LangGraph, OpenAI Agents SDK, CrewAI (stub)	✅	Mar 2026 — civitas-contrib, not this repo; `civitas/adapters/` does not exist in python-civitas

Phase 4 — Platform Maturation¶

M4.1b — Dynamic Agent Spawning¶

Status: ✅ Completed — April 2026 | Priority: 🔴 High

Agents spawn and decommission other agents at runtime. Enables LLM-driven orchestrators that create specialist agents on demand. See design spec.

Design decisions locked: - DynamicSupervisor is a separate class from Supervisor (Erlang-faithful separation — ONE_FOR_ONE only, starts empty) - DynamicSupervisor is declared as a static child in topology YAML; its children are dynamic - self.spawn() targets the nearest ancestor DynamicSupervisor — no explicit target at the call site - on_spawn_requested is a governance veto hook on DynamicSupervisor (return False to deny) - max_children enforces blast radius per DynamicSupervisor

Open design questions (being resolved): - ~~Q2 — Restart semantics~~ → transient default; no escalation on exhaustion; on_child_terminated hook - Q3 — on_spawn_requested placement (supervisor vs agent vs both) - ~~Q4 — Limit semantics~~ → both: max_children (concurrent) + max_total_spawns (lifetime budget) - ~~Q5 — Despawn semantics~~ → despawn() hard stop + stop(drain, timeout) soft stop (awaitable, timeout fallback to hard stop) - ~~Q6 — Cross-process spawning~~ → bus message protocol from day one; in-process v0.4; cross-process v0.5 (homogeneous deployments) - ~~Q7 — topology show live state~~ → TopologyServer(GenServer) JSON HTTP endpoint; CLI pings /topology; falls back to static YAML if unreachable

Deliverable	Status
`DynamicSupervisor` class — starts empty, ONE_FOR_ONE, `max_children` + `max_total_spawns` limits	✅
`type: dynamic_supervisor` in topology YAML	✅
`self.spawn(AgentClass, name, config)` — nearest ancestor routing	✅
`self.despawn(name)` — hard stop; `self.stop(drain, timeout)` — soft stop	✅
`on_spawn_requested` governance hook on `DynamicSupervisor`	✅
`on_child_terminated` notification hook on spawning agent	✅
`Runtime.spawn()` / `Runtime.despawn()` / `Runtime.stop_agent()` — external entry points	✅
`SpawnError` added to error hierarchy	✅
38 unit + integration tests	✅
`TopologyServer(GenServer)` — supervised JSON HTTP management endpoint	✅
`topology show` pings `TopologyServer`; falls back to static YAML	✅
`examples/dynamic_spawning.py`	✅

M4.2 — Security Hardening¶

Status: ✅ Completed — v0.4 | Priority: 🔴 High

Design approved. Splits into five independently shippable sub-milestones — see docs/design/security-hardening.md for full rationale, design decisions, and resolved questions.

Recommended delivery order: a → c → d → e → b.

M4.2a — Identity & Signing¶

Status: ✅ Complete

Deliverable	Status
`civitas/security/` package: `IdentityConfig`, `SigningConfig`, `SecurityConfig`	✅
`AgentIdentity`: Ed25519 keypair generation, OpenSSH-style storage (`id_ed25519` / `id_ed25519.pub`)	✅
`KeyRegistry`: public key lookup by agent name	✅
`MessageSigner`: sign outgoing envelopes (v=2 wire format), verify incoming	✅
`NonceCache`: bounded LRU replay protection (10k entries)	✅
`SignatureError` — new `CivitasError` subclass	✅
`SigningSerializer` wrapping `MsgpackSerializer`	✅
Multi-node key distribution: public keys in topology YAML; spawn-message vouching for dynamic agents	✅
`security:` YAML block parsing in `Runtime.from_config()`	✅
InProcess transport: signing bypassed entirely (D9 performance rule)	✅
`signing.allow_unsigned: true` escape hatch for rolling upgrades	✅
Unit + integration tests ≥90% coverage on new code	✅

M4.2b — Transport mTLS¶

Status: ✅ Complete

Deliverable	Status
ZMQ CURVE: server keypair on proxy, client keypairs on Workers	✅
NATS TLS + nkeys: Ed25519-based subject auth, TLS cert/key/CA config	✅
`security.transport` YAML block plumbing into ZMQ and NATS transports	✅
`civitas security init` CLI — scaffold keys and config for ZMQ/NATS deployments	✅

M4.2c — Credential Isolation¶

Status: ✅ Complete

Deliverable	Status
`${VAR_NAME}` env-var substitution in `Runtime.from_config()`	✅
Unset variable raises `ConfigurationError` with clear message	✅
`civitas.secrets.SecretsProvider` protocol + file/env/Vault implementations	✅
Per-agent `credentials:` block in topology YAML	✅
Plugin handles: `self.llm("anthropic")` resolves per-agent credential at call time	✅

M4.2d — Tool Sandbox¶

Status: ✅ Complete

Deliverable	Status
Bubblewrap wrapper for MCP subprocess execution on Linux	✅
`sandbox:` YAML block per MCP server (network, filesystem allowlists)	✅
Refuse-to-start when `sandbox.enabled: true` and `bwrap` unavailable	✅
Clear error messages with per-distro install instructions	✅

M4.2e — Audit Log¶

Status: ✅ Complete

Deliverable	Status
`civitas.audit` module: `AuditEvent` TypedDict, `AuditSink` protocol	✅
`JsonlFileSink`: batched fsync (100ms / 100 events), `sync_writes` option, SIGHUP rotation	✅
`NullSink` for tests	✅
Emission at chokepoints: `MessageBus.route()`, `MCPTool.execute()`, sandbox violations, secret access	✅
`SyslogSink` and `OtlpSink` implementations	✅

M4.3 — Codebase Security & Enterprise Posture¶

Status: ✅ Completed — April 2026 | Priority: 🔴 High

Complements M4.2. Where M4.2 hardens the runtime (mTLS, message signing, credential isolation, sandboxing), M4.3 hardens the codebase and supply chain so enterprises have a clear security story before adoption: known vulnerabilities tracked, dependencies scanned, secrets never committed, a published threat model, and a documented disclosure process.

The deliverables are split across tooling (CI-enforced scanners), documentation (threat model, security architecture, adoption checklist), and process (disclosure policy, release notes, third-party audit).

Deliverable	Status
SAST in CI — Bandit + Semgrep on every PR, fail build on `HIGH`+	✅
Dependency scanning — `pip-audit` in CI + Dependabot weekly	✅
SBOM generation — CycloneDX SBOM published with every release	✅
Secret scanning — `gitleaks` pre-commit hook + CI job on full history	✅
`docs/security/threat-model.md` — STRIDE analysis per runtime component	✅
`docs/security/architecture.md` — security model (trust boundaries, supervision, transport isolation)	✅
`SECURITY.md` — responsible disclosure policy, contact, supported versions, response SLAs	✅
`docs/security/enterprise-checklist.md` — adoption checklist (deployment hardening, config review, audit log integration)	✅
External security audit before v1.0 — fix all `HIGH`+ findings, publish summary	⏳ Deferred to pre-v1.0
Continuous posture — CVE watch on runtime deps, security release notes, CVSS-scored advisories	⏳ Ongoing process

M4.4 — Capability-Aware Registry¶

Status: ✅ Completed — May 2026 | Priority: 🟡 Medium

Agents declare capability tags at the class level; the registry supports filtered lookups; agents can route to any capable peer without knowing its name.

Deliverable	Status
`RoutingEntry.capabilities` + `RoutingEntry.capability_metadata` fields	✅
`LocalRegistry.register()` / `register_remote()` accept capabilities	✅
`find_by_capability(tag)` — all agents (local + remote) with that tag	✅
`find_by_capabilities(tags, match="any"\\|"all")` — multi-tag filtered lookups	✅
`AgentProcess.capabilities` / `capability_metadata` class-level declarations	✅
`AgentProcess.send_capable(capability, payload)` — fire-and-forget to any capable agent	✅
`CapabilityNotFoundError` raised when no registered agent declares the tag	✅
YAML `capabilities:` / `capability_metadata:` block overrides class-level defaults	✅
Distributed propagation: Worker announcements carry capabilities; `_on_remote_register` populates remote entries	✅
`RegistryListener` hook: async callbacks fired after every register/deregister (Presidium integration point)	✅
`LocalRegistry.add_listener()` / `remove_listener()` — fire-and-forget tasks with error logging	✅
Public exports: `RoutingEntry`, `RegistryListener`, `CapabilityNotFoundError` from `civitas` top-level	✅
29 unit tests covering all registry operations, listener lifecycle, and `send_capable`	✅

Design notes¶

Boundary with Presidium: Civitas capability tags are operational routing data — plain strings by convention (e.g., "text.summarize"). Presidium owns the controlled vocabulary, human-readable descriptions, and governance metadata. Presidium plugs in via the RegistryListener hook — it receives every register/deregister event with full capability info and maintains its own authoritative Agent Registry.

Distributed topology: Every node (Runtime and Worker) has a complete capability view of the deployment. Worker announcements include capabilities and capability_metadata; the Runtime's _on_remote_register handler populates register_remote() entries. send_capable() thus works transparently across process boundaries.

Tag format: plain strings, dot-namespaced by convention ("domain.action"). No enum enforcement — Presidium owns the controlled vocabulary and Civitas treats tags as opaque routing keys.

HTTP Gateway¶

Status: ✅ Completed — April 2026

Supervised edge process bridging external HTTP traffic into the Civitas message bus. HTTP/1.1 + HTTP/2 (uvicorn) and HTTP/3 / QUIC (aioquic) in v0.4. gRPC deferred to v0.5. See design spec.

Deliverable	Status
`HTTPGateway(AgentProcess)` — ASGI app, request translation, route table	✅
HTTP/1.1 + HTTP/2 via uvicorn[standard] — uvloop + httptools (`civitas[http]`)	✅
HTTP/3 / QUIC via aioquic — `Alt-Svc` header, 0-RTT (`civitas[http3]`)	✅
TLS config from topology YAML / env vars	✅
Topology YAML support (`type: http_gateway`)	✅
Graceful drain on supervisor shutdown	✅
≥ 20 unit tests + ≥ 5 integration tests	✅
`examples/http_gateway.py`	✅
gRPC via grpclib / grpcio	⏸️ v0.5
Custom `.proto` loading from `proto_dir`	⏸️ v0.5

Implementation checklist¶

Gateway API Surface¶

Status: ✅ Completed — April 2026

Declarative routes, Pydantic request/response validation, middleware chain, and auto-generated OpenAPI 3.1 docs on top of HTTPGateway. See design spec.

Deliverable	Status
`@route` decorator — documents HTTP method + path on agent handler (YAML is authoritative for wiring)	✅
Path parameter extraction into `message.payload`	✅
`@contract` decorator — Pydantic request/response validation, 422 error shape	✅
`GatewayRequest` / `GatewayResponse` / `NextMiddleware` types	✅
Global + route-scoped middleware chain	✅
Stateful GenServer middleware via `request.gateway.call()`	✅
Auto-generated OpenAPI 3.1 spec at `GET /openapi.json`	✅
Swagger UI at `GET /docs`, ReDoc at `GET /redoc`	✅
YAML-declared routes and schemas (no decorators required)	✅
`civitas topology validate` cross-checks YAML routes against `@route` decorators	✅
≥ 15 unit tests + ≥ 3 integration tests	✅

Routing authority: YAML is the single source of truth for gateway wiring. @route stores metadata on the method object only — it is never read by the gateway at runtime. Its value is (1) colocated documentation of intent and (2) a machine-checkable annotation that civitas topology validate cross-references against YAML to warn on drift.

Implementation checklist¶

Postgres StateStore + Migration¶

Status: ✅ Completed — May 2026 | Priority: 🔴 High | Corrected — July 2026

Correction (July 2026): This section originally described PostgresStateStore itself as a civitas-core deliverable. Per boundary.md, state store implementations (SQLite, Postgres, Redis) are a civitas-contrib concern — only the StateStore protocol, InMemoryStateStore (the trivial default), the plugin loader's lazy resolution, and the civitas state migrate CLI are core's job. Table corrected below.

SQLite works for single-process deployments but breaks under concurrent cross-process writes (ZMQ Level 2+, NATS Level 3). PostgresStateStore extends the StateStore protocol — switching backends is a topology YAML change with no agent code changes, and no top-level civitas import ever references asyncpg directly.

Deliverable	Status	Lives in
`StateStore` protocol extended with `list_agents()` and `close()`	✅	civitas core
`InMemoryStateStore.list_agents()` / `close()`	✅	civitas core
Plugin loader entry `type: postgres` → lazy `civitas_contrib.plugins.postgres_store.PostgresStateStore` import	✅	civitas core (resolution only)
`@runtime_checkable StateStore` — `isinstance()` checks work	✅	civitas core
`civitas state migrate <src> <dst>` — dry-run by default, `--execute` to apply; lazy-imports `PostgresStateStore` from civitas-contrib	✅	civitas core
`_parse_dsn()` — `sqlite:<path>`, `.db`/`.sqlite` extension, `postgresql://` URL	✅	civitas core (`cli/state.py`)
`PostgresStateStore` — `asyncpg` backend, connection pool, `civitas_agent_state` JSONB table	✅	civitas-contrib — not this repo
`civitas[postgres]` optional extra — `asyncpg>=0.29`	❌ removed from core	use `civitas-contrib[postgres]`
Helpful `ImportError`/`ConfigurationError` with install hint if civitas-contrib not installed	✅	civitas core (lazy-import pattern)
20 unit tests covering protocol, migrate CLI DSN parsing, and mocked contrib import	✅	civitas core
Zero-downtime dual-write migration	⏸️ Deferred — maintenance-window copy is sufficient for v0.4
PgBouncer deployment guide	⏸️ Deferred to docs pass
MySQL StateStore (`aiomysql` backend)	✅ Done (civitas-contrib 0.2.0) — `civitas_contrib.plugins.mysql_store.MySQLStateStore`; loader `type: mysql`; real-MySQL tested

MySQL StateStore — deferred because Postgres covers the multi-process persistence gap and asyncpg is a better async foundation. If ever built, it belongs in civitas-contrib alongside the other state store implementations, following the same lazy plugin-loader pattern (type: mysql loader entry resolving to civitas_contrib.plugins.mysql_store.MySQLStateStore, mysql:// DSN in _parse_dsn).

v0.4.0 Release Fixes¶

Status: ✅ Completed — July 2026

Two bugs reported against v0.3.0, found by a downstream project building against civitas at HEAD. Both fixed and folded into the v0.4.0 release alongside the Phase 4 work above — the v0.4.0 changes were already sitting on main, unreleased, when these were reported.

Deliverable	Status
GH #6 — Route-scoped gateway middleware wired into ASGI dispatch	✅
GH #7 — `civitas/py.typed` marker added	✅

GH #6 — Route-scoped gateway middleware is parsed but never executed¶

RouteEntry.middleware (a route's own middleware: list in topology YAML) was parsed into RouteEntry objects but never read by GatewayASGI. The dispatch layer built its middleware chain once at construction time from config.middleware (global only) — the matched route's .middleware field was never consulted. A route declaring its own auth/guard middleware (e.g. an admin-only route on an otherwise public gateway) would silently run without that guard, with no error or warning.

This contradicted docs/gateway.md and this file's own Gateway API Surface checklist, both of which stated route-scoped middleware runs after global middleware, before contract validation.

Fix: - GatewayASGI._handle_http() now matches the route before building the middleware chain. - Route-scoped middleware (entry.middleware, resolved via the existing load_middleware() loader) is appended after global middleware when building the chain, restoring the documented order: global → route-scoped → contract validation → bus dispatch. - Resolved route middleware callables are cached per RouteEntry (by object identity) so they are loaded once, not on every request. - Unresolvable route middleware paths are logged and skipped, matching the existing behavior for global middleware — never raises at request time. - Corrected the two checklist items below (Gateway API Surface, "Wire into ASGI" and "Tests") that had been checked off despite this gap.

GH #7 — Missing `py.typed` marker despite "Typing :: Typed" classifier + mypy --strict¶

pyproject.toml declares the "Typing :: Typed" classifier and the package runs mypy --strict internally, but no civitas/py.typed marker file existed in the source tree or the published 0.3.0 wheel. Downstream projects running their own mypy --strict got error: ... missing library stubs or py.typed marker [import-untyped] for every import from civitas.

Fix: - Added empty civitas/py.typed. - [tool.hatch.build.targets.wheel] packages = ["civitas"] picks it up automatically — verified present in the built wheel (unzip -l dist/*.whl) as part of the release checklist below.

v0.5.0 — Released¶

Status: ✅ Released — July 2026 (buckets A + B + C)

Scope was three buckets; a fourth candidate (D) was explicitly deferred to a future version — see below. Bucket A (correctness & hardening) ✅, Bucket B (durable suspension) ✅, Bucket C (doc hygiene) ✅ — all done and fully tested. A msgpack>=1.2.1 security bump (GHSA-6v7p-g79w-8964) also landed on this line. Cutting the v0.5.0 release tag is the maintainer's call.

A — Correctness & hardening¶

Status: ✅ Completed — July 2026

Seven items from context/known-issues.md (private cross-repo tracker). Each was re-verified against the current codebase — via grep, not taken on faith — immediately before fixing, since one originally-scoped item (F01-2, span leak on serializer error in bus.route()) turned out already fixed, and F04-2 (below) turned out already fixed too, only found because a search for its old, differently-worded issue text missed the actual _KNOWN_CONFIG_KEYS implementation on first pass.

ID	Priority	Issue	Resolution
FD-01	🔴 High	`MetricsCollector` not wired to real event sources — dashboard always showed 0 for message flow, restarts	`civitas.observability.metrics.MetricsSink` protocol added; injected via `ComponentSet`/`Runtime.set_metrics()`. Wires `message_handled` + `agent_error` in `AgentProcess._dispatch()`, `message_sent` in `send()`/`ask()`. `llm_call` is not auto-wired — `llm_span()` is a bare context manager with no interception point for `ModelResponse` token/cost data without a larger redesign of how `self.llm` is wrapped; that's a real follow-up, not silently claimed done here.
FD-03	🟡 Medium	`civitas/cli/dashboard.py` monkey-patched `runtime._root_supervisor._handle_crash` directly	`Supervisor.add_crash_callback()` + `Runtime.on_crash()` public hook added, invoked from `_handle_crash()` before the restart strategy runs. `cli/dashboard.py` monkeypatch removed.
F04-2	—	~~`Runtime.from_config` silently accepts unknown topology YAML keys~~	Already fixed — `Runtime._KNOWN_CONFIG_KEYS` + the `ConfigurationError` raise already existed in `from_config_dict()`. No code change; added the missing regression test only.
F01-3	🟡 Medium	`Message.ttl` declared and documented, never enforced	`ttl` field (optional float, seconds) added to `Message`; enforced in `Mailbox.get()` — expired messages are discarded with a warning and the search continues.
F11-5	🟡 Medium	`on_stop()` not called when `on_start()` raises	`AgentProcess._start()` now wraps `on_start()`; on failure, closes the open `civitas.agent.start` span with the error, sets `CRASHED`, runs `on_stop()` + MCP client cleanup, then re-raises. The pre-existing test asserting the old behavior (`on_stop` not called) was inverted to assert the new one.
FD-07	🟢 Low	`Worker` processes didn't receive exporters from topology YAML	Fixed together with FD-09 (same underlying gap — see below). `exporters` now flows through `Worker.__init__` and `cli/run.py`'s `_run_worker()`.
FD-09	🟢 Low	Two parallel OTEL span export paths coexist	Scope note: rather than the originally-sketched "write an `OTLPExportBackend` that converts `SpanData` to OTEL wire format," which is a genuinely large, failure-prone undertaking (hand-rolling OTLP span construction) for a bug-fix pass, the actual fix makes the two paths mutually exclusive: `Tracer.__init__` skips the direct `TracerProvider` entirely whenever a `span_queue` is supplied. `build_component_set()` now builds a `SpanQueue` + `FanOutBackend` only when `plugins.exporters` is configured in YAML, and `Runtime`/`Worker` own starting/stopping the `OTELAgent` task. This makes `plugins.exporters: [{type: console}]` actually work for the first time — previously dead code, per `civitas/plugins/loader.py`'s own docstring example (`type: otel`) which was never backed by a real implementation. The existing OTLP env-var auto-detect path (`OTEL_EXPORTER_OTLP_ENDPOINT`) is untouched and still uses the direct `TracerProvider` path when no `plugins.exporters` are configured — zero behavior change for that default case.

B — Durable suspension¶

Status: ✅ Done (design + implementation)

agent.suspend() / agent.resume() — integration point #8 in boundary.md's eight-point Civitas→Presidium contract, required for Presidium's human-in-the-loop (HITL) approval flow, where an agent must pause, durably persist enough state to resume later (possibly after a process restart), and resume when Presidium's policy engine or a human approves.

Design spec: docs/design/durable-suspension.md (FINAL DESIGN S1–S10). Delivered per that spec:

ProcessStatus.SUSPENDED re-introduced fully-wired (removed as dead API in F02-6), with every transition defined and tested — the governing constraint that F02-6 flagged.
Suspension pauses dispatch only (no coroutine snapshotting). suspend() is a non-blocking flag actioned at the message-loop boundary; while suspended only the priority queue is drained so business messages stay buffered (FIFO + backpressure preserved).
Durable marker persisted inside self.state (reserved key _civitas.suspended) via the existing checkpoint() path — no StateStore protocol change, so all contrib stores work unchanged. Durable only with a persistent store (same caveat as checkpoint()).
Write-ahead suspend (pause in-memory first, then persist; never falls back to RUNNING); approver-gated resume; marker cleared on permanent removal (despawn / restarts exhausted) but kept on graceful shutdown / crash-restart. Supervisor _stop() and restart strategies handle SUSPENDED.
Suspend/resume emit spans + AuditEvents (resume records the approver). A suspended agent does not count as crashed. ask() into a suspended agent times out (fail-fast deferred).

C — Doc hygiene (completed as part of scoping this release)¶

docs/milestones.md had drifted from boundary.md's repo-ownership split in four places, all corrected in this pass:

M3.4 MCP Integration — table claimed MCPClient/MCPTool/civitas[mcp] as civitas-core deliverables; they live in Fabrica (civitas-contrib). Corrected to show only the types + connect_mcp() lazy-import point as core.
Postgres StateStore + Migration — table claimed PostgresStateStore itself as civitas-core; it lives in civitas-contrib. Corrected to show only the protocol, loader resolution, and civitas state migrate CLI as core.
Infrastructure & Release — Framework adapters — claimed LangGraph/OpenAI adapters as civitas-core with CrewAI merely "planned"; civitas/adapters/ doesn't exist in this repo at all — all three (including CrewAI as a stub) live in civitas-contrib. Corrected.
Phase 5 — Prompt Library & Playground, Skills Gateway — framed as "Civitas-side features"; boundary.md assigns both to civitas-contrib. Corrected with explicit "Lives in" callouts matching how Fabrica's entry already read.

D — Explicitly deferred to a future version¶

Not python-civitas's job per boundary.md, and not touched in v0.5.0: Prompt Library & Playground, Skills Gateway, CrewAI adapter full implementation, MySQL StateStore, Fabrica. All belong to civitas-contrib or the separate civitas-forge repo — revisit there, not here.

v0.6.0 — Gateway Completion (Released)¶

Status: ✅ Released — 2026-07-04 (v0.6.0, PyPI)

The HTTP Gateway shipped in v0.4 with HTTP/1.1, HTTP/2, and HTTP/3 (QUIC), but several planned transport and middleware features were deferred across the gateway design docs (http-gateway.md Phases 3–4, gateway-api-surface.md). v0.6.0 completes the gateway as a coherent theme. A design refresh across those two docs should precede implementation (they were written pre-v0.4 and predate the shipped ASGI/middleware layer).

#	Deliverable	Priority	Source
G1	gRPC gateway — generic `civitas.Agent` service proxying any agent by name; `Invoke`/`Cast` unary RPCs with a `Struct` payload; committed `.proto` + `_pb2` stubs; health + server reflection; `civitas[grpc]` (grpcio default). `Stream` (server-streaming) deferred to G3; per-agent `proto_dir` loading is a non-goal	✅ Done	grpc-gateway.md
G2	WebSocket upgrade — long-lived bidirectional sessions: inbound frames `cast()` to an agent, agent streams back over the same socket (`ws_routes`)	✅ Done	gateway-streaming.md
G3	SSE / true streaming responses — `mode: "stream"` routes stream agent output as Server-Sent Events (replaces the `{"chunks": [...]}` workaround); also completes the gRPC `Stream` RPC deferred in G1	✅ Done	gateway-streaming.md
G4	Rate-limiting middleware — `RateLimiter` GenServer + `rate_limit` middleware (`civitas.gateway.ratelimit`)	✅ Done	gateway-api-surface.md
G5	Auth middleware — first-party API-key auth (`civitas.gateway.auth.require_api_key`, fail-closed, `CIVITAS_GATEWAY_API_KEY`); JWT (opt-in `civitas[jwt]`) + mTLS remain integration points	✅ Done (API-key; JWT/mTLS deferred)	gateway-api-surface.md
G6	File uploads — `multipart/form-data` parsed at the ASGI edge; files delivered base64 under `__files__`	✅ Done	gateway-api-surface.md
G7	HTTP/2 server push	⛔ Won't do — Server Push was removed from Chrome (2022) and is effectively dead across browsers; use SSE/WebSocket (G2/G3) for server-initiated data	http-gateway.md
G8	gRPC reflection service — generic reflection for the gRPC surface	✅ Done (shipped in G1)	grpc-gateway.md
G9	Evaluate quiche-python (Rust QUIC) as a drop-in for aioquic	✅ Done (evaluated) — no official/production-ready Python binding exists today; stay on aioquic, revisit when one matures	http-gateway.md

Status (2026-07-04): feature-complete. G1–G6 and G8 shipped; G7 dropped (HTTP/2 Server Push is dead in browsers — use G2/G3 instead); G9 evaluated (no viable Rust QUIC Python binding — stay on aioquic). v0.6.0 Gateway Completion is ready to release.

Non-goals for v0.6.0: business logic in the gateway, load balancing, request queuing — the gateway stays a thin translate-and-route edge (http-gateway.md Non-Goals). A bus-native streaming primitive (agent-to-agent stream() across all transports) is also out of scope for v0.6.0 — G2/G3 use gateway-mediated streaming; the first-class version shipped in v0.7.1 as R7 (see bus-native-streaming.md, #22).

v0.7.0 — Spawn Maturation & Gateway Auth (Released)¶

Released 2026-07-05. R1 (non-blocking spawn, #14), R2 (spawn_into, #16), R3 (JWT+mTLS auth + fail-open fix, #18), R4 (encrypted StateStore, #19), R5 (per-agent spawn quotas, #21), R6 (cross-process spawn, #20) all shipped. R7 (bus-native streaming, #15) deferred as a stretch item.

Status: ✅ Released — R1–R6 shipped in v0.7.0 (2026-07-05); R7 shipped in v0.7.1; R8 shipped in v0.7.2; R9 shipped in v0.7.3; v0.7.4 (2026-07-21) is a security patch: click ≥ 8.3.3 (PYSEC-2026-2132, transitive via typer, not exploitable in civitas — click.edit() is never called) + restored the Semgrep SARIF pipeline in the Security workflow (the action's generateSarif input was dropped upstream, silently disabling SAST uploads — fail-open, now CLI-invoked).

Theme: finish what v0.6.0 deferred. The largest coherent cluster is dynamic-spawn maturation (the #8 / #9 / #10 follow-ups + quotas + cross-process), plus completing gateway auth and one data-at-rest security item. This also lays groundwork for a possible future self-healing capability, which builds on supervision + dynamic spawn + telemetry (under investigation).

#	Deliverable	Priority	Source
R1	✅ Done (PR #14) — Non-blocking dynamic spawn: `spawn(wait=False)` / `spawn_nowait()`; `on_start()` runs in-task; failures via `on_child_terminated`. Design: `non-blocking-spawn.md` (Oracle + Momus reviewed).	🔴 High	GH #9 (from #8)
R2	✅ Done (PR #16) — `spawn_into(supervisor_name, …)` public cross-tree spawn helper	🟡 Medium	GH #10
R3	✅ Done (PR #18) — First-party JWT auth (opt-in `civitas[jwt]`) + mTLS client-cert auth + middleware fail-open fix	🟡 Medium	v0.6.0 §G5
R4	✅ Done (PR #19) — Encrypted `StateStore` at rest (`civitas[encryption]`)	🟡 Medium	design/security-hardening.md
R5	✅ Done (PR #21) — Per-agent spawn quotas (beyond the global `max_children`)	🟢 Low	design/dynamic-spawning.md Non-Goals
R6	✅ Done (PR #20) — Cross-process dynamic spawning (ZMQ / NATS)	🟡 Medium	design/dynamic-spawning.md Non-Goals
R7	Bus-native streaming primitive (`AgentProcess.stream()`, agent-to-agent, no transport change)	✅ Done (v0.7.1)	#22 · bus-native-streaming.md
R8	WS/gRPC gateway auth — JWT (`Sec-WebSocket-Protocol` subprotocol) and gRPC (`ServerInterceptor` + mTLS transport wiring, Health/Reflection carve-out) auto-inherit from the existing HTTP JWT/mTLS config; fail-closed startup validations for the new insecure-config combinations.	✅ Done (v0.7.2)	#17 · gateway-ws-grpc-auth.md
R9	HTTP mTLS via trusted reverse proxy — `require_client_cert` was always non-functional against uvicorn (never populates the ASGI TLS extension); new opt-in `mtls_source="proxy_header"` trusts an RFC 9440 `Client-Cert` header from a trusted-CIDR proxy instead, feeding the unchanged DN-allowlist authorizer. `direct` mode (default) is unchanged, still non-functional.	✅ Done (v0.7.3)	#25 · gateway-http-mtls-proxy.md
R10	HTTP mTLS in `direct` mode, for real — R9 explicitly left `direct` mode non-functional; this closes that other half. New `TlsAwareHttpToolsProtocol` (a thin `HttpToolsProtocol` subclass) reads the real peer certificate straight off the TLS transport (`ssl_object.getpeercert(binary_form=True)`, verified empirically before implementing) and populates the same ASGI extension shape `_client_cert_from_scope()` already expects — no change to the existing, correct DN-authorization logic. Surfaced again from `civitas-io/presidium`'s M7 work, which needs a genuinely self-hostable, single-process server (no mandatory reverse proxy). Verified against Presidium's own real mTLS test suite too, via a local editable install (not a committed dependency change). Released as v0.11.2 -- whose own release verification (a real fresh-venv install) immediately found `import civitas` failed entirely (a lazy-import discipline break, `_tls_protocol.py` pulling in `uvicorn` eagerly), fixed and re-released same-day as v0.11.3, live and verified on PyPI.	✅ Done (2026-08-23)	#25 (reopened, closed again) · gateway-http-mtls-direct.md
R11	`streamable_http` MCP transport — `MCPServerConfig.transport` gained a third real value, `"streamable_http"`, using the official `mcp` SDK's `streamable_http_client`. Real motivating case: most current remote MCP servers (and, confirmed separately, AgentGateway's own MCP proxy endpoint) expose a single `/mcp` endpoint over Streamable HTTP, not classic `sse`. The actual client construction lives in `civitas-io/fabrica`'s `MCPClient.connect()`, verified end to end against a real running Streamable HTTP MCP server, not mocked -- including a real anyio/asyncio cancellation interop finding along the way. Real performance benchmarks run on the homelab (AMD Ryzen 9 3900X): `streamable_http` p50 2.01ms/673 calls-s @ concurrency=10 vs. `stdio` 0.69ms/2356 and `sse` 1.32ms/991 -- checked against two independently-published industry benchmarks (TM Dev Lab, Stacklok/ToolHive), with an honest reconciliation of why direct ranking isn't valid (different network topology, different workload).	✅ Done (2026-08-24)	#26 (closed) · `civitas-io/fabrica`'s `SPIKE-mcp-transport-benchmark.md`

Suggested cut line: R1–R2 (spawn follow-ups) are the headline; R3 (auth) + R4 (encrypted store) are strong companions; R5–R9 are opportunistic and can slip to a later patch.

v0.8.0 — Supervision Core Hardening (Released)¶

Status: ✅ Released 2026-07-23. Scoped from the 2026-07-21 actor-model architecture review; shipped as five work packages on dev/v0.8.0, merged via PR #38. Closed #27–#35. Design: supervision-hardening.md. Theme: make the advertised OTP guarantee true — v0.1–v0.7 built the runtime outward (transports, spawning, gateway, security); v0.8.0 turns inward and fixes the supervision core those layers stand on. Prerequisite for the Medicus self-healing demo (restart-as-remediation needs trustworthy restarts).

#	Deliverable	Priority	Source
H1	✅ Done (dev/v0.8.0, `e758f72`) — Escalation restarts the escalated subtree under ONE_FOR_ONE; budget window cleared on supervisor restart (fresh incarnation rule)	🔴 P0	#28 · design D2
H2	✅ Done (dev/v0.8.0, `e758f72`) — Serialized, observable crash handling — per-supervisor crash queue (strictly sequential, stale-incarnation skip); restart failures logged at ERROR + escalated via the parent's queue; crash-drop window closed	🔴 P0	#30 · design D4
H3	✅ Done (dev/v0.8.0, `e758f72`) — Registration snapshot preserved across restart — `reregister_preserving()` at 3 Supervisor paths + Worker	🔴 P0	#29 · design D3
H4	✅ Done (dev/v0.8.0) — Priority heartbeats (stopgap) — `_agency.heartbeat` at `priority=1` (busy agents ack between messages; suspended agents ack while staying suspended); threshold breaches enqueue on the crash queue so restart backoff no longer stalls the monitor loop	🔴 P1	#31 · design D5
H5	✅ Done (dev/v0.8.0, staged (b) per §2.1) — Restart state reset: `self.state` reset before checkpoint restore (only checkpointed state survives — suspend marker S7 intact); ctor-capture groundwork (`__new__` records the child spec) for v0.9 fresh-instance restart; AGENTS.md contract finalized	🔴 P1	design D1 (A1) · xfail tests
H6	✅ Done (dev/v0.8.0) — Opt-in `handle_timeout` watchdog — a hung async `handle()` becomes a visible crash through the normal `on_error` path; YAML per-agent; span attr for hung-vs-buggy triage	🔴 P1	design D5 (A7)
H7	✅ Done (dev/v0.8.0) — `on_stop()` exception containment during normal shutdown — contained + ERROR-logged, agent reaches STOPPED, shutdown completes	🟡 P1	#27
H8	✅ Done (dev/v0.8.0) — `ErrorAction.RETRY` retries in place — FIFO preserved, fresh `handle_timeout` per attempt, STOP aborts mid-retry; `RETRY_AFTER` delay-lane idea recorded as deferred (design §5)	🟡 P2	#32 · design D8
H9	✅ Done (dev/v0.8.0) — `_runtime` sink (bare subscription): WARNING + drop for send, fail-fast error reply for ask; glob patterns exclude `_`-prefixed system names (C6 slice of H13)	🟡 P2	#33 · design D8
H10	✅ Done (dev/v0.8.0) — `LocalRegistry.register_b64` deleted (zero callers; keys live in `KeyRegistry` only)	🟡 P2	#34 · design D8
H11	✅ Done (dev/v0.8.0) — Truth sweep: README + getting-started + index + plugins (extras/imports → civitas-contrib); 5 orphaned guide pages + security docs added to site nav	🟡 P2	#35
H12	✅ Done (dev/v0.8.0) — "Delivery semantics & hazards" in messaging.md; restart-contract + escalation + handle_timeout + suspension sections in supervision.md; NEW `recipes.md` (when-to-use decision guide) + `agents-guide.md` + `llms.txt` (coding-agent-consumable docs)	🟡 P2	design D7 (A4/A5/A8/C8)
H13	Hygiene batch — monotonic clocks for windows/TTL, broadcast glob excludes `_agency.*`, bus accessor methods (encapsulation)	🟢 Stretch	design D8 (C2/C6/C7)

Exit criteria: all six strict-xfail tests in tests/unit/test_actor_model_gaps.py converted to plain passing tests (each Hn fix flips its test in the same PR); H2 adds a concurrent-crash stress test (N children crashing in the same tick under ONE_FOR_ALL → exactly one restart cycle).

Suggested cut line: H1–H4 (the P0/P1 correctness cluster) are the headline and must ship together; H5–H7 are strong companions; H8–H13 are opportunistic and can slip to v0.8.x patches.

Explicitly deferred to v0.9+: D6 (unify static Supervisor + DynamicSupervisor into one actor-based engine — needs its own design/plan cycle), D5-structural (per-process out-of-band liveness), C3 (in-process serialization fast path), B4 (DynSup wait=True head-of-line move).

v0.8.1 — Verification Perimeter (Released)¶

Status: ✅ Released 2026-07-24. Scoped from the 2026-07-23 full test/coverage review; shipped as V1–V7 on dev/v0.8.1, merged via PR #44. Closed #39–#43. Five shipped defects found (all invisible until something finally executed the code). Plan: .sisyphus/plans/verification-perimeter-v0.8.1.md (no design doc — test infrastructure, not runtime semantics). Theme: a gate that doesn't run is indistinguishable from a gate that passes — closes the fail-open verification class (3^rd instance: Semgrep SARIF, example Dockerfile, now the integration suite).

#	Deliverable	Priority	Source
V1	✅ Done (dev/v0.8.1) — Integration tests gate CI: required job, full tests/integration (~12 s); its FIRST run caught a real env bug (Rich help width). nats-server install deferred (guards skip)	🔴 P0	#39
V2	✅ Done (dev/v0.8.1) — 3 dead modules revived: contrib `importorskip` guards + core-only fixtures; collection 3 errors → 0; suite 157 passed / 23 honest skips	🔴 P0	#40
V3	✅ Done (dev/v0.8.1) — Root-caused + fixed cross-process spawn E2E: TWO ZMQ subscription-propagation races since R6 (announce outran child-topic propagation; per-request reply topics raced their own first use). Fixed via subscription-settle barrier before announce + stable per-transport reply prefix. 5/5 green on macOS AND Linux (docker-verified); design addendum in cross-process-spawn.md	🔴 P0	#41
V4	✅ Done (dev/v0.8.1) — omit-list audit: 10 stale entries deleted; ToolRegistry + model.py 100% (were omitted / 0%); loader 96%	🟡 P1	#42
V5	✅ Done (dev/v0.8.1) — 19 CliRunner tests; cli/* measured (except run.py live paths). CAUGHT: `civitas version` hardcoded '0.1.0' in every release since M3.1. Coverage 92→87.6% over ~900 newly-measured stmts (honest direction)	🟡 P1	#42
V6	✅ Done (dev/v0.8.1) — tested it, and the first test found that HTTP/3 had never worked: `StreamReset` imported from the wrong aioquic module → ImportError on first event (the #25 pattern). Fixed + first-ever QUIC loopback GET green; h3.py measured	🟡 P2	#43
V7	Hygiene: process/runtime miss-range top-ups, un-awaited-coroutine test warning	🟢 Stretch	review §F4

Sequencing: V3 → V2 → V1 (gate lands green) → V4 → V5 → V6 → release. Branch dev/v0.8.1, accumulating release PR (v0.8.0 model). Closes #39–#42 (+#43 if V6 tests; else docs-demote).

v0.8.2 — Hygiene (Released)¶

Status: ✅ Released 2026-07-24 (PR #45). Closed the last unwatched gate (13 NATS tests now run in CI) + v0.8.1 residue.

#	Deliverable	Priority
G1	✅ Done (dev/v0.8.2) — nats-server v2.14.3 pinned+sha256 in CI; 14/14 NATS tests pass (13 had never run anywhere); fixture port genuinely random now	🟡 Medium
G2	✅ Done (dev/v0.8.2) — init auto-splits paths; basename-only identifier validation; docs/cli.md updated; 4 new tests	🟢 Low
G3	✅ Done (dev/v0.8.2) — deploy 88%, topology 89%, dashboard 39% (args), state at its contrib-gated ceiling (49%, documented); total 89.7%	🟢 Low

v0.9.0 — Supervision Endgame (Released)¶

Status: ✅ Released 2026-07-24. The full close-out of the 2026-07-23 architecture review — the regression harness (tests/unit/test_actor_model_gaps.py) now has zero expected failures. Design: supervision-endgame.md — ✅ ACCEPTED (Q1–Q4 ratified). Plan: .sisyphus/plans/supervision-endgame-v0.9.0.md (E1–E5); E4 (the largest package) had its own dedicated plan with two explicit halt-checks — neither ever triggered, so all four packages shipped together as planned rather than splitting into v0.9.1.

#	Deliverable	Priority
E1	✅ Done — extracted `RestartEngine`, one restart-accounting engine shared by `Supervisor` and `DynamicSupervisor` (previously two divergent implementations). B3: backoff now derives from restart-window occupancy, decaying naturally once the window empties, not from a lifetime counter that never forgot	High
E2	✅ Done — D1a, fresh-instance restart: a restart now builds a NEW object from the original constructor call — flips the LAST strict-xfail tracker, closing finding A1. Behavior change: object references held across a restart go stale by design (route by name)	High
E3	✅ Done — D5, per-process liveness: supervisors probe a Worker's process-level health channel instead of pinging every agent's mailbox. The A6 false-positive (a busy-but-healthy remote agent force-restarted) is now a green end-to-end test over real ZMQ; dead-task detection dropped from a full starvation cycle to ~1 probe interval	High
E4	✅ Done — D6, supervisor actorization + B4: every `Supervisor` is now an addressable actor with its own mailbox (crash processing rides it, not a bespoke queue); new `civitas.supervision.status` introspection query; suspending a supervisor is hard-rejected (a paused subtree manager is a footgun); `DynamicSupervisor` `wait=True` spawns no longer block the supervisor's other traffic while a child's `on_start()` is slow	Medium

Sequencing: E1 → E2 → E3 → E4 (Phases A–D, each with its own verification pass) → E5 (this entry). Branch dev/v0.9.0. Two structural findings surfaced and were resolved before code landed in each case (documented as design-doc addenda, not retrofitted): D-E4-6 (Supervisor.stop() vs. an inherited method name collision) and D-E4-8 (stop()'s shutdown ordering had to reverse once crash-processing moved onto the supervisor's own loop, to preserve the pre-existing "no resurrection after stop" guarantee).

v0.9.1 — Post-endgame Polish (Released)¶

Status: ✅ Released 2026-07-28. Coverage top-ups, and a full Textual TUI rebuild of civitas dashboard ("civitas top") that attaches to an already-running topology over HTTP instead of spawning its own runtime. Design: dashboard-v2.md — ✅ ACCEPTED, fully implemented. Plan: .sisyphus/plans/dashboard-v2.md (Phases A–G).

#	Deliverable	Priority
—	✅ Done — `process.py` (88%→92%) / `runtime.py` (87%→91%) coverage top-ups: 22 new tests covering `llm_span()`/`tool_span()`'s tracer-present path, `connect_mcp()`'s error paths, `spawn_into()`'s validation/error paths, suspend/resume/despawn checkpoint-failure branches, and message-signing wiring	Low
A–D	✅ Done — `TopologyServer` enrichment: `restart_count`, `crashes_in_window`, `capabilities`, `uptime_seconds`, `process_id`; new `GET /metrics` (auto-provisioned `MetricsCollector`) and `GET /processes` (psutil resource stats via the existing D5 health-probe wire protocol) endpoints; closed FD-01 (`llm_span()` now always feeds metrics, independent of tracing)	Medium
E–F	✅ Done — the Textual app itself (Mockup B's dense three-pane grid: tree \| detail \| resources), chosen after building and comparing two real runnable mockups; `civitas dashboard <topology.yaml>` rewritten to YAML-driven remote-attach only (breaking CLI change, documented); new `civitas[dashboard]` extra	Medium
G	✅ Done — verification sweep (1373/1373 unit+integration, macOS + Linux Docker), `docs/cli.md` rewrite, CHANGELOG entry, runnable demo at `examples/dashboard_demo/`	—

Found along the way (not planned, surfaced by actually running things): two dead metrics hooks (llm_call()/FD-01, agent_restarted()) that existed and were unit-tested but were never actually called from civitas/; TopologyServer has zero authentication today (acceptable for read-only endpoints, tracked as the v0.9.2 prerequisite gate for any write action); three silently broken API calls in examples/dynamic_spawning.py, fixed, exposing that no example file in this repo has any test coverage (tracked, v0.9.2). Control-plane items (auth, suspend/resume-as-write, kill/restart, mailbox introspection) were deliberately scoped OUT after a capability-scope discussion, to be designed properly rather than added reactively — see v0.9.5 below (the original single "v0.9.2" grab-bag was later split into v0.9.2–v0.9.5, 2026-07-28).

v0.9.2 — Examples Completeness (Released)¶

Status: ✅ Released 2026-07-28. A smoke test proving every example actually runs, 8 new examples for real, previously-undemonstrated features, and two real product bugs found and tracked (not papered over) along the way. This release is "Cluster 3 minus the CI matrix" from the 2026-07-28 roadmap split — the original single "v0.9.2" backlog written right after v0.9.1 shipped was four unrelated kinds of work, separated into v0.9.2–v0.9.5, each its own coherently-scoped release.

#	Deliverable	Priority
—	✅ Done — examples smoke test (`tests/integration/test_examples_smoke.py`): 30 tests across three shapes (run-to-completion, long-running-then-signaled, paired long-running processes), plus a self-checking test so a future example can't silently ship untracked	High
—	✅ Done — 8 new examples: `non_blocking_spawn.py`, `supervision_introspection.py`, `custom_plugin.py`, `streaming_response.py`, `secured_messaging.py`, `grpc_gateway.py`, `gateway_auth.py`, `cross_process_spawn/` — every one verified running end-to-end on macOS and Linux (Docker), not just written and assumed correct	Medium
—	✅ Done — `examples/README.md`, a full index of every example (existing + new), what each demonstrates, and how to run the smoke test; linked from the top-level `README.md`	—

Found along the way (not planned, surfaced by actually running things), each real enough to get its own tracked entry below rather than a quiet inline fix: three more silently-broken example API calls (stateful_workflow.py's wrong SQLiteStateStore import path, level2_multi_process/run_worker.py's nonexistent Worker.from_config(), both frameworks/*.py examples' wrong civitas.adapters.* import path — all fixed); and two genuine, previously- unexercised product bugs, NOT example bugs, each needing its own future investigation: Runtime.from_config()/civitas run --topology doesn't filter process:-tagged nodes (builds every node locally regardless, duplicating whatever a real Worker process builds for itself); and message signing over a real ZMQ transport silently times out an agent-to-agent ask() round trip even with allow_unsigned=True set, with no existing test ever having exercised signing over a real transport end-to-end. Both detailed, root-caused, and fixed in v0.9.2.1 below.

v0.9.2.1 — Bugfix Release (Released)¶

Status: ✅ Released 2026-07-28. Both real product bugs found building v0.9.2, fully root-caused and fixed, with real regression tests added for the exact gap that let each ship silently.

#	Deliverable	Priority
—	✅ Done — message signing + ZMQ/NATS transport fix: new `Transport.set_serializer()`, called from `Runtime.start()`'s signing-wiring, so the transport's own private serializer reference (used by `request()`'s internal reply_to round-trip) gets swapped to the signing one too, not just the Runtime's and the Bus's. New `tests/integration/test_signed_transport.py` proves a real signed `ask()` completes over both real ZMQ and real NATS	High
—	✅ Done — `process:`-tag filtering fix: new `process_filter` keyword on `Runtime.from_config()`/`from_config_dict()` (`"*"` default = build everything, unchanged; `None` = untagged nodes only; a named string = that process's nodes only, matching `Worker`'s own filtering). `civitas/cli/run.py`'s supervisor role now uses `process_filter=None`. New `TestProcessFilter` test class in `tests/unit/test_runtime.py` (5 tests, including nested-supervisor transparency and `dynamic_supervisor`'s node-level tag shape)	High
—	✅ Done — `examples/secured_messaging.py` gained a real, live, signed `ask()` demo (Part 3) now that it actually works; `examples/deployment/level2_multi_process/run_supervisor.py` updated to use `process_filter=None`, matching the fix	—

Both bugs were found via direct instrumentation of the actual wire bytes and CLI behavior, not guessed — see the v0.9.2 entry above for how they originally surfaced, and civitas/transport/__init__.py's Transport.set_serializer docstring for the full signing root-cause writeup.

v0.9.3 — Telemetry: OTEL Linkage, Prometheus, Grafana, Native Storage, Query Layer, TUI (Released)¶

Status: ✅ Released 2026-07-29, as a single release (built and verified as six sequential capabilities — Track A: A1-A3, Track B: B1-B3 — each with its own real, live verification, then shipped together as one v0.9.3). A design conversation first scoped "telemetry dashboard" (the original grab-bag item) into these six small, sequential capabilities plus one deferred item (B4, Part 2 below) — see each subsection for the full finding and fix.

A1 — OTEL trace linkage¶

Scoped as a verification exercise ("does trace continuity survive a network hop"); live verification found something more fundamental — confirmed via direct instrumentation, not assumed from reading code, that OTEL spans had never linked to each other at all, even within a single process.

#	Deliverable	Priority
—	✅ Done — Root cause: `Tracer._make_span()` called `self._otel_tracer.start_span(name, attributes=...)` with no `context=` parameter at all — every real OTEL span became its own isolated root trace with a random OTEL-assigned trace_id and `parent_id: null`, regardless of civitas's own correct `trace_id`/`span_id`/`parent_span_id` bookkeeping on `Span`/`Message`. A real Jaeger/Grafana/Datadog view (`docs/observability.md` Mode 3) would have shown every `send`/`recv`/`llm.chat`/`tool.execute`/etc. span as a disconnected single-span "trace" — no tree, no request-flow view, the one thing distributed tracing exists to do	High
—	✅ Done — Fix: a new `_otel_parent_context()` helper builds a real (non-recording, "remote") OTEL `SpanContext` from civitas's own `trace_id`/`parent_span_id` — the standard extracted-context pattern every OTEL propagator uses — and `_make_span()` passes it as `context=` to `start_span()`. Since OTEL mints its own span_id/trace_id and there's no public API to force a specific one, OTEL's real assigned ID is made authoritative for civitas's own `Span.trace_id`/`span_id` too, and `MessageBus.route()`/`request()` (`civitas/bus.py`) copy that back onto the outgoing `Message` before it hits the wire — so a downstream hop's `handle_span`/`recv_span` parents to a span OTEL actually emitted, not a dangling made-up ID, across process/transport boundaries too	High
—	✅ Done — Verified with a REAL 2-OS-process ZMQ round trip (not a mock): `examples/deployment/level2_multi_process`'s `frontend`/`worker_a`/`worker_b` driven by an ad-hoc driver agent, OTEL console-exported spans from both processes captured and cross-checked — every span belonging to the actual message flow shares one `trace_id` across both processes with correct parent-child links end-to-end (confirmed `worker_a`'s `recv` span correctly parents to `frontend`'s real `send` span's OTEL-assigned ID); `civitas.agent.start`/`stop` lifecycle spans correctly remain standalone roots (no causal parent, as expected)	—
—	✅ Done — Regression tests: `tests/unit/test_observability.py` (parent-context linkage, root-span shape, malformed-ID fallback never raises, OTEL-authoritative ID overwrite) and `tests/unit/test_bus.py` (`route()` really mutates the outgoing `Message` before serialization, and what was sent is what arrived) — pinning the mechanism at unit-test speed so a regression doesn't need a live 2-process repro to catch	—
—	✅ Done — `docs/observability.md`'s existing (previously aspirational, now finally true) claim about Jaeger showing "a single distributed trace per request... linked by parent-child relationships" now carries a transparency note pointing at this fix	—

Known, deliberately-accepted limitation (documented in _otel_parent_context()'s docstring, not worked around): a caller-supplied non-empty trace_id with NO parent_span_id is discarded in favor of a fresh OTEL-minted one when OTEL is active, since OTEL's own SpanContext model has no way to express "root span, but honor this specific trace_id" via the public API. No real call site in this codebase hits this today — every caller derives trace_id and parent_span_id together, from the same message/span.

A2 — Prometheus metrics¶

Real Prometheus text-format metrics exposition at the standard /metrics scrape path.

#	Deliverable	Priority
—	✅ Done — `GET /metrics` on `TopologyServer` now serves real, hand-rolled Prometheus text-format exposition (`civitas/observability/prometheus_export.py`) at the standard scrape path — no `metrics_path` override needed in a Prometheus `scrape_configs` entry. Hand-rolled deliberately over the `prometheus_client` library after weighing the trade-off: the data shape only ever needs counters and gauges (no real histograms/summaries — `AgentMetrics` tracks a running sum + count, not buckets, so `_sum`/`_count` pairs are exposed honestly rather than faking the special histogram/summary quantile machinery), keeping full spec correctness achievable by hand (proper label-value escaping, `+Inf`/`-Inf`/`NaN` float formatting) without a new dependency/extras group	High
—	✅ Done — Breaking change, deliberate: civitas's own JSON metrics snapshot (used internally by `civitas top`) moved from `/metrics` to `GET /snapshot` — "never wise to break standards in OSS projects" (2026-07-29 decision). Updated everywhere: `civitas/dashboard/app.py`'s polling client, `docs/cli.md`, `docs/observability.md` (new "Prometheus metrics" section with the full metric reference table and a Grafana recipe), and a `docs/design/dashboard-v2.md` addendum (§13) recording the deviation from that design doc's original §3.2 spec	High
—	✅ Done — Metrics exposed: `civitas_messages_handled_total`/`_sent_total`, `civitas_message_latency_ms_sum`/`_count`, `civitas_agent_errors_total`/`_restarts_total`, `civitas_llm_tokens_in_total`/`_out_total`/`_cost_usd_total` (the actual cost-tracking value proposition — only emitted for agents that have actually made an LLM call, mirroring `MetricsCollector.llm_call()`'s own established FD-01 discipline), `civitas_agent_status` (enum-pattern gauge), `civitas_runtime_uptime_seconds`. Deliberately drops `total_messages`/`total_cost_usd` (redundant — Prometheus's own `sum()` over the per-agent series gives the same number)	—
—	✅ Done — Unplanned second finding, fixed same-session: live verification (a real Prometheus server actually scraping the endpoint, not just eyeballed output) surfaced that `MetricsCollector.agent_status_changed()` — present since v0.9.1 — had never been called from anywhere in the runtime; a plainly-running agent's exposed status came back `"unknown"` forever (the JSON `/snapshot` endpoint never even exposed `.status` at all, so nothing had surfaced this until the new Prometheus gauge did). Fixed by routing every `AgentProcess` status transition through one new choke point (`_set_status()` in `civitas/process.py`, replacing ~10 direct assignment sites) — guarded via `getattr` so a user-supplied custom `MetricsSink` implementing only the required Protocol methods (`agent_status_changed` was never actually part of that Protocol) keeps working unchanged, confirmed by a real hand-rolled-fake regression test	High
—	✅ Done — Verified against a REAL local Prometheus server (not mocked): installed via `brew install prometheus`, pointed a real `scrape_configs` target at a live civitas `TopologyServer` with zero `metrics_path` override, confirmed `"health": "up"` via `/api/v1/targets`, and confirmed real PromQL queries (`civitas_messages_handled_total`, `civitas_agent_status`) return correct live values via `/api/v1/query`	—
—	✅ Done — New `tests/unit/test_prometheus_export.py` (16 tests: escaping, float formatting, per-family HELP/TYPE lines, LLM-series suppression for non-LLM agents, well-formed-line structural check) plus a new end-to-end HTTP-level test in `tests/unit/test_topology_server.py` and two new tests in `tests/unit/test_process.py` proving the status-wiring fix (a real `MetricsCollector` end-to-end, and a hand-rolled fake WITHOUT `agent_status_changed` proving it never crashes)	—

A3 — Grafana stack, completing Track A¶

#	Deliverable	Priority
—	✅ Done — Scope correction from the original backlog wording: "example OTel-collector config" wasn't actually applicable — civitas's `/metrics` (A2) is scraped directly by Prometheus (pull-based); an OTel Collector is only relevant to the separate trace/OTLP push path already documented in `docs/observability.md`'s Mode 3. A fully-provisioned Prometheus + Grafana `docker-compose` stack is the more directly useful, actually-runnable deliverable for the metrics side — shipped instead	—
—	✅ Done — `examples/observability/grafana/`: a `docker-compose.yml` bringing up Prometheus (scraping civitas's standard `/metrics`) and Grafana, both fully provisioned via Grafana's own datasource/dashboard provisioning mechanism — zero manual clicking after `docker compose up`. Dashboard (`provisioning/dashboards/civitas.json`, a standard Grafana export) has 8 panels: message throughput, error rate, LLM cost over time (per agent/model — the actual cost-tracking value proposition), average latency (honest sum/count division, not a fabricated histogram), agent status table, and total-spend/restarts/uptime stat panels	High
—	✅ Done — Verified fully end-to-end, not just JSON-schema-validated: ran `examples/dashboard_demo/` (already-existing, already generates realistic cost/latency/restart/error data via `ChattyWorker`/`FlakyWorker`) as the real scrape target, brought up the real `docker compose` stack, confirmed via Prometheus's own `/api/v1/targets` that the scrape target reports `"health": "up"`, confirmed via Grafana's `/api/datasources` and `/api/search` that both the datasource and dashboard auto-provisioned correctly, and confirmed via a live PromQL query that real non-zero cost data (e.g. `chatty` agent accumulating real `$0.249` over the run) flows all the way through	—
—	✅ Done — `docs/observability.md`'s "Prometheus metrics" section and `examples/README.md`'s index both updated to point at the new example; the example's own `README.md` documents the two-terminal quick start, a full panel/query reference table, how to point it at your own app instead of the demo, and how to import the dashboard JSON into an existing Grafana instance	—

Track A is fully shipped as part of this release.

B1 — Native telemetry storage¶

A civitas-native persistent span store for small/local deployments. Design-first: full design conversation and decision log in docs/design/telemetry-native.md before any code.

#	Deliverable	Priority
—	✅ Done — `civitas/observability/sqlite_backend.py`'s `SQLiteBackend` — a real `ExportBackend` implementation (no protocol changes; plugs into the already-existing `SpanQueue → OTELAgent → ExportBackend` path, composable with other exporters via the already-existing `FanOutBackend`). One SQLite file per fixed-size time window (`window_days`, default 30) rather than one growing file with row-level deletes — retention removes whole files, not rows. Hot fields (`agent_name`, `llm_model`, `llm_tokens_in`/`_out`, `llm_cost_usd`) promoted to real, indexed SQL columns for fast `GROUP BY`/`SUM()` aggregation, while the full `attributes` dict is also kept (`attributes_json`) for drill-down. New `civitas[telemetry]` extras group (`aiosqlite`)	High
—	✅ Done, unplanned root-cause fix found live — while writing the attribute-normalization logic, discovered that `AgentProcess.llm_span()`'s spans (`civitas.llm.chat` — the actually-used, ergonomic API real agent code calls, confirmed via `examples/dashboard_demo/agents.py`) had never carried any agent identity at all, in either the OTEL/Jaeger export path (A1) or this new storage backend. Confirmed by directly inspecting a real span's attributes, not assumed. Fixed at the root in `civitas/process.py` — `civitas.agent.name` added to that span's attributes. The separate, lower-level `Tracer.start_llm_span()` API (no `AgentProcess`/agent context available to it architecturally) was left as-is, documented as a real but different-in-kind limitation	High
—	✅ Done — Verified with a REAL `Runtime` running real agents (`exporters=[SQLiteBackend(...)]`), confirmed by directly querying the actual `.db` file with a fresh `aiosqlite` connection — not mocked. Confirmed the LLM cost-tracking value proposition specifically: a real `civitas.llm.chat` span's cost/tokens/model land correctly in the promoted SQL columns, queryable directly	—
—	✅ Done — New `tests/unit/test_sqlite_backend.py` (24 tests: every normalization case across both LLM span shapes plus the deliberate `NULL`-on-no-match fallback, window-index/filename round-tripping, retention sweep including the edge case of a span written directly into an already-expired window) and `tests/integration/test_sqlite_backend_integration.py` (real `Runtime`, real file, real query)	—
—	✅ Done — `docs/observability.md` gained a "Native SQLite storage" section, and a real documentation gap was fixed alongside it: the existing "LLM spans" attribute reference had only ever documented ONE of the two real LLM span shapes (`Tracer.start_llm_span()`'s), never `AgentProcess.llm_span()`'s (`civitas.llm.chat`) — now both are documented, distinctly	—
—	Deliberately deferred, not neglected — multi-process aggregation (design doc §7): each OS process would produce its own separate file set; a concrete future answer (reuse civitas's own message bus, following the `_agency.health_probe` precedent) is sketched, not left as "TBD"	—

B2 — Telemetry query/aggregation layer¶

A query/aggregation layer over B1's SQLite store. Design conversation (not a full standalone design doc — more mechanical than B1's genuine new architectural decision) captured in docs/design/telemetry-native.md's §13.

#	Deliverable	Priority
—	✅ Done — `civitas/observability/sqlite_query.py`'s `SQLiteQueryEngine` — a pure, read-only query API deliberately decoupled from any UI/CLI (B3's job, still undecided). Four methods shipped: `cost_over_time`, `message_rate_over_time` (bucketed by time, `bucket_seconds` caller-chosen), `cost_by_agent`, `cost_by_model` (whole-range totals). Real dataclasses (`CostBucket`, `MessageRateBucket`) for results, matching the `SpanData`/`RuntimeSnapshot` precedent — not raw tuples	High
—	✅ Done — Cross-window queries (a time range spanning more than one of B1's window files) use SQLite's native `ATTACH DATABASE` — confirmed working through `aiosqlite` directly (including parameter-bound `ATTACH DATABASE ?`, not just a literal path) — one real SQL query across N attached files, not N round trips merged in Python, exactly as the design doc's §3 anticipated and deferred to B2	High
—	✅ Done — Handles the trickiest real case correctly: a single time bucket whose spans landed in two different window files. A double `GROUP BY` (once per attached window file, once in an outer re-aggregation over the `UNION ALL`) merges these into one correct row instead of two separate per-file ones — verified with a real, carefully-constructed test (a window boundary that does NOT coincide with the chosen bucket boundary, confirmed empirically before writing the test, not assumed)	High
—	✅ Done — Small shared refactor: `SQLiteBackend`'s private `_index_from_filename` promoted to a module-level `index_from_filename()` function (alongside the existing `window_index`/`window_filename`) so `SQLiteQueryEngine` doesn't need to reach into `SQLiteBackend`'s internals	—
—	✅ Done — Verified against a REAL `Runtime` running real agents (`exporters=[SQLiteBackend(...)]`), queried by a real `SQLiteQueryEngine` — not synthetic `SpanData`. New `tests/unit/test_sqlite_query.py` (9 tests) and `tests/integration/test_sqlite_query_integration.py`	—
—	Explicit decision: ship 4 methods now, evaluate more later. Design doc §13 records 6 candidate query methods considered but not built (latency percentiles, error rate over time, restart/crash timeline, trace/span drill-down, top-N queries, model-comparison-over-time) — tracked so the list isn't lost, not a commitment to build all of them	—

B3 — Telemetry TUI, completing Track B¶

The Textual TUI over B1/B2's native SQLite store. Full design + decisions in docs/design/telemetry-native.md's §14.

#	Deliverable	Priority
—	✅ Done — Confirmed empirically BEFORE committing to the approach, not assumed: real charts genuinely render inside a Textual app via `textual-plotext` (a real, installable, actively-maintained package) — verified with an actual headless render (`app.export_screenshot()`) showing a correctly-axis-labeled line chart before any of `civitas telemetry`'s own code was written	—
—	✅ Done — `civitas telemetry <db-dir>` (`civitas/dashboard/telemetry_app.py`) — a NEW, separate Textual app from `civitas top` (different attach model: reads a local SQLite directory directly, no live process required, unlike `civitas top`'s live HTTP attach). Reuses `civitas top`'s palette and `@work`-based periodic-poll-worker pattern, adapted to re-query SQLite instead of polling HTTP	High
—	✅ Done — Panels: `CostChart`/`MessageRateChart` (real line charts, capped at the top 6 series by total value — a real multi-agent/multi-model deployment's cardinality would make a terminal legend unreadable well before that), `StatPanel` (total spend/messages/top-agent), `CostBreakdownTable` (per-agent + per-model), `TimeRangeBar`	High
—	✅ Done — Time range: both a `--since` launch flag (duration shorthand OR absolute ISO datetime) AND interactive in-TUI switching (h/d/w/m preset keys + `r` for immediate manual refresh), per explicit direction. Periodic refresh (`--refresh`, default 30s) shipped for v1 — reusing `civitas top`'s own polling precedent turned out not to be the hard path originally hedged as a fallback-to-one-shot option	High
—	✅ Done — New `civitas[telemetry]` dependency: `textual-plotext`, folded into the existing extra (not a separate one) — the TUI is meaningless without the SQLite store it reads from anyway	—
—	✅ Done — Verified end-to-end against REAL data: a real `Runtime` + `SQLiteBackend` writing live while a real headless Textual pilot drove the actual TUI, confirmed correct totals/charts/keybinding behavior via rendered screenshot text extraction, not just "it didn't crash." New `tests/unit/test_telemetry_time.py` (14 tests), `tests/unit/test_telemetry_widgets.py` (10 tests), `tests/integration/test_telemetry_app.py` (5 tests, including a real live-running-Runtime scenario and a genuine "data outside the query range" case)	—
—	Deferred, tracked (not built now, see v0.9.3.7–9 in Part 2) — log/event viewer (needs a new B2 query method), live tick chart animation, scrollable/paginated breakdown table for larger deployments	—

Track B's originally-scoped work (B1-B3) is fully shipped as part of this release. B4 (placement + pluggable-backend refactor) remains explicitly deferred — see Part 2 below.

Also included in this release: a full documentation pass across the whole arc (README.md's CLI list and observability section, docs/index.md, examples/README.md, examples/dashboard_demo/) — found and fixed real gaps (the CLI list was missing both civitas dashboard and civitas telemetry; the observability section only told the original OTLP/Jaeger story). examples/dashboard_demo/topology.yaml now also feeds a SQLiteBackend exporter — one demo topology serves civitas top, Grafana, and civitas telemetry from the same live run, verified end-to-end including dynamically-spawned children's cost data.

v0.9.4 — Dashboard TUI Polish (Released)¶

Status: ✅ Released 2026-07-30, as a single release (five small, self-contained additions to the existing live civitas top TUI — no auth needed, no new design surface — built and verified one item at a time, then shipped together as this one release, matching the v0.9.3 precedent).

Also closed by this walkthrough, not previously cross-referenced: dashboard-v2.md's original P1 item "historical charts/sparklines (message-rate, cost-over-time)" is exactly what v0.9.3's B2 (SQLiteQueryEngine.cost_over_time/message_rate_over_time) and B3 (civitas telemetry's real charts) shipped — a different app (civitas telemetry, not civitas top) than originally envisioned, but the same underlying need. Considered done, not tracked separately.

Item	Priority	Source	Status
Dashboard layout: optional focus/expand mode for the detail pane (Mockup A's wide-detail idea)	Low	design/dashboard-v2.md §7.0 (§14 addendum) — Mockup B (dense three-pane grid) shipped as the default in v0.9.1; Mockup A's core idea kept as an opt-in mode, not discarded	✅ Done — a dedicated `f` keybinding (not Enter — confirmed `Tree.NodeSelected` can't distinguish click from Enter), widens `AgentDetailPanel` 1fr→3fr at the expense of the other two panes (which shrink but stay visible, never hidden). Verified via real measured widget widths in a headless Textual test, not just the CSS flag
Dashboard: multi-cluster / multi-topology view	Low	design/dashboard-v2.md P2 (§15 addendum)	✅ Done — `civitas dashboard` accepts multiple topology files, each attached to and polled concurrently, switchable via tabs, all already-live in the background (instant switching, no fetch-on-demand delay). Required extracting the per-cluster view+poll-worker logic into a new, independently reusable `ClusterView` widget. Found and fixed a real, live-discovered gap along the way: `TabbedContent`'s own tab-selector bar grabs default keyboard focus, not any tab's content — confirmed by inspecting `app.focused` directly, not assumed — which would have silently broken the "f" focus-toggle binding (item above) in multi-cluster mode. Verified against two REAL, concurrently-running `Runtime`+`TopologyServer` processes with genuinely distinct agent sets, confirming real cross-cluster data isolation
Dashboard: "session length" (LLM conversation turns/duration)	Low	design/dashboard-v2.md P1 (§16 addendum)	✅ Done — defined as THIS INCARNATION's LLM engagement (`AgentProcess.session_turn_count`/`session_duration_seconds`, reset on restart, matching `uptime_seconds`'s own precedent exactly) rather than a genuine cross-restart concept — that bigger idea is real and tracked separately, not folded in here (see "Tracked idea — explicit cross-restart `session_id` concept" below). Only counts real LLM usage (FD-01's discipline); exposed via `/topology` like `uptime_seconds`, not routed through `MetricsCollector`. Verified end-to-end against a real running `dashboard_demo` agent, not just unit-tested
Dashboard: network I/O per process	Low	design/dashboard-v2.md P1 (§17 addendum)	❌ Investigated, declined — a genuinely different category from "blocked": there's nothing to wait for, the underlying capability doesn't exist affordably on any platform. Confirmed empirically, not assumed: Linux's `/proc/<pid>/net/dev` is byte-for-byte identical to system-wide `/proc/net/dev` (per-namespace, not per-process, for the normal non-containerized case) — real attribution needs eBPF/root. macOS's `nettop -P -L 1 -x` DOES work (verified with real output) but is an undocumented, private CSV format with no stability contract. Windows has no clean per-process network counter in the standard taxonomy either. Every path needs root, heuristic packet-capture attribution, or fragile undocumented-CLI-parsing — none fit this project's lean-dependency philosophy. Not built, not planned
Dashboard: distinct HITL-wait vs. governance-suspend visual signal	Low	design/dashboard-v2.md §6 option B (§18 addendum)	✅ Done — the cross-repo blocker (Presidium owns the policy that would populate a meaningful `reason`) was broken by a civitas-side API, not an invented heuristic: `AgentProcess.suspend()`/`Runtime.suspend()`/the `_agency.suspend` wire payload all gained an additive, backward-compatible `category: SuspendCategory` parameter (`HITL_APPROVAL`/`GOVERNANCE_PAUSE`/`OTHER`, default `OTHER`), persisted in the same durable suspend marker, plus a `suspend_for_approval()` convenience wrapper. Rendered as a distinct blue (not the originally-suggested cyan — already reserved as `TOPOLOGY_ACCENT`) in the tree/detail panel, only for `hitl_approval`. `examples/dashboard_demo/` gained a real `ApprovalWorker` agent; verified end-to-end against it — a real running `/topology` endpoint returning `suspend_category: hitl_approval`, and a real headless Textual pilot + exported SVG confirming the actual rendered blue ink (`#0000ff`), not just markup text

v0.9.5 — Control-Plane Auth Foundation: TopologyServer/HTTPGateway Merge (Released)¶

Status: ✅ Released 2026-07-30, as a single release. The prerequisite gate for the v0.9.5 AuthN/AuthZ backlog (below, now renumbered v0.9.6): TopologyServer had zero authentication and existed as a separate, hand-rolled asyncio.start_server HTTP server that reinvented what HTTPGateway already does with an audited AuthN stack. Rather than bolt a second auth implementation onto it, the two were merged — TopologyServer is now an ordinary agent behind HTTPGateway, inheriting its API-key/JWT/mTLS auth. Design-first: full investigation, four reviewed decisions, and a phased migration plan in docs/design/topology-gateway-merge.md.

Breaking change (deliberate, D6): the TopologyServer class is removed — no longer importable from civitas. YAML users are unaffected (type: topology_server nodes keep working, now building a TopologyAgent + HTTPGateway sub-supervisor under the hood). Direct-construction (non-YAML) Python callers migrate to constructing an HTTPGateway(GatewayConfig(topology_agent=...)) + TopologyAgent themselves.

#	Deliverable	Verified
1	`GatewayResponse` raw-body escape hatch (route-scoped, for Prometheus `/metrics`' plain text) — plus a real pre-existing bug fixed in passing: `traceparent` was dropped on non-streaming responses	unit
2	`TopologyAgent(GenServer)` over a shared `_TopologyIntrospection` base — same privileged `Runtime` injection, reached via `handle_call()` instead of a socket	byte-for-byte builder-parity tests
3	`HTTPGateway` auto-registers the seven fixed introspection routes (`topology_agent`/`topology_prefix`/`topology_middleware`), `/health` auth-free by default	unit + ASGI end-to-end
4	`type: topology_server` builds a `TopologyAgent` + `HTTPGateway` dedicated sub-supervisor (Option A, addendum D3a) — zero YAML migration for the common case	real `dashboard_demo` run (all endpoints, Prometheus `/metrics`, bare-array `/agents`, dashboard client)
5	The actual payoff: introspection endpoints inherit real auth — `/topology`/`/metrics` gated (401/200), `/health` exempt, no-auth-block stays wide-open	real Runtime, real API-key auth
6	`TopologyServer` class removed; ~35 test call-sites migrated to `TopologyAgent`/the real gateway	full suite

Deferred, tracked: attach_to (mount introspection routes onto an existing http_gateway rather than a dedicated internal one) — D6c, not built; and the dashboard client learning to send auth headers so civitas dashboard/top can attach to an auth-protected endpoint end-to-end — D7, necessary follow-on for auth to be usable from the TUI, tracked under v0.9.6 below.

v0.9.6 — Dashboard Control-Plane Write Actions (Released)¶

Status: ✅ Released 2026-07-31, as a single release.

The auth prerequisite shipped as v0.9.5 (the TopologyServer/HTTPGateway merge, see Part 1). Every item below built directly on the merged, auth-capable introspection gateway, each as an auto-registered route carrying the node's auth middleware, with the authenticated principal recorded as the audit actor (control-plane-writes.md D2 — the honest binding). civitas ships the seam + honest audit + safe localhost default, never AuthZ (customers bring their own SCIM/IdP/OPA middleware; single devs bring nothing). Full design + rationale in docs/design/control-plane-writes.md.

Item	Priority	Status
AuthNZ integration seam + honest audit binding + safe default	High	✅ Done — middleware sets `request.auth["principal"]={"id":...}` (bring-your-own-auth); the authenticated principal flows into every write action's `AuditEvent` under a reserved, unspoofable `__principal__` key; no auth → `{"id":"unauthenticated"}` + localhost bind as the control, plus a loud non-localhost-no-auth warning. Reference example `examples/control_plane_auth.py`. Verified: `initiated_by="alice"` recorded even when the body smuggles `"attacker"`
Dashboard/API: suspend/resume an agent	Medium	✅ Done — `POST /agents/{name}/suspend`\|`/resume`; principal → `initiated_by`/`approver` in the audit
Dashboard client sends auth headers (D7)	High	✅ Done — `fetch_json`/`civitas dashboard`/`topology show` gained `--header 'Name: Value'` (scheme-agnostic) so the TUI can attach to an auth-protected endpoint. Verified 401 without / 200 with
Dashboard/API: kill / force-restart an agent	Low	✅ Done — `POST /agents/{name}/restart`; the OTP "let it crash" (`_agency.force_restart` → agent raises → supervisor restarts per its existing policy), audited `agent.force_restart`
Mailbox introspection (peek)	Low	✅ Done — `GET /agents/{name}/mailbox`; new non-destructive `Mailbox.peek()`, returns message metadata only (never payloads), same-process agents
Mailbox: inject a message	Low	✅ Done — `POST /agents/{name}/mailbox`; rejects reserved `_agency.`/`civitas.` types (no privilege escalation), audited `mailbox.inject` with the actor + type (never payload)
`attach_to`: mount introspection routes onto an existing `http_gateway` (D6c)	Low	✅ Done — a `topology_server` node with `config:{attach_to:<gw>}` builds only the `TopologyAgent`; a separate `http_gateway` with `config:{topology_agent:<name>}` serves its routes on that gateway's own port. Single-pass, linked by name
Deployment-shape reporting (transport/mode + per-process container hint)	Medium	✅ Done — surfaced in review: single-vs-multi-process was only inferable from `/processes` row counts and containerization was invisible. `/processes` now returns an explicit `deployment: {transport, mode}` (read from the live bus transport) and a per-process `container: {containerized, orchestrator}` hint (cheap cached heuristics, cross-platform-safe). Pure read-only reporting — distinct from the container management declined below
Mailbox: remove one specific in-flight message	Low	❌ Investigated, declined — mechanically feasible but breaks the at-most-once/FIFO delivery guarantee and the sender's mental model. The real use case (poison-pill wedging an agent) is already covered by force-restart (drops the whole mailbox + fresh incarnation); surgical single-message removal doesn't justify undermining a core guarantee. Revisit only with a use case force-restart can't serve
Per-agent process/container awareness beyond `process_id` (e.g. Docker)	Low	❌ Declined — recommended AGAINST (couples the runtime to a deployment concern better owned by container-native tooling); unchanged

v0.10.0 — HITL Polish (Released)¶

Status: ✅ Released 2026-07-31, as a single release (HITL half of the original "HITL & Streaming polish" bundle; the streaming R7 items are split out to their own future release — see Part 2). Three orthogonal HITL rough-edges on the durable-suspension primitive; full design in docs/design/hitl-polish.md.

Item	Status
Durable suspension: indefinite `ask()` (HITL approvals take hours/days)	✅ Done (D1) — `ask()`/`Runtime.ask()`/`bus.request()`/all 3 transports widen `timeout: float \\| None`; `None`/`-1`/`<=0` = wait indefinitely; default 30s unchanged. Uniform because every transport already funnels through `asyncio.timeout(None)`. Verified end-to-end: an indefinite ask into a SUSPENDED agent stays pending, then returns the real reply on resume
Durable suspension: fail-fast `ask()` into a suspended agent	✅ Done (D2) — opt-in `ask(..., fail_if_suspended=True)` raises the new `AgentSuspendedError` instantly instead of buffering; default off (strictly additive). Registry learns suspension via a name set updated at the single `_set_status()` choke point; consulted only when the flag is set
Durable suspension: crash-while-suspended restart-budget exemption	✅ Done (D3) — `RestartEngine.exempt_verdict()`; both static + dynamic crash paths skip the budget window for a crash whose incarnation carried the durable suspend marker (restart still happens, into SUSPENDED). Scoped — a RUNNING crash still counts

Part 2 — Backlog¶

Status: 🗂️ Tracked — the active todo list: everything not yet done. New work lands here first (a design doc if warranted), then moves into Part 1 — Shipped once released. Owner column: core = python-civitas, else the target repo.

Now open — tracked issue (python-civitas)¶

Public documentation reliability (see the next section, below) shipped 2026-08-26 -- all 4 action items done, same day it was opened. The M-LAST performance-benchmarking item (see Part 2 -- Backlog below) remains complete -- real benchmarks, real results, docs/design/performance-benchmark.md.

Public documentation reliability (Done)¶

Status: ✅ Done -- all 4 action items shipped | Priority: 🔴 High | Owner: core

Trigger: a real, code-verified audit of every public-facing doc (README.md, docs/*.md, AGENTS.md, docs/llms.txt, docs/agents-guide.md) found the public docs surface extensively stale relative to source -- broken imports (TopologyServer, removed at v0.9.5, still imported in AGENTS.md), two docs directly contradicting each other on real runtime behavior (mailbox blocking vs. non-blocking, architecture.md vs. faq.md), a fabricated civitas[mcp] pip extra and an entirely invented "Civitas as an MCP server" class in docs/mcp.md, a wrong pricing-table model ID in docs/plugins.md that silently makes real cost computation return None, and more -- full findings list in this session's own transcript, not duplicated here. Root cause, confirmed: CI lints/type-checks/tests civitas/ and smoke-tests examples/*.py (tests/integration/test_examples_smoke.py exists precisely because standalone examples used to rot the same way) -- but zero mechanism checks any prose or code-fence in docs/, README.md, or AGENTS.md. That gap is structural, not accidental: two independent past renames (TopologyServer→TopologyAgent; a state-store import path moving from contrib into core) each independently show up as unfixed in 2-3 different docs, meaning refactors here are not currently accompanied by any doc-sweep step.

Resolved via a 5-advisor LLM council (Contrarian/First Principles/Expansionist/Outsider/ Executor, peer-reviewed -- ⅘ independent reviewers picked the same response strongest, 5/5 flagged the same blind spot). Core finding: "docs" here is a category error, not a separate artifact class from code -- every public doc claim (an import path, a pip extra, a YAML schema, a pricing string) is a checkable claim about running code, the exact same category examples/*.py already got a real CI gate for. The fix is to extend that existing pattern, not invent a docs-specific discipline. Explicitly rejected: treating "docs for coding agents" as requiring separate content from "docs for humans" -- the failures are audience-agnostic (a broken import breaks the same way for both); the one place agent-facing docs should differ is structure/density (a lean router an agent can act on precisely), not correctness.

Action items, in the council's own priority order:

✅ Hand-fix the concretely-identified breakages -- no new tooling required, stops active harm immediately. TopologyServer→TopologyAgent import fix (with the v0.9.5 HTTPGateway-routing architecture note, in topology.md/supervision.md/cli.md/ observability.md), the mailbox blocking-vs-non-blocking contradiction (canonicalize on faq.md's correct version), the fabricated civitas[mcp] extra and invented civitas.mcp.server.MCPServer class in docs/mcp.md (rewritten: real fabrica-backed client, real single top-level mcp: servers: YAML schema applying to every agent, real civitas.mcp.call span shape, real dead-MCPToolError gotcha), the wrong pricing-table model ID in docs/plugins.md (claude-opus-4-6, full real table) plus the entirely missing OpenAIProvider/GeminiProvider/MistralProvider docs and the fact LiteLLMProvider is a NotImplementedError stub, the stale sqlite import path, the docs/agents-guide.md v0.8.0-vs-v0.9.0 restart-contract version contradiction (source confirms v0.9.0), missing civitas security init zmq/nats + civitas state migrate + topology show --header CLI docs, the --transport default's real supervisor-vs-worker-mode split, the wrong docs_enabled tri-state default and missing GatewayConfig/GatewayRequest fields, the false "no WebSocket / no auth" claims in docs/gateway.md (both real and shipped) plus the undocumented RateLimiter, the wrong Alt-Svc header format and default message type, the Transport protocol's 3 missing methods (unsubscribe/wait_subscribed/set_serializer) in docs/transports.md, the missing _agency.force_restart system message, and the phantom civitas/adapters/ + civitas/plugins/anthropic.py file listings in both CONTRIBUTING.md and docs/contributing.md (plus their drifted pre-commit-hook descriptions, now reconciled). Full list of what was actually found and fixed lives in each fixed doc's own real content, not restated here.

New real bug surfaced while fixing docs/gateway.md, not just a doc error — since resolved (2026-08-27): RouteTable.merge_contracts_from() (the mechanism that's supposed to wire @route/@contract decorator metadata into request validation) was only ever exercised directly against a manually-constructed RouteTable in tests/unit/test_gateway.py — HTTPGateway.__init__ built and owned its RouteTable as a private _route_table with no public way to call merge_contracts_from() on it or inject a pre-merged table, and civitas topology validate never called the related RouteTable.from_class() either, despite its own docstring claiming that's its exclusive caller. Ran a 5-advisor LLM council on the fix (wire it up via an explicit config field, auto-resolve via the registry at startup, or delete the dead mechanism) — all 5 peer reviewers independently ranked the same advisor's answer strongest, and the council's own recommendation was to verify real usage before choosing, since the choice hinges entirely on a fact nobody had checked. Did that verification: grepped and git-blamed @route/@contract usage across every repo in the org (python-civitas, civitas-contrib, presidium, fabrica, kordon, tessera, prx) — found zero real usage of @contract anywhere, including in civitas's own examples (examples/http_gateway.py/gateway_auth.py used @route as decoration only, never paired with @contract); presidium, a real, substantial downstream consumer of HTTPGateway/GatewayConfig, builds all its routing through YAML/config and never imports the decorators. Confirmed dead code, not a live feature with a wiring bug. Removed entirely: @route, @contract, RouteTable.from_class(), RouteTable.merge_contracts_from(), RouteEntry.request_schema/response_schema, civitas/gateway/contracts.py, and the OpenAPI schema-generation branches that depended on them (see CHANGELOG.md [Unreleased] for the full breaking-change entry). docs/gateway.md now documents that there is no built-in request/response validation and shows how to validate inside handle() directly. 2. ✅ Restructure AGENTS.md into a thin router -- cut from 910 to 304 lines (not quite the aspirational ~100, but every cut line was genuinely duplicated SDK-reference content: Quick Import Reference, the full Repository Layout tree, and the Core API sections for AgentProcess/Message/Supervisor/Runtime/Topology YAML/Tools/LLM calls/multi-agent patterns -- all now covered, and kept accurate, in docs/concepts.md, docs/agents-guide.md, docs/messaging.md, docs/supervision.md, docs/topology.md, docs/plugins.md, and CONTRIBUTING.md. What stayed: org structure + the one load-bearing dependency-direction rule, environment setup, code style, testing rules, public SDK stability rules, the Karpathy behavioral guidelines, and the 18-item Anti-Patterns list -- none of that duplicates anything in docs/, so it doesn't rot the same way. Added an explicit pointer at the top distinguishing this file (contributing to civitas core) from docs/agents-guide.md (building on top of civitas as a library), plus a PR-checklist line requiring the matching docs/*.md page to be updated in the same PR as any SDK-visible change. 3. ✅ Extend test_examples_smoke.py's exact philosophy to markdown: tests/integration/test_docs_codeblocks.py extracts every ``python fence fromdocs/.md,README.md,AGENTS.md,CONTRIBUTING.md(excludingdocs/milestones.mditself -- a historical log, not current reference), parses each withast(a hard failure on invalid syntax), and for everycivitas.import checks the module actually exists and the imported symbol is a real attribute on it.civitas_contrib./fabrica.imports are best-effort -- verified if that package happens to be installed, skipped (not failed) if not, since this repo's own CI deliberately never installs them (the exact cross-repo blind spot the monorepo-vs-separate council decision above chose not to solve this way). No CI wiring needed --tests/integrationalready runs in the existingintegrationjob in.github/workflows/ci.yml`, so this is live immediately.

First real catch, on the very first run: docs/concepts.md's own "Transport" section had an independent, second, already-stale copy of the Transport protocol (invalid Python syntax -- bodies omitted -- and only 5 of the real 10 methods, the exact "five methods" miscount already fixed in docs/transports.md during item 1). Replaced with a link to docs/transports.md#transport-protocol instead of a second copy. Confirms the checker earns its keep: same repo, same audit pass, and a second independent copy of the same fact had already drifted. 4. ✅ Add mkdocs build --strict to .github/workflows/docs.yml (a new "Build docs (strict...)" step before the gh-deploy step) so broken internal links/cross-references fail CI instead of silently deploying. Ran it locally first and fixed every real warning it found before wiring it in (fix-then-gate, not gate-then-fail): six docs/milestones.md links to docs/design/*.md written with a wrong ../ prefix (docs/milestones.md is itself inside docs/, so design/x.md is correct, not ../design/x.md); five links pointing at repo-root files (CHANGELOG.md, SECURITY.md, AGENTS.md, examples/dynamic_spawning.py, and 4 civitas/*.py references) that were relative paths assuming those files ship inside the built site -- they don't (mkdocs's docs_dir is docs/ only), so these were dead links on the actual deployed site today, not just a strict-mode nitpick; one wrong relative path to design/gateway-streaming.md from docs/streaming.md; and four same-page/cross-page anchor slugs that didn't match MkDocs's real heading-to-anchor slugification (double hyphens where headings have &/—, and two docs/cli.md → docs/observability.md anchors referencing an old heading name entirely). All fixed with either corrected relative paths (within docs/) or full GitHub blob URLs (for anything genuinely outside docs/). uv run mkdocs build --strict now passes clean. 5. Deliberately not building: a cross-doc-agreement linter for claims a code-fence can't verify (behavioral assertions, version numbers) -- the fix there is structural (make there be exactly one place each fact lives, cross-linked) not a second linter reconciling copies forever. 6. Deliberately deferred, not forgotten: a generalized/reusable doc-verification tool, an "agent-legible docs" positioning story, and a formal versioning-against-releases scheme -- real, legitimate future value, revisit only once items 1-4 are done and stable.

#26 (Streamable HTTP MCP transport) -- Done (2026-08-24), moved to Part 1 above as R11.

#25 (reopened, direct-mode half) -- Done (2026-08-23), moved to Part 1 below. mtls_source="direct" HTTP mTLS was still non-functional against uvicorn (never populated the ASGI TLS extension, uvicorn#400); a real client with a valid, trusted, allowlisted certificate was incorrectly rejected 401. Fixed with a custom uvicorn HTTP protocol subclass reading the real peer certificate off the TLS transport. See gateway-http-mtls-direct.md.

27–#35 closed by v0.8.0; #39–#43 closed by¶

v0.8.1. The 2026-07 architecture review is fully closed by v0.9.0 — zero xfail trackers remain in tests/unit/test_actor_model_gaps.py. Coverage top-ups and the dashboard rebuild are closed by v0.9.1. The examples smoke test + 8 new examples are closed by v0.9.2.

v0.9.3 follow-ups — Telemetry (Planned)¶

Scoped 2026-07-28 after a design conversation surfaced that two genuinely separate capabilities were both hiding under the single word "telemetry dashboard": (A) civitas already emits rich OTEL spans (cost, tokens, latency, per-agent/model) that already export cleanly to mature external tools (Jaeger/Grafana/Datadog) via one env var — docs/observability.md Mode 3 — so a chunk of "Option A" already exists and just needs hardening/completing; (B) a genuinely new civitas-native, zero-dependency, cost-focused view has real value for small/local deployments that don't want to stand up Jaeger just to see "what did this run cost me" — but requires building persistence that doesn't exist anywhere today (spans have no durable store; they're printed or OTLP-exported and then gone).

Decision: do both, built as six small, sequential capabilities (Track A: A1-A3, Track B: B1-B3), each independently verified with real, live checks before moving to the next — Track A first (cheap, low-risk, and A1 did itself surface a real bug per this project's pattern of "verify, don't assume"), Track B after, with its own dedicated design doc before B1's storage code landed (same rigor dashboard-v2 got). All six shipped together as a single release, v0.9.3 (2026-07-29 decision — ship the whole arc as one release rather than a version per capability). What remains below is genuine follow-up work, not part of that release.

Track A — harden/complete what already half-exists:

A1, A2, and A3 all shipped together as part of v0.9.3 — see Part 1 above (§A1-A3) for full findings and fixes. Track A is now fully shipped.

Track B — the native, cost-focused, zero-dependency view:

B1, B2, and B3 all shipped together as part of v0.9.3 too — see Part 1 above (§B1-B3). Remaining Track B items:

#	Capability	Scope
v0.9.3.6 (B4)	Deferred by explicit decision (2026-07-29), documented not silently dropped — two real design questions surfaced right after B1 shipped, both real enough to track but not worth a mid-flight refactor of already-working, already-tested code: (1) placement — `SQLiteBackend` writes durable data to disk, matching this project's own "persistence backends live in `civitas-contrib`" precedent (`SQLiteStateStore` already lives there) rather than core `python-civitas`, which currently only ships zero-I/O feature machinery (`ExportBackend`/`FanOutBackend`/`ConsoleBackend`); (2) pluggability — SQLite is one backend among several a user may eventually want (Postgres, etc.) — the telemetry-specific logic (span normalization, schema shape) should be separated from the storage mechanism itself so alternative backends don't have to reimplement normalization, "build like a library so others can use the capability." Full writeup in `docs/design/telemetry-native.md`'s §12 addendum.	✅ Done (v0.11.0) — see below
v0.11.1 (B4 cont.)	Driver-backed SpanStores + MySQL StateStore (civitas-contrib 0.2.0 + core loader). `PostgresSpanStore` (`[postgres]`), `MySQLSpanStore` (`[mysql]`), and `MySQLStateStore` (`[mysql]`) — all against the B4 `SpanStore`/`StateStore` protocols, reusing core's public `normalize_span`. Single `civitas_spans` table + `DOUBLE` epoch times so bucketed queries match core's `SQLiteSpanStore` exactly (verified by a cross-backend equivalence suite against real Dockerized Postgres/MySQL via testcontainers). Core loader gains `exporter` types `sqlite`/`postgres`/`mysql` (SpanStores as declarative exporters) and state `type: mysql`	✅ Done
v0.11.0 (B4)	SpanStore protocol + contrib boundary rule (design `spanstore-and-contrib-boundary.md`). Resolves both v0.9.3.6 questions: (1) pluggability — `SpanStore` extends `ExportBackend` with the query surface (one protocol per backend, read+write share one schema, can't drift); `normalize_span()` is now public API every backend imports; `SQLiteBackend`+`SQLiteQueryEngine` merged into `SQLiteSpanStore` (old names kept as aliases); `InMemorySpanStore` added as reference impl + test double, with a parametrized conformance suite proving the seam is backend-agnostic. (2) placement — adopted a cleaner contrib boundary rule (needs a third-party runtime dependency → contrib, not touches disk), which keeps `SQLiteSpanStore` in core AND moved `SQLiteStateStore` back from contrib to core (stdlib sqlite3, zero new deps; YAML `type: sqlite` now works without contrib). Driver-backed Postgres/MySQL span+state stores stay in contrib (future, demand-driven). Contrib deprecation shim ships as a separate civitas-contrib release gated on core v0.11.0	✅ Done
v0.9.3.7	Log/event viewer for `civitas telemetry` — ✅ Done (v0.10.1): added the two individual-span query methods it needed (`recent_spans` feed + `spans_in_trace` §13 drill-down, with a `SpanRecord`), then an `EventLogTable` panel (time/event/agent/status/duration, newest-first, ok/error coloured, count in title, scrolls natively). Wired into `civitas telemetry` and verified end-to-end against a real Runtime+SQLiteBackend run	Done
v0.9.3.8	Live tick animation for `CostChart`/`MessageRateChart` — split into two findings (v0.10.1): (1) tweened intra-poll frames — ❌ declined: interpolating fake cost/rate values between real polls is dishonest on a cost dashboard; the per-poll redraw IS the update, and the charts already draw straight line segments between real points only. (2) range-adaptive bucket granularity — ✅ done: the real reason the chart looked dead was that both queries defaulted to a fixed bucket (1 day for cost) regardless of the selected range, so a 1h view collapsed to a single point. Added `TimeRange.bucket_seconds()` (1m/5m/1h/6h/1d, following the window) threaded into both queries — a 1h view now shows ~60 real points, verified end-to-end (12 spans over an hour → 12 points vs. 1 before)	Done
v0.9.3.9	Scrollable/paginated `CostBreakdownTable` — ✅ Done (v0.10.1): the "would overflow" premise was inaccurate — `CostBreakdownTable` is a `ScrollView` and already scrolls natively at any cardinality (verified: 100 rows in a 20-row viewport, `max_scroll_y=81`). The real large-deployment gap was a scroll affordance, added as an agent/model count in the border title ("Breakdown (30 agents, 12 models)")	Done

Tracked idea — explicit cross-restart `session_id` concept (not scheduled)¶

Raised in conversation (2026-07-30) while scoping v0.9.4's simple "session length" signal (above) — deliberately NOT folded into that small, incarnation-scoped feature. This is a genuinely bigger, cross-cutting idea: real session identity with explicit boundaries and continuation relationships across restarts ("session 2 continues session 1 after a crash-restart"), not just "how long has this incarnation been running."

What it would add, concretely:

A real session_id (likely a UUID) that can PERSIST across restarts when appropriate — the runtime (or the agent) recognizing "this is a continuation of an interrupted session" vs "this is genuinely new," which today's incarnation-scoped signal cannot do (it deliberately resets on every restart, full stop).
Explicit boundary triggers, plural — today's signal has exactly one (a restart). A real concept needs at least three: (a) idle timeout (no LLM call in N minutes → session ends even without a restart), (b) an explicit agent-declared boundary (e.g. "user said goodbye", "task complete" — an opt-in API, not automatic), © restart/crash (today's only signal).
Propagation through the existing message-causality mechanism — a civitas.session_id attribute threaded through messages/spans the same way trace_id already propagates via inheritance from _current_message in send()/ask(). This is what would let EVERY message and span in one logical session share an identity, not just the initiating agent.
A real telemetry connection, not just a dashboard cosmetic: if spans carried civitas.session_id, SQLiteBackend's normalize_span() (§4) could promote it to a real column, and SQLiteQueryEngine could gain a cost_by_session/turns_by_session query — genuinely more useful paired with the ALREADY-tracked, not-yet-built "trace/span drill-down" candidate method (§13's list): "show me this session's full trace timeline," not just one span.
Cross-restart continuation would need StateStore — persisting {session_id, ended_at} on shutdown (graceful or crash-recovery-detected), read back in on_start()/_restore_state() to decide whether to continue the prior session or start fresh (with some "recent enough to count as interrupted" threshold needing its own decision).

Real open questions a proper design would need to resolve, not yet decided:

The crux of the whole thing: does "session" mean "one incarnation's engagement" (today's simple version, and what this whole idea has been implicitly assuming) or "one logical conversation with a specific counterparty" — which a SINGLE incarnation could have MANY of concurrently (e.g. a customer-support agent handling 5 different users' conversations at once)? If the latter (which feels like the more honest, generally-useful definition), session_id can't be a single scalar on the agent at all — it needs to be keyed per-correspondent/per-task, a meaningfully different (and bigger) shape than anything sketched above.
Who generates/owns the session boundary — automatic runtime heuristics, or an explicit opt-in agent API? Automatic is more "batteries included" but risks getting boundaries wrong for use cases the runtime structurally can't know about (see the per-correspondent question above).
Does this belong in core civitas, or is it more naturally a civitas_contrib pattern/mixin agents opt into? Also possibly touches v0.9.5's AuthN/AuthZ conversation ("who is this conversation with" is an identity-adjacent question too) — worth raising there, not deciding now.

Not scheduled against any version — tracked here so the idea isn't lost, deliberately not assigned to v0.9.4 (too big, violates that release's own "no new design surface" framing) or v0.9.5 (a different kind of design conversation, though possibly a discussion point there too).

Tracked — architecture decomposition / refactor pass (not scheduled)¶

Raised 2026-07-31. The runtime has grown organically across the whole v0.7–v0.9 arc; several core modules have accreted multiple responsibilities and are now large enough that a deliberate decomposition pass is worth a dedicated session (design-first, behavior-preserving — the ~1.6:1 test-to-source ratio makes this safe to do under a green suite). Grounded in real size, not a vibe:

Module	Lines	Responsibilities that have accreted
`civitas/process.py`	~1975	`AgentProcess` + `Mailbox` + `ProcessStatus`/`SuspendCategory`/`_ForcedRestart` + the message loop + `_dispatch`/retry + suspend/resume/force-restart + durable-state restore + MCP client wiring + streaming hooks — a genuine god-module
`civitas/supervisor.py`	~1705	`Supervisor` + `DynamicSupervisor` + restart-engine wiring + heartbeat monitor + remote-child (D5) probing + crash-event queue
`civitas/runtime.py`	~1360	`Runtime` + the whole `from_config` YAML loader (`_build_node`, now 6+ node types incl. the v0.9.⅚ topology_server pair + attach_to) + component wiring + control-plane entry points

Together ~5k lines (~27% of source). Candidate cuts (to be designed, not prejudged): split Mailbox and the suspend/force-restart control machinery out of process.py; separate DynamicSupervisor and the heartbeat/remote-liveness concern from supervisor.py; extract the from_config topology loader (the _build_node dispatch) out of runtime.py into its own loader module. Explicitly NOT a rewrite — a decomposition of already-working, already-tested code, one behavior-preserving move at a time, each verified against the existing suite. Not scheduled against a version yet; a natural fit before the v1.0.0 GA gates (an external security auditor reviewing 2k-line god-modules is a worse experience than reviewing focused ones).

Bus-native streaming polish (R7 remainder) (Planned)¶

Split from the original v0.10.0 "HITL & Streaming" bundle when the HITL half shipped as v0.10.0 (Part 1). Both refine the already-shipped bus-native streaming feature; neither is a defect.

Item	Priority	Source
R7: credit-based stream backpressure (`civitas.stream.credit` reserved)	Low	design/bus-native-streaming.md §8 Q5
R7: immediate `StreamInterrupted` on producer loss	✅ Done (v0.10.1)	design/bus-native-streaming.md §8b addendum — producer keeps a per-stream `recipient` index; on teardown (stop/force-restart/crash) sends `producer_stopped` → `StreamInterrupted` to each consumer, so a consumer mid-stream fails immediately instead of waiting out `idle_timeout` (still the backstop for a hard-killed peer). Verified end-to-end

v1.0.0 — GA gates (Planned)¶

Item	Priority	Notes
External security audit (fix all HIGH+; publish summary)	🟡 Medium	hard blocker for declaring 1.0
Postgres: zero-downtime dual-write migration	🟢 Low	production-ops for GA
Postgres: PgBouncer deployment guide	🟢 Low	docs pass
ZMQ at-least-once route establishment — go/no-go review	🟢 Low	sub-ms residual after v0.8.1 settle-barrier; build only if reproduced (design/cross-process-spawn.md addendum)
CI matrix: macOS + Windows runners (today: Ubuntu only)	🟡 Medium	moved from the original v0.9.2 grab-bag (2026-07-28 roadmap split) — production ZMQ defaults are already Windows-safe (`tcp://`), but 4 test files use `ipc://` (Unix-only) and nothing has ever been CI-verified outside Linux; revisit sooner if a real Windows/macOS user need arises before GA

Continuous (every release, no version): CVE watch / CVSS advisories — enforced by the Security workflow (pip-audit --strict caught PYSEC-2026-2132 in practice).

v1.1+ — Enterprise ladder (Planned, demand-driven)¶

Item	Priority	Source
Fine-grained ACL DSL (overlaps M4.4 capabilities)	🟡 Medium	design/security-hardening.md
HSM / TPM-backed signing keys	🟢 Low	design/security-hardening.md
PKI / CA integration (cert issuance)	🟢 Low	design/security-hardening.md
Visual Topology Editor (drag-drop UI)	🟢 Low	§M4.1
Fiddler eval exporter: two-way guardrail receive	🟢 Low	§M2.6

Ideas (not yet specced)¶

Item	Priority	Where tracked
Medicus self-healing hero demo (P0+P1: detect → diagnose → verified PR) — flagship example; supersedes the Telegram personal assistant (which drops to a minor gateway+skills sample)	🟡 Medium	design/medicus-demo.md
Self-healing / autonomous remediation agent — monitor (metrics/audit/OTEL/crash) → diagnose (LLM) → sandbox-verify → canary-deploy → auto-rollback, under staged autonomy + safety gates	🟡 Medium	design/self-healing.md
Worker-level restart-with-new-code (blue-green drain) — the deploy primitive enabling self-healing & near-zero-downtime code updates (Python has no safe in-place reload)	🟡 Medium	design/self-healing.md

Other repos (versioned by their owning repo — tracked here for visibility only — per `boundary.md`)¶

Item	Owner	Status	Where tracked
`CivitasMCPServer` — expose an agent tree as an MCP server	fabrica	⏸️ not started anywhere	§M3.4
CrewAI adapter — full implementation (stub raises `NotImplementedError` today)	civitas-contrib	⏳ stub	§Infrastructure & Release
~~MySQL StateStore~~	civitas-contrib	✅ Done (`civitas-contrib` 0.2.0) — row stale, kept struck through rather than silently deleted; see §Postgres StateStore's own ✅ record	§Postgres StateStore
Prompt Library & Playground	civitas-contrib	💡 idea (🔴 high), spec unwritten	§Phase 5
Skills Gateway	civitas-contrib	💡 idea, spec unwritten	§Phase 5
Fabrica — Tools Gateway / `find_tools` (RFC 0001)	civitas-forge	💡 idea (🔴 high), spec unwritten	§Phase 5, rfc/0001
LLM Gateway (governed: rate limits, budgets, grant routing)	presidium	⏸️ moved	§Phase 5
Credential-propagation RFC (per-user OAuth for retrieved tools)	cross-repo	⏸️ future RFC	rfc/0001 §out-of-scope

Recently shipped (moved out of this backlog; see Part 1 for detail): verification perimeter v0.8.1 (#39–#43, PR #44), supervision core hardening v0.8.0 (#27–#35, PR #38), cross-process dynamic spawning (#20), per-agent spawn quotas (#21), encrypted StateStore at rest (#19), first-party JWT + mTLS gateway auth for HTTP (#18), non-blocking dynamic spawn + spawn_into() (#14, #16), bus-native streaming (#22), WS/gRPC gateway auth (#17), HTTP mTLS via reverse proxy (#25).

M4.1 — Visual Topology Editor¶

Status: ⏸️ Deferred | Priority: 🟢 Low

Web-based drag-and-drop editor for designing agent topologies visually.

Deliverable	Status
Drag-and-drop agent/supervisor canvas	⏸️
Visual message flow connections	⏸️
Supervision strategy configuration via UI	⏸️
Export to valid Civitas topology YAML	⏸️
Round-trip: imported YAML renders correctly	⏸️

M-LAST — Real Performance Benchmarking (do this last, deliberately)¶

Status: ✅ Complete, 2026-08-25. Real benchmarks against a real, standalone civitas server, on real hardware (a MacBook + a separate Linux homelab host, direct Tailscale connection), a real k6 load profile replicating TM Dev Lab's own exact methodology shape, and a real, explicit ranking against their own published table. Full results, methodology, and honest limitations (including a real, encountered cross-host firewall constraint for the message-bus benchmark specifically, and a real, unreproduced anomaly worth a future root-cause pass): docs/design/performance-benchmark.md. Real, reusable harness checked in at benchmarks/ (not deleted after use). Headline finding: civitas's own HTTPGateway throughput at a directly-comparable CPU-bound workload (936.8 req/sec) beats Node.js and Python/FastMCP in TM Dev Lab's own published table, despite civitas's own run paying a real cross-host network-hop latency tax their same-host setup never had to pay -- a real, previously-unmeasured, positive result, reported alongside the equally real fact that civitas's average latency is higher than all four of their published implementations.

Original scoping, kept for the record below:

Status: ⏸️ Deferred, scoped | Priority: 🟢 Low — explicitly sequenced after everything else in this backlog, not before. Scoped 2026-08-24, directly informed by a real spike (civitas-io/fabrica's SPIKE-mcp-transport-benchmark.md) and the real industry research done alongside it — both the methodology gaps found in that spike and the two credible published benchmarks it was checked against (TM Dev Lab's multi-language MCP benchmark, Stacklok/ToolHive's transport benchmark) directly shape what "done properly" means here.

Why deferred to last, on purpose: a real, comparable benchmark needs a stable target — running it now, mid-backlog, means re-running it every time a real perf-relevant change lands elsewhere in this list (gateway, transport, supervision). Doing it once, last, against a settled surface is the more honest use of the effort.

What "properly scoped" means, learned directly from the real gaps found in the MCP transport spike — do NOT repeat these shortcuts here:

A real, independent load generator (k6, Locust, or equivalent) driving genuinely separate connections/sessions against a real running civitas process — not concurrent asyncio tasks sharing one connection inside the same process as the code under test. The MCP spike's own single-shared-session throughput plateau (never scaling past ~5 concurrent callers) is a direct, named consequence of skipping this the first time; don't repeat it here.
A real network hop, not pure loopback — at minimum client and server as separate processes on the same host talking over a real socket; ideally client and server on separate real hosts (the homelab is available), since loopback-only numbers were shown to diverge meaningfully from real, published, network-inclusive numbers in the MCP spike's own reconciliation section.
Realistic workloads, not workload-free echo/no-op calls — at least one CPU-light path (e.g. a simple agent handle() round trip) and one representative real path (e.g. a gateway HTTP request through to a responding agent, matching how TlsAwareHttpToolsProtocol/ HTTPGateway are actually used) so results reflect real usage, not pure transport overhead in isolation.
State the actual concurrency model precisely in the results themselves — real OS threads vs. asyncio tasks vs. separate processes vs. separate real client machines, spelled out before any number is reported, matching the "honest methodology disclosure" pattern the MCP spike added only after being asked directly. Don't make that same doc-back-and-forth necessary again.
Pick ONE existing published methodology to genuinely replicate the shape of (TM Dev Lab's k6/Docker/50-VU setup is the more directly comparable of the two, since it already covers multiple languages/runtimes at the SAME workload and load profile) so the eventual comparison is a real, apples-to-apples ranking — not another disclosed-as-not-comparable exercise. Named explicitly here so scoping this later doesn't silently drift back into an easier, non-comparable shape.
Cover the real surfaces that matter for this repo specifically: HTTPGateway/mTLS request throughput (the R10 work), the message bus under zmq/nats transports at real concurrency, and Supervisor/DynamicSupervisor spawn latency under load — not just MCP (that surface already has its own real, if narrowly-scoped, numbers from the spike above).

Deliverable, when this is picked up: a docs/design/performance-benchmark.md design doc first (methodology, exact tool, exact comparison target, real hardware spec) — matching this project's own "design doc before implementation" convention — then a real, reproducible harness (kept, not deleted, per this org's spike-code convention) and a findings doc with real, dated numbers, an honest limitations section, and an explicit ranking against the one chosen published methodology from item 5 above.

Phase 5 — Agentic Platform¶

Civitas provides the runtime primitives. Governance lives in Presidium — an interface library that defines governance protocols (PolicyEngine, AgentRegistry, CredentialProvider, etc.) with lightweight defaults (CEL policy engine, in-memory registry) in the core package, and adapters for existing products (OPA, Vault, LiteLLM) plus reference implementations for novel components in presidium-contrib.

Presidium follows the same pattern as Civitas: protocols in core, implementations in contrib. Every component works as an in-process library (single-process deployments) or as a service (distributed deployments via Civitas GenServers or standalone HTTP). See Civitas-Presidium Boundary for the full architecture.

The items below are ideas across the wider Civitas product line that complement Presidium's governance layer. Most of them are not python-civitas's job — see the "Lives in" callout on each. Per boundary.md (2026-05-08), Prompt Library and Skills Gateway are civitas-contrib ("Dev tooling" / "skills routing layer"), same as Fabrica is its own repo. This file previously implied otherwise; corrected July 2026.

Prompt Library & Playground¶

Status: 💡 Idea — to be specced | Priority: 🔴 High | Lives in: civitas-contrib, not python-civitas

Correction (July 2026): Originally framed here as a "Civitas-side feature." boundary.md lists "Prompt library" under civitas-contrib ownership ("Dev tooling"). If built, PromptStore would be a civitas_contrib-namespaced GenServer subclass depending on this repo's GenServer base class — not a civitas/ module itself. The civitas playground CLI reference below would need to become a civitas-contrib CLI plugin or a documented pattern, not a core subcommand.

Prompts as first-class versioned entities, stored and served by a supervised PromptStore(GenServer). Agents load instructions by name rather than hardcoding strings — prompt changes never require a code deploy. The playground (CLI + dashboard tab) lets you test a prompt version against a live agent before promoting it.

This is one of the strongest SaaS upgrade stories: the OSS PromptStore runs in your deployment; a hosted version adds a web UI for non-engineers, team collaboration, cross-deployment promotion, and output analytics.

Idea	Notes
`PromptStore(GenServer)` — versioned prompt storage on the bus	Agents call `call("prompt_store", {"agent": "assistant", "slot": "system"})`
SQLite backend (runtime-mutable) + YAML dir backend (git-tracked)	User chooses per deployment
Named version aliases — `latest`, `stable`, `experimental`	Pinned per agent per environment in topology YAML
Per-agent, per-slot prompt mapping	Each agent can have multiple slots: `system`, `few_shot`, `tools`
Hot-swap support — reload prompt without restarting agent	Agent subscribes to prompt update events
`civitas playground` CLI — interactive session with a specified prompt version	Test against live runtime before promoting
Dashboard tab — side-by-side prompt diff, test messages, output comparison	Lightweight eval harness backed by EvalLoop (M2.5)
A/B traffic splitting between prompt versions	Random split; metrics tracked via OTEL spans
SaaS layer — web UI, team collaboration, cross-deployment promotion, analytics	`design/prompt-library.md — to be written`
Spec	design/prompt-library.md — to be written

LLM Gateway¶

Status: ⏸️ Moved to Presidium

Model routing without governance (multi-provider fallback for reliability) is a thin Civitas utility — CompositeModelProvider. It is not a full gateway.

The full governed LLM gateway — per-agent rate limits, cost tracking, budget enforcement, grant-based provider routing — belongs in Presidium. It is implemented via the GovernedModelProvider protocol in the presidium core package, with the LiteLLMProxyAdapter and PortkeyAdapter available in presidium-contrib. It wraps any Civitas ModelProvider via the plugin protocol and enforces governance policy before delegating to the underlying provider.

Civitas provides the ModelProvider protocol (integration point 2 for Presidium). Civitas does not provide rate limiting, budgets, or grant-based routing — those are governance concerns.

Residual Civitas utility: CompositeModelProvider — a simple ordered fallback chain (primary → fallback) for reliability. No governance, no per-agent tracking. Infrastructure, not governance.

See Presidium for the governed implementation (GovernedModelProvider in core, LiteLLMProxyAdapter in contrib). See docs/design/civitas-presidium-boundary.md for the full boundary definition.

Fabrica — Tools Gateway¶

Status: 💡 Idea — to be specced | Priority: 🔴 High

Product: Fabrica (pip install fabrica) — lives in civitas-io/civitas-forge, not in python-civitas.

Fabrica solves the tool schema token problem: passing all tool schemas to every LLM call is token-expensive and degrades selection accuracy beyond ~20–30 tools. Instead of N schemas, the LLM receives one find_tools(query) meta-tool and retrieves only the schema it needs.

Fabrica aggregates tool sources (local ToolStore, MCP servers, Composio, custom), serves a unified namespace, and exposes a retrieval interface. Civitas agents connect to it as a tool source — any other LLM framework can too.

Dependency chain: M3.4 (MCP plumbing) → M4.4 (ToolStore) → Fabrica (retrieval)

See RFC 0001 (docs/rfc/0001-tool-retrieval.md) for the formal problem statement and proposed interface standard.

Idea	Notes
`find_tools(query)` meta-tool — one schema sent to LLM, not N	Keyword backend (default) + embedding backend (`fabrica[search]`)
Tool source aggregation — local ToolStore, MCP servers, Composio, custom	Pluggable `ToolSource` protocol
Unified tool namespace across all sources	`gateway://source/tool_name` address scheme
Per-source credential isolation	Each source has its own auth config; agents never see other sources' secrets
Tool call sandboxing	Filesystem + network isolation for untrusted tool execution
Health monitoring + circuit breaker per source	Unhealthy sources removed from routing automatically
MCP-compatible interface	Fabrica itself exposes `list_tools` + `call_tool` — any MCP client can connect
Civitas integration — `ToolSource` plugin pointing at Fabrica	`civitas[fabrica]` extra
SaaS upgrade path — hosted Fabrica with team tool registry, analytics	Future
Spec	`civitas-forge/packages/fabrica/` — to be created

Skills Gateway¶

Status: 💡 Idea — to be specced | Priority: 🟡 Medium | Lives in: civitas-contrib, not python-civitas

Correction (July 2026): Originally framed here as a "Civitas-side feature." boundary.md lists "Skills gateway" under civitas-contrib ownership ("skills routing layer"). It would consume this repo's Capability-Aware Registry (M4.4) and MessageBus as a dependency, the same way civitas-contrib's provider plugins and adapters consume civitas core today — it would not be implemented inside civitas/.

A supervised registry of composable agent workflows — "skills" — that can be discovered and invoked by name or capability. A skill is a named, versioned sequence of tool calls, LLM steps, or sub-agent invocations exposed as a single callable unit on the bus.

Extends the Capability-Aware Registry (M4.4): where M4.4 answers "which agent can do X?", the Skills Gateway answers "invoke skill X, wherever it runs."

Idea	Notes
`@skill` decorator — declare a reusable workflow on any agent	Versioned, named, queryable by capability tags
Skill discovery by capability / input type	`gateway.find_skill("summarise", input_type="text/html")`
Cross-agent skill composition	Skills can invoke other skills; gateway handles routing
Skill versioning with semver + forward compatibility	Old callers work when a skill is upgraded
Local + remote skill sources	Skills can live in the local registry or a remote Civitas deployment
Hosted skills marketplace	Future SaaS layer — shared skills across organisations
Spec	design/skills-gateway.md — to be written

Milestones¶

Status legend¶

Part 1 — Shipped¶

Overview¶

Phase 1 — Core Runtime¶

Phase 2 — Ecosystem¶

M2.1 — ZMQ Multi-Process Transport¶

M2.2 — NATS Distributed Transport¶

M2.3 — OTEL Observability¶

M2.5 — EvalLoop (Local)¶

Implementation checklist¶

M2.6 — Remote Eval Exporters¶

Phase 3 — Developer Experience¶

M3.1–M3.3 — CLI and Dashboard¶

M3.4 — MCP Integration¶

M3.5 — GenServer¶

Implementation checklist¶

Infrastructure & Release¶

Phase 4 — Platform Maturation¶

M4.1b — Dynamic Agent Spawning¶

M4.2 — Security Hardening¶

M4.2a — Identity & Signing¶

M4.2b — Transport mTLS¶

M4.2c — Credential Isolation¶

M4.2d — Tool Sandbox¶

M4.2e — Audit Log¶

M4.3 — Codebase Security & Enterprise Posture¶

M4.4 — Capability-Aware Registry¶

Design notes¶

HTTP Gateway¶

Implementation checklist¶

Gateway API Surface¶

Implementation checklist¶

Postgres StateStore + Migration¶

v0.4.0 Release Fixes¶

GH #6 — Route-scoped gateway middleware is parsed but never executed¶

GH #7 — Missing py.typed marker despite "Typing :: Typed" classifier + mypy --strict¶

v0.5.0 — Released¶

A — Correctness & hardening¶

B — Durable suspension¶

C — Doc hygiene (completed as part of scoping this release)¶

D — Explicitly deferred to a future version¶

v0.6.0 — Gateway Completion (Released)¶

v0.7.0 — Spawn Maturation & Gateway Auth (Released)¶

v0.8.0 — Supervision Core Hardening (Released)¶

v0.8.1 — Verification Perimeter (Released)¶

v0.8.2 — Hygiene (Released)¶

v0.9.0 — Supervision Endgame (Released)¶

v0.9.1 — Post-endgame Polish (Released)¶

v0.9.2 — Examples Completeness (Released)¶

v0.9.2.1 — Bugfix Release (Released)¶

v0.9.3 — Telemetry: OTEL Linkage, Prometheus, Grafana, Native Storage, Query Layer, TUI (Released)¶

A1 — OTEL trace linkage¶

A2 — Prometheus metrics¶

A3 — Grafana stack, completing Track A¶

B1 — Native telemetry storage¶

B2 — Telemetry query/aggregation layer¶

B3 — Telemetry TUI, completing Track B¶

v0.9.4 — Dashboard TUI Polish (Released)¶

v0.9.5 — Control-Plane Auth Foundation: TopologyServer/HTTPGateway Merge (Released)¶

v0.9.6 — Dashboard Control-Plane Write Actions (Released)¶

v0.10.0 — HITL Polish (Released)¶

Part 2 — Backlog¶

Now open — tracked issue (python-civitas)¶

Public documentation reliability (Done)¶

27–#35 closed by v0.8.0; #39–#43 closed by¶

v0.9.3 follow-ups — Telemetry (Planned)¶

Tracked idea — explicit cross-restart session_id concept (not scheduled)¶

Tracked — architecture decomposition / refactor pass (not scheduled)¶

Bus-native streaming polish (R7 remainder) (Planned)¶

v1.0.0 — GA gates (Planned)¶

v1.1+ — Enterprise ladder (Planned, demand-driven)¶

Ideas (not yet specced)¶

Other repos (versioned by their owning repo — tracked here for visibility only — per boundary.md)¶

M4.1 — Visual Topology Editor¶

M-LAST — Real Performance Benchmarking (do this last, deliberately)¶

Phase 5 — Agentic Platform¶

Prompt Library & Playground¶

LLM Gateway¶

Fabrica — Tools Gateway¶

GH #7 — Missing `py.typed` marker despite "Typing :: Typed" classifier + mypy --strict¶

Tracked idea — explicit cross-restart `session_id` concept (not scheduled)¶

Other repos (versioned by their owning repo — tracked here for visibility only — per `boundary.md`)¶