Compose Stack Ownership

See deploy/README.md for the full authoritative stack list and layering.

Quick summary: - core.yml — always running: postgres, redis, redpanda, openbao - ritual.yml — always running: ritual-server, ritual-consumer, ceremony-warden - platform.yml — always running: keycloak, forgejo, matrix - observability.yml — always running: grafana, prometheus, dozzle - apps.yml — always running: forge-bridge, slack-bridge, audit-events, lore-drift, git-poller

Legacy stacks (docker-compose.yml, docker-compose.prod-full.yml) are deprecated — do not use for new deployments.

bin/forge-deploy-check           # report drift between tracked files and live host
bin/forge-deploy-check --sync    # apply tracked files to host

Kafka Broker Policy

Never use 10.10.0.1:19092 for TLS clients. The Redpanda TLS cert is issued for spel.toilville.dev. IP addresses break hostname validation.

The host firewall must also allow inbound 19092/tcp on toilville-forge. That rule is persisted via netfilter-persistent and is required for the public Kafka listener to survive reboots.

SASL credentials: secret/forge/kafka-scram in OpenBao.

Validation:

bin/forge-kafka-check   # fail if any TLS client uses an IP broker

Vault / OpenBao

• Internal URL (Docker gateway): http://172.22.0.1:8200
• WireGuard URL (host access): http://10.10.0.1:8200
• Public URL: https://vault.toilville.dev (via nginx proxy → 10.10.0.1)
• Systemd service: openbao on toilville-forge

Hook System

bin/forge-regen-hooks
git add generated/hooks/
git commit -m "regen: hooks from canonical template"

Bastia Grant Scopes

Managed in core.bastia_grant_scopes and core.bastia_grant_policies.

Narrow helpers in bin/forge-privileged-op wrap the common operations. Do not add raw bao, ssh, or scp to Claude’s allowlist — use bounded helpers instead.

Privilege Operations

bin/forge-privileged-op vault-read <path>
bin/forge-privileged-op vault-write <path> <json>
bin/forge-privileged-op remote-copy <local> <host:path>
bin/forge-privileged-op remote-restart <service>
bin/forge-privileged-op service-restart <service>
bin/forge-privileged-op secret-patch <path> <field> <value>
bin/forge-privileged-op db-exec <sql-file>
bin/forge-privileged-op rotate-forge-ci-password

Each operation: requests grant → executes → revokes. Fail-closed if Bastia denies.

Local Cache

cr-sqlite cache lives at ~/.ritual/schema.db. Freshness tiers:

bin/forge-local-cache-check          # human-readable report
bin/forge-local-cache-check --json   # machine-readable

System Health Check

bin/ritual-system-doctor   # run all checks
bin/forge-kafka-check      # Kafka policy validation
bin/forge-deploy-check     # deploy drift check
bin/forge-local-cache-check # cache freshness

How to submit a request or idea

Send an email to the Toilville forge inbox. Write your request in plain English in the subject line. The forge reads it, creates a work item, and routes it for review.

Examples of things you can send: - Review the latest design mockup - Can you approve the Q2 budget estimate? - Update the sprint goal for this week - Help — the integration is broken again

You’ll receive a reply if clarification is needed. Decisions and outcomes are recorded automatically.

How to contribute code or documentation

Open a pull request or issue in the Forgejo repository at git.toilville.dev. The forge monitors all repos in the toilville organization and creates work items automatically when you:

• Open a new issue
• Submit a pull request
• Push a commit to a tracked branch
• Leave a review comment

You don’t need to mention the forge or use any special syntax. Just work normally in git.

How decisions get made

When a request requires a group decision, you may receive a decision parlay — a structured prompt asking you to vote, object, or add context.

• You can respond by email reply or by commenting in the relevant git thread.
• Votes, objections, and clarifications are recorded in the lineage trail.
• A decision is recorded once the threshold is met (unanimous, majority, or steward call).
• You can review past decisions by asking: What did we decide about X?

How to request a service operation

If you need something restarted, redeployed, or updated on the forge infrastructure, send an email or open an issue describing what you need and why. The request is routed for approval before anything runs.

Do not ask for raw access to production servers. Bounded operation requests are the correct path.

What you’ll never need to know

These are implementation details — you do not need to understand them to contribute:

• Kafka topics or broker addresses
• OpenBao / Vault secret paths
• Docker compose files
• Claude hook internals
• Bastia grant scopes

If you find yourself needing these, ask an operator — it likely means there’s a missing bounded helper or form that should exist.

Getting help

Send an email with your question. Or open a Forgejo issue in toilville/spelwork tagged help.

Top 5 high-impact findings

1. Hardcoded schemas (canonical violations only)

2. Dead code / orphans

3. Recommendations — ranked

1. Worktree cleanup (S, ~1.1 GB reclaimed). Inspect agent-a9739024/services/ritual-consumer/ first; merge or discard, then git worktree prune the rest.
2. Migrate spelwork-ap AGE bootstrap DDL to a new versioned migration (e.g. migrations/3XX_ap_steward_topology.sql). Replace the .rs CREATE TABLE with a schema-existence assertion.
3. Migrate dynseal provenance DDL the same way — pattern_attestations, information_provenance, supply_chain_log.
4. Pre-commit hook: flag CREATE TABLE / ALTER TABLE outside migrations/ and **/tests/. Cheap enforcement, prevents regression.
5. Compliance DB ownership: decide whether deploy/compliance-schema.sql is canonical for the SOC2 audit DB, or whether it should move under migrations/ with a schema-name prefix. Document the answer in docs/src/compliance.md.
6. services/forge-api/routers/wishes.py — source the status enum from the manifest pattern catalog instead of inline list literal. Optional, low-impact.
7. Investigate services/billing-ingest/ — confirm active or archive.
8. files.zip — delete the orphan archive.

4. Pragmatic exceptions (flagged but OK to leave)

• forge-kv-cache/cold.rs:68 (blocks table) — embedded SQLite cache, intentional schema isolation. Now external: git.toilville.dev/Toilville/forge-kv-cache.
• services/ritual-consumer/main.py:80 (retry_queue) — local SQLite resilience buffer when Kafka unavailable
• Flink .sql files (flink-jobs/*.sql) — Kafka streaming virtual tables, not DB DDL
• core/core/tools/*.sql — one-off debug queries
• 11 Dockerfile copies / 6 main.py copies across services — standard polyrepo pattern, not duplication

5. Summary metrics

• 7 SQL DDL violations (3 critical-path runtime, 3 service-local provenance, 1 script)
• 233 hardcoded table-name string references across ~14 services
• 3 hardcoded enum-value sites
• 6 stale .sql files outside migrations (3 OK Flink, 3 ambiguous compliance/utility)
• 1.2 GB reclaimable from agent worktrees
• 1 possibly-orphan service (billing-ingest)
• 0 obviously unused dependencies (sampled)
• 0 retired services
• 17 GB total .spel/ size, of which 2.7 GB is properly-ignored build cache + virtualenv + worktrees