1. Business impact
| Function | Impact if unavailable | Priority |
|---|---|---|
| Decision API used by customers' AI agents | Agents with SDK calls cannot get decisions; the SDK fails closed (blocks), except low-impact actions the customer explicitly allowed during an outage | 1 |
| Approvals and panel | Customers cannot approve actions or change policies | 1 |
| Audit log | Loss would remove customers' evidence | 1 (integrity) |
| Public vendor lookup and marketing site | Lost leads and lookups; no customer data at risk | 2 |
The main threats considered: loss of the VPS or its disk, accidental or malicious deletion of data, compromise of the server, prolonged outage at the hosting provider, and the Founder being unavailable.
2. Recovery targets (honest)
| Scenario | RPO (data loss) | RTO (time to restore) |
|---|---|---|
| Application failure, bad deploy | None | 1 hour (rollback, policy 03) |
| Database corruption or deletion, server intact | Up to 24 hours (last daily dump) | 4 hours |
| Loss of the VPS or its disk | Up to 24 hours (last off-site copy) | 24 hours (target; the data restore itself took about 1 minute in the drill below - the rest is rebuilding the server) |
The database restore was proven by the drill in section 3; the full rebuild of a new server has not been drilled yet (commitment 4).
3. Backups as they exist today (in place)
- On the server: a daily PostgreSQL backup runs at 00:15 (Brasília time) with
pg_dumpin custom format; 14 daily copies are kept on the same disk as the database. - Off-site (since 2026-09-28): at 00:45 (Brasília time) a package with the latest dump, the stored files (vendor evidence, organization logos, audit-chain anchors in
storage/app), the server.envand the database owner credential is encrypted on the server with age and uploaded to Backblaze B2 (region us-east, United States), outside Hostinger. - Encryption: the server holds only the public key; the private key that opens the backups is kept offline by the Founder, in two places, and was removed from the server. A compromise of the server does not expose older backups, and Backblaze cannot read them.
- Immutability and retention: the bucket has Object Lock (compliance mode) with a 30-day default retention: no one, including us, can delete or overwrite a copy during those 30 days. A lifecycle rule hides each copy after 32 days and deletes it one day later, so off-site copies are kept for at most 35 days. The key the server uses is limited to this one bucket and can only write (no read, list or delete).
- Each run is logged in
/var/log/semantiqwall-offsite.log; a run fails (and logs an error) if the newest dump is older than 26 hours. - Restore drill, 2026-09-28 (passed, 59 seconds): with a temporary read-only key that expired after one hour, the latest off-site copy was downloaded from Backblaze, decrypted with the offline private key and restored to a separate database, entirely in memory (
/dev/shm) and deleted afterwards. Results:pg_restorewith 0 errors; 42 of 42 tables; organizations, users, agents, policies, cases and vendors with the same row counts as production; stored files 7 of 7;.envidentical to the server's; audit trail identical to production up to event #218 (production had 2 later events from deploys after the dump). The script isdeploy/restore-drill.sh. - Hostinger backups (extra layer, confirmed 2026-09-28): the hosting provider takes a weekly backup of the whole VPS (schedule "weekly" in the Hostinger panel; 2 copies held). They are managed by Hostinger, stored within Hostinger and not encrypted by us, so they do not replace the off-site copy.
- Not covered: Caddy configuration (small, rebuilt from the repository's setup script). The code is safe in the private GitHub repository.
4. Commitments (owner: Founder)
- Restore drill every quarter with
deploy/restore-drill.sh(next by 2026-12-31), recording the result in the security register. - Alert on a failed off-site run (together with uptime monitoring, policy 04, target 2026-10-31).
- Written rebuild runbook for a new VPS from scratch, using
deploy/setup-server.shanddeploy/first-deploy.sh(target 2026-11-30).
5. Disaster recovery steps
- Declare an incident (policy 05) and set severity.
- Decide: restore in place (server intact) or rebuild (server lost or compromised). A compromised server is always rebuilt, never cleaned.
- Rebuild: create a new Ubuntu 24.04 VPS; apply the server baseline (policy 03); run the setup and first-deploy scripts from the repository; create a new deploy key; restore
.envfrom the encrypted off-site copy (decrypted with the offline private key) and rotate every secret that was on the old server (policy 02). - Restore data:
pg_restorethe most recent good dump; restore vendor evidence files; run migrations; run the audit-chain verification against the anchors from the off-site copy. - Point DNS to the new server; Caddy obtains certificates automatically.
- Check login, MFA, API decisions and approvals; then tell customers the service is back and what, if anything, was lost.
- Post-incident review (policy 05).
6. Communication during a disruption
- Customers are informed by e-mail to each organization owner, sent through the transactional e-mail provider or, if that is down, from contato@semantiqwall.com.
- Commitment: a public status page (owner: Founder, target 2027-03-31).
7. Resilience limits we disclose
- A single VPS in a single datacenter; no standby server and no geographic redundancy. The datacenter is in Brazil (Campinas, São Paulo); the off-site backup copy is in the United States (Backblaze, us-east).
- A single person operates the service. If the Founder is unavailable, no one else can restore it. Commitment: a sealed emergency-access procedure (where the credentials are kept and who may use them) with a trusted person, and a second operator at the first technical hire (target 2027-03-31).
- Physical and environmental controls of the datacenter (power, cooling, fire, access) are Hostinger's responsibility; we have not yet obtained its attestation (policy 09).
8. Testing and review
The plan is exercised at least once a year (a full rebuild drill on a scratch VPS) and after any significant infrastructure change. Results and fixes go in the security register.