Docs / Operations
Backups and restore
A backup is one complete, consistent image of one database. A restore creates a new database.
Available View Markdown
# Backups and restore
> Status: Available. A backup is one complete, consistent image of one database. A restore creates a new database.
| | |
|---|---|
| **Works** | Whole-database backup while writes continue. Image verification. Scheduled backups with retention and restore drills. Restore into a new database. |
| **Differs** | A restore never overwrites the original. The restored database has a new name, a new ID and a new token. |
| **Missing** | Point-in-time restore. Incremental backups. Backups that skip idle databases. |
| **Operational impact** | A scheduled backup activates each database and copies it in full. |
| **Evidence** | Covered by the project's backup tests. How large a database fits in one backup window has not been measured. |
Put backups in a second bucket, ideally at a second provider. Losing the bucket loses everything since the last finished backup.
## Take a backup
```sh
image="s3:$PWD/backup.env#images/app/$(date -u +%Y%m%dT%H%M%SZ)"
gauntlet backup --source https://n1.example.net:7101/db/app --store "$image"
gauntlet verify-image --store "$image"
```
- Use a new prefix for every backup. A non-empty destination is refused.
- The database keeps serving writes while the backup runs.
- Keep the printed receipt.
- A backup that cannot finish copying in time fails cleanly. It never leaves a partial image marked ready.
## Schedule backups
`gauntlet backups` runs from cron on a backup host. It backs up every database on an interval, applies retention, and runs restore drills.
```sh
*/15 * * * * gauntlet backups run --config /etc/gauntlet/backups.json
30 3 * * * gauntlet backups drill --config /etc/gauntlet/backups.json --cache-dir /srv/gauntlet-cache/drill
0 * * * * gauntlet backups status --config /etc/gauntlet/backups.json >/dev/null
```
The defaults are a daily backup, keeping 7 images and 30 days.
## Restore
A restore creates a **new database with a new name, a new ID and a new token**.
1. Verify the image: `gauntlet verify-image --store "$image"`.
2. Make a working copy of the image if the original must stay an archive. Serving an image turns it into a live cluster.
3. Serve the copy on loopback with `gauntlet node`, wait for `/ready`, then run `gauntlet restore-database --store 's3:#gauntlet/beta' --name app-restored --source http://127.0.0.1:7201`.
4. Get the new token: `gauntlet admin token app-restored`.
5. Check the data, then point clients at the new name and token.
**Do a restore drill before you need one.** An image you have never restored is not yet a backup.
## Coming
- **Restore to a transaction ID** (P10.17): exact, one cut across shards, into a new identity.
- **Backups that skip unchanged databases** without waking them (P10.50).
See [branches and point-in-time recovery](https://gauntlet-db.stavros.xyz/docs/concepts/branches-and-recovery.md).
---
Source: https://gauntlet-db.stavros.xyz/docs/operations/backups
Connect this project to my existing Gauntlet database.
Context:
- Documentation index for agents: https://gauntlet-db.stavros.xyz/llms.txt
- Quickstart: https://gauntlet-db.stavros.xyz/docs/quickstart.md
- SQL compatibility and error codes: https://gauntlet-db.stavros.xyz/docs/sql/compatibility.md
- Database: the one already configured for this project
- Protocol: PostgreSQL wire or the native HTTP JSON API, whichever this project already uses
- Connection values are in environment variables: GAUNTLET_DATABASE_URL (PostgreSQL wire),
GAUNTLET_HTTP_URL (HTTP base URL) and GAUNTLET_DATABASE_TOKEN (the database token).
These names are a convention, not a shipped SDK setting. Keep this project's own names if it has them.
Before editing, read the Gauntlet documentation above and inspect how this project
already talks to its database. Gauntlet is pre-beta software; where the documentation
and my deployment disagree, tell me instead of guessing.
What Gauntlet is:
- A distributed SQL database. SQL is the SQLite family, run by the Turso engine.
- The PostgreSQL wire protocol is a transport only. There is no PostgreSQL SQL dialect,
no pg_catalog or information_schema, no extensions, and no ORM is certified yet.
- libSQL client compatibility is not available yet. Do not assume it.
Connecting:
- PostgreSQL wire: dbname is the database name, the user name is ignored, and the
password is the database token. Authentication is SCRAM-SHA-256. Keep TLS
verification on (sslmode=verify-full with the deployment's CA certificate).
- HTTP: POST <GAUNTLET_HTTP_URL>/db/<name>/sql with a JSON body
{"sql": "...", "params": [...]} and the header "Authorization: Bearer <token>".
POST /db/<name>/tx runs a list of statements atomically.
SQL rules that differ from PostgreSQL:
- Every table needs an explicit primary key, and INSERT supplies the key value.
- CREATE TABLE is a standalone statement outside a transaction. There is no ALTER TABLE,
DROP TABLE or CREATE INDEX; an index comes only from a UNIQUE constraint at creation.
- Parameters are $1, ?1 or ?. Names are single-part (no schema.table).
- Column types follow SQLite affinity. In prepared statements ordinary columns are
described as text; CAST(x AS INTEGER) when the driver needs a number.
Errors you must handle (SQLSTATE):
- 40001: conflict. Roll back and retry the whole transaction. Locking is no-wait, so
this is normal under contention.
- 40003: commit outcome unknown. Do NOT retry. Look the transaction up with
GET /db/<name>/txn/<txid> and replay only if the status is `aborted`.
- 57P03: cannot serve right now. Roll back, then retry the whole transaction with backoff.
- 53300: too many connections. Back off; keep the pool below the node's limit.
Credentials and scope:
- Use only the database token for this one database. Keep it server-side and out of
source, logs, browser bundles and this conversation.
- Never use or ask for a deployment admin token, a client token or bucket credentials.
- Do not create, drop, back up, restore or branch databases, and do not rotate credentials.
Task:
1. Add the smallest connection integration that fits this project.
2. Run one explicit `SELECT 1` connectivity check. It may activate this database if it
is idle; do not probe or wake any other database.
3. Do not change the schema or write application data unless I ask for it.
Report the files changed, the protocol and client used, the result of the check, and
anything this project needs that Gauntlet does not support.- Works
- Whole-database backup while writes continue. Image verification. Scheduled backups with retention and restore drills. Restore into a new database.
- Differs
- A restore never overwrites the original. The restored database has a new name, a new ID and a new token.
- Missing
- Point-in-time restore. Incremental backups. Backups that skip idle databases.
- Operational impact
- A scheduled backup activates each database and copies it in full.
- Evidence
- Covered by the project's backup tests. How large a database fits in one backup window has not been measured.
Put backups in a second bucket, ideally at a second provider. Losing the bucket loses everything since the last finished backup.
#Take a backup
Shell
image="s3:$PWD/backup.env#images/app/$(date -u +%Y%m%dT%H%M%SZ)"
gauntlet backup --source https://n1.example.net:7101/db/app --store "$image"
gauntlet verify-image --store "$image"- Use a new prefix for every backup. A non-empty destination is refused.
- The database keeps serving writes while the backup runs.
- Keep the printed receipt.
- A backup that cannot finish copying in time fails cleanly. It never leaves a partial image marked ready.
#Schedule backups
gauntlet backups runs from cron on a backup host. It backs up every database on an interval, applies retention, and runs restore drills.
Shell
*/15 * * * * gauntlet backups run --config /etc/gauntlet/backups.json
30 3 * * * gauntlet backups drill --config /etc/gauntlet/backups.json --cache-dir /srv/gauntlet-cache/drill
0 * * * * gauntlet backups status --config /etc/gauntlet/backups.json >/dev/nullThe defaults are a daily backup, keeping 7 images and 30 days.
#Restore
A restore creates a new database with a new name, a new ID and a new token.
- Verify the image:
gauntlet verify-image --store "$image". - Make a working copy of the image if the original must stay an archive. Serving an image turns it into a live cluster.
- Serve the copy on loopback with
gauntlet node, wait for/ready, then rungauntlet restore-database --store 's3:#gauntlet/beta' --name app-restored --source http://127.0.0.1:7201. - Get the new token:
gauntlet admin token app-restored. - Check the data, then point clients at the new name and token.
Do a restore drill before you need one. An image you have never restored is not yet a backup.
#Coming
- Restore to a transaction ID (P10.17): exact, one cut across shards, into a new identity.
- Backups that skip unchanged databases without waking them (P10.50).