gem

Storage backends

How stores work

Register one store per destination. The dump is streamed to all of them at once from a single pg_dump process, so adding a second destination costs upload bandwidth and no extra load on the database.

The :s3 and :r2 stores need the aws-sdk-s3 gem. It stays out of the gem's dependencies, so add it yourself. The :file store needs nothing extra.

$ bundle add aws-sdk-s3

S3

config.register(:s3) do |store|
  store.bucket        = "db-backups"
  store.region        = "eu-central-1"
  store.storage_class = :standard_ia
end

setting	env	notes
bucket	n/a	required
region	AWS_REGION	required by AWS, defaults to `auto` on R2
access_key_id	AWS_ACCESS_KEY_ID	leave blank to use the default AWS credential chain
secret_access_key	AWS_SECRET_ACCESS_KEY	must be set or blank together with the key id
storage_class	BB_S3_STORAGE_CLASS	defaults to `standard_ia`
part_size	BB_S3_PART_SIZE	multipart chunk in bytes, defaults to 8 MB
thread_count	BB_S3_THREAD_COUNT	parallel part uploads, defaults to 2
endpoint	n/a	set it to point the S3 store at MinIO, Spaces or any compatible service

standard_ia is the default because backups are written once and read almost never, which is exactly what infrequent-access pricing is for. Any of standard, onezone_ia, intelligent_tiering, glacier_ir, glacier, deep_archive, reduced_redundancy, express_onezone, outposts and snow are accepted. An unrecognised value fails validation before anything uploads. Set it to an empty string to use the bucket default.

Glacier classes are cheap to write and slow to read. A backup you cannot restore within your recovery window is not much of a backup, so check the retrieval time before choosing one.

Cloudflare R2

R2 is the S3 store with two differences: endpoint is required, and region defaults to auto.

config.register(:r2) do |store|
  store.endpoint          = "https://<account-id>.r2.cloudflarestorage.com"
  store.bucket            = "db-backups"
  store.access_key_id     = ENV["R2_ACCESS_KEY_ID"]
  store.secret_access_key = ENV["R2_SECRET_ACCESS_KEY"]
end

No egress fees make R2 a good second destination. Generate the access key pair in the Cloudflare dashboard under R2, API tokens.

Local file

The :file store writes the dump to a directory instead of an object store. It needs no extra gem and takes a single required setting, path, with no environment default.

config.register(:file) do |store|
  store.path = "/mnt/backups"
end

The dump is written to path under the same database/YYYY/MM/DD-HHMMSS.dump key the object stores use. The directory tree is created on the first run. A dump under min_size is rejected and the partial file removed, as with the other stores.

Credentials

The initializer is committed, so keep secrets out of it. Two supported approaches:

Leave access_key_id and secret_access_key unset. The AWS SDK then uses its default credential chain, which picks up an IAM instance role or task role with no secrets on disk at all.
Read them from the environment of the process that runs the backup, as the R2 example above does.

Setting only one of the pair fails validation. A half-configured credential otherwise turns into a confusing permission error at 3am.

The bucket policy needs PutObject for uploads and DeleteObject so the gem can remove a partial upload after a failed dump.

Several stores at once

config.register(:s3) do |store|
  store.bucket = "db-backups"
  store.region = "eu-central-1"
end

config.register(:r2) do |store|
  store.endpoint = "https://<account-id>.r2.cloudflarestorage.com"
  store.bucket   = "db-backups"
end

The dump is read once and fanned out to every store in parallel, each on its own thread. Every store gets the same key.

Stores fail independently. If one upload fails and another succeeds, the run reports partial_success, exits non-zero, and the successful copy stays where it is. You still have a backup, and you still hear about the destination that broke.

When a dump fails

If pg_dump itself dies partway, everything already uploaded is a truncated copy of a broken stream. The gem deletes those objects rather than leave a file that looks like a backup and restores like garbage. The same cleanup runs when a store rejects a dump for being under min_size.

Tuning uploads

config.register(:s3) do |store|
  store.bucket       = "db-backups"
  store.region       = "eu-central-1"
  store.part_size    = 16 * 1024 * 1024
  store.thread_count = 4
end

Larger parts and more threads help on a fast link with a large database. Both cost memory, roughly part size times thread count per store, so raise them deliberately on a small instance.

gem