feat: backup improvements (AlbyHub restic, Gitea dump, Immich MinIO sync) #16

Closed
klaatu wants to merge 4 commits from backup-improvements into main
Collaborator

Implements three backup improvements from issue #14:

1. AlbyHub Restic Backup (Hosea)

  • Adds services.restic.backups.albyhub to hosts/unstable/hosea/default.nix
  • Backs up /chain/alby to rest:https://nas1.shire-zebra.ts.net:30248/albyhub
  • Stops AlbyHub before backup and restarts after for LDK state consistency
  • Uses agenix secret albyhub-restic-password for repo password
  • Prunes snapshots: 7 daily, 4 weekly, 3 monthly
  • Action required: Run agenix -e secrets/albyhub-restic-password.age before deploying

2. Gitea Dump CronJob (Kubernetes)

  • Adds manifests/gitea/dump-cronjob.yaml
  • Runs daily at 2 AM (offset from 4 AM Immich backup)
  • Mounts Gitea data PVC (read-only) and NFS share nas1.shire-zebra.ts.net:/mnt/all/backups/gitea-dumps
  • Runs gitea dump as zip, then cleans up dumps older than 30 days
  • Added to manifests/gitea/kustomization.yaml

3. Immich MinIO Sync (Kubernetes)

  • Updates manifests/immich/backup.yaml
  • Adds ExternalSecret immich-minio-creds pulling from Bitwarden item dcbcf704-7dce-48d7-bbd1-b3a801875b3d
  • CronJob now uses init container (immich-go) to stage files into emptyDir, then main container (rclone/rclone:latest) syncs staged files to s3:immich on nas1 MinIO (http://nas1.shire-zebra.ts.net:9000)
  • Removes NFS write dependency for the photo archive

Closes #14

Implements three backup improvements from issue #14: ## 1. AlbyHub Restic Backup (Hosea) - Adds `services.restic.backups.albyhub` to `hosts/unstable/hosea/default.nix` - Backs up `/chain/alby` to `rest:https://nas1.shire-zebra.ts.net:30248/albyhub` - Stops AlbyHub before backup and restarts after for LDK state consistency - Uses agenix secret `albyhub-restic-password` for repo password - Prunes snapshots: 7 daily, 4 weekly, 3 monthly - **Action required:** Run `agenix -e secrets/albyhub-restic-password.age` before deploying ## 2. Gitea Dump CronJob (Kubernetes) - Adds `manifests/gitea/dump-cronjob.yaml` - Runs daily at 2 AM (offset from 4 AM Immich backup) - Mounts Gitea data PVC (read-only) and NFS share `nas1.shire-zebra.ts.net:/mnt/all/backups/gitea-dumps` - Runs `gitea dump` as zip, then cleans up dumps older than 30 days - Added to `manifests/gitea/kustomization.yaml` ## 3. Immich MinIO Sync (Kubernetes) - Updates `manifests/immich/backup.yaml` - Adds ExternalSecret `immich-minio-creds` pulling from Bitwarden item `dcbcf704-7dce-48d7-bbd1-b3a801875b3d` - CronJob now uses init container (`immich-go`) to stage files into emptyDir, then main container (`rclone/rclone:latest`) syncs staged files to `s3:immich` on nas1 MinIO (`http://nas1.shire-zebra.ts.net:9000`) - Removes NFS write dependency for the photo archive Closes #14
klaatu added 4 commits 2026-04-04 06:10:05 +00:00
Author
Collaborator

Reviewed — looks solid. A few notes:

AlbyHub Restic (hosea)

  • Stop/start wrapper for LDK consistency is the right call
  • rest:https:// with the Tailscale hostname is correct
  • ⚠️ The backupPrepareCommand / backupCleanupCommand run as root — systemctl stop/start will work, but if the backup itself fails mid-run, backupCleanupCommand still fires (that is the intended NixOS restic behavior), so AlbyHub will come back up regardless. That is fine.
  • ⚠️ Action item before deploy: agenix -e secrets/albyhub-restic-password.age — noted in the PR body, just confirming I see it
  • Minor: RandomizedDelaySec = "1h" means the backup could run anywhere from 00:00–01:00. That is fine for daily pruning cadence.

Gitea dump CronJob

  • Read-only PVC mount is the right safety choice
  • find -mtime +30 -delete cleanup is clean
  • ⚠️ The image is pinned to gitea/gitea:1.25.4 — worth keeping this in sync with whatever version the Helm chart deploys, otherwise gitea dump -c /data/gitea/conf/app.ini may complain about config schema mismatches. Consider using the same image tag variable or a latest-tracking tag if you do not pin the chart version.
  • ⚠️ securityContext.runAsUser: 1000 — verify that uid 1000 in the gitea container has read access to /data/gitea/conf/app.ini and write access to the NFS share. The Gitea Helm chart typically runs as uid 1000 (git user), so this should be fine, but worth a quick check on the NFS export permissions.

Immich → MinIO sync

  • ExternalSecret pulling from the same Bitwarden item as Gitea MinIO creds — clean reuse
  • init container staging pattern avoids writing to NFS, good
  • ⚠️ rclone/rclone:latest — unpinned image in production. Not a blocker, but consider pinning to a digest or tag for reproducibility
  • ⚠️ First run will pull all photos into the emptyDir stage volume before rclone syncs. Depending on your photo library size, the pod may hit default ephemeral storage limits. If Immich has a lot of data, you may want to add resources.limits.ephemeral-storage or use a larger PVC instead of emptyDir for staging.
  • ℹ️ --s3-no-check-bucket skips bucket existence check — fine if the immich bucket already exists on nas1 MinIO.

Overall: LGTM with the caveats above. The agenix secret creation is the only hard blocker before deploying to Hosea. Everything else is advisory. Merging when you are ready. 🌌

Reviewed — looks solid. A few notes: **AlbyHub Restic (hosea)** - ✅ Stop/start wrapper for LDK consistency is the right call - ✅ `rest:https://` with the Tailscale hostname is correct - ⚠️ The `backupPrepareCommand` / `backupCleanupCommand` run as root — `systemctl stop/start` will work, but if the backup itself fails mid-run, `backupCleanupCommand` still fires (that is the intended NixOS restic behavior), so AlbyHub will come back up regardless. That is fine. - ⚠️ **Action item before deploy:** `agenix -e secrets/albyhub-restic-password.age` — noted in the PR body, just confirming I see it - Minor: `RandomizedDelaySec = "1h"` means the backup could run anywhere from 00:00–01:00. That is fine for daily pruning cadence. **Gitea dump CronJob** - ✅ Read-only PVC mount is the right safety choice - ✅ `find -mtime +30 -delete` cleanup is clean - ⚠️ The image is pinned to `gitea/gitea:1.25.4` — worth keeping this in sync with whatever version the Helm chart deploys, otherwise `gitea dump -c /data/gitea/conf/app.ini` may complain about config schema mismatches. Consider using the same image tag variable or a `latest`-tracking tag if you do not pin the chart version. - ⚠️ `securityContext.runAsUser: 1000` — verify that uid 1000 in the gitea container has read access to `/data/gitea/conf/app.ini` and write access to the NFS share. The Gitea Helm chart typically runs as uid 1000 (git user), so this should be fine, but worth a quick check on the NFS export permissions. **Immich → MinIO sync** - ✅ ExternalSecret pulling from the same Bitwarden item as Gitea MinIO creds — clean reuse - ✅ init container staging pattern avoids writing to NFS, good - ⚠️ `rclone/rclone:latest` — unpinned image in production. Not a blocker, but consider pinning to a digest or tag for reproducibility - ⚠️ First run will pull all photos into the `emptyDir` stage volume before rclone syncs. Depending on your photo library size, the pod may hit default ephemeral storage limits. If Immich has a lot of data, you may want to add `resources.limits.ephemeral-storage` or use a larger PVC instead of emptyDir for staging. - ℹ️ `--s3-no-check-bucket` skips bucket existence check — fine if the `immich` bucket already exists on nas1 MinIO. **Overall:** LGTM with the caveats above. The agenix secret creation is the only hard blocker before deploying to Hosea. Everything else is advisory. Merging when you are ready. 🌌
Author
Collaborator

Closing in favor of PR #17, which is the stronger implementation. Key reasons:

Restic secrets: #17 reuses the existing restic-env.age / restic-pw.age secrets (no new secret to create before deploying). #16 required a brand-new albyhub-restic-password.age that does not exist yet — a hard blocker I flagged in my earlier review.

Gitea dump CronJob: #17 separates the dump into an init container and cleanup into a main container (cleaner separation of concerns), uses --skip-log --skip-custom-dir --skip-db flags for a lighter repo-only dump, and retains the 30 most recent dumps by count rather than by mtime (more predictable). Also includes successfulJobsHistoryLimit / failedJobsHistoryLimit.

Immich backup: #17 adds a proper 400Gi Longhorn PVC for staging (vs emptyDir in #16 which could hit ephemeral storage limits), bakes the rclone config into the Secret as rclone.conf (cleaner than passing credentials as env vars), pins rclone to 1.68 (vs latest), and adds --transfers 4 --checkers 8 for reasonable parallelism.

Overall #17 is more production-ready. Closing this one.

Closing in favor of PR #17, which is the stronger implementation. Key reasons: **Restic secrets:** #17 reuses the existing `restic-env.age` / `restic-pw.age` secrets (no new secret to create before deploying). #16 required a brand-new `albyhub-restic-password.age` that does not exist yet — a hard blocker I flagged in my earlier review. **Gitea dump CronJob:** #17 separates the dump into an init container and cleanup into a main container (cleaner separation of concerns), uses `--skip-log --skip-custom-dir --skip-db` flags for a lighter repo-only dump, and retains the 30 most recent dumps by count rather than by mtime (more predictable). Also includes `successfulJobsHistoryLimit` / `failedJobsHistoryLimit`. **Immich backup:** #17 adds a proper 400Gi Longhorn PVC for staging (vs emptyDir in #16 which could hit ephemeral storage limits), bakes the rclone config into the Secret as `rclone.conf` (cleaner than passing credentials as env vars), pins rclone to `1.68` (vs `latest`), and adds `--transfers 4 --checkers 8` for reasonable parallelism. Overall #17 is more production-ready. Closing this one.
klaatu closed this pull request 2026-04-04 06:19:45 +00:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: greg/nixos#16