klaatu
67f12856e1
fix: set MC_CONFIG_DIR to /tmp/.mc so mc can write config as non-root
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-04-04 18:18:43 +00:00
Greg Hellings
475c0eda64
fix: minio secrets exposed
buildbot/nix-effects Build done.
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-04-04 13:11:31 -05:00
greg
828d61d818
Merge pull request 'feat: backup improvements — AlbyHub Restic, Gitea dump CronJob, Immich → MinIO' ( #17 ) from klaatu/nixos:feature/backup-improvements into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/17
Reviewed-by: greg <gitea@local.domain >
2026-04-04 18:00:16 +00:00
klaatu
0218447008
fix: address review feedback on backup improvements
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
- manifests/gitea/dump-cronjob.yaml: Remove --skip-log, --skip-custom-dir,
--skip-db flags to make backup complete
- manifests/gitea/dump-cronjob.yaml: Replace NFS volume + cleanup container
with S3 upload to backup-gitea bucket using MinIO client (mc).
30-day lifecycle set via mc ilm. Uses minio_key/minio_secret from
existing gitea-config secret.
- hosts/unstable/hosea/default.nix: Replace raw services.restic.backups.albyhub
block with greg.backup.jobs.albyhub using the greg.backup module.
Remove manual age.secrets.restic-env and age.secrets.restic-pw entries
since the greg.backup module declares them.
2026-04-04 02:05:03 -05:00
klaatu
2741be72c7
feat: add update-manifests workflow
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-04-04 06:18:14 +00:00
klaatu
e8beb7f2a9
feat: add update-flake-lock workflow
2026-04-04 06:18:12 +00:00
klaatu
b2728bf1de
feat: add gitea-runner NixOS module
2026-04-04 06:18:10 +00:00
klaatu
d65dcb51a3
feat: add gitea-runner secret placeholder
2026-04-04 06:18:07 +00:00
klaatu
87bc9a21d9
feat: add gitea-runner kustomization
2026-04-04 06:18:04 +00:00
klaatu
ca25dde780
feat: add gitea-runner FluxCD HelmRelease
2026-04-04 06:18:01 +00:00
klaatu
c61900f9e6
feat: add gitea-runner namespace manifest
2026-04-04 06:17:59 +00:00
klaatu
1077d357ca
feat(immich): switch backup to rclone sync → MinIO immich bucket on nas1
2026-04-04 06:13:43 +00:00
klaatu
6b84824a53
feat(gitea): add dump-cronjob to kustomization
2026-04-04 06:13:00 +00:00
klaatu
c74d647ec3
feat(gitea): add daily dump CronJob writing to NAS1 NFS
2026-04-04 06:12:56 +00:00
klaatu
58a5c95c32
feat(hosea): add Restic backup for AlbyHub /chain/alby
2026-04-04 06:12:31 +00:00
Greg Hellings
9a26c75884
chore: allow outgoing webhooks
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-04-03 23:49:27 -05:00
Greg Hellings
276b939252
fix: expose buildbot
2026-04-03 23:49:12 -05:00
greg
3eb44f2958
Merge pull request 'feat: expose Buildbot via nginx proxy on LAN' ( #13 ) from klaatu/nixos:feat/buildbot-proxy into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/13
2026-04-04 03:45:51 +00:00
klaatu
9bbdb74f85
fix: merge conflict — keep both nebula.enable and buildbot proxies
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-04-03 22:18:17 -05:00
greg
35b6058c09
Merge pull request 'feat: Nebula mesh network overlay' ( #12 ) from klaatu/nixos:feat/nebula-mesh into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/12
2026-04-04 03:03:53 +00:00
Greg Hellings
88d9234332
fix: no default routes
2026-04-02 08:11:32 -05:00
root
eabead05ab
feat: expose Buildbot via nginx proxy on LAN
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
Add greg.proxies entries on jeremiah for buildbot.home and
buildbot.thehellings.lan, both proxying to http://localhost:8010/ .
Nginx handles auth passthrough - Buildbot's own Gitea OAuth applies.
Add DNS aliases in genesis hosts file:
- buildbot.thehellings.lan → 10.42.1.8 (LAN)
- buildbot.home → 100.102.186.39 (Tailscale)
2026-04-01 17:31:38 -05:00
Greg Hellings
a1bef073cf
fix: generate linode certs
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-04-01 17:20:46 -05:00
root
f35e8c755c
fix: only set listen.port on lighthouse/relay nodes (regular nodes use OS-assigned port 0)
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-proxmoxtemplate Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build Build done.
2026-04-01 17:14:12 -05:00
root
71ae25a58c
fix: move unsafe_routes to settings.tun.unsafe_routes (correct NixOS nebula option path)
buildbot/nix-eval Build done.
2026-04-01 17:13:37 -05:00
root
0e61efe5b5
fix: rename tun.dev to tun.device (correct NixOS option name)
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-build Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
2026-04-01 17:07:47 -05:00
root
84e4c68f0e
fix: move unsafeRoutes to module default, genesis overrides to []
...
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build Build done.
Per review feedback:
- nebula module now defaults unsafeRoutes to [{route=10.42.0.0/16 via=10.157.0.2}]
so all regular nodes get home LAN routing automatically
- genesis overrides unsafeRoutes=[] since it IS the routing node (avoids loop)
- exodus and all k3s nodes (hosea, isaiah, jeremiah, zeke) simplified to
nebula.enable = true only, relying on the new default
2026-04-01 16:45:59 -05:00
greg
559155d411
Merge branch 'main' into feat/nebula-mesh
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build Build done.
2026-04-01 21:36:08 +00:00
Greg Hellings
2bb0177ffa
chore: add encrypted keys and unencrypted certs
buildbot/nix-eval Build done.
buildbot/nix-build Build started.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.hm-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
2026-04-01 16:33:59 -05:00
Greg Hellings
15ed4ae5f7
chore: allow click.discord.com
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-04-01 15:49:32 -05:00
root
792217f640
feat: add Nebula mesh network overlay
...
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build Build done.
Introduces a greg.nebula NixOS module and enables it across all managed
hosts for the nebula.thehellings.com overlay (CIDR: 10.157.0.0/16).
Architecture:
- linode: lighthouse + relay (public internet, UDP 4242)
- genesis: regular node + unsafe_routes router for 10.42.0.0/16 (home LAN)
- hosea, isaiah, jeremiah, zeke, exodus: regular nodes with unsafe_routes
pointing to genesis to reach the home LAN
Changes:
- modules/nixos/nebula.nix: new greg.nebula module
- isLighthouse / isRelay options
- unsafeRoutes option (tun.unsafe_routes)
- routesSubnet option: enables IP forwarding + nftables masquerade NAT
on the gateway host (genesis) so Nebula peers reach 10.42.0.0/16
- agenix secret reference per-host (secrets/nebula/<name>.key.age)
- opens UDP/4242 in the firewall
- modules/nixos/default.nix: import nebula.nix
- hosts/unstable/linode/default.nix: greg.nebula.isLighthouse = true
- hosts/unstable/genesis/default.nix: greg.nebula.routesSubnet = "10.42.0.0/16"
- hosts/unstable/{hosea,isaiah,jeremiah,zeke,exodus}/default.nix:
greg.nebula.enable = true with unsafeRoutes via genesis
- network.json: add nebulaIp field for each managed host
- secrets/secrets.nix: declare nebula/<host>.key.age entries
- secrets/nebula/README.md: full PKI bootstrap guide (CA, certs, agenix)
2026-03-28 23:24:52 -05:00
greg
ba1914e48c
Merge pull request 'feat: add top-5 fullest PVs panel to Kubernetes dashboard' ( #11 ) from klaatu/nixos:feat/k8s-volume-dashboard into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/11
2026-03-27 15:25:44 +00:00
root
07b3601206
feat: add top-5 fullest PVs panel to Kubernetes dashboard
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
Add a horizontal bar gauge showing the disk usage % of the 5 most-full
PersistentVolumes in the cluster, using kubelet volume stats already
scraped by Prometheus:
topk(5, kubelet_volume_stats_used_bytes
/ kubelet_volume_stats_capacity_bytes * 100)
Labels show namespace/PVC name. Color thresholds: green <70%, yellow
70–90%, red >90%. Placed at the bottom of the dashboard (y=46).
Bump dashboard version to 3.
2026-03-27 09:06:19 -05:00
Greg Hellings
93458eb6d4
chore: slight updates to Network dashboard
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-03-27 09:03:07 -05:00
greg
6f7b3c238e
Merge pull request 'fix: correct unpoller metric names in Network & UniFi dashboard' ( #10 ) from klaatu/nixos:fix/network-dashboard-metrics into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/10
Reviewed-by: greg <gitea@local.domain >
2026-03-27 13:51:18 +00:00
root
de0b8fb07e
fix: correct unpoller metric names in network dashboard
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
All metrics had wrong prefix (unifipoller_ → unpoller_) and several
had incorrect names entirely. Corrected against unpoller master source
(pkg/promunifi/):
- unifipoller_device_uptime_seconds → unpoller_device_uptime_seconds
- unifipoller_port_receive_bytes_total → unpoller_device_port_receive_bytes_total
(port metrics live under device_ namespace: unpoller_device_port_*)
- unifipoller_port_transmit_bytes_total → unpoller_device_port_transmit_bytes_total
- unifipoller_device_wan_receive_bytes_total → unpoller_wan_max_rx_bytes_rate
(no per-device WAN bytes total counter; WAN ns is unpoller_wan_*;
rate metrics are already instantaneous gauges, no rate() wrapper)
- unifipoller_device_wan_transmit_bytes_total → unpoller_wan_max_tx_bytes_rate
- unifipoller_client_wifi_tx_rate_bps → count(unpoller_client_uptime_seconds{wired="false"})
(no wifi-specific rate metric; wired label distinguishes client types)
- unifipoller_client_wired_tx_rate_bps → count(unpoller_client_uptime_seconds{wired="true"})
- unifipoller_client_receive_bytes_total → unpoller_client_receive_bytes_total
- unifipoller_client_transmit_bytes_total → unpoller_client_transmit_bytes_total
Bump dashboard version to 3.
2026-03-27 08:44:12 -05:00
greg
3111548c99
Merge pull request 'feat: kube-state-metrics, Restic ExternalSecret, updated dashboards' ( #9 ) from klaatu/nixos:fix/observability-complete into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/9
Reviewed-by: greg <gitea@local.domain >
2026-03-27 13:27:30 +00:00
Greg Hellings
841224b20f
fix: update external-secrets apiVersion
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-03-27 08:27:03 -05:00
klaatu and Greg Hellings
45536f4a8d
feat: update kubernetes and network Grafana dashboards
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-03-27 08:19:45 -05:00
klaatu and Greg Hellings
4717763fed
feat: add restic ExternalSecret for Prometheus basic auth
2026-03-27 08:19:45 -05:00
klaatu and Greg Hellings
53f33eb13a
feat: add kube-state-metrics
2026-03-27 08:19:45 -05:00
klaatu and Greg Hellings
b18807b1ef
feat: add kube-state-metrics.yaml and restic-secret.yaml to kustomization
2026-03-27 08:19:45 -05:00
klaatu and Greg Hellings
2c0b4b1472
feat: add kube_state_metrics job, restic basic_auth, remove kea job
2026-03-27 08:19:40 -05:00
klaatu and Greg Hellings
d713e5104c
feat: mount restic-credentials secret into Prometheus
2026-03-27 08:18:53 -05:00
Greg Hellings
e9a4662620
fix: open kubelet/cadvisor prometheus ports
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-03-26 05:58:52 -05:00
Greg Hellings
eba1a0c1ca
fix: make restic https
2026-03-26 05:56:21 -05:00
Greg Hellings
4eb5d6873a
chore: remove unused kea configuration
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-03-26 05:45:16 -05:00
greg
71f553a6f2
Merge pull request 'fix: add missing shire-zebra.ts.net DNS entries; update exodus Tailscale IP' ( #8 ) from klaatu/nixos:fix/tailscale-dns-entries into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/8
2026-03-26 02:48:34 +00:00
root
8e218a71e0
fix: add missing shire-zebra.ts.net DNS entries; update exodus Tailscale IP
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
genesis, zeke, linode, and exodus were missing shire-zebra.ts.net
aliases in the genesis dnsmasq hosts file. Only hosea, isaiah,
jeremiah, gitlab, matrix, chronicles, and nas1 had them. Since
genesis is the DNS server for the whole network, Prometheus (and
everything else) couldn't resolve those four hostnames, causing
instant connection failures on scrape.
Also update exodus Tailscale IP from 100.80.99.48 to 100.70.99.91
(stale IP in both network.json and the hosts file).
2026-03-25 21:12:51 -05:00
greg
fe65f34c05
Merge pull request 'fix: disable Grafana enforce_domain to allow LAN proxy access' ( #6 ) from klaatu/nixos:fix/grafana-enforce-domain into main
...
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/6
2026-03-25 22:42:16 +00:00