Greg Hellings
2bb0177ffa
chore: add encrypted keys and unencrypted certs
buildbot/nix-eval Build done.
buildbot/nix-build Build started.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.hm-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
2026-04-01 16:33:59 -05:00
root
792217f640
feat: add Nebula mesh network overlay
...
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build Build done.
Introduces a greg.nebula NixOS module and enables it across all managed
hosts for the nebula.thehellings.com overlay (CIDR: 10.157.0.0/16).
Architecture:
- linode: lighthouse + relay (public internet, UDP 4242)
- genesis: regular node + unsafe_routes router for 10.42.0.0/16 (home LAN)
- hosea, isaiah, jeremiah, zeke, exodus: regular nodes with unsafe_routes
pointing to genesis to reach the home LAN
Changes:
- modules/nixos/nebula.nix: new greg.nebula module
- isLighthouse / isRelay options
- unsafeRoutes option (tun.unsafe_routes)
- routesSubnet option: enables IP forwarding + nftables masquerade NAT
on the gateway host (genesis) so Nebula peers reach 10.42.0.0/16
- agenix secret reference per-host (secrets/nebula/<name>.key.age)
- opens UDP/4242 in the firewall
- modules/nixos/default.nix: import nebula.nix
- hosts/unstable/linode/default.nix: greg.nebula.isLighthouse = true
- hosts/unstable/genesis/default.nix: greg.nebula.routesSubnet = "10.42.0.0/16"
- hosts/unstable/{hosea,isaiah,jeremiah,zeke,exodus}/default.nix:
greg.nebula.enable = true with unsafeRoutes via genesis
- network.json: add nebulaIp field for each managed host
- secrets/secrets.nix: declare nebula/<host>.key.age entries
- secrets/nebula/README.md: full PKI bootstrap guide (CA, certs, agenix)
2026-03-28 23:24:52 -05:00
Greg Hellings
1a578ff032
fix: add grafana file and open Grafana port
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
2026-03-25 11:40:00 -05:00
root
ab6e1d24fe
feat: comprehensive observability stack
...
- Add restic_backups and mattermost scrape jobs to Prometheus config
- Add UnPoller deployment+service+secret for UniFi metrics (unpoller.yaml)
- Add unifi scrape job targeting unpoller in-cluster
- Add 4 Grafana dashboards as NixOS-managed etc files on hosea:
- system-health: CPU, memory, disk, network per host
- backup-health: restic backup freshness and duration
- kubernetes: pod counts, restart rates, node resources
- network: DNS queries, DHCP leases, UniFi ports, ping latency
- Add grafana-api-token agenix secret placeholder on hosea
- Open firewall ports 9100/9427/9558 for prometheus exporters in baseline.nix
2026-03-25 09:23:31 -05:00
Greg Hellings
69ebf78d6c
fix: add new agenix file
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
2026-03-25 06:35:20 -05:00
root
7fee882488
fix: security and monitoring improvements
...
buildbot/nix-eval Build done.
Issue 1 - Grafana secret key:
- Replace hardcoded '123456789' secret_key in hosea Grafana config
with an agenix-managed secret ($__file interpolation)
- Add age.secrets.grafana-secret-key declaration to hosea config
- Register grafana-secret-key.age in secrets/secrets.nix (publicKeys=everyone)
- NOTE: The .age file still needs to be created by running:
agenix -e secrets/grafana-secret-key.age
and entering a strong random secret (e.g. from 'pwgen -s 64 1')
Issue 2 - Prometheus stale targets:
- Remove retired vm-gitlab host from all three scrape jobs
(node, systemd, ping) in manifests/monitoring/config.yaml
- linode was already present in all scrape jobs
Issue 3 - Prometheus alerting gaps:
- Wire up rule_files in prometheus.yml pointing at alerts.yml
- Add manifests/monitoring/alerts.yaml with alerting rules:
* HostDown (node unreachable > 2m) - critical
* HighCPULoad (>90% for 10m) - warning
* LowDiskSpace (<10% free) - warning
* CriticalDiskSpace (<5% free) - critical
* HighMemoryUsage (>90% for 10m) - warning
* ResticBackupStaleness (no success > 26h) - warning
* PodCrashLooping - warning
* PodNotReady (>10m) - warning
- Mount alerts ConfigMap as separate volume subPath in deployment
so prometheus.yml and alerts.yml are both accessible at
/etc/prometheus/ without conflicts
2026-03-25 06:02:17 -05:00
Greg Hellings
85a338220c
Rekey with Exodus system key
2026-01-18 23:16:54 -06:00
Greg Hellings
c51e2e5810
fix for Genesis and Isaiah
...
buildbot/nix-eval Build done. (2 warnings)
buildbot/nix-build Build done.
buildbot/nix-effects Build done.
Get gitea-actions running on Isaiah
Fix adblock upate on Genesis
2025-12-30 20:21:16 -06:00
Greg Hellings
8452caf7ac
Rekey from Jeremiah
buildbot/nix-eval Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.hm-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-exodus Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-isaiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-genesis Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-hosea Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-jeremiah Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-linode Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-icdm-root Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-iso Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-zeke Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-vm-gitlab Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.nixos-proxmoxtemplate Build done.
buildbot/nix-build Build done.
buildbot/nix-build gitea:greg/nixos#checks.x86_64-linux.pkg-upgrade-pg-cluster Build done.
2025-12-29 13:36:53 -06:00
Greg Hellings
489bcb7e51
Update for new Gitea instance
2025-12-29 01:50:06 -06:00
Greg Hellings
89a0a01aab
Rekey secrets
2025-12-28 17:43:16 -06:00
Greg Hellings
2910ffce5b
Rekey secrets
2025-12-26 23:29:28 -06:00
Greg Hellings
20e6be10d0
Remove remote workers
2025-12-16 17:11:34 -06:00
Greg Hellings
337accaeeb
Enable buildbot-nix
2025-12-06 23:17:00 -06:00
Greg Hellings
459889d1d2
Add compose and attic configurations
...
Add compose directory to store docker/podman compose files for running
on the TrueNAS box
Add attic configurations to auto-listen on all my home boxes to upload
builds
Update nix-conf to support reading from the Attic server
2025-10-16 17:25:43 -05:00
Greg Hellings
3f587cdf8e
Use restic for backups
...
Now that I have a working Restic backup server, use that for backups
rather than rsyncing to Synology
2025-10-15 00:02:48 -05:00
Greg Hellings
16e98c44b8
Change backups to Restic
2025-10-13 21:09:27 -05:00
Greg Hellings
f9f1357b8e
Rename of Jude to Zeke
...
Since Jude is a now a server, we rename it to Zeke, for Ezekiel
Update of tailscale to auto-register nodes
Update kubernetes to install the etcd application
Update Zeke IP address and DNS bindings
2025-07-02 12:01:27 -05:00
Greg Hellings
676b100e99
Update gitlab-runner configs
2025-06-11 23:54:32 -05:00
Greg Hellings
e13bee750c
Make gitlab use remote postgres
2025-06-11 21:29:20 -05:00
Greg Hellings
d7f53e53a3
Expost postgres cluster
...
Expose the postgres cluster outside of the Kubernetes cluster
Give myself the option of connecting to postgres from my own machines,
so that I can hopefully leverage the improved performance
Update some file linting issues in Nix
Add necessary gitlab keys so it can be updated properly the next time I
deploy to it
Add DNS entries for postgres
Update the configuraiton of Traefik to expose the appropriate ports
2025-05-28 00:10:50 -05:00
Greg Hellings
26d4e9b771
Configure Dendrite on kubernetes
...
Configure a matrix database
Configure Dendrite into Kubernetes
Update vps configuration to new settings
2025-05-26 23:37:24 -05:00
Greg Hellings
ce5f604636
External Secrets setup
...
Setup the external secrets helm charts
Convert hard coded secrets to external secrets
Add BitWarden CLI for serving external secrets
2025-05-26 17:10:01 -05:00
Greg Hellings
ffd2dd78bc
Rework how gitlab-runners are defined
2025-05-21 22:25:11 -05:00
Greg Hellings
4cd0de0109
Isaiah gets new kubernetes runner
2025-05-20 22:51:42 -05:00
Greg Hellings
641c9fd7ac
Rename Jeremiah key
2025-05-20 22:27:28 -05:00
Greg Hellings
d8798bc17d
Remove isaiah-podman and docker-auth gitlab keys
2025-05-20 22:13:31 -05:00
Greg Hellings
82f09cc33e
Enable Kubernetes on Jeremiah
2025-05-19 22:01:49 -05:00
Greg Hellings
fa302301af
Isaiah spin up
...
Bring Isaiah back into commission, this time as the core of my k3s
cluster. Standing up the basic serverInit, and also creating the core
server components that are needed.
2025-05-19 20:10:21 -05:00
Greg Hellings
98efad2780
Rekey secrets to include gitlab
2025-04-14 15:04:54 -05:00
Greg Hellings
0607fc4f14
Enable pushing builds to S3
2025-01-21 16:51:04 -06:00
Greg Hellings
206d22aeb2
Update secrets to add new server
2024-11-20 00:18:52 -06:00
Greg Hellings
395bfc25bc
Finish configuring vm-matrix
2024-11-20 00:11:12 -06:00
Greg Hellings
f6fdeb1789
Mass rename of Myself to Isaiah
2024-11-18 21:11:17 -06:00
Greg Hellings
a71eb7485a
Completely reformat to satisfy nixfmt
2024-10-19 01:20:46 -05:00
Greg Hellings
229188214f
Support for deadnix pre-commit check
2024-10-03 15:21:56 -05:00
Greg Hellings
78c802d32c
A complete reformatting to match nixpkgs-fmt
2024-10-03 11:26:39 -05:00
Greg Hellings
4d4beaf620
Try to migrate minio to NAS
...
Try to move as much of the S3 load onto the NAS as I am able to manage,
incluing pointing s3.thehellings.lan towards the NAS
2024-08-16 22:46:05 -05:00
Greg Hellings
f9b4b7133b
Make backups more fun with syncthing
2024-08-15 04:48:29 +00:00
Greg Hellings
403a765244
Merge branch 'main' of gh:greg-hellings/nixos-config
2024-06-24 23:25:41 -05:00
Greg Hellings
d80b7cb098
Add Jeremiah runner
2024-06-21 13:02:13 -05:00
Greg Hellings
8cfd27d6ca
Update cache for vbox runner
2024-06-20 22:37:14 -05:00
Greg Hellings
f94103e856
Add locally hosted S3 cache
2024-06-20 22:12:38 -05:00
Greg Hellings
7c0d5cf8cd
Adding Jeremiah key
2024-06-20 17:41:11 -05:00
Greg Hellings
1b88076d97
Ceph on Jeremiah
2024-05-07 10:13:51 -05:00
Greg Hellings
4cd66ec1b5
Add gitlab-runner to Linode
2024-05-01 19:46:34 +00:00
Greg Hellings
84b91d086d
Update secrets
2024-04-21 15:03:39 -05:00
Greg Hellings
f787fc93fa
Add Jeremiah
2024-04-16 08:34:06 -05:00
Greg Hellings
338efa36ac
QEmu Working, again...
2024-04-04 13:53:54 -05:00
Greg Hellings
55408c55ee
Improve performance for gitlab-runner
2024-04-03 15:31:02 -05:00