4 Commits

Include renames

Author	SHA1	Message	Date
klaatuandGreg Hellings	d713e5104c	feat: mount restic-credentials secret into Prometheus	2026-03-27 08:18:53 -05:00
root	7fee882488	fix: security and monitoring improvements ... Issue 1 - Grafana secret key: - Replace hardcoded '123456789' secret_key in hosea Grafana config with an agenix-managed secret ($__file interpolation) - Add age.secrets.grafana-secret-key declaration to hosea config - Register grafana-secret-key.age in secrets/secrets.nix (publicKeys=everyone) - NOTE: The .age file still needs to be created by running: agenix -e secrets/grafana-secret-key.age and entering a strong random secret (e.g. from 'pwgen -s 64 1') Issue 2 - Prometheus stale targets: - Remove retired vm-gitlab host from all three scrape jobs (node, systemd, ping) in manifests/monitoring/config.yaml - linode was already present in all scrape jobs Issue 3 - Prometheus alerting gaps: - Wire up rule_files in prometheus.yml pointing at alerts.yml - Add manifests/monitoring/alerts.yaml with alerting rules: * HostDown (node unreachable > 2m) - critical * HighCPULoad (>90% for 10m) - warning * LowDiskSpace (<10% free) - warning * CriticalDiskSpace (<5% free) - critical * HighMemoryUsage (>90% for 10m) - warning * ResticBackupStaleness (no success > 26h) - warning * PodCrashLooping - warning * PodNotReady (>10m) - warning - Mount alerts ConfigMap as separate volume subPath in deployment so prometheus.yml and alerts.yml are both accessible at /etc/prometheus/ without conflicts	2026-03-25 06:02:17 -05:00
Greg Hellings	08b0ebf05f	Fix Kubernetes access ... Give the pods the proper authorization Clarify the jobs endpoint names	2025-11-22 21:22:51 -06:00
Greg Hellings	82acff5857	Add Prometheus scraping to Kubernetes	2025-11-22 20:15:40 -06:00