Issue 1 - Grafana secret key:
- Replace hardcoded '123456789' secret_key in hosea Grafana config
with an agenix-managed secret ($__file interpolation)
- Add age.secrets.grafana-secret-key declaration to hosea config
- Register grafana-secret-key.age in secrets/secrets.nix (publicKeys=everyone)
- NOTE: The .age file still needs to be created by running:
agenix -e secrets/grafana-secret-key.age
and entering a strong random secret (e.g. from 'pwgen -s 64 1')
Issue 2 - Prometheus stale targets:
- Remove retired vm-gitlab host from all three scrape jobs
(node, systemd, ping) in manifests/monitoring/config.yaml
- linode was already present in all scrape jobs
Issue 3 - Prometheus alerting gaps:
- Wire up rule_files in prometheus.yml pointing at alerts.yml
- Add manifests/monitoring/alerts.yaml with alerting rules:
* HostDown (node unreachable > 2m) - critical
* HighCPULoad (>90% for 10m) - warning
* LowDiskSpace (<10% free) - warning
* CriticalDiskSpace (<5% free) - critical
* HighMemoryUsage (>90% for 10m) - warning
* ResticBackupStaleness (no success > 26h) - warning
* PodCrashLooping - warning
* PodNotReady (>10m) - warning
- Mount alerts ConfigMap as separate volume subPath in deployment
so prometheus.yml and alerts.yml are both accessible at
/etc/prometheus/ without conflicts
This commit is contained in:
@@ -21,6 +21,11 @@ in
|
||||
./hardware-configuration.nix
|
||||
];
|
||||
|
||||
age.secrets.grafana-secret-key = {
|
||||
file = ../../../secrets/grafana-secret-key.age;
|
||||
owner = "grafana";
|
||||
};
|
||||
|
||||
# Bootloader
|
||||
boot = {
|
||||
loader = {
|
||||
@@ -124,7 +129,7 @@ in
|
||||
];
|
||||
};
|
||||
settings = {
|
||||
security.secret_key = "123456789";
|
||||
security.secret_key = "$__file{${config.age.secrets.grafana-secret-key.path}}";
|
||||
server = {
|
||||
domain = "${config.networking.hostName}.shire-zebra.ts.net";
|
||||
enforce_domain = true;
|
||||
|
||||
Reference in New Issue
Block a user