632 Commits

Include renames

Author	SHA1	Message	Date
Greg Hellings	7579b2b7a1	fix: Grafana network port	2026-03-25 12:24:03 -05:00
Greg Hellings	1a578ff032	fix: add grafana file and open Grafana port	2026-03-25 11:40:00 -05:00
root	4eebf07f6b	fix: scope prometheus exporter firewall rules to home hosts only ... Move allowedTCPPorts for node/ping/systemd/blackbox exporters from hosts/baseline.nix (applies to ALL hosts including linode) into modules/nixos/home.nix (applies only to greg.home = true hosts). This prevents ports 9100/9115/9427/9558 from being opened on linode, which is internet-facing. All LAN hosts set greg.home = true and will still get the ports opened; linode sets greg.home = false and will not.	2026-03-25 09:46:26 -05:00
root	df2f9d69fe	fix: address PR feedback + expand infrastructure coverage ... - Replace unpoller-credentials plain Secret with ExternalSecret pulling from Bitwarden item 15bfc957-5de5-49b2-ab6c-b41800e71564 via bitwarden-login ClusterSecretStore (username + password) - Pull UP_UNIFI_DEFAULT_USER from secret as well (was hardcoded) - Add OpenWRT access point (10.42.0.3) to genesis DNS hosts file as ap.thehellings.lan (discovered during network scan) - Add blackbox exporter to genesis (port 9115, ICMP module) for probing non-NixOS infrastructure that has no node_exporter: OpenWRT AP, Joel, pve1, chronicles, nas1, odoo, mattermost, homeassistant, USW-Pro-HD-24 UniFi switch - Add infra_ping scrape job to Prometheus using blackbox prober targeting all non-NixOS LAN hosts via genesis blackbox exporter	2026-03-25 09:25:16 -05:00
root	ab6e1d24fe	feat: comprehensive observability stack ... - Add restic_backups and mattermost scrape jobs to Prometheus config - Add UnPoller deployment+service+secret for UniFi metrics (unpoller.yaml) - Add unifi scrape job targeting unpoller in-cluster - Add 4 Grafana dashboards as NixOS-managed etc files on hosea: - system-health: CPU, memory, disk, network per host - backup-health: restic backup freshness and duration - kubernetes: pod counts, restart rates, node resources - network: DNS queries, DHCP leases, UniFi ports, ping latency - Add grafana-api-token agenix secret placeholder on hosea - Open firewall ports 9100/9427/9558 for prometheus exporters in baseline.nix	2026-03-25 09:23:31 -05:00
greg	0050f866bf	Merge pull request 'fix: security and monitoring improvements (issues 1-3)' (#1) from klaatu/nixos:fix/security-and-monitoring into main ... Reviewed-on: https://src.thehellings.com/greg/nixos/pulls/1	2026-03-25 12:09:28 +00:00
Greg Hellings	0f8d15a41c	feat: Enable prowlarr	2026-03-25 06:28:27 -05:00
root	7fee882488	fix: security and monitoring improvements ... Issue 1 - Grafana secret key: - Replace hardcoded '123456789' secret_key in hosea Grafana config with an agenix-managed secret ($__file interpolation) - Add age.secrets.grafana-secret-key declaration to hosea config - Register grafana-secret-key.age in secrets/secrets.nix (publicKeys=everyone) - NOTE: The .age file still needs to be created by running: agenix -e secrets/grafana-secret-key.age and entering a strong random secret (e.g. from 'pwgen -s 64 1') Issue 2 - Prometheus stale targets: - Remove retired vm-gitlab host from all three scrape jobs (node, systemd, ping) in manifests/monitoring/config.yaml - linode was already present in all scrape jobs Issue 3 - Prometheus alerting gaps: - Wire up rule_files in prometheus.yml pointing at alerts.yml - Add manifests/monitoring/alerts.yaml with alerting rules: * HostDown (node unreachable > 2m) - critical * HighCPULoad (>90% for 10m) - warning * LowDiskSpace (<10% free) - warning * CriticalDiskSpace (<5% free) - critical * HighMemoryUsage (>90% for 10m) - warning * ResticBackupStaleness (no success > 26h) - warning * PodCrashLooping - warning * PodNotReady (>10m) - warning - Mount alerts ConfigMap as separate volume subPath in deployment so prometheus.yml and alerts.yml are both accessible at /etc/prometheus/ without conflicts	2026-03-25 06:02:17 -05:00
Greg Hellings	e21ed89cfe	fix: use upstream kiwix-server	2026-03-10 16:52:27 -05:00
Greg Hellings	01ce9d74da	chore: update flake lock	2026-03-10 08:46:10 -05:00
Greg Hellings	b1bfb337d6	chore: reduce eval workers on Jeremiah to reduce OOM	2026-03-04 10:53:12 -06:00
Greg Hellings	d7c6f23280	chore: radarr installed	2026-03-04 10:53:12 -06:00
Greg Hellings	46138b90c2	chore: enable transmission on hosea	2026-03-04 10:53:12 -06:00
Greg Hellings	426a85e357	chore: remove GUI/steam from jeremiah	2026-02-19 19:56:15 -06:00
Greg Hellings	e514982fb6	chore: remove proxmox from jeremiah	2026-02-19 19:55:13 -06:00
Greg Hellings	c563b57974	opt: remove dead code	2026-02-19 19:50:28 -06:00
Greg Hellings	61d4f4872e	chore: silence deprecation warning	2026-02-19 19:49:49 -06:00
Greg Hellings	c6b973738b	fix: paths to secrets files because of new folder structure	2026-02-19 08:06:54 -06:00
Greg Hellings	1e3117cd30	fix: wsl builds updated to latest	2026-02-17 17:04:11 -06:00
Greg Hellings	e2181f124a	chore: dynamic hosts, builder targets ... Make hosts read from directories, to limit manual changes Rename vm-gitlab -> gitlab A few reformatting and lint changes due to altered files Create builder targets for Darwin builders	2026-02-17 16:11:29 -06:00
Greg Hellings	22beae0368	fix: remove deprecated refs	2026-02-11 09:18:22 -06:00
Greg Hellings	3489609266	Update isaiah runner capacity	2026-02-02 10:41:41 -06:00
Greg Hellings	f58ef063be	Add nfs-utils everywhere for mounting	2026-02-02 10:41:04 -06:00
Greg Hellings	e5d0db1eac	Force pull the act_runner containers	2026-01-25 21:16:13 -06:00
Greg Hellings	8e75fc2243	Require network-online for adblock script	2026-01-08 10:54:08 -06:00
Greg Hellings	c51e2e5810	fix for Genesis and Isaiah ... Get gitea-actions running on Isaiah Fix adblock upate on Genesis	2025-12-30 20:21:16 -06:00
Greg Hellings	125c898c72	Update DNS entry for genesis	2025-12-29 17:25:35 -06:00
Greg Hellings	83e58d8015	Remove gitlab-runner from linode	2025-12-29 01:57:59 -06:00
Greg Hellings	489bcb7e51	Update for new Gitea instance	2025-12-29 01:50:06 -06:00
Greg Hellings	8e602da466	Expose new node drive to longhorn	2025-12-28 23:40:19 -06:00
Greg Hellings	8c99a92d77	Use metadata in more places	2025-12-28 21:18:13 -06:00
Greg Hellings	c5cff5667d	Update Genesis for hardware support	2025-12-26 23:50:53 -06:00
Greg Hellings	ca05aed2ff	A few minor tweaks for network settings	2025-12-26 23:06:36 -06:00
Greg Hellings	925d0018e6	Move pubkey info to network.json	2025-12-26 23:06:03 -06:00
Greg Hellings	4db5b1b2a6	Slight formatting fix in Jeremiah	2025-12-25 22:54:13 -06:00
Greg Hellings	44ea82fe2c	Remove btc and use Albyhub	2025-12-23 18:13:25 -06:00
Greg Hellings	856d4ec627	Fix building with Colmena. ... Apply overlays and nixpkgs imports once Do not set hostPlatform explicitly, set it from the network.json file	2025-12-22 15:27:36 -06:00
Greg Hellings	36526b4562	Simplify nixpkgs eval	2025-12-18 10:51:09 -06:00
Greg Hellings	20e6be10d0	Remove remote workers	2025-12-16 17:11:34 -06:00
Greg Hellings	cca9b6bebe	Only use as remote builders	2025-12-16 16:59:05 -06:00
Greg Hellings	553957c35e	Move more data into metadata	2025-12-16 16:48:11 -06:00
Greg Hellings	1a528f10de	Simplify nixpkgs eval	2025-12-16 09:07:55 -06:00
Greg Hellings	c77c4a8ff6	Add more buildbot listeners	2025-12-16 08:15:37 -06:00
Greg Hellings	854aed86e0	Updates to Flakes and Zeke ... Add /dev/sda1 to Zeke Update Flakes Add initial colmena support Add colmena to shells Update kubernetes pins	2025-12-15 22:18:29 -06:00
Greg Hellings	ee3467b2ee	Update to Nextcloud32	2025-12-10 08:38:56 -06:00
Greg Hellings	c446dcbaa7	Increase memory but limited workers for eval	2025-12-10 08:22:21 -06:00
Greg Hellings	2cb2e2f3d1	Increase memory for eval	2025-12-10 07:45:11 -06:00
Greg Hellings	01e6aa22d5	Simplify evaluations a bit	2025-12-07 17:50:55 -06:00
Greg Hellings	fa555f0298	Return 444 for AI bots	2025-12-07 16:27:06 -06:00
Greg Hellings	e5a3c24058	Do not register roots	2025-12-07 15:41:10 -06:00