fix: use LAN IPs for non-k8s hosts in Prometheus scrape config #7

Closed
klaatu wants to merge 1 commits from klaatu/nixos:fix/prometheus-scrape-addresses into main
Collaborator

Root cause

Prometheus runs as a pod and only has Tailscale routes for the k8s nodes it runs on (isaiah, jeremiah, zeke) and hosea. Scraping genesis, exodus, and linode via *.shire-zebra.ts.net fails instantly (~1ms, connection refused) because those Tailscale routes do not exist inside the cluster network namespace.

This was visible in the metrics: working hosts showed 100-230ms scrape durations (real network round-trip); failing hosts showed ~1ms (instant refusal).

Fix

Host Old New Reason
genesis genesis.shire-zebra.ts.net 10.42.1.5 LAN host, no k8s, LAN IP reachable
zeke zeke.shire-zebra.ts.net keep k8s node, Tailscale route exists
exodus exodus.shire-zebra.ts.net 100.80.99.48 Laptop, no fixed LAN IP, use TS IP directly
linode linode.shire-zebra.ts.net 100.109.86.8 VPS, no LAN IP, use TS IP directly

Also fixes kea and dnsmasq scrape jobs (genesis-only) to use the LAN IP.

Opened by Klaatu 🌌

## Root cause Prometheus runs as a pod and only has Tailscale routes for the k8s nodes it runs on (isaiah, jeremiah, zeke) and hosea. Scraping genesis, exodus, and linode via `*.shire-zebra.ts.net` fails instantly (~1ms, connection refused) because those Tailscale routes do not exist inside the cluster network namespace. This was visible in the metrics: working hosts showed 100-230ms scrape durations (real network round-trip); failing hosts showed ~1ms (instant refusal). ## Fix | Host | Old | New | Reason | |------|-----|-----|--------| | genesis | `genesis.shire-zebra.ts.net` | `10.42.1.5` | LAN host, no k8s, LAN IP reachable | | zeke | `zeke.shire-zebra.ts.net` | keep | k8s node, Tailscale route exists | | exodus | `exodus.shire-zebra.ts.net` | `100.80.99.48` | Laptop, no fixed LAN IP, use TS IP directly | | linode | `linode.shire-zebra.ts.net` | `100.109.86.8` | VPS, no LAN IP, use TS IP directly | Also fixes kea and dnsmasq scrape jobs (genesis-only) to use the LAN IP. > Opened by Klaatu 🌌
klaatu added 1 commit 2026-03-26 01:58:47 +00:00
fix: use LAN IPs for non-k8s hosts in Prometheus scrape config
buildbot/nix-eval Build done. (1 warning)
buildbot/nix-build Build done.
e76dfee1d8
Prometheus runs as a pod and only has Tailscale routes to the three
k8s nodes it runs on (isaiah, jeremiah, zeke) plus hosea. Scraping
genesis, exodus, and linode via *.shire-zebra.ts.net fails instantly
(~1ms, connection refused) because those Tailscale routes don't exist
inside the cluster network.

Fix by host type:
- k8s nodes (isaiah, jeremiah, zeke, hosea): keep Tailscale hostnames
- LAN hosts (genesis): use LAN IP 10.42.1.5
- Roaming/VPS (exodus, linode): use Tailscale IPs directly (100.x.x.x)
  since they have no fixed LAN address but the k8s nodes do route
  to other Tailscale IPs via the subnet router

Apply same logic to kea, dnsmasq (genesis-only jobs) and the
systemd/ping exporter jobs.
greg closed this pull request 2026-03-26 02:47:19 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: greg/nixos#7