feat(observability): ship pod logs to Loki via Grafana Alloy
Adds a Grafana Alloy DaemonSet that tails honeydue-namespace pod logs from /var/log/pods and pushes them to Loki at obs.88oakapps.com, reusing the existing OBS_INGEST_TOKEN (14-day retention). - deploy-k3s/manifests/observability/alloy-logs.yaml — DaemonSet + RBAC + token Secret + Alloy config. Runs as root (/var/log/pods is 0750 root:root) but otherwise locked down: all caps dropped, read-only root filesystem, seccomp RuntimeDefault, read-only hostPath mount. - network-policies.yaml — allow-egress-from-alloy-logs (DNS + k8s API + obs HTTPS), mirroring the vmagent egress policy. - 03-deploy.sh — applies alloy-logs with the OBS_INGEST_TOKEN substitution and waits for the DaemonSet rollout. The Loki container, nginx /loki/api/v1/push route, and Grafana Loki datasource live on the obs server and are not repo-managed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -372,3 +372,57 @@ spec:
|
||||
ports:
|
||||
- port: 8000
|
||||
protocol: TCP
|
||||
|
||||
---
|
||||
# alloy-logs egress — Grafana Alloy discovers honeydue pods via the k8s API
|
||||
# and pushes their logs to Loki at obs.88oakapps.com. Same k3s NetworkPolicy
|
||||
# DNAT gotcha as vmagent: API-server traffic is policy-checked as
|
||||
# dst=<node_public_ip>:6443, so an explicit :6443 rule is required.
|
||||
# Alloy reads log FILES from a hostPath, so it needs no ingress and no
|
||||
# egress to pod :8000/:8080 — only DNS, the API server, and obs HTTPS.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-egress-from-alloy-logs
|
||||
namespace: honeydue
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: alloy-logs
|
||||
policyTypes:
|
||||
- Egress
|
||||
egress:
|
||||
# DNS (cluster-internal)
|
||||
- to:
|
||||
- namespaceSelector: {}
|
||||
ports:
|
||||
- port: 53
|
||||
protocol: UDP
|
||||
- port: 53
|
||||
protocol: TCP
|
||||
# k8s API server via ClusterIP (pre-DNAT view)
|
||||
- to:
|
||||
- ipBlock:
|
||||
cidr: 10.43.0.0/16
|
||||
ports:
|
||||
- port: 443
|
||||
protocol: TCP
|
||||
# k8s API server post-DNAT (real path k3s NetPol enforcer sees) — REQUIRED
|
||||
- to:
|
||||
- ipBlock:
|
||||
cidr: 0.0.0.0/0
|
||||
except:
|
||||
- 10.42.0.0/16
|
||||
ports:
|
||||
- port: 6443
|
||||
protocol: TCP
|
||||
# HTTPS to public (log push to obs.88oakapps.com via Cloudflare)
|
||||
- to:
|
||||
- ipBlock:
|
||||
cidr: 0.0.0.0/0
|
||||
except:
|
||||
- 10.42.0.0/16
|
||||
- 10.43.0.0/16
|
||||
ports:
|
||||
- port: 443
|
||||
protocol: TCP
|
||||
|
||||
Reference in New Issue
Block a user