Monitoring
Observability stack: VictoriaMetrics, VictoriaLogs, Grafana, OpenTelemetry, Metrics Server.
The monitoring stack provides full observability for the cluster:
- VictoriaMetrics — time-series metrics storage (Prometheus-compatible)
- VictoriaLogs — log storage (Loki-compatible)
- Grafana — dashboards and visualization
- OpenTelemetry Collector — metrics and log collection pipeline
- Metrics Server — resource metrics for
kubectl topand HPAs - argocd-monitor — ServiceMonitors for the Argo CD components
Alertmanager is not separate: it ships inside the VictoriaMetrics k8s-stack chart.
Data Flow
Nodes/Pods
│
├── Container logs ──→ OTel Agent DaemonSet ──→ VictoriaLogs
├── Host metrics ────→ OTel Agent DaemonSet ──→ VictoriaMetrics
├── kubelet metrics ──→ OTel Agent DaemonSet ──→ VictoriaMetrics
└── k8s events ─────→ OTel Gateway ──────────→ VictoriaLogs
│
Grafana queries ◄──── User| Page | What it covers |
|---|---|
| VictoriaMetrics | Metrics storage, VMAgent scraping, and Alertmanager |
| VictoriaLogs | Log storage and LogQL queries |
| Grafana | Dashboards, datasources, and Authentik OIDC |
| OpenTelemetry | The agent/gateway collector pipelines |
| Metrics Server | kubectl top and HPA support |
| Argo CD Monitor | ServiceMonitors for Argo CD's components |