Monitoring

Observability stack: VictoriaMetrics, VictoriaLogs, Grafana, OpenTelemetry, Metrics Server.

The monitoring stack provides full observability for the cluster:

  • VictoriaMetrics — time-series metrics storage (Prometheus-compatible)
  • VictoriaLogs — log storage (Loki-compatible)
  • Grafana — dashboards and visualization
  • OpenTelemetry Collector — metrics and log collection pipeline
  • Metrics Server — resource metrics for kubectl top and HPAs
  • argocd-monitor — ServiceMonitors for the Argo CD components

Alertmanager is not separate: it ships inside the VictoriaMetrics k8s-stack chart.

Data Flow

Nodes/Pods

    ├── Container logs ──→ OTel Agent DaemonSet ──→ VictoriaLogs
    ├── Host metrics ────→ OTel Agent DaemonSet ──→ VictoriaMetrics
    ├── kubelet metrics ──→ OTel Agent DaemonSet ──→ VictoriaMetrics
    └── k8s events ─────→ OTel Gateway ──────────→ VictoriaLogs

                                               Grafana queries ◄──── User
PageWhat it covers
VictoriaMetricsMetrics storage, VMAgent scraping, and Alertmanager
VictoriaLogsLog storage and LogQL queries
GrafanaDashboards, datasources, and Authentik OIDC
OpenTelemetryThe agent/gateway collector pipelines
Metrics Serverkubectl top and HPA support
Argo CD MonitorServiceMonitors for Argo CD's components