feat: add PrometheusRule alerts for all services

28 alert rules across 9 PrometheusRule files covering infrastructure
(Longhorn, cert-manager), data (PostgreSQL, OpenBao, OpenSearch),
storage (SeaweedFS), devtools (Gitea), identity (Hydra, Kratos),
media (LiveKit), and mesh (Linkerd golden signals for all services).

Severity routing: critical alerts fire to Matrix + email, warnings
to Matrix only (AlertManager config updated in separate commit).
This commit is contained in:
2026-03-24 12:20:55 +00:00
parent 74bb59cfdc
commit 3fc54c8851
15 changed files with 363 additions and 2 deletions

View File

@@ -0,0 +1,28 @@
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: openbao-alerts
namespace: data
labels:
role: alert-rules
spec:
groups:
- name: openbao
rules:
- alert: VaultSealed
expr: vault_core_unsealed == 0
for: 1m
labels:
severity: critical
annotations:
summary: "OpenBao/Vault is sealed"
description: "OpenBao/Vault is sealed — automatic unseal may have failed"
- alert: VaultDown
expr: up{job=~".*openbao.*"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "OpenBao/Vault is down"
description: "OpenBao instance {{ $labels.namespace }}/{{ $labels.pod }} is down."