feat: add PrometheusRule alerts for all services
28 alert rules across 9 PrometheusRule files covering infrastructure (Longhorn, cert-manager), data (PostgreSQL, OpenBao, OpenSearch), storage (SeaweedFS), devtools (Gitea), identity (Hydra, Kratos), media (LiveKit), and mesh (Linkerd golden signals for all services). Severity routing: critical alerts fire to Matrix + email, warnings to Matrix only (AlertManager config updated in separate commit).
This commit is contained in:
@@ -12,3 +12,5 @@ resources:
|
||||
- seaweedfs-filer-pvc.yaml
|
||||
- vault-secrets.yaml
|
||||
- seaweedfs-remote-sync.yaml
|
||||
- seaweedfs-servicemonitor.yaml
|
||||
- seaweedfs-alertrules.yaml
|
||||
|
||||
46
base/storage/seaweedfs-alertrules.yaml
Normal file
46
base/storage/seaweedfs-alertrules.yaml
Normal file
@@ -0,0 +1,46 @@
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: PrometheusRule
|
||||
metadata:
|
||||
name: seaweedfs-alerts
|
||||
namespace: storage
|
||||
labels:
|
||||
role: alert-rules
|
||||
spec:
|
||||
groups:
|
||||
- name: seaweedfs
|
||||
rules:
|
||||
- alert: SeaweedFSMasterDown
|
||||
expr: up{job=~".*seaweedfs.*", app="seaweedfs-master"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "SeaweedFS master is down"
|
||||
description: "SeaweedFS master {{ $labels.namespace }}/{{ $labels.pod }} is down."
|
||||
|
||||
- alert: SeaweedFSVolumeDown
|
||||
expr: up{job=~".*seaweedfs.*", app="seaweedfs-volume"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "SeaweedFS volume server is down"
|
||||
description: "SeaweedFS volume server {{ $labels.namespace }}/{{ $labels.pod }} is down."
|
||||
|
||||
- alert: SeaweedFSFilerDown
|
||||
expr: up{job=~".*seaweedfs.*", app="seaweedfs-filer"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "SeaweedFS filer is down"
|
||||
description: "SeaweedFS filer {{ $labels.namespace }}/{{ $labels.pod }} is down."
|
||||
|
||||
- alert: SeaweedFSDiskLow
|
||||
expr: (seaweedfs_disk_free_bytes / (seaweedfs_disk_free_bytes + seaweedfs_disk_used_bytes)) < 0.15
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "SeaweedFS disk space low"
|
||||
description: "SeaweedFS disk on {{ $labels.instance }} has less than 15% free space."
|
||||
Reference in New Issue
Block a user