feat: La Suite email/messages, buildkitd, monitoring, vault and storage updates
- Add Messages (email) service: backend, frontend, MTA in/out, MPA, SOCKS proxy, worker, DKIM config, and theme customization - Add Collabora deployment for document collaboration - Add Drive frontend nginx config and values - Add buildkitd namespace for in-cluster container builds - Add SeaweedFS remote sync and additional S3 buckets - Update vault secrets across namespaces (devtools, lasuite, media, monitoring, ory, storage) with expanded credential management - Update monitoring: rename grafana→metrics OAuth2Client, add Prometheus remote write and additional scrape configs - Update local/production overlays with resource patches - Remove stale login-ui resource patch from production overlay
This commit is contained in:
255
AGENTS.md
Normal file
255
AGENTS.md
Normal file
@@ -0,0 +1,255 @@
|
||||
# Sunbeam Infrastructure
|
||||
|
||||
Kubernetes manifests for a self-hosted collaboration platform. Single-node k3s cluster deployed via Kustomize + Helm. All YAML, no Terraform, no Pulumi.
|
||||
|
||||
## How to Deploy
|
||||
|
||||
```bash
|
||||
# Local (Lima VM on macOS):
|
||||
sunbeam up # full stack bring-up
|
||||
sunbeam apply # re-apply all manifests
|
||||
sunbeam apply lasuite # apply single namespace
|
||||
|
||||
# Or raw kustomize:
|
||||
kustomize build --enable-helm overlays/local | sed 's/DOMAIN_SUFFIX/192.168.5.2.sslip.io/g' | kubectl apply --server-side -f -
|
||||
```
|
||||
|
||||
There is no `make`, no CI pipeline, no Terraform. Manifests are applied with `sunbeam apply` (which runs kustomize build + sed + kubectl apply).
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- **Do NOT add Helm charts when plain YAML works.** Most resources here are plain YAML. Helm is only used for complex upstream charts (Kratos, Hydra, CloudNativePG, etc.) that have their own release cycles. A new Deployment, Service, or ConfigMap should be plain YAML.
|
||||
- **Do NOT create Ingress resources.** This project does not use Kubernetes Ingress. Routing is handled by Pingora (a custom reverse proxy) configured via a TOML ConfigMap in `base/ingress/`. To expose a new service, add a `[[routes]]` entry to the Pingora config.
|
||||
- **Do NOT introduce Terraform, Pulumi, or any IaC tool.** Everything is Kustomize.
|
||||
- **Do NOT modify overlays when the change belongs in base.** Base holds the canonical config; overlays only hold environment-specific patches. If something applies to both local and production, it goes in base.
|
||||
- **Do NOT add RBAC, NetworkPolicy, or PodSecurityPolicy resources.** Linkerd service mesh handles mTLS. RBAC is managed at the k3s level.
|
||||
- **Do NOT create new namespaces** without being asked. The namespace layout is intentional.
|
||||
- **Do NOT hardcode domains.** Use `DOMAIN_SUFFIX` as a placeholder — it gets substituted at deploy time.
|
||||
- **Never commit TLS keys or secrets.** Secrets are managed by OpenBao + Vault Secrets Operator, not stored in this repo.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
base/ # Canonical manifests (environment-agnostic)
|
||||
{namespace}/ # One directory per Kubernetes namespace
|
||||
kustomization.yaml # Resources, patches, helmCharts
|
||||
namespace.yaml # Namespace definition with Linkerd injection
|
||||
vault-secrets.yaml # VaultAuth + VaultStaticSecret + VaultDynamicSecret
|
||||
{app}-deployment.yaml # Deployments
|
||||
{app}-service.yaml # Services
|
||||
{app}-config.yaml # ConfigMaps
|
||||
{chart}-values.yaml # Helm chart values
|
||||
patch-{what}.yaml # Strategic merge patches
|
||||
|
||||
overlays/
|
||||
local/ # Lima VM dev overlay (macOS)
|
||||
kustomization.yaml # Selects base dirs, adds local patches + image overrides
|
||||
patch-*.yaml / values-*.yaml
|
||||
production/ # Scaleway server overlay
|
||||
kustomization.yaml
|
||||
patch-*.yaml / values-*.yaml
|
||||
|
||||
scripts/ # Bash automation (local-up.sh, local-down.sh, etc.)
|
||||
secrets/ # TLS cert placeholders (gitignored)
|
||||
```
|
||||
|
||||
## Manifest Conventions — Follow These Exactly
|
||||
|
||||
### kustomization.yaml
|
||||
|
||||
```yaml
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
namespace: {namespace-name}
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- {app}-deployment.yaml
|
||||
- {app}-service.yaml
|
||||
- vault-secrets.yaml
|
||||
|
||||
# Helm charts only for complex upstream projects
|
||||
helmCharts:
|
||||
- name: {chart}
|
||||
repo: https://...
|
||||
version: "X.Y.Z"
|
||||
releaseName: {name}
|
||||
namespace: {ns}
|
||||
valuesFile: {chart}-values.yaml
|
||||
|
||||
patches:
|
||||
- path: patch-{what}.yaml
|
||||
target:
|
||||
kind: Deployment
|
||||
name: {name}
|
||||
```
|
||||
|
||||
### Namespace definition
|
||||
|
||||
Every namespace gets Linkerd injection:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: {name}
|
||||
annotations:
|
||||
linkerd.io/inject: enabled
|
||||
```
|
||||
|
||||
### Deployments
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {app}
|
||||
namespace: {ns}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: {app}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: {app}
|
||||
spec:
|
||||
containers:
|
||||
- name: {app}
|
||||
image: {app} # Short name — Kustomize images: section handles registry
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: {app}-config # Bulk env from ConfigMap
|
||||
env:
|
||||
- name: DB_PASSWORD # Individual secrets from VSO-synced K8s Secrets
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {app}-db-credentials
|
||||
key: password
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /__lbheartbeat__
|
||||
port: 8000
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 20
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /__heartbeat__
|
||||
port: 8000
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 10
|
||||
resources:
|
||||
limits:
|
||||
memory: 512Mi
|
||||
cpu: 500m
|
||||
requests:
|
||||
memory: 128Mi
|
||||
cpu: 100m
|
||||
```
|
||||
|
||||
Key patterns:
|
||||
- `image: {app}` uses a short name — the actual registry is set via `images:` in the overlay kustomization.yaml
|
||||
- Django apps use `/__lbheartbeat__` (liveness) and `/__heartbeat__` (readiness)
|
||||
- Init containers run migrations: `python manage.py migrate --no-input`
|
||||
- `envFrom` for ConfigMaps, individual `env` entries for secrets
|
||||
|
||||
### Secrets (Vault Secrets Operator)
|
||||
|
||||
Secrets are NEVER stored in this repo. They flow: OpenBao → VaultStaticSecret/VaultDynamicSecret CRD → K8s Secret → Pod env.
|
||||
|
||||
```yaml
|
||||
# VaultAuth — one per namespace, always named vso-auth
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultAuth
|
||||
metadata:
|
||||
name: vso-auth
|
||||
namespace: {ns}
|
||||
spec:
|
||||
method: kubernetes
|
||||
mount: kubernetes
|
||||
kubernetes:
|
||||
role: vso
|
||||
serviceAccount: default
|
||||
|
||||
---
|
||||
# Static secrets (OIDC keys, Django secrets, etc.)
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultStaticSecret
|
||||
metadata:
|
||||
name: {secret-name}
|
||||
namespace: {ns}
|
||||
spec:
|
||||
vaultAuthRef: vso-auth
|
||||
mount: secret
|
||||
type: kv-v2
|
||||
path: {openbao-path}
|
||||
refreshAfter: 30s
|
||||
destination:
|
||||
name: {k8s-secret-name}
|
||||
create: true
|
||||
overwrite: true
|
||||
transformation:
|
||||
excludeRaw: true
|
||||
templates:
|
||||
KEY_NAME:
|
||||
text: '{{ index .Secrets "openbao-key" }}'
|
||||
|
||||
---
|
||||
# Dynamic secrets (DB credentials, rotate every 5m)
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultDynamicSecret
|
||||
metadata:
|
||||
name: {app}-db-credentials
|
||||
namespace: {ns}
|
||||
spec:
|
||||
vaultAuthRef: vso-auth
|
||||
mount: database
|
||||
path: static-creds/{app}
|
||||
allowStaticCreds: true
|
||||
refreshAfter: 5m
|
||||
rolloutRestartTargets:
|
||||
- kind: Deployment
|
||||
name: {app}
|
||||
destination:
|
||||
name: {app}-db-credentials
|
||||
create: true
|
||||
transformation:
|
||||
templates:
|
||||
password:
|
||||
text: 'postgresql://{{ index .Secrets "username" }}:{{ index .Secrets "password" }}@postgres-rw.data.svc.cluster.local:5432/{db}'
|
||||
```
|
||||
|
||||
### Shared ConfigMaps
|
||||
|
||||
Apps in `lasuite` namespace share these ConfigMaps via `envFrom`:
|
||||
- `lasuite-postgres` — DB_HOST, DB_PORT, DB_ENGINE
|
||||
- `lasuite-valkey` — REDIS_URL, CELERY_BROKER_URL
|
||||
- `lasuite-s3` — AWS_S3_ENDPOINT_URL, AWS_S3_REGION_NAME
|
||||
- `lasuite-oidc-provider` — OIDC endpoints (uses DOMAIN_SUFFIX)
|
||||
|
||||
### Overlay patches
|
||||
|
||||
Patches in overlays are strategic merge patches. Name them `patch-{what}.yaml` or `values-{what}.yaml`:
|
||||
```yaml
|
||||
# overlays/local/patch-oidc-verify-ssl.yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: lasuite-oidc-provider
|
||||
namespace: lasuite
|
||||
data:
|
||||
OIDC_VERIFY_SSL: "false"
|
||||
```
|
||||
|
||||
## What NOT to Do
|
||||
|
||||
- Don't create abstract base layers, nested overlays, or "common" directories. The structure is flat: `base/{namespace}/` and `overlays/{env}/`.
|
||||
- Don't use `configMapGenerator` or `secretGenerator` — ConfigMaps are plain YAML resources, secrets come from VSO.
|
||||
- Don't add `commonLabels` or `commonAnnotations` in kustomization.yaml — labels are set per-resource.
|
||||
- Don't use JSON patches when a strategic merge patch works.
|
||||
- Don't wrap simple services in Helm charts.
|
||||
- Don't add comments explaining what standard Kubernetes fields do. Only comment non-obvious decisions.
|
||||
- Don't change Helm chart versions without being asked — version pinning is intentional.
|
||||
- Don't add monitoring/alerting rules unless asked — monitoring lives in `base/monitoring/`.
|
||||
- Don't split a single component across multiple kustomization.yaml directories.
|
||||
Reference in New Issue
Block a user