Files
sbbb/AGENTS.md
Sienna Meridian Satterwhite ccfe8b877a feat: La Suite email/messages, buildkitd, monitoring, vault and storage updates
- Add Messages (email) service: backend, frontend, MTA in/out, MPA, SOCKS
  proxy, worker, DKIM config, and theme customization
- Add Collabora deployment for document collaboration
- Add Drive frontend nginx config and values
- Add buildkitd namespace for in-cluster container builds
- Add SeaweedFS remote sync and additional S3 buckets
- Update vault secrets across namespaces (devtools, lasuite, media,
  monitoring, ory, storage) with expanded credential management
- Update monitoring: rename grafana→metrics OAuth2Client, add Prometheus
  remote write and additional scrape configs
- Update local/production overlays with resource patches
- Remove stale login-ui resource patch from production overlay
2026-03-10 19:00:57 +00:00

256 lines
8.3 KiB
Markdown

# Sunbeam Infrastructure
Kubernetes manifests for a self-hosted collaboration platform. Single-node k3s cluster deployed via Kustomize + Helm. All YAML, no Terraform, no Pulumi.
## How to Deploy
```bash
# Local (Lima VM on macOS):
sunbeam up # full stack bring-up
sunbeam apply # re-apply all manifests
sunbeam apply lasuite # apply single namespace
# Or raw kustomize:
kustomize build --enable-helm overlays/local | sed 's/DOMAIN_SUFFIX/192.168.5.2.sslip.io/g' | kubectl apply --server-side -f -
```
There is no `make`, no CI pipeline, no Terraform. Manifests are applied with `sunbeam apply` (which runs kustomize build + sed + kubectl apply).
## Critical Rules
- **Do NOT add Helm charts when plain YAML works.** Most resources here are plain YAML. Helm is only used for complex upstream charts (Kratos, Hydra, CloudNativePG, etc.) that have their own release cycles. A new Deployment, Service, or ConfigMap should be plain YAML.
- **Do NOT create Ingress resources.** This project does not use Kubernetes Ingress. Routing is handled by Pingora (a custom reverse proxy) configured via a TOML ConfigMap in `base/ingress/`. To expose a new service, add a `[[routes]]` entry to the Pingora config.
- **Do NOT introduce Terraform, Pulumi, or any IaC tool.** Everything is Kustomize.
- **Do NOT modify overlays when the change belongs in base.** Base holds the canonical config; overlays only hold environment-specific patches. If something applies to both local and production, it goes in base.
- **Do NOT add RBAC, NetworkPolicy, or PodSecurityPolicy resources.** Linkerd service mesh handles mTLS. RBAC is managed at the k3s level.
- **Do NOT create new namespaces** without being asked. The namespace layout is intentional.
- **Do NOT hardcode domains.** Use `DOMAIN_SUFFIX` as a placeholder — it gets substituted at deploy time.
- **Never commit TLS keys or secrets.** Secrets are managed by OpenBao + Vault Secrets Operator, not stored in this repo.
## Directory Structure
```
base/ # Canonical manifests (environment-agnostic)
{namespace}/ # One directory per Kubernetes namespace
kustomization.yaml # Resources, patches, helmCharts
namespace.yaml # Namespace definition with Linkerd injection
vault-secrets.yaml # VaultAuth + VaultStaticSecret + VaultDynamicSecret
{app}-deployment.yaml # Deployments
{app}-service.yaml # Services
{app}-config.yaml # ConfigMaps
{chart}-values.yaml # Helm chart values
patch-{what}.yaml # Strategic merge patches
overlays/
local/ # Lima VM dev overlay (macOS)
kustomization.yaml # Selects base dirs, adds local patches + image overrides
patch-*.yaml / values-*.yaml
production/ # Scaleway server overlay
kustomization.yaml
patch-*.yaml / values-*.yaml
scripts/ # Bash automation (local-up.sh, local-down.sh, etc.)
secrets/ # TLS cert placeholders (gitignored)
```
## Manifest Conventions — Follow These Exactly
### kustomization.yaml
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: {namespace-name}
resources:
- namespace.yaml
- {app}-deployment.yaml
- {app}-service.yaml
- vault-secrets.yaml
# Helm charts only for complex upstream projects
helmCharts:
- name: {chart}
repo: https://...
version: "X.Y.Z"
releaseName: {name}
namespace: {ns}
valuesFile: {chart}-values.yaml
patches:
- path: patch-{what}.yaml
target:
kind: Deployment
name: {name}
```
### Namespace definition
Every namespace gets Linkerd injection:
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: {name}
annotations:
linkerd.io/inject: enabled
```
### Deployments
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {app}
namespace: {ns}
spec:
replicas: 1
selector:
matchLabels:
app: {app}
template:
metadata:
labels:
app: {app}
spec:
containers:
- name: {app}
image: {app} # Short name — Kustomize images: section handles registry
envFrom:
- configMapRef:
name: {app}-config # Bulk env from ConfigMap
env:
- name: DB_PASSWORD # Individual secrets from VSO-synced K8s Secrets
valueFrom:
secretKeyRef:
name: {app}-db-credentials
key: password
livenessProbe:
httpGet:
path: /__lbheartbeat__
port: 8000
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /__heartbeat__
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
resources:
limits:
memory: 512Mi
cpu: 500m
requests:
memory: 128Mi
cpu: 100m
```
Key patterns:
- `image: {app}` uses a short name — the actual registry is set via `images:` in the overlay kustomization.yaml
- Django apps use `/__lbheartbeat__` (liveness) and `/__heartbeat__` (readiness)
- Init containers run migrations: `python manage.py migrate --no-input`
- `envFrom` for ConfigMaps, individual `env` entries for secrets
### Secrets (Vault Secrets Operator)
Secrets are NEVER stored in this repo. They flow: OpenBao → VaultStaticSecret/VaultDynamicSecret CRD → K8s Secret → Pod env.
```yaml
# VaultAuth — one per namespace, always named vso-auth
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: vso-auth
namespace: {ns}
spec:
method: kubernetes
mount: kubernetes
kubernetes:
role: vso
serviceAccount: default
---
# Static secrets (OIDC keys, Django secrets, etc.)
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: {secret-name}
namespace: {ns}
spec:
vaultAuthRef: vso-auth
mount: secret
type: kv-v2
path: {openbao-path}
refreshAfter: 30s
destination:
name: {k8s-secret-name}
create: true
overwrite: true
transformation:
excludeRaw: true
templates:
KEY_NAME:
text: '{{ index .Secrets "openbao-key" }}'
---
# Dynamic secrets (DB credentials, rotate every 5m)
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
name: {app}-db-credentials
namespace: {ns}
spec:
vaultAuthRef: vso-auth
mount: database
path: static-creds/{app}
allowStaticCreds: true
refreshAfter: 5m
rolloutRestartTargets:
- kind: Deployment
name: {app}
destination:
name: {app}-db-credentials
create: true
transformation:
templates:
password:
text: 'postgresql://{{ index .Secrets "username" }}:{{ index .Secrets "password" }}@postgres-rw.data.svc.cluster.local:5432/{db}'
```
### Shared ConfigMaps
Apps in `lasuite` namespace share these ConfigMaps via `envFrom`:
- `lasuite-postgres` — DB_HOST, DB_PORT, DB_ENGINE
- `lasuite-valkey` — REDIS_URL, CELERY_BROKER_URL
- `lasuite-s3` — AWS_S3_ENDPOINT_URL, AWS_S3_REGION_NAME
- `lasuite-oidc-provider` — OIDC endpoints (uses DOMAIN_SUFFIX)
### Overlay patches
Patches in overlays are strategic merge patches. Name them `patch-{what}.yaml` or `values-{what}.yaml`:
```yaml
# overlays/local/patch-oidc-verify-ssl.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: lasuite-oidc-provider
namespace: lasuite
data:
OIDC_VERIFY_SSL: "false"
```
## What NOT to Do
- Don't create abstract base layers, nested overlays, or "common" directories. The structure is flat: `base/{namespace}/` and `overlays/{env}/`.
- Don't use `configMapGenerator` or `secretGenerator` — ConfigMaps are plain YAML resources, secrets come from VSO.
- Don't add `commonLabels` or `commonAnnotations` in kustomization.yaml — labels are set per-resource.
- Don't use JSON patches when a strategic merge patch works.
- Don't wrap simple services in Helm charts.
- Don't add comments explaining what standard Kubernetes fields do. Only comment non-obvious decisions.
- Don't change Helm chart versions without being asked — version pinning is intentional.
- Don't add monitoring/alerting rules unless asked — monitoring lives in `base/monitoring/`.
- Don't split a single component across multiple kustomization.yaml directories.