259 lines
7.1 KiB
Markdown
259 lines
7.1 KiB
Markdown
|
# Serving Looks in Production 👠
|
|||
|
|
|
|||
|
|
The Super Boujee Business Box ✨ runs on a single Scaleway Elastic Metal server in Paris. One box, one bill, European data sovereignty by default. No multi-cloud, no multi-region, no per-seat fees — just a server that belongs to you.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## The Server
|
|||
|
|
|
|||
|
|
| Property | Value |
|
|||
|
|
|----------|-------|
|
|||
|
|
| **Provider** | Scaleway Elastic Metal |
|
|||
|
|
| **Region** | Paris (PAR1/PAR2) |
|
|||
|
|
| **RAM** | 64GB minimum |
|
|||
|
|
| **Storage** | Local NVMe |
|
|||
|
|
| **Network** | Public IPv4, configurable reverse DNS |
|
|||
|
|
| **Monthly cost** | ~€50–80 (server) + ~€7–16 (services) |
|
|||
|
|
|
|||
|
|
### External Scaleway Services
|
|||
|
|
|
|||
|
|
| Service | Purpose | Cost |
|
|||
|
|
|---------|---------|------|
|
|||
|
|
| Object Storage | PostgreSQL backups (barman), cold asset overflow | ~€5–10/mo |
|
|||
|
|
| Transactional Email (TEM) | Outbound SMTP relay | ~€1/mo |
|
|||
|
|
| Generative APIs | AI inference for all components | ~€1–5/mo |
|
|||
|
|
|
|||
|
|
Total: under €100/month for the entire collaboration suite. That's less than three Slack seats. 💅
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## k3s Installation
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik" sh -
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Traefik is disabled because we use Pingora. k3s gives us single-binary Kubernetes without the overhead of a full cluster setup.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Deployment Flow
|
|||
|
|
|
|||
|
|
### First-Time Setup
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 1. Install k3s (above)
|
|||
|
|
|
|||
|
|
# 2. Configure sunbeam
|
|||
|
|
sunbeam config set \
|
|||
|
|
--domain sunbeam.pt \
|
|||
|
|
--host admin@{SERVER_IP} \
|
|||
|
|
--infra-dir /path/to/sbbb \
|
|||
|
|
--acme-email ops@sunbeam.pt
|
|||
|
|
|
|||
|
|
# 3. Bring up core infrastructure
|
|||
|
|
sunbeam up # cert-manager, Linkerd, TLS
|
|||
|
|
|
|||
|
|
# 4. Deploy everything
|
|||
|
|
sunbeam apply # all namespaces
|
|||
|
|
|
|||
|
|
# 5. Seed credentials
|
|||
|
|
sunbeam seed # all secrets into OpenBao
|
|||
|
|
|
|||
|
|
# 6. Bootstrap services
|
|||
|
|
sunbeam bootstrap # Gitea orgs, repos, service accounts
|
|||
|
|
|
|||
|
|
# 7. Verify
|
|||
|
|
sunbeam verify # OpenBao + VSO sync test
|
|||
|
|
sunbeam check # functional health probes
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Ongoing Deployments
|
|||
|
|
|
|||
|
|
The daily driver. Most deploys are just one command.
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Deploy all namespaces
|
|||
|
|
sunbeam apply
|
|||
|
|
|
|||
|
|
# Deploy specific namespace
|
|||
|
|
sunbeam apply lasuite
|
|||
|
|
sunbeam apply monitoring
|
|||
|
|
sunbeam apply matrix
|
|||
|
|
|
|||
|
|
# Build, push, and deploy a service
|
|||
|
|
sunbeam build sol --push --deploy
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Kustomize Structure
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
sbbb/
|
|||
|
|
├── base/ # Canonical manifests (environment-agnostic)
|
|||
|
|
│ ├── data/ # PostgreSQL, Valkey, OpenSearch, OpenBao, SearXNG
|
|||
|
|
│ ├── devtools/ # Gitea
|
|||
|
|
│ ├── ingress/ # Pingora proxy
|
|||
|
|
│ ├── lasuite/ # All La Suite apps + Hive + Postfix
|
|||
|
|
│ ├── matrix/ # Tuwunel + Sol☀️
|
|||
|
|
│ ├── media/ # LiveKit
|
|||
|
|
│ ├── mesh/ # Linkerd
|
|||
|
|
│ ├── monitoring/ # Prometheus, Grafana, Loki, Tempo, Alloy
|
|||
|
|
│ ├── ory/ # Kratos + Hydra
|
|||
|
|
│ ├── storage/ # SeaweedFS
|
|||
|
|
│ ├── cert-manager/ # Certificate management
|
|||
|
|
│ ├── longhorn/ # Volume management
|
|||
|
|
│ └── vso/ # Vault Secrets Operator
|
|||
|
|
├── overlays/
|
|||
|
|
│ ├── local/ # Local dev patches (resource limits, mkcert)
|
|||
|
|
│ └── production/ # Production patches (SSL, full resources)
|
|||
|
|
├── scripts/ # Bash automation
|
|||
|
|
└── docs/ # You are here ✨
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### The Pattern
|
|||
|
|
|
|||
|
|
- **Base** holds canonical config — deployments, services, configmaps, values files, vault secrets, alert rules, service monitors
|
|||
|
|
- **Overlays** hold only patches — resource overrides, domain-specific values, environment differences
|
|||
|
|
- **DOMAIN_SUFFIX** placeholder gets replaced by `sed` at deploy time
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# What sunbeam apply does under the hood
|
|||
|
|
kustomize build overlays/production \
|
|||
|
|
| sed 's/DOMAIN_SUFFIX/sunbeam.pt/g' \
|
|||
|
|
| kubectl apply --server-side -f -
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Deployment Phases
|
|||
|
|
|
|||
|
|
The stack deploys in order of dependency — foundation first, glamour later:
|
|||
|
|
|
|||
|
|
### Phase 1 — Core Infrastructure
|
|||
|
|
1. Kubernetes & networking (k3s, Linkerd)
|
|||
|
|
2. Secrets (OpenBao, Vault Secrets Operator)
|
|||
|
|
3. Database (CloudNativePG)
|
|||
|
|
4. Cache (Valkey)
|
|||
|
|
5. Storage (SeaweedFS)
|
|||
|
|
|
|||
|
|
### Phase 2 — Identity & Edge
|
|||
|
|
6. Ory Kratos & Hydra
|
|||
|
|
7. Pingora edge proxy
|
|||
|
|
8. cert-manager (Let's Encrypt)
|
|||
|
|
|
|||
|
|
### Phase 3 — Applications
|
|||
|
|
9. Docs (Collabora)
|
|||
|
|
10. Meet (LiveKit)
|
|||
|
|
11. Drive
|
|||
|
|
12. Messages (full email stack)
|
|||
|
|
13. Calendars
|
|||
|
|
14. People
|
|||
|
|
15. Projects
|
|||
|
|
16. Find
|
|||
|
|
17. Integration navbar
|
|||
|
|
|
|||
|
|
### Phase 4 — Connectivity & Sync
|
|||
|
|
18. Hive (Drive ↔ S3 sync)
|
|||
|
|
19. Postfix (outbound SMTP)
|
|||
|
|
|
|||
|
|
### Phase 5 — Matrix & AI
|
|||
|
|
20. Tuwunel (Matrix homeserver)
|
|||
|
|
21. Sol☀️ (AI agent)
|
|||
|
|
|
|||
|
|
### Phase 6 — Development Tools
|
|||
|
|
22. Gitea
|
|||
|
|
23. SearXNG
|
|||
|
|
|
|||
|
|
### Phase 7 — Observability
|
|||
|
|
24. Prometheus + Grafana
|
|||
|
|
25. Loki + Alloy
|
|||
|
|
26. Tempo
|
|||
|
|
27. AlertManager
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## DNS Setup
|
|||
|
|
|
|||
|
|
All subdomains point to the server's public IP via A records:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
docs.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
meet.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
drive.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
mail.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
messages.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
people.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
src.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
auth.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
cal.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
projects.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
metrics.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
livekit.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
s3.sunbeam.pt → A → {SERVER_IP}
|
|||
|
|
# ... and the rest
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Email DNS
|
|||
|
|
|
|||
|
|
Email requires additional records:
|
|||
|
|
|
|||
|
|
| Record | Value |
|
|||
|
|
|--------|-------|
|
|||
|
|
| **MX** | Points to server IP (messages-mta-in listens on port 25) |
|
|||
|
|
| **SPF** | `v=spf1 ip4:{SERVER_IP} include:tem.scaleway.com ~all` |
|
|||
|
|
| **DKIM** | Generated by Postfix/Rspamd, published as TXT record |
|
|||
|
|
| **DMARC** | `v=DMARC1; p=quarantine; rua=mailto:dmarc@sunbeam.pt` |
|
|||
|
|
| **PTR** | Reverse DNS configured in Scaleway console |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## TLS Certificates
|
|||
|
|
|
|||
|
|
cert-manager handles Let's Encrypt certificates in production:
|
|||
|
|
|
|||
|
|
- ACME HTTP-01 challenges routed by Pingora itself (watches Ingress objects for solver pods)
|
|||
|
|
- Certificates stored in K8s Secret `pingora-tls`
|
|||
|
|
- Pingora watches the Secret and hot-reloads on renewal (zero downtime)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Backups
|
|||
|
|
|
|||
|
|
### PostgreSQL
|
|||
|
|
|
|||
|
|
- **Tool:** barman (continuous archiving)
|
|||
|
|
- **Destination:** Scaleway Object Storage
|
|||
|
|
- **Retention:** 30 days (see [ops.md](ops.md) for the 7-year cold storage gap)
|
|||
|
|
- **Recovery:** CloudNativePG handles point-in-time recovery
|
|||
|
|
|
|||
|
|
### Secrets
|
|||
|
|
|
|||
|
|
- OpenBao (Vault) data backed up separately
|
|||
|
|
- Root token and unseal keys stored securely offline
|
|||
|
|
|
|||
|
|
### Object Storage
|
|||
|
|
|
|||
|
|
- SeaweedFS volumes on local NVMe
|
|||
|
|
- Game assets also exist in Drive (Hive keeps them synced — bless that fairy)
|
|||
|
|
- Consider periodic snapshots for critical buckets
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Image Registry
|
|||
|
|
|
|||
|
|
Container images are built and stored in the Gitea registry at `src.sunbeam.pt`:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Build and push
|
|||
|
|
sunbeam build proxy --push
|
|||
|
|
sunbeam build sol --push
|
|||
|
|
|
|||
|
|
# Images tagged as
|
|||
|
|
src.sunbeam.pt/studio/proxy:latest
|
|||
|
|
src.sunbeam.pt/studio/sol:latest
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
For arm64 targets, `sunbeam mirror` rebuilds amd64-only La Suite images for multi-arch support. We don't discriminate against architectures. ✨
|