studio/wfe
2
0
Fork 0
Code Issues Pull Requests Actions Packages Releases 10 Activity

feat(wfe-server): Dockerfile and configuration reference

Browse Source 
Multi-stage alpine build targeting sunbeam-remote buildx builder.
Comprehensive README documenting all config options, env vars,
auth methods (static tokens, OIDC/JWT, webhook HMAC), and backends.
This commit is contained in:
Sienna Meridian Satterwhite
2026-04-06 21:01:28 +01:00
parent 556c9bc4b3
commit 6f4700ef89
3 changed files with 313 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
.dockerignore Normal file
View File

@@ -0,0 +1,10 @@
target/
.git/
*.md
!README.md
.envrc
TODO.md
.cargo/
.github/
test/
workflows.yaml

30
Dockerfile Normal file
View File

@@ -0,0 +1,30 @@
# Stage 1: Build
FROM rust:alpine AS builder
RUN apk add --no-cache musl-dev protobuf-dev openssl-dev openssl-libs-static pkgconfig
WORKDIR /build
COPY . .
# Configure the sunbeam cargo registry (workspace deps reference it)
RUN mkdir -p .cargo && printf '[registries.sunbeam]\nindex = "sparse+https://src.sunbeam.pt/api/packages/studio/cargo/"\n' > .cargo/config.toml
RUN cargo build --release --bin wfe-server \
-p wfe-server \
--features "wfe-yaml/rustlang,wfe-yaml/buildkit,wfe-yaml/containerd,wfe-yaml/kubernetes" \
&& strip target/release/wfe-server
# Stage 2: Runtime
FROM alpine:3.21
RUN apk add --no-cache ca-certificates tini
COPY --from=builder /build/target/release/wfe-server /usr/local/bin/wfe-server
RUN adduser -D -u 1000 wfe
USER wfe
EXPOSE 50051 8080
ENTRYPOINT ["tini", "--"]
CMD ["wfe-server"]

273
wfe-server/README.md Normal file
View File

@@ -0,0 +1,273 @@
# wfe-server
Headless workflow server with gRPC API, HTTP webhooks, and OIDC authentication.
## Quick Start
```bash
# Minimal (SQLite + in-memory queue)
wfe-server
# Production (Postgres + Valkey + OpenSearch + OIDC)
wfe-server \
--db-url postgres://wfe:secret@postgres:5432/wfe \
--queue valkey --queue-url redis://valkey:6379 \
--search-url http://opensearch:9200
```
## Docker
```bash
docker build -t wfe-server .
docker run -p 50051:50051 -p 8080:8080 wfe-server
```
## Configuration
Configuration is layered: **CLI flags > environment variables > TOML config file**.
### CLI Flags / Environment Variables
| Flag | Env Var | Default | Description |
|------|---------|---------|-------------|
| `--config` | - | `wfe-server.toml` | Path to TOML config file |
| `--grpc-addr` | `WFE_GRPC_ADDR` | `0.0.0.0:50051` | gRPC listen address |
| `--http-addr` | `WFE_HTTP_ADDR` | `0.0.0.0:8080` | HTTP listen address (webhooks) |
| `--persistence` | `WFE_PERSISTENCE` | `sqlite` | Persistence backend: `sqlite` or `postgres` |
| `--db-url` | `WFE_DB_URL` | `wfe.db` | Database URL or file path |
| `--queue` | `WFE_QUEUE` | `memory` | Queue backend: `memory` or `valkey` |
| `--queue-url` | `WFE_QUEUE_URL` | `redis://127.0.0.1:6379` | Valkey/Redis URL |
| `--search-url` | `WFE_SEARCH_URL` | *(none)* | OpenSearch URL (enables search) |
| `--workflows-dir` | `WFE_WORKFLOWS_DIR` | *(none)* | Directory to auto-load YAML workflows |
| `--auth-tokens` | `WFE_AUTH_TOKENS` | *(none)* | Comma-separated static bearer tokens |
### TOML Config File
```toml
# Network
grpc_addr = "0.0.0.0:50051"
http_addr = "0.0.0.0:8080"
# Auto-load workflow definitions from this directory
workflows_dir = "/etc/wfe/workflows"
# --- Persistence ---
[persistence]
backend = "postgres" # "sqlite" or "postgres"
url = "postgres://wfe:secret@postgres:5432/wfe"
# For SQLite:
# backend = "sqlite"
# path = "/data/wfe.db"
# --- Queue / Locking ---
[queue]
backend = "valkey" # "memory" or "valkey"
url = "redis://valkey:6379"
# --- Search ---
[search]
url = "http://opensearch:9200" # Enables workflow + log search
# --- Authentication ---
[auth]
# Static bearer tokens (simple API auth)
tokens = ["my-secret-token"]
# OIDC/JWT authentication (e.g., Ory Hydra, Keycloak, Auth0)
oidc_issuer = "https://auth.sunbeam.pt/"
oidc_audience = "wfe-server" # Expected 'aud' claim
# Webhook HMAC secrets (per source)
[auth.webhook_secrets]
github = "whsec_github_secret_here"
gitea = "whsec_gitea_secret_here"
# --- Webhooks ---
# Each trigger maps an incoming webhook event to a workflow.
[[webhook.triggers]]
source = "github" # "github" or "gitea"
event = "push" # GitHub/Gitea event type
match_ref = "refs/heads/main" # Optional: only trigger on this ref
workflow_id = "ci" # Workflow definition to start
version = 1
[webhook.triggers.data_mapping]
repo = "$.repository.full_name" # JSONPath from webhook payload
commit = "$.head_commit.id"
branch = "$.ref"
[[webhook.triggers]]
source = "gitea"
event = "push"
workflow_id = "deploy"
version = 1
```
## Persistence Backends
### SQLite
Single-file embedded database. Good for development and single-node deployments.
```toml
[persistence]
backend = "sqlite"
path = "/data/wfe.db"
```
### PostgreSQL
Production-grade. Required for multi-node deployments.
```toml
[persistence]
backend = "postgres"
url = "postgres://user:password@host:5432/dbname"
```
The server runs migrations automatically on startup.
## Queue Backends
### In-Memory
Default. Single-process only -- workflows are lost on restart.
```toml
[queue]
backend = "memory"
```
### Valkey / Redis
Production-grade distributed queue and locking. Required for multi-node.
```toml
[queue]
backend = "valkey"
url = "redis://valkey:6379"
```
Provides both `QueueProvider` (work distribution) and `DistributedLockProvider` (workflow-level locking).
## Search
Optional. When configured, enables:
- Full-text workflow log search via `SearchLogs` RPC
- Workflow instance indexing for filtered queries
```toml
[search]
url = "http://opensearch:9200"
```
## Authentication
### Static Bearer Tokens
Simplest auth. Tokens are compared in constant time.
```toml
[auth]
tokens = ["token1", "token2"]
```
Use with: `Authorization: Bearer token1`
### OIDC / JWT
For production. The server discovers JWKS keys from the OIDC issuer and validates JWT tokens on every request.
```toml
[auth]
oidc_issuer = "https://auth.sunbeam.pt/"
oidc_audience = "wfe-server"
```
Security properties:
- Algorithm derived from JWK (prevents algorithm confusion attacks)
- Symmetric algorithms rejected (RS256, RS384, RS512, ES256, ES384, PS256 only)
- OIDC issuer must use HTTPS
- Fail-closed: server won't start if OIDC discovery fails
Use with: `Authorization: Bearer <jwt-token>`
### Webhook HMAC
Webhook endpoints validate payloads using HMAC-SHA256.
```toml
[auth.webhook_secrets]
github = "your-github-webhook-secret"
gitea = "your-gitea-webhook-secret"
```
## gRPC API
13 RPCs available on the gRPC port (default 50051):
| RPC | Description |
|-----|-------------|
| `StartWorkflow` | Start a new workflow instance |
| `GetWorkflow` | Get workflow instance by ID |
| `ListWorkflows` | List workflow instances with filters |
| `SuspendWorkflow` | Pause a running workflow |
| `ResumeWorkflow` | Resume a suspended workflow |
| `TerminateWorkflow` | Stop a workflow permanently |
| `RegisterDefinition` | Register a workflow definition |
| `GetDefinition` | Get a workflow definition |
| `ListDefinitions` | List all registered definitions |
| `PublishEvent` | Publish an event for waiting workflows |
| `WatchLifecycle` | Server-streaming: lifecycle events |
| `StreamLogs` | Server-streaming: real-time step output |
| `SearchLogs` | Full-text search over step logs |
## HTTP Webhooks
Webhook endpoint: `POST /webhooks/{source}`
Supported sources:
- `github` -- GitHub webhook payloads with `X-Hub-Signature-256` HMAC
- `gitea` -- Gitea webhook payloads with `X-Gitea-Signature` HMAC
- `generic` -- Any JSON payload (requires bearer token auth)
Payload size limit: 2MB.
## Workflow YAML Auto-Loading
Point `workflows_dir` at a directory of `.yaml` files to auto-register workflow definitions on startup.
```toml
workflows_dir = "/etc/wfe/workflows"
```
File format: see [wfe-yaml](../wfe-yaml/) for the YAML workflow definition schema.
## Ports
| Port | Protocol | Purpose |
|------|----------|---------|
| 50051 | gRPC (HTTP/2) | Workflow API |
| 8080 | HTTP/1.1 | Webhooks |
## Health Check
The gRPC port responds to standard gRPC health checks. For HTTP health, any non-webhook GET to port 8080 returns 404 (the server is up if it responds).
## Environment Variable Reference
All configuration can be set via environment variables:
```bash
WFE_GRPC_ADDR=0.0.0.0:50051
WFE_HTTP_ADDR=0.0.0.0:8080
WFE_PERSISTENCE=postgres
WFE_DB_URL=postgres://wfe:secret@postgres:5432/wfe
WFE_QUEUE=valkey
WFE_QUEUE_URL=redis://valkey:6379
WFE_SEARCH_URL=http://opensearch:9200
WFE_WORKFLOWS_DIR=/etc/wfe/workflows
WFE_AUTH_TOKENS=token1,token2
RUST_LOG=info # Tracing filter (debug, info, warn, error)
```