# wfe-server Headless workflow server with gRPC API, HTTP webhooks, and OIDC authentication. ## Quick Start ```bash # Minimal (SQLite + in-memory queue) wfe-server # Production (Postgres + Valkey + OpenSearch + OIDC) wfe-server \ --db-url postgres://wfe:secret@postgres:5432/wfe \ --queue valkey --queue-url redis://valkey:6379 \ --search-url http://opensearch:9200 ``` ## Docker ```bash docker build -t wfe-server . docker run -p 50051:50051 -p 8080:8080 wfe-server ``` ## Configuration Configuration is layered: **CLI flags > environment variables > TOML config file**. ### CLI Flags / Environment Variables | Flag | Env Var | Default | Description | |------|---------|---------|-------------| | `--config` | - | `wfe-server.toml` | Path to TOML config file | | `--grpc-addr` | `WFE_GRPC_ADDR` | `0.0.0.0:50051` | gRPC listen address | | `--http-addr` | `WFE_HTTP_ADDR` | `0.0.0.0:8080` | HTTP listen address (webhooks) | | `--persistence` | `WFE_PERSISTENCE` | `sqlite` | Persistence backend: `sqlite` or `postgres` | | `--db-url` | `WFE_DB_URL` | `wfe.db` | Database URL or file path | | `--queue` | `WFE_QUEUE` | `memory` | Queue backend: `memory` or `valkey` | | `--queue-url` | `WFE_QUEUE_URL` | `redis://127.0.0.1:6379` | Valkey/Redis URL | | `--search-url` | `WFE_SEARCH_URL` | *(none)* | OpenSearch URL (enables search) | | `--workflows-dir` | `WFE_WORKFLOWS_DIR` | *(none)* | Directory to auto-load YAML workflows | | `--auth-tokens` | `WFE_AUTH_TOKENS` | *(none)* | Comma-separated static bearer tokens | ### TOML Config File ```toml # Network grpc_addr = "0.0.0.0:50051" http_addr = "0.0.0.0:8080" # Auto-load workflow definitions from this directory workflows_dir = "/etc/wfe/workflows" # --- Persistence --- [persistence] backend = "postgres" # "sqlite" or "postgres" url = "postgres://wfe:secret@postgres:5432/wfe" # For SQLite: # backend = "sqlite" # path = "/data/wfe.db" # --- Queue / Locking --- [queue] backend = "valkey" # "memory" or "valkey" url = "redis://valkey:6379" # --- Search --- [search] url = "http://opensearch:9200" # Enables workflow + log search # --- Authentication --- [auth] # Static bearer tokens (simple API auth) tokens = ["my-secret-token"] # OIDC/JWT authentication (e.g., Ory Hydra, Keycloak, Auth0) oidc_issuer = "https://auth.sunbeam.pt/" oidc_audience = "wfe-server" # Expected 'aud' claim # Webhook HMAC secrets (per source) [auth.webhook_secrets] github = "whsec_github_secret_here" gitea = "whsec_gitea_secret_here" # --- Webhooks --- # Each trigger maps an incoming webhook event to a workflow. [[webhook.triggers]] source = "github" # "github" or "gitea" event = "push" # GitHub/Gitea event type match_ref = "refs/heads/main" # Optional: only trigger on this ref workflow_id = "ci" # Workflow definition to start version = 1 [webhook.triggers.data_mapping] repo = "$.repository.full_name" # JSONPath from webhook payload commit = "$.head_commit.id" branch = "$.ref" [[webhook.triggers]] source = "gitea" event = "push" workflow_id = "deploy" version = 1 ``` ## Persistence Backends ### SQLite Single-file embedded database. Good for development and single-node deployments. ```toml [persistence] backend = "sqlite" path = "/data/wfe.db" ``` ### PostgreSQL Production-grade. Required for multi-node deployments. ```toml [persistence] backend = "postgres" url = "postgres://user:password@host:5432/dbname" ``` The server runs migrations automatically on startup. ## Queue Backends ### In-Memory Default. Single-process only -- workflows are lost on restart. ```toml [queue] backend = "memory" ``` ### Valkey / Redis Production-grade distributed queue and locking. Required for multi-node. ```toml [queue] backend = "valkey" url = "redis://valkey:6379" ``` Provides both `QueueProvider` (work distribution) and `DistributedLockProvider` (workflow-level locking). ## Search Optional. When configured, enables: - Full-text workflow log search via `SearchLogs` RPC - Workflow instance indexing for filtered queries ```toml [search] url = "http://opensearch:9200" ``` ## Authentication ### Static Bearer Tokens Simplest auth. Tokens are compared in constant time. ```toml [auth] tokens = ["token1", "token2"] ``` Use with: `Authorization: Bearer token1` ### OIDC / JWT For production. The server discovers JWKS keys from the OIDC issuer and validates JWT tokens on every request. ```toml [auth] oidc_issuer = "https://auth.sunbeam.pt/" oidc_audience = "wfe-server" ``` Security properties: - Algorithm derived from JWK (prevents algorithm confusion attacks) - Symmetric algorithms rejected (RS256, RS384, RS512, ES256, ES384, PS256 only) - OIDC issuer must use HTTPS - Fail-closed: server won't start if OIDC discovery fails Use with: `Authorization: Bearer ` ### Webhook HMAC Webhook endpoints validate payloads using HMAC-SHA256. ```toml [auth.webhook_secrets] github = "your-github-webhook-secret" gitea = "your-gitea-webhook-secret" ``` ## gRPC API 13 RPCs available on the gRPC port (default 50051): | RPC | Description | |-----|-------------| | `StartWorkflow` | Start a new workflow instance | | `GetWorkflow` | Get workflow instance by ID | | `ListWorkflows` | List workflow instances with filters | | `SuspendWorkflow` | Pause a running workflow | | `ResumeWorkflow` | Resume a suspended workflow | | `TerminateWorkflow` | Stop a workflow permanently | | `RegisterDefinition` | Register a workflow definition | | `GetDefinition` | Get a workflow definition | | `ListDefinitions` | List all registered definitions | | `PublishEvent` | Publish an event for waiting workflows | | `WatchLifecycle` | Server-streaming: lifecycle events | | `StreamLogs` | Server-streaming: real-time step output | | `SearchLogs` | Full-text search over step logs | ## HTTP Webhooks Webhook endpoint: `POST /webhooks/{source}` Supported sources: - `github` -- GitHub webhook payloads with `X-Hub-Signature-256` HMAC - `gitea` -- Gitea webhook payloads with `X-Gitea-Signature` HMAC - `generic` -- Any JSON payload (requires bearer token auth) Payload size limit: 2MB. ## Workflow YAML Auto-Loading Point `workflows_dir` at a directory of `.yaml` files to auto-register workflow definitions on startup. ```toml workflows_dir = "/etc/wfe/workflows" ``` File format: see [wfe-yaml](../wfe-yaml/) for the YAML workflow definition schema. ## Ports | Port | Protocol | Purpose | |------|----------|---------| | 50051 | gRPC (HTTP/2) | Workflow API | | 8080 | HTTP/1.1 | Webhooks | ## Health Check The gRPC port responds to standard gRPC health checks. For HTTP health, any non-webhook GET to port 8080 returns 404 (the server is up if it responds). ## Environment Variable Reference All configuration can be set via environment variables: ```bash WFE_GRPC_ADDR=0.0.0.0:50051 WFE_HTTP_ADDR=0.0.0.0:8080 WFE_PERSISTENCE=postgres WFE_DB_URL=postgres://wfe:secret@postgres:5432/wfe WFE_QUEUE=valkey WFE_QUEUE_URL=redis://valkey:6379 WFE_SEARCH_URL=http://opensearch:9200 WFE_WORKFLOWS_DIR=/etc/wfe/workflows WFE_AUTH_TOKENS=token1,token2 RUST_LOG=info # Tracing filter (debug, info, warn, error) ```