wfe/wfe-server/README.md

wfe-server

Headless workflow server with gRPC API, HTTP webhooks, and OIDC authentication.

Quick Start

# Minimal (SQLite + in-memory queue)
wfe-server

# Production (Postgres + Valkey + OpenSearch + OIDC)
wfe-server \
  --db-url postgres://wfe:secret@postgres:5432/wfe \
  --queue valkey --queue-url redis://valkey:6379 \
  --search-url http://opensearch:9200

Docker

docker build -t wfe-server .
docker run -p 50051:50051 -p 8080:8080 wfe-server

Configuration

Configuration is layered: CLI flags > environment variables > TOML config file.

CLI Flags / Environment Variables

Flag	Env Var	Default	Description
--config	-	wfe-server.toml	Path to TOML config file
--grpc-addr	WFE_GRPC_ADDR	0.0.0.0:50051	gRPC listen address
--http-addr	WFE_HTTP_ADDR	0.0.0.0:8080	HTTP listen address (webhooks)
--persistence	WFE_PERSISTENCE	sqlite	Persistence backend: sqlite or postgres
--db-url	WFE_DB_URL	wfe.db	Database URL or file path
--queue	WFE_QUEUE	memory	Queue backend: memory or valkey
--queue-url	WFE_QUEUE_URL	redis://127.0.0.1:6379	Valkey/Redis URL
--search-url	WFE_SEARCH_URL	(none)	OpenSearch URL (enables search)
--workflows-dir	WFE_WORKFLOWS_DIR	(none)	Directory to auto-load YAML workflows
--auth-tokens	WFE_AUTH_TOKENS	(none)	Comma-separated static bearer tokens

TOML Config File

# Network
grpc_addr = "0.0.0.0:50051"
http_addr = "0.0.0.0:8080"

# Auto-load workflow definitions from this directory
workflows_dir = "/etc/wfe/workflows"

# --- Persistence ---
[persistence]
backend = "postgres"                          # "sqlite" or "postgres"
url = "postgres://wfe:secret@postgres:5432/wfe"
# For SQLite:
# backend = "sqlite"
# path = "/data/wfe.db"

# --- Queue / Locking ---
[queue]
backend = "valkey"                            # "memory" or "valkey"
url = "redis://valkey:6379"

# --- Search ---
[search]
url = "http://opensearch:9200"                # Enables workflow + log search

# --- Authentication ---
[auth]
# Static bearer tokens (simple API auth)
tokens = ["my-secret-token"]

# OIDC/JWT authentication (e.g., Ory Hydra, Keycloak, Auth0)
oidc_issuer = "https://auth.sunbeam.pt/"
oidc_audience = "wfe-server"                  # Expected 'aud' claim

# Webhook HMAC secrets (per source)
[auth.webhook_secrets]
github = "whsec_github_secret_here"
gitea = "whsec_gitea_secret_here"

# --- Webhooks ---

# Each trigger maps an incoming webhook event to a workflow.
[[webhook.triggers]]
source = "github"                             # "github" or "gitea"
event = "push"                                # GitHub/Gitea event type
match_ref = "refs/heads/main"                 # Optional: only trigger on this ref
workflow_id = "ci"                            # Workflow definition to start
version = 1

[webhook.triggers.data_mapping]
repo = "$.repository.full_name"               # JSONPath from webhook payload
commit = "$.head_commit.id"
branch = "$.ref"

[[webhook.triggers]]
source = "gitea"
event = "push"
workflow_id = "deploy"
version = 1

Persistence Backends

SQLite

Single-file embedded database. Good for development and single-node deployments.

[persistence]
backend = "sqlite"
path = "/data/wfe.db"

PostgreSQL

Production-grade. Required for multi-node deployments.

[persistence]
backend = "postgres"
url = "postgres://user:password@host:5432/dbname"

The server runs migrations automatically on startup.

Queue Backends

In-Memory

Default. Single-process only -- workflows are lost on restart.

[queue]
backend = "memory"

Valkey / Redis

Production-grade distributed queue and locking. Required for multi-node.

[queue]
backend = "valkey"
url = "redis://valkey:6379"

Provides both QueueProvider (work distribution) and DistributedLockProvider (workflow-level locking).

Search

Optional. When configured, enables:

• Full-text workflow log search via SearchLogs RPC
• Workflow instance indexing for filtered queries

[search]
url = "http://opensearch:9200"

Authentication

Static Bearer Tokens

Simplest auth. Tokens are compared in constant time.

[auth]
tokens = ["token1", "token2"]

Use with: Authorization: Bearer token1

OIDC / JWT

For production. The server discovers JWKS keys from the OIDC issuer and validates JWT tokens on every request.

[auth]
oidc_issuer = "https://auth.sunbeam.pt/"
oidc_audience = "wfe-server"

Security properties:

• Algorithm derived from JWK (prevents algorithm confusion attacks)
• Symmetric algorithms rejected (RS256, RS384, RS512, ES256, ES384, PS256 only)
• OIDC issuer must use HTTPS
• Fail-closed: server won't start if OIDC discovery fails

Use with: Authorization: Bearer <jwt-token>

Webhook HMAC

Webhook endpoints validate payloads using HMAC-SHA256.

[auth.webhook_secrets]
github = "your-github-webhook-secret"
gitea = "your-gitea-webhook-secret"

gRPC API

13 RPCs available on the gRPC port (default 50051):

RPC	Description
StartWorkflow	Start a new workflow instance
GetWorkflow	Get workflow instance by ID
ListWorkflows	List workflow instances with filters
SuspendWorkflow	Pause a running workflow
ResumeWorkflow	Resume a suspended workflow
TerminateWorkflow	Stop a workflow permanently
RegisterDefinition	Register a workflow definition
GetDefinition	Get a workflow definition
ListDefinitions	List all registered definitions
PublishEvent	Publish an event for waiting workflows
WatchLifecycle	Server-streaming: lifecycle events
StreamLogs	Server-streaming: real-time step output
SearchLogs	Full-text search over step logs

HTTP Webhooks

Webhook endpoint: POST /webhooks/{source}

Supported sources:

• github -- GitHub webhook payloads with X-Hub-Signature-256 HMAC
• gitea -- Gitea webhook payloads with X-Gitea-Signature HMAC
• generic -- Any JSON payload (requires bearer token auth)

Payload size limit: 2MB.

Workflow YAML Auto-Loading

Point workflows_dir at a directory of .yaml files to auto-register workflow definitions on startup.

workflows_dir = "/etc/wfe/workflows"

File format: see wfe-yaml for the YAML workflow definition schema.

Ports

Port	Protocol	Purpose
50051	gRPC (HTTP/2)	Workflow API
8080	HTTP/1.1	Webhooks

Health Check

The gRPC port responds to standard gRPC health checks. For HTTP health, any non-webhook GET to port 8080 returns 404 (the server is up if it responds).

Environment Variable Reference

All configuration can be set via environment variables:

WFE_GRPC_ADDR=0.0.0.0:50051
WFE_HTTP_ADDR=0.0.0.0:8080
WFE_PERSISTENCE=postgres
WFE_DB_URL=postgres://wfe:secret@postgres:5432/wfe
WFE_QUEUE=valkey
WFE_QUEUE_URL=redis://valkey:6379
WFE_SEARCH_URL=http://opensearch:9200
WFE_WORKFLOWS_DIR=/etc/wfe/workflows
WFE_AUTH_TOKENS=token1,token2
RUST_LOG=info                    # Tracing filter (debug, info, warn, error)