Commit Graph

113 Commits

Author SHA1 Message Date
34209470c3 feat(wfe-server): full feature set, debian base, name resolution in gRPC
Proto changes:

* Add `name` to `WorkflowInstance`, `WorkflowSearchResult`,
  `RegisteredDefinition`, and `DefinitionSummary` messages.
* Add optional `name` override to `StartWorkflowRequest` and echo the
  assigned name back in `StartWorkflowResponse`.
* Document that `GetWorkflowRequest.workflow_id` accepts UUID or
  human name.

gRPC handler changes:

* `start_workflow` honors the optional name override and reads the
  instance back to return the assigned name to clients.
* `get_workflow` flows through `WorkflowHost::get_workflow`, which
  already falls back from UUID to name lookup.
* `stream_logs`, `watch_lifecycle`, and `search_logs` resolve
  name-or-UUID up front so the LogStore/lifecycle bus (keyed by
  UUID) subscribe to the right instance.
* `register_workflow` propagates the definition's display name into
  `RegisteredDefinition.name`.

Crate build changes:

* Enable the full executor feature set on wfe-yaml —
  `rustlang,buildkit,containerd,kubernetes,deno` — so the shipped
  binary recognizes every step type users can write.
* Dockerfile switched from `rust:alpine` to `rust:1-bookworm` +
  `debian:bookworm-slim` runtime. `deno_core` bundles a v8 binary
  that only ships glibc; alpine/musl can't link it without building
  v8 from source.
2026-04-07 19:07:52 +01:00
d88af54db9 feat(wfe-yaml): optional display name on workflow spec + schema tests
Add an optional `name` field to `WorkflowSpec` so YAML authors can
declare a human-friendly display name alongside the existing slug
`id`. The compiler copies it through to `WorkflowDefinition.name`,
which surfaces in definitions listings, run tables, and JSON output.
Slug `id` remains the primary lookup key.

Also adds a small smoke test for the schema generators to catch
regressions in `generate_json_schema` / `generate_yaml_schema`.
2026-04-07 19:07:30 +01:00
be0b93e959 feat(wfe): auto-assign workflow names + ensure store + name-or-UUID lookups
Three related host.rs changes that together make the 1.9 name support
end-to-end functional.

1. `WorkflowHost::start()` now calls `persistence.ensure_store_exists()`.
   The method existed on the trait and was implemented by every
   provider but nothing ever invoked it, so the Postgres/SQLite schema
   was never auto-created on startup — deployments failed on first
   persist with `relation "wfc.workflows" does not exist`.

2. New `start_workflow_with_name` entry point accepting an optional
   caller-supplied name override. The normal `start_workflow` is now a
   thin wrapper that passes `None` (auto-assign). The default path
   calls `next_definition_sequence(definition_id)` and formats the
   result as `{definition_id}-{N}` before persisting. Sub-workflow
   children also get auto-assigned names via HostContextImpl.

3. `get_workflow`/`suspend_workflow`/`resume_workflow`/
   `terminate_workflow` now accept either a UUID or a human-friendly
   name. `get_workflow` tries the UUID index first, then falls back to
   name lookup. A new `resolve_workflow_id` helper returns the
   canonical UUID so the gRPC log/lifecycle streams (which are keyed
   by UUID internally) can translate before subscribing.
2026-04-07 19:01:02 +01:00
9af1a0d276 feat(persistence): name column, name lookup, definition sequence counter
Land the `name` field and `next_definition_sequence` counter in the
two real persistence backends. Both providers:

* Add `name TEXT NOT NULL UNIQUE` to the `workflows` table.
* Add a `definition_sequences` table (`definition_id, next_num`) with
  an atomic UPSERT + RETURNING to give the host a race-free monotonic
  counter for `{def_id}-{N}` name generation.
* INSERT/UPDATE queries now include `name`; SELECT row parsers hydrate
  it back onto `WorkflowInstance`.
* New `get_workflow_instance_by_name` method for name-based lookups
  used by grpc handlers.

Postgres includes a DO-block migration that back-fills `name` from
`id` on pre-existing deployments so the NOT NULL + UNIQUE invariant
holds retroactively; callers can overwrite with a real name on the
next persist.
2026-04-07 18:58:25 +01:00
d9b9c5651e feat(wfe-core): human-friendly workflow names
Add a `name` field to both `WorkflowDefinition` (optional display name
declared in YAML, e.g. "Continuous Integration") and `WorkflowInstance`
(required, unique alongside the UUID primary key). Instance names are
auto-assigned as `{definition_id}-{N}` via a per-definition monotonic
counter so the 42nd run of `ci` becomes `ci-42`.

Persistence trait gains two methods:

* `get_workflow_instance_by_name` — name-based lookup for Get/Cancel/
  Suspend/Resume/Watch/Logs RPCs so callers can address instances
  interchangeably as either UUID or human name.
* `next_definition_sequence` — atomic per-definition counter used by
  the host at start time to allocate the next N.

This commit wires the in-memory test provider and touches the deno
bridge test helper; the real postgres/sqlite impls follow in the next
commit. UUIDs remain the primary key throughout — names are a second
unique index, never a replacement.
2026-04-07 18:58:12 +01:00
883471181d fix(wfe-kubernetes): run scripts under /bin/bash for pipefail support
Kubernetes step jobs with a `run:` block were invoked via
`/bin/sh -c <script>`. On debian-family base images that resolves to
dash, which rejects `set -o pipefail` ("Illegal option") and other
bashisms (arrays, process substitution, `{1..10}`). The first line of
nearly every real CI script relies on `set -euo pipefail`, so the
steps were failing with exit code 2 before running a single command.

Switch to `/bin/bash -c` so `run:` scripts can rely on the bash
feature set. Containers that lack bash should use the explicit
`command:` form instead.
2026-04-07 18:55:10 +01:00
02a574b24e style: apply cargo fmt workspace-wide
Pure formatting pass from `cargo fmt --all`. No logic changes. Separating
this out so the 1.9 release feature commits that follow show only their
intentional edits.
2026-04-07 18:44:21 +01:00
3915bcc1ec fix(wfe-core): sub-workflow inherits parent workflow data
SubWorkflowStep was hard-coding `inputs: serde_json::Value::Null` from
the YAML compiler, so every `type: workflow` step kicked off a child
instance with an empty data object. Scripts in child workflows then
saw empty `$REPO_URL`, `$COMMIT_SHA`, etc. and failed immediately.

Now: when no explicit inputs are set, the child inherits the parent
workflow's data (when it's an object). Scripts in child workflows can
reference the same top-level inputs the parent was started with without
every `type: workflow` step needing to re-declare them.
2026-04-07 18:38:41 +01:00
da26f142ee chore: ignore local dev sqlite + generated schema artifacts
Local dev runs with the SQLite backend leave `wfe.db{,-shm,-wal}` files
in the repo root, and `workflows.schema.yaml` is a generated artifact
we prefer to fetch from the running server's `/schema/workflow.yaml`
endpoint rather than checking in.
2026-04-07 18:37:30 +01:00
1b873d93f3 feat(wfe-server): gRPC reflection, auto-generated schema endpoints, Dockerfile
- tonic-reflection for gRPC service discovery
  - /schema/workflow.json (JSON Schema from schemars derives)
  - /schema/workflow.yaml (same schema in YAML)
  - /schema/workflow.proto (raw proto file)
  - Multi-stage alpine Dockerfile with all executor features
  - Comprehensive configuration reference (wfe-server/README.md)
  - Release script (scripts/release.sh)
  - Bumped to 1.8.1
v1.8.1
2026-04-06 23:47:42 +01:00
6f4700ef89 feat(wfe-server): Dockerfile and configuration reference
Multi-stage alpine build targeting sunbeam-remote buildx builder.
Comprehensive README documenting all config options, env vars,
auth methods (static tokens, OIDC/JWT, webhook HMAC), and backends.
2026-04-06 21:01:28 +01:00
556c9bc4b3 fix: pin icu_calendar <2.2 to work around temporal_rs/deno_core incompatibility v1.8.0 2026-04-06 19:42:45 +01:00
ad921fd14b fix(wfe-yaml): add missing license, repository, homepage fields 2026-04-06 19:41:20 +01:00
ae1cec2998 refactor: remove vendored Go repos, keep only .proto files 2026-04-06 19:41:02 +01:00
2f6dba296f chore: bump version to 1.8.0, update CHANGELOG 2026-04-06 18:14:03 +01:00
ae049cf2d3 fix(wfe-containerd): replace nerdctl CLI shelling with proper containerd gRPC API calls 2026-04-06 18:09:48 +01:00
e00db9e58d feat(wfe-containerd): ContainerdServiceProvider with nerdctl backend 2026-04-06 18:00:28 +01:00
51f2611e9d test(wfe-kubernetes): service provider integration tests with 100% coverage 2026-04-06 18:00:01 +01:00
84686672ea feat(wfe-yaml): services block in workflow YAML definitions 2026-04-06 17:59:56 +01:00
e5db02b4f8 feat(wfe-kubernetes):
KubernetesServiceProvider for infrastructure services
2026-04-06 17:59:50 +01:00
4dfcc61143 feat(wfe): capability-based workflow routing and service lifecycle in dequeue loop 2026-04-06 17:59:44 +01:00
affcf1bca8 feat(wfe-core): add ServiceDefinition types and ServiceProvider trait 2026-04-06 17:59:38 +01:00
22d3f569df feat(wfe-yaml): add kubernetes/k8s step type with lazy
client creation
2026-04-06 17:09:23 +01:00
8f3539ed34 refactor(wfe-kubernetes): add lazy client mode for factory-based step creation 2026-04-06 17:09:11 +01:00
a9b0993ffb test(wfe-kubernetes): integration tests against Lima k3s cluster 2026-04-06 16:42:15 +01:00
6a341e426c feat(wfe-kubernetes): KubernetesStep executor (StepBody impl) 2026-04-06 16:42:09 +01:00
85a83e7580 feat(wfe-kubernetes): log streaming and resource cleanup 2026-04-06 16:42:04 +01:00
d62dc0f349 feat(wfe-kubernetes): Job manifest builder and output parsing 2026-04-06 16:41:55 +01:00
1574342e92 feat(wfe-kubernetes): scaffold crate with config, client, and namespace modules 2026-04-06 16:41:48 +01:00
2c679229db chore: add wfe-kubernetes to workspace, add kube/k8s-openapi deps 2026-04-06 16:41:36 +01:00
a4d0f2a547 fix: add version + registry to wfe-deno path dep for publishing v1.7.0 2026-04-06 13:57:56 +01:00
4b8e544ab8 chore: bump version to 1.7.0, update
CHANGELOG
2026-04-05 22:29:05 +01:00
9a08882e28 feat(wfe-deno): Deno bindings for the WFE workflow engine 2026-04-05 22:06:07 +01:00
afb91c66bd feat(wfe-core): make WorkflowBuilder steps and last_step fields public
Needed by wfe-deno to build workflow definitions via ops without
going through the pub(crate) StepBuilder constructor.
2026-04-05 22:05:04 +01:00
ead883f714 chore: bump version to 1.6.3, update CHANGELOG v1.6.3 2026-04-05 19:55:44 +01:00
ac45011794 fix(wfe-core): propagate step_name into execution pointers
Sets step_name on execution pointers when advancing to next steps,
compensation steps, and parallel branch children so that runtime
consumers can identify steps by name without lookup.
2026-04-05 19:55:12 +01:00
2b244348ca chore: bump version to 1.6.2, update CHANGELOG v1.6.2 2026-04-05 12:45:25 +01:00
de66fef2d6 feat(wfe-core): add add_step_typed() and make wire_outcome public
Adds WorkflowBuilder::add_step_typed<S>() for adding named, configured
steps directly — needed for parallel branch closures in the CLI.
Makes wire_outcome() public so callers can wire custom step graphs.
2026-04-05 12:44:00 +01:00
6c16c89379 fix: add version + registry to wfe-server path deps for publishing v1.6.1 2026-04-05 12:01:02 +01:00
e515ffbe0c chore: bump version to 1.6.1, update CHANGELOG 2026-04-05 11:55:42 +01:00
978109d3fc feat(wfe-core): add step config API for attaching runtime JSON config
Adds StepBuilder::config() to attach arbitrary JSON configuration to
individual steps, readable at runtime via context.step.step_config.
Bumps version to 1.6.1.
2026-04-05 11:52:40 +01:00
17a50d776b chore: bump version to 1.6.0, update CHANGELOG v1.6.0 2026-04-01 14:39:21 +01:00
550dcd1f0c chore: add wfe-server crates to workspace, update test contexts
Add wfe-server-protos and wfe-server to workspace members.
Update StepExecutionContext constructions with log_sink: None
in buildkit and containerd test files.
2026-04-01 14:37:40 +01:00
cbbeaf6d67 feat(wfe-server): headless workflow server with gRPC, webhooks, and OIDC auth
Single-binary server exposing the WFE engine over gRPC (13 RPCs) with
HTTP webhook support (GitHub, Gitea, generic events).

Features:
- gRPC API: workflow CRUD, lifecycle event streaming, log streaming,
  log search via OpenSearch
- HTTP webhooks: HMAC-SHA256 verified GitHub/Gitea webhooks with
  configurable triggers that auto-start workflows
- OIDC/JWT auth: discovers JWKS from issuer, validates with asymmetric
  algorithm allowlist to prevent algorithm confusion attacks
- Static bearer token auth with constant-time comparison
- Lifecycle event broadcasting via tokio::broadcast
- Log streaming: real-time stdout/stderr via LogSink trait, history
  replay, follow mode
- Log search: full-text search via OpenSearch with workflow/step/stream
  filters
- Layered config: CLI flags > env vars > TOML file
- Fail-closed on OIDC discovery failure, fail-loud on config parse errors
- 2MB webhook payload size limit
- Blocked sensitive env var injection (PATH, LD_PRELOAD, etc.)
2026-04-01 14:37:25 +01:00
6dffb91626 feat(wfe-server-protos): add gRPC service definitions for workflow server
13 RPCs in wfe.v1.Wfe service: RegisterWorkflow, StartWorkflow,
GetWorkflow, CancelWorkflow, SuspendWorkflow, ResumeWorkflow,
SearchWorkflows, PublishEvent, WatchLifecycle (stream),
StreamLogs (stream), SearchLogs, ListDefinitions.
2026-04-01 14:35:57 +01:00
c63bf7b814 feat(wfe-yaml): add log streaming to shell executor + security hardening
Shell step streaming: when LogSink is present, uses cmd.spawn() with
tokio::select! to interleave stdout/stderr line-by-line. Respects
timeout_ms with child.kill() on timeout. Falls back to buffered mode
when no LogSink.

Security: block sensitive env var overrides (PATH, LD_PRELOAD, etc.)
from workflow data injection. Proper error handling for pipe capture.

4 LogSink regression tests + 2 env var security regression tests.
2026-04-01 14:33:53 +01:00
7a9af8015e feat(wfe-core): add LogSink trait and wire lifecycle publisher into executor
LogSink trait for real-time step output streaming. Added to
StepExecutionContext as optional field (backward compatible).
Threaded through WorkflowExecutor and WorkflowHostBuilder.

Wired LifecyclePublisher.publish() into executor at 5 points:
StepStarted, StepCompleted, Error, Completed, Terminated.
Also added lifecycle events to host start/suspend/resume/terminate.
2026-04-01 14:33:27 +01:00
d437e6ff36 chore: add CHANGELOG.md for v1.5.0
Full changelog covering v1.0.0, v1.4.0, and v1.5.0 releases.
Also fix containerd integration test default address to handle
Lima socket forwarding gracefully.

879 tests passing. 88.8% coverage on wfe-rustlang.
v1.5.0
2026-03-29 17:13:14 +01:00
93f1b726ce chore: bump version to 1.5.0
Bump workspace version and all internal crate references to 1.5.0.
Add wfe-rustlang to workspace members and dependencies.
2026-03-29 17:08:41 +01:00
c58c5d3eff chore: update Lima VM config and CI pipeline for v1.5.0
Lima wfe-test VM: Alpine with system containerd + BuildKit from apk,
TCP socat proxy for reliable gRPC transport, probes with sudo for
socket permission fixes. 2 core / 4GB / 20GB.

CI pipeline: add wfe-rustlang to feature-tests, package, and publish
steps. Container tests use TCP proxy (http://127.0.0.1:2500) instead
of Unix socket forwarding. Containerd tests set WFE_IO_DIR for shared
filesystem support.
2026-03-29 16:58:03 +01:00