Commit Graph

18 Commits

Author SHA1 Message Date
ce68e4beed test(wfe-yaml): coverage pass to 90%+ and fix duration parsing bug
Added 51 tests: compiler hooks/parallel/error behavior (20),
validation error paths (15), shell integration tests (7),
lib.rs file loading (5), interpolation edge cases (4).

Fixed parse_duration_ms: "ms" suffix was unreachable because
strip_suffix('s') matched first. Now checks "ms" before "s".

Coverage: 40% → 90.3%. 326 total workspace tests.
2026-03-25 21:42:26 +00:00
b89563af63 feat(wfe-yaml): add YAML workflow definitions with shell executor
Concourse-CI-inspired YAML format for defining workflows. Compiles
to standard WorkflowDefinition + step factories.

Features:
- Schema parsing with serde_yaml (YamlWorkflow, YamlStep, StepConfig)
- ((var.path)) interpolation from config maps at load time
- YAML anchors (&anchor/*alias) fully supported
- Validation at load time (no runtime surprises)
- Shell executor: runs commands via tokio::process, captures stdout,
  parses ##wfe[output name=value] annotations for structured outputs
- Compiler: sequential wiring, parallel blocks, on_failure/on_success/
  ensure hooks, error behavior mapping
- Public API: load_workflow(), load_workflow_from_str()
- 23 tests (schema, interpolation, compiler, e2e)
2026-03-25 21:32:00 +00:00
8d0f83da3c feat(wfe-core): add output_data to ExecutionResult and register_step_factory to host
Core plumbing for YAML workflow support:
- Add output_data field to ExecutionResult for step output capture
- Executor merges output_data into workflow.data after step completion
- Add register_step_factory(key, factory) to WorkflowHost for
  dynamic step registration by external crates like wfe-yaml
2026-03-25 21:16:09 +00:00
c8582eb514 feat: add OpenTelemetry tracing support behind otel feature flag
- Add tracing::instrument spans to executor (workflow.execute),
  host (workflow.start, event.publish, event.process)
- Add otel feature flag to wfe-core and wfe crates
- Add wfe/src/otel.rs helper for OTLP exporter initialization
- Dependencies: tracing-opentelemetry, opentelemetry, opentelemetry_sdk,
  opentelemetry-otlp (all optional behind otel feature)
- Step execution stays at info level, executor internals at debug
2026-03-25 20:41:34 +00:00
bd51517e9f fix: demote per-round executor log to debug level
The "Executing workflow" message fires every execution round (many
times per workflow) and is noisy at INFO. Step start/completion
remains at INFO.
2026-03-25 20:35:11 +00:00
88fc6bf7ad feat: add executor tracing, auto-register primitives, and Default impls
- Add info!-level tracing to workflow executor: logs each execution
  round, each step run (with type and name), step completion, and
  workflow completion
- WorkflowHost.start() now auto-registers all built-in primitive step
  types so users don't need to register them manually
- Add #[derive(Default)] to all primitive steps and PollEndpointConfig
- Add tracing-subscriber to wfe crate for the pizza example
- Pizza example now shows full step-by-step execution logs
2026-03-25 20:32:47 +00:00
6d57f8ef22 test: add end-to-end tests and pizza workflow example
E2E tests covering: linear workflows, conditional branching, while loops,
foreach iteration, parallel branches, event-driven waiting, delays,
error handling (retry/suspend/terminate), saga compensation, versioning.

Pizza workflow example demonstrating the full API: parallel prep,
quality checks with retry, oven timer events, and delivery routing.

252 tests total, 93%+ coverage, zero clippy warnings.
2026-03-25 20:14:48 +00:00
c74b9b6ad7 feat(wfe-opensearch): add OpenSearch search index provider
OpenSearchIndex implementing SearchIndex trait. Index mapping for
workflow documents. Bool queries with multi_match for terms, term
filters for status/reference, range filters for dates. Pagination support.
2026-03-25 20:14:24 +00:00
f95bef3883 feat(wfe-valkey): add Valkey provider for locks, queues, and lifecycle events
ValkeyLockProvider: SET NX EX for acquisition, Lua script for safe release.
ValkeyQueueProvider: LPUSH/RPOP for FIFO queues.
ValkeyLifecyclePublisher: PUBLISH to per-instance and global channels.

Connections obtained once during construction (no per-operation TCP handshakes).
2026-03-25 20:14:07 +00:00
b2c37701b1 feat(wfe-postgres): add PostgreSQL persistence provider
PostgresPersistenceProvider using sqlx with wfc schema. JSONB for
complex fields, TIMESTAMPTZ for dates. Transactions for atomicity.
Proper error propagation on deserialization (no unwrap_or_default).
Status conversion returns Result for unknown values.
2026-03-25 20:13:44 +00:00
f39766cc3d feat(wfe-sqlite): add SQLite persistence provider
SqlitePersistenceProvider using sqlx with in-memory and file-based
SQLite. Schema with 6 tables, 8 indexes, created via ensure_store_exists.
Transactions for multi-table operations. Passes persistence_suite tests.
2026-03-25 20:11:16 +00:00
8b946e86e3 feat(wfe): add WorkflowHost, registry, sync runner, and purger
WorkflowHost orchestrates the engine: background workflow and event
consumers, start/stop lifecycle, workflow CRUD, event publishing.
WorkflowHostBuilder (owned-self, returns Result).

InMemoryWorkflowRegistry with version support. SyncWorkflowRunner
for testing. WorkflowPurger stub.

Event consumer acquires distributed locks before modifying workflows
to prevent lost updates from concurrent event processing.
2026-03-25 20:11:06 +00:00
a61e68d2a9 feat(wfe-core): add workflow executor with result processing and error handling
WorkflowExecutor: acquire lock, load instance, run steps, process
results, persist, release lock. StepRegistry for resolving step types.

ResultProcessor handles: next, outcome, persist, branch, sleep,
wait_for_event, poll_endpoint. ErrorHandler implements Retry (with
max_retries), Suspend, Terminate, and Compensate behaviors.
2026-03-25 20:10:45 +00:00
456c3c5b2e feat(wfe-core): add fluent workflow builder API
Owned-self builder pattern (no lifetime parameters). WorkflowBuilder
chains start_with/then/end_workflow to produce WorkflowDefinition.

StepBuilder supports: name, id, on_error, compensate_with, then,
then_fn, wait_for, delay, if_do, while_do, for_each, saga, parallel.

ParallelBuilder for branching with join semantics. InlineStep for
closure-based steps. Step config stored on WorkflowStep.step_config.
2026-03-25 20:10:33 +00:00
d0a3f0e185 feat(wfe-core): add step primitives for workflow control flow
12 step primitives implementing StepBody: DecideStep, IfStep, WhileStep,
ForEachStep, SequenceStep, DelayStep, WaitForStep, ScheduleStep,
RecurStep, PollEndpointStep, SagaContainerStep, EndStep.

Each primitive handles its state machine via persistence_data and
branch creation for container steps.
2026-03-25 20:10:03 +00:00
eea8bdb824 feat(wfe-core): add test support with in-memory providers and test suites
InMemoryPersistenceProvider, InMemoryLockProvider, InMemoryQueueProvider,
InMemoryLifecyclePublisher behind test-support feature flag.

Shared test suite macros: persistence_suite!, lock_suite!, queue_suite!
that run the same tests against any provider implementation.
2026-03-25 20:09:17 +00:00
d87d888787 feat(wfe-core): add models, traits, and error types
Core domain models: WorkflowInstance, ExecutionPointer, WorkflowDefinition,
WorkflowStep, Event, EventSubscription, ScheduledCommand, ExecutionError,
LifecycleEvent, PollEndpointConfig. All serde-serializable.

Provider traits: PersistenceProvider (composite of WorkflowRepository,
EventRepository, SubscriptionRepository, ScheduledCommandRepository),
DistributedLockProvider, QueueProvider, SearchIndex, LifecyclePublisher,
WorkflowMiddleware, StepMiddleware, WorkflowRegistry.

StepBody trait with StepExecutionContext for workflow step implementations.
WorkflowData marker trait (blanket impl for Serialize + DeserializeOwned).
2026-03-25 20:07:50 +00:00
098564db51 chore: scaffold workspace with nextest and docker compose
Workspace with 6 crates: wfe-core, wfe-sqlite, wfe-postgres,
wfe-opensearch, wfe-valkey, and wfe (umbrella). Nextest profiles
for default, integration, and CI. Docker compose for PostgreSQL,
Valkey, and OpenSearch integration tests.
2026-03-25 20:05:34 +00:00