rust-rewrite #2

siennathesane · 2026-03-21T14:40:24Z

siennathesane commented

2026-03-21 14:40:24 +00:00

No description provided.

siennathesane added 39 commits 2026-03-21 14:40:25 +00:00

feat: add tuwunel/matrix support with OpenSearch ML post-apply hooks c82f15b190

- Add matrix to MANAGED_NS and tuwunel to restart/build targets
- Add post-apply hooks for matrix namespace:
  - _patch_tuwunel_oauth2_redirect: reads client_id from hydra-maester
    Secret and patches OAuth2Client redirectUris dynamically
  - _inject_opensearch_model_id: reads model_id from ingest pipeline
    and writes to ConfigMap for tuwunel deployment env var injection
- Add post-apply hook for data namespace:
  - _ensure_opensearch_ml: idempotently registers/deploys all-mpnet-base-v2
    (768-dim) model, creates ingest + hybrid search pipelines
- Add tuwunel secrets to OpenBao seed (OIDC, TURN, registration token)
- Refactor secret seeding to only write dirty paths (avoid VSO churn)
- Add ACME email fallback from config when not provided via CLI flag

refactor: cross-platform tool downloads, configurable infra dir and ACME email d5b963253b

- Make tool downloads platform-aware (darwin/linux, arm64/amd64)
- Add buildctl to bundled tools
- Add get_infra_dir() with config fallback for REPO_ROOT resolution
- Add ACME email to sunbeam config (set/get)
- Add REGISTRY_HOST_IP substitution in kustomize builds
- Update Kratos admin identity schema to employee
- Fix logs command to use production tunnel and context

feat: Rust rewrite scaffolding with embedded kustomize+helm 80c67d34cb

Phase 0 of Python-to-Rust CLI rewrite:

- Cargo.toml with all dependencies (kube-rs, reqwest, russh, rcgen, lettre, etc.)
- build.rs: downloads kustomize v5.8.1 + helm v4.1.0 at compile time, embeds as bytes, sets SUNBEAM_COMMIT from git
- src/main.rs: tokio main with anyhow error formatting
- src/cli.rs: full clap derive struct tree matching all Python argparse subcommands
- src/config.rs: SunbeamConfig serde struct, load/save ~/.sunbeam.json
- src/output.rs: step/ok/warn/table with exact Python format strings
- src/tools.rs: embedded kustomize+helm extraction to cache dir
- src/kube.rs: parse_target, domain_replace, context management
- src/manifests.rs: filter_by_namespace with full test coverage
- Stub modules for all remaining features (cluster, secrets, images, services, checks, gitea, users, update)

23 tests pass, cargo check clean.

feat: Phase 1 foundations — kube-rs client, OpenBao HTTP client, self-update 42c2a74928

kube.rs:
- KubeClient with lazy init from kubeconfig + context selection
- SSH tunnel via subprocess (port 2222, forward 16443->6443)
- Server-side apply for multi-document YAML via kube-rs discovery
- Secret get/create, namespace ensure, exec in pod, rollout restart
- Domain discovery from gitea-inline-config secret
- kustomize_build with embedded binary, domain/email/registry substitution
- kubectl and bao CLI passthrough commands

openbao.rs:
- Lightweight Vault/OpenBao HTTP API client using reqwest
- System ops: seal-status, init, unseal
- KV v2: get, put, patch, delete with proper response parsing
- Auth: enable method, write policy, write roles
- Database secrets engine: config, static roles
- Replaces all kubectl exec bao shell commands from Python version

update.rs:
- Self-update from latest mainline commit via Gitea API
- CI artifact download with SHA256 checksum verification
- Atomic self-replace (temp file + rename)
- Background update check with hourly cache (~/.local/share/sunbeam/)
- Enhanced version command with target triple and build date

build.rs:
- Added SUNBEAM_TARGET and SUNBEAM_BUILD_DATE env vars

35 tests pass.

feat: Phase 2 feature modules + comprehensive test suite (142 tests) ec235685bf

services.rs:
- Pod status with unicode icons, grouped by namespace
- VSO sync status (VaultStaticSecret/VaultDynamicSecret via kube-rs DynamicObject)
- Log streaming via kube-rs log_stream + futures::AsyncBufReadExt
- Pod get in YAML/JSON format
- Rollout restart with namespace/service filtering

checks.rs:
- 11 health check functions (gitea, postgres, valkey, openbao, seaweedfs, kratos, hydra, people, livekit)
- AWS4-HMAC-SHA256 S3 auth header generation using sha2 + hmac
- Concurrent execution via tokio JoinSet
- mkcert root CA trust for local TLS

secrets.rs:
- Stub with cmd_seed/cmd_verify (requires live cluster for full impl)

users.rs:
- All 10 Kratos identity operations via reqwest + kubectl port-forward
- Welcome email via lettre SMTP through port-forwarded postfix
- Employee onboarding with auto-assigned ID, HR metadata
- Offboarding with Kratos + Hydra session revocation

gitea.rs:
- Bootstrap without Lima VM: admin password, org creation, OIDC auth source
- Gitea API via kubectl exec curl

images.rs:
- BuildEnv detection, buildctl build + push via port-forward
- Per-service builders for all 17 build targets
- Deploy rollout, node image pull, uv Dockerfile patching
- Mirror scaffolding (containerd operations marked TODO)

cluster.rs:
- Pure K8s cmd_up: cert-manager, linkerd, rcgen TLS certs, core service wait
- No Lima VM operations

manifests.rs:
- Full cmd_apply: kustomize build, two-pass convergence, ConfigMap restart detection
- Pre-apply cleanup, webhook wait, mkcert CA, tuwunel OAuth2 redirect patch

Test coverage: 142 tests across 14 modules (44 in checks, 27 in cli, 13 in images, 12 in tools, 12 in services, 11 in users, 10 in manifests, 9 in kube, 9 in cluster, 7 in update, 6 in gitea, 4 in openbao, 3 in output, 2 in config).

refactor: add thiserror error tree and tracing logging cc0b6a833e

SunbeamError enum with typed variants (Kube, Config, Network, Secrets,
Build, Identity, ExternalTool, Io, Json, Yaml, Other) each mapping to
a process exit code. ResultExt trait replaces anyhow's .context().

main.rs initializes tracing-subscriber with RUST_LOG env filter and
routes all errors to exit codes via SunbeamError::exit_code().

Removes anyhow dependency.

refactor: migrate all modules from anyhow to SunbeamError 7fd8874d99

Replace anyhow::{bail, Context, Result} with crate::error::{Result,
SunbeamError, ResultExt} across all modules. Each module uses the
appropriate error variant (Kube, Secrets, Build, Identity, etc).

feat: implement secrets.rs with OpenBao HTTP API bc5eeaae6e

Full cmd_seed implementation using openbao::BaoClient:
- OpenBao init/unseal via HTTP API (no kubectl exec)
- KV v2 seeding with get_or_create pattern and dirty-path tracking
- Kubernetes auth method + VSO policy configuration
- Database secrets engine with vault PG user and static roles
- DKIM key generation via rsa + pkcs8 crates
- Kratos admin identity seeding via port-forward + reqwest

cmd_verify: VSO E2E test with test sentinel, sync poll, cleanup.

feat: implement OpenSearch ML setup and model_id injection 503e407243

ensure_opensearch_ml: cluster settings, model registration/deployment
(all-mpnet-base-v2), ingest + search pipelines for hybrid BM25+neural.

inject_opensearch_model_id: reads model_id from ingest pipeline,
writes to matrix/opensearch-ml-config ConfigMap.

os_api helper: kube exec curl inside opensearch pod.

refactor: deduplicate constants, fix secret key mismatch, add VSS pruning bcfb443757

- New src/constants.rs: single source for MANAGED_NS (includes monitoring)
  and GITEA_ADMIN_USER, imported by all modules that previously had copies
- Fix checks.rs reading wrong key names from gitea-admin-credentials secret
- Add VaultStaticSecret pruning in pre_apply_cleanup (H1)
- Fix cert_manager_present check (was always true after canonicalize)
- Add warnings for silent failures in pre_apply_cleanup
- Fix os_api dead variable assignment
- Set TLS private key permissions to 0600
- Redact Gitea admin password in print_urls

fix: SSH tunnel leak, cmd_bao injection, discovery cache, DNS async 6ec0666aa1

- Store SSH tunnel child in static Mutex (was dropped immediately)
- cmd_bao: use env(1) for VAULT_TOKEN instead of sh -c (no shell injection)
- Cache API discovery across kube_apply documents (was per-doc roundtrip)
- Replace blocking ToSocketAddrs with tokio::net::lookup_host
- Remove double YAML->JSON->string->JSON serialization in kube_apply
- ResultExt::ctx now preserves all SunbeamError variants

fix: CNPG readiness, DKIM SPKI format, kv_patch, container name 24e98b4e7d

- Check CNPG Cluster CRD status.phase instead of pod Running phase
- DKIM public key: use SPKI format (BEGIN PUBLIC KEY) matching Python
- Use kv_patch instead of kv_put for dirty paths (preserves external fields)
- Vault KV only written when password is newly generated
- Gitea exec passes container name Some("gitea")
- Fix openbao comment (400 not 409)

fix: rewrite users.rs to fully async (was blocking tokio runtime) e95ee4f377

Replace all blocking I/O with async equivalents:
- tokio::process::Command instead of std::process::Command
- tokio::time::sleep instead of std::thread::sleep
- reqwest::Client (async) instead of reqwest::blocking::Client
- All helper functions (api, find_identity, generate_recovery, etc.) now async
- PortForward::Drop uses start_kill() (sync SIGKILL) for cleanup
- send_welcome_email wrapped in spawn_blocking for lettre sync transport

fix: S3 auth signature tested against AWS reference vector 019c73e300

Refactor s3_auth_headers into deterministic s3_auth_headers_at that
accepts a timestamp. Add test with AWS example credentials and fixed
date verifying canonical request, string-to-sign, and final signature.

fix: employee ID pagination, add async tests dff4588e52

- next_employee_id now paginates through all identities (was limited to 200)
- Add #[tokio::test] tests: ensure_tunnel noop, BaoClient connection error,
  check_update_background returns quickly when forge URL empty

fix: stdin password, port-forward retry, seed advisory lock aad469e9c6

- set-password reads from stdin when password arg omitted
- Port-forward proxy retries on pod restart instead of failing
- cmd_seed acquires PID-based advisory lockfile to prevent concurrent runs

feat: OAuth2 CLI authentication with PKCE and token caching d4421d3e29

New src/auth.rs module:
- Authorization Code + PKCE flow via localhost redirect
- OIDC discovery from Hydra well-known endpoint
- Browser-based login (opens system browser automatically)
- Token caching at ~/.local/share/sunbeam/auth.json (0600 perms)
- Automatic refresh when access token expires (refresh valid 7 days)
- get_token() for use by other modules (pm, etc.)
- cmd_auth_login/logout/status subcommands

feat: unified project management across Planka and Gitea 5bdb78933f

New src/pm.rs module with sunbeam pm subcommand:
- Planka client: cards, boards, lists, comments, assignments
  via OIDC token exchange for Planka JWT
- Gitea client: issues, comments, labels, milestones
  via OAuth2 Bearer token
- Unified Ticket type with p:/g: ID prefixes
- pm list: parallel fetch from both sources, merged display
- pm show/create/comment/close/assign across both systems
- Auth via crate::auth::get_token() (Hydra OAuth2)

fix: install rustls ring crypto provider at startup 184ad85c60

Rustls 0.23 requires an explicit CryptoProvider. Enable the ring
feature and call install_default() before any TLS operations.

fix: auth login domain resolution with --domain flag 7fab2a7f3c

Domain resolves from: --domain flag > cached token > config
production_host > cluster discovery. Clear error when none available.

fix: URL-encode scope parameter with %20 instead of + 43b5a4eef9

fix: auth login UX — timeout, Ctrl+C, suppress K8s error, center HTML 1029ff0747

- 5-minute timeout on callback wait (Ctrl+C now works)
- Skip K8s client_id lookup when no cluster configured (removes noisy ERROR)
- Center the success page HTML to match Sunbeam Studios branding

fix: use predictable client_id via pre-seeded K8s secret 3a5e1c62ba

Pre-create oidc-sunbeam-cli secret with CLIENT_ID=sunbeam-cli before
hydra-maester reconciles. No cluster access needed at login time.

feat: kubectl-style contexts with per-domain auth tokens 88b02acdd1

Config now supports named contexts (like kubectl), each bundling
domain, kube-context, ssh-host, infra-dir, and acme-email. Legacy
flat config auto-migrates to a "production" context on load.

- sunbeam config set --domain sunbeam.pt --host user@server
- sunbeam config use-context production
- sunbeam config get (shows all contexts)

Auth tokens stored per-domain (~/.local/share/sunbeam/auth/{domain}.json)
so local and production don't clobber each other. pm and auth commands
read domain from active context instead of K8s cluster discovery.

refactor: remove --env flag, use --context like kubectl ded0ab442e

Context resolution: --context flag > current-context from config > "local".
No more production/local distinction in the CLI flags — the context
determines everything (domain, kube-context, ssh-host, infra-dir).

Remove Env enum entirely. Production detection is now "context has ssh-host".

feat: split auth into sso/git, Planka token exchange, board discovery ffc0fe917b

Auth:
- sunbeam auth login runs SSO (Hydra OIDC) then Git (Gitea PAT)
- SSO callback auto-redirects browser to Gitea token page
- sunbeam auth sso / sunbeam auth git for individual flows
- Gitea PAT verified against API before saving

Planka:
- Token exchange via /api/access-tokens/exchange-using-token endpoint
- Board discovery via GET /api/projects
- String IDs (snowflake) handled throughout

Config:
- kubectl-style contexts: --context flag > current-context > "local"
- Removed --env flag
- Per-domain auth token storage

feat: complete pm subcommands with board discovery and user resolution c6aa1bd8ce

Planka:
- Board discovery via GET /api/projects (no hardcoded IDs)
- String IDs (snowflake) throughout — TicketRef::Planka holds String
- Create auto-discovers first board/list, or matches --target by name
- Close finds "Done"/"Closed" list and moves card automatically
- Assign resolves users via search, supports "me" for self-assign
- Ticket IDs use p:/g: short prefixes

Gitea:
- Assign uses PATCH on issue (not POST /assignees which needs collaborator)
- Create requires --target org/repo

All pm subcommands tested against live Planka + Gitea instances.

feat: user provisioning — mailbox, Projects, welcome email f75f61f238

Onboarding now provisions app-level accounts:
- create_mailbox: Django ORM via kubectl exec into messages-backend
- setup_projects_user: knex.js via kubectl exec into projects pod
- Welcome email includes job title and department when provided

Offboarding cleans up:
- delete_mailbox: removes mailbox + Django user
- cleanup_projects_user: soft-deletes Planka user + memberships

All provisioning is best-effort (warns on failure, doesn't block).

feat: --no-cache build flag and Sol build target 8d6e815a91

- Add --no-cache flag to sunbeam build (passes --no-cache to buildctl)
- Add Sol (virtual librarian) as a build target
- Wire no_cache through all build functions and dispatch

feat: Python upstream — onboard/offboard, mailbox, Projects, --no-cache b92c6ad18c

Python changes that were ported to Rust in preceding commits:
- User onboard/offboard with mailbox + Projects provisioning
- Welcome email with job title/department
- --no-cache build flag
- Date validation, apply confirmation, build targets

chore: suppress dead_code warning on exit code constants b6daf608af

refactor: workspace scaffolding — sunbeam-sdk + sunbeam binary crate 2ffedb95cb

Convert the single binary crate into a Cargo workspace with two members:
sunbeam-sdk (library) and sunbeam (thin binary). Moves build.rs to the
SDK with adjusted .git/HEAD path for the nested layout.

refactor: SDK core modules — error, config, output, constants b92700d363

Move foundational modules into sunbeam-sdk. All crate-internal references
remain unchanged since these are sibling modules within the SDK crate.

refactor: SDK kube, openbao, and tools modules 8e51e0b3ae

Move kube (client, apply, exec, secrets, kustomize_build) into kube/
submodule with tools.rs as a child. Move openbao BaoClient into
openbao/ submodule.

refactor: SDK images and secrets modules with submodule splits bc65b9157d

Split images.rs (1809L) into mod.rs + builders.rs (per-service build
functions). Split secrets.rs (1727L) into mod.rs + seeding.rs (KV
get_or_create, seed_openbao) + db_engine.rs (PostgreSQL static roles).
Moves BuildTarget enum from cli.rs into images/mod.rs with conditional
clap::ValueEnum derive behind the "cli" feature.

refactor: SDK users, pm, and checks modules with submodule splits 6c7e1cd064

Split users.rs (1157L) into mod.rs + provisioning.rs (mailbox,
projects user, welcome email). Split pm.rs (1664L) into mod.rs +
planka.rs (PlankaClient) + gitea_issues.rs (GiteaClient). Split
checks.rs (1214L) into mod.rs + probes.rs (11 check functions + S3).

refactor: SDK small command modules — services, cluster, manifests, gitea, update, auth 8e5d295902

refactor: binary crate — thin main.rs + cli.rs dispatch e0961cce73

Slim binary that depends on sunbeam-sdk for all logic. Replaces 62
crate:: refs with sunbeam_sdk::. Tracing filter updated to include
sunbeam_sdk=info.

feat: Python upstream — Sol bot registration TODO 3ef3fc0255

siennathesane merged commit 3ef3fc0255 into mainline

2026-03-21 14:40:42 +00:00

siennathesane deleted branch rust-rewrite

2026-03-21 14:40:42 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: studio/cli#2