Sienna Meridian Satterwhite
b3a38767e0
CLAUDE.md: updated source layout with orchestrator, grpc, code_index, breadcrumbs modules. Deployment: added gRPC service, startup flowchart, new secrets and troubleshooting. Conversations: updated lifecycle to show orchestrator path and gRPC session keys.
86 lines
4.9 KiB
Markdown
86 lines
4.9 KiB
Markdown
# Sol — Developer Context
|
|
|
|
## Build & Test
|
|
|
|
```sh
|
|
cargo build --release # debug: cargo build
|
|
cargo test # unit tests, no external services needed
|
|
cargo build --release --target x86_64-unknown-linux-gnu # cross-compile for production
|
|
```
|
|
|
|
Integration tests (require `.env` with `SOL_MISTRAL_API_KEY`):
|
|
|
|
```sh
|
|
cargo test --test integration_test
|
|
```
|
|
|
|
Docker (multi-stage, vendored deps):
|
|
|
|
```sh
|
|
docker build -t sol .
|
|
```
|
|
|
|
Production build + deploy via sunbeam CLI:
|
|
|
|
```sh
|
|
sunbeam build sol --push --deploy
|
|
```
|
|
|
|
## Private Cargo Registry
|
|
|
|
`mistralai-client` is published to Sunbeam's private Gitea cargo registry at `src.sunbeam.pt`. The crate is vendored into `vendor/` for Docker builds. Registry config lives in `.cargo/config.toml` (not checked in — generated by Dockerfile and locally via `cargo vendor vendor/`).
|
|
|
|
## Vendor Workflow
|
|
|
|
After adding/updating deps:
|
|
|
|
```sh
|
|
cargo vendor vendor/
|
|
```
|
|
|
|
This updates the vendored sources. Commit `vendor/` changes alongside `Cargo.lock`.
|
|
|
|
## Key Architecture Notes
|
|
|
|
- **`chat_blocking()` workaround**: The Mistral client's `chat_async` holds a `std::sync::MutexGuard` across `.await`, making the future `!Send`. All chat calls use `chat_blocking()` which runs `client.chat()` via `tokio::task::spawn_blocking`.
|
|
- **Transport-agnostic orchestrator**: The `orchestrator` module emits `OrchestratorEvent`s via broadcast channel. It has no knowledge of Matrix or gRPC. Transport bridges subscribe to events and translate to their protocol.
|
|
- **Two input paths**: Matrix sync loop and gRPC `CodeAgent` service. Both feed `GenerateRequest` into the orchestrator.
|
|
- **Tool dispatch routing**: `ToolSide::Server` tools execute locally in Sol. `ToolSide::Client` tools are relayed to the gRPC client (sunbeam code TUI) via oneshot channels.
|
|
- **Conversations API**: `ConversationRegistry` with persistent state (SQLite-backed), agents, function call loop. Enabled via `agents.use_conversations_api`.
|
|
- **deno_core sandbox**: `run_script` tool spins up a fresh V8 isolate per invocation with `sol.*` host API bindings. Timeout via V8 isolate termination. Output truncated to 4096 chars.
|
|
- **Code indexing**: Tree-sitter symbol extraction (Rust, Python, TypeScript, Go, Java) from Gitea repos and `sunbeam code` IndexSymbols. Stored in OpenSearch `sol_code` index.
|
|
|
|
## K8s Context
|
|
|
|
- Namespace: `matrix`
|
|
- Deployment: `sol` (Recreate strategy — single replica, SQLite can't share)
|
|
- PVC: `sol-data` (1Gi RWO) mounted at `/data` — holds `sol.db` + `matrix-state/`
|
|
- ConfigMap: `sol-config` — `sol.toml` + `system_prompt.md` (subPath mounts)
|
|
- Secrets: `sol-secrets` via VaultStaticSecret from OpenBao `secret/sol` (5 keys: `matrix-access-token`, `matrix-device-id`, `mistral-api-key`, `gitea-admin-username`, `gitea-admin-password`)
|
|
- Vault auth: Sol authenticates to OpenBao via K8s auth (role `sol-agent`, policy `sol-agent`) for storing user impersonation tokens at `secret/sol-tokens/{localpart}/{service}`
|
|
- Build target: `x86_64-unknown-linux-gnu` (Scaleway amd64 server)
|
|
- Base image: `gcr.io/distroless/cc-debian12:nonroot`
|
|
|
|
## Source Layout
|
|
|
|
- `src/main.rs` — startup, component wiring, gRPC server, backfill, agent recreation + sneeze
|
|
- `src/sync.rs` — Matrix event handlers, context hint injection for new conversations
|
|
- `src/config.rs` — TOML config with serde defaults (8 sections: matrix, opensearch, mistral, behavior, agents, services, vault, grpc)
|
|
- `src/context.rs` — `ResponseContext`, `derive_user_id`, `localpart`
|
|
- `src/conversations.rs` — `ConversationRegistry` (room→conversation mapping, SQLite-backed, reset_all)
|
|
- `src/persistence.rs` — SQLite store (WAL mode, 3 tables: `conversations`, `agents`, `service_users`)
|
|
- `src/matrix_utils.rs` — message extraction, reply/edit/thread detection, image download
|
|
- `src/time_context.rs` — time utilities
|
|
- `src/tokenizer.rs` — token counting
|
|
- `src/orchestrator/` — transport-agnostic event-driven pipeline (engine, events, tool dispatch)
|
|
- `src/grpc/` — gRPC CodeAgent service (server, session, auth, bridge to orchestrator events)
|
|
- `src/brain/` — evaluator (engagement decision), responder (response generation), personality, conversation manager
|
|
- `src/agents/` — registry (instructions hash + automatic recreation), definitions (orchestrator + domain agents)
|
|
- `src/sdk/` — vault client (K8s auth), token store (Vault-backed), gitea client (PAT auto-provisioning), kratos client
|
|
- `src/memory/` — schema, store, extractor
|
|
- `src/tools/` — registry + dispatch, search, code_search, room_history, room_info, script, devtools (gitea), identity (kratos), web_search (searxng), research (parallel micro-agents), bridge
|
|
- `src/code_index/` — schema, gitea repo walker, tree-sitter symbol extraction, OpenSearch indexer
|
|
- `src/breadcrumbs/` — adaptive code context injection (project outline + hybrid search)
|
|
- `src/archive/` — schema, indexer
|
|
- `proto/code.proto` — gRPC service definition (CodeAgent: Session + ReindexCode)
|