docs: update CLAUDE.md, deployment, and conversations docs

CLAUDE.md: updated source layout with orchestrator, grpc, code_index, breadcrumbs modules. Deployment: added gRPC service, startup flowchart, new secrets and troubleshooting. Conversations: updated lifecycle to show orchestrator path and gRPC session keys.
2026-03-24 12:58:35 +00:00
parent 2949ea354f
commit b3a38767e0
3 changed files with 94 additions and 45 deletions
--- a/docs/conversations.md
+++ b/docs/conversations.md
@@ -8,7 +8,7 @@ The Conversations API path provides persistent, server-side conversation state p
 sequenceDiagram
    participant M as Matrix Sync
    participant E as Evaluator
-    participant R as Responder
+    participant O as Orchestrator
    participant CR as ConversationRegistry
    participant API as Mistral Conversations API
    participant T as ToolRegistry
@@ -17,14 +17,14 @@ sequenceDiagram
    M->>E: message event
    E-->>M: MustRespond/MaybeRespond

-    M->>R: generate_response_conversations()
-    R->>CR: send_message(room_id, input, is_dm)
+    M->>O: GenerateRequest
+    O->>CR: send_message(conversation_key, input, is_dm)

-    alt new room (no conversation)
+    alt new conversation
        CR->>API: create_conversation(agent_id?, model, input)
        API-->>CR: ConversationResponse + conversation_id
        CR->>DB: upsert_conversation(room_id, conv_id, tokens)
-    else existing room
+    else existing conversation
        CR->>API: append_conversation(conv_id, input)
        API-->>CR: ConversationResponse
        CR->>DB: update_tokens(room_id, new_total)
@@ -32,20 +32,20 @@ sequenceDiagram

    alt response contains function_calls
        loop up to max_tool_iterations (5)
-            R->>T: execute(name, args)
-            T-->>R: result string
-            R->>CR: send_function_result(room_id, entries)
+            O->>T: execute(name, args)
+            T-->>O: result string
+            O->>CR: send_function_result(room_id, entries)
            CR->>API: append_conversation(conv_id, FunctionResult entries)
            API-->>CR: ConversationResponse
            alt more function_calls
-                Note over R: continue loop
+                Note over O: continue loop
            else text response
-                Note over R: break
+                Note over O: break
            end
        end
    end

-    R-->>M: response text (or None)
+    O-->>M: response text (or None)
    M->>M: send to Matrix room
    M->>M: fire-and-forget memory extraction
 ```
@@ -59,6 +59,8 @@ Each Matrix room maps to exactly one Mistral conversation:

 The mapping is stored in `ConversationRegistry.mapping` (HashMap in-memory, backed by SQLite `conversations` table).

+For gRPC coding sessions, the conversation key is the project path + branch, creating a dedicated conversation per coding context.
+
 ## ConversationState

 ```rust
@@ -101,7 +103,7 @@ This means conversation history is lost on compaction. The archive still has the

 ### startup recovery

-On initialization, `ConversationRegistry::new()` calls `store.load_all_conversations()` to restore all room→conversation mappings from SQLite. This means conversations survive pod restarts.
+On initialization, `ConversationRegistry::new()` calls `store.load_all_conversations()` to restore all room-to-conversation mappings from SQLite. This means conversations survive pod restarts.

 ### SQLite schema

--- a/docs/deployment.md
+++ b/docs/deployment.md
@@ -7,7 +7,7 @@ Sol runs as a single-replica Deployment in the `matrix` namespace. SQLite is the
 ```mermaid
 flowchart TD
    subgraph OpenBao
-        vault[("secret/sol<br/>matrix-access-token<br/>matrix-device-id<br/>mistral-api-key")]
+        vault[("secret/sol<br/>matrix-access-token<br/>matrix-device-id<br/>mistral-api-key<br/>gitea-admin-username<br/>gitea-admin-password")]
    end

    subgraph "matrix namespace"
@@ -18,6 +18,7 @@ flowchart TD
        deploy[Deployment<br/>sol]
        init[initContainer<br/>fix-permissions]
        pod[Container<br/>sol]
+        svc[Service<br/>sol-grpc<br/>port 50051]
    end

    vault --> |VSO sync| vss
@@ -29,6 +30,7 @@ flowchart TD
    cm --> |subPath mounts| pod
    pvc --> |/data| init
    pvc --> |/data| pod
+    svc --> |gRPC| pod
 ```

 ## manifests
@@ -49,6 +51,7 @@ replicas: 1

 - Resources: 256Mi request / 512Mi limit memory, 100m CPU request
 - `enableServiceLinks: false` — avoids injecting service env vars that could conflict
+- Ports: 50051 (gRPC)

 **Environment variables** (from Secret `sol-secrets`):

@@ -57,6 +60,8 @@ replicas: 1
 | `SOL_MATRIX_ACCESS_TOKEN` | `matrix-access-token` |
 | `SOL_MATRIX_DEVICE_ID` | `matrix-device-id` |
 | `SOL_MISTRAL_API_KEY` | `mistral-api-key` |
+| `SOL_GITEA_ADMIN_USERNAME` | `gitea-admin-username` |
+| `SOL_GITEA_ADMIN_PASSWORD` | `gitea-admin-password` |

 Fixed env vars:

@@ -115,17 +120,19 @@ spec:

 The `rolloutRestartTargets` field means VSO will automatically restart the Sol deployment when secrets change in OpenBao.

-Three keys synced from OpenBao `secret/sol`:
+Five keys synced from OpenBao `secret/sol`:

 - `matrix-access-token`
 - `matrix-device-id`
 - `mistral-api-key`
+- `gitea-admin-username`
+- `gitea-admin-password`

 ## `/data` mount layout

 ```
 /data/
-├── sol.db           SQLite database (conversations + agents tables, WAL mode)
+├── sol.db           SQLite database (conversations, agents, service_users — WAL mode)
 └── matrix-state/    Matrix SDK sqlite state store (E2EE keys, sync tokens)
 ```

@@ -140,7 +147,9 @@ Store secrets at `secret/sol` in OpenBao KV v2:
 openbao kv put secret/sol \
  matrix-access-token="syt_..." \
  matrix-device-id="DEVICE_ID" \
-  mistral-api-key="..."
+  mistral-api-key="..." \
+  gitea-admin-username="..." \
+  gitea-admin-password="..."
 ```

 These are synced to K8s Secret `sol-secrets` by the Vault Secrets Operator.
@@ -162,23 +171,36 @@ The Docker build cross-compiles to `x86_64-unknown-linux-gnu` on macOS. The fina

 ## startup sequence

-1. Initialize `tracing_subscriber` with `RUST_LOG` env filter (default: `sol=info`)
-2. Load config from `SOL_CONFIG` path
-3. Load system prompt from `SOL_SYSTEM_PROMPT` path
-4. Read 3 secret env vars (`SOL_MATRIX_ACCESS_TOKEN`, `SOL_MATRIX_DEVICE_ID`, `SOL_MISTRAL_API_KEY`)
-5. Build Matrix client with E2EE sqlite store, restore session
-6. Connect to OpenSearch, ensure archive + memory indices exist
-7. Initialize Mistral client
-8. Build components: Personality, ConversationManager, ToolRegistry, Indexer, Evaluator, Responder
-9. Backfill conversation context from archive (if `backfill_on_join` enabled)
-10. Open SQLite database (fallback to in-memory on failure)
-11. Initialize AgentRegistry + ConversationRegistry (load persisted state from SQLite)
-12. If `use_conversations_api` enabled: ensure orchestrator agent exists on Mistral server
-13. Backfill reactions from Matrix room timelines
-14. Start background index flush task
-15. Start Matrix sync loop
-16. If SQLite failed: send `*sneezes*` to all joined rooms
-17. Log "Sol is running", wait for SIGINT
+```mermaid
+flowchart TD
+    start[Start] --> tracing[Init tracing<br/>RUST_LOG env filter]
+    tracing --> config[Load config + system prompt]
+    config --> secrets[Read env vars<br/>access token, device ID, API key]
+    secrets --> matrix[Build Matrix client<br/>E2EE sqlite store, restore session]
+    matrix --> opensearch[Connect OpenSearch<br/>ensure archive + memory + code indices]
+    opensearch --> mistral[Init Mistral client]
+    mistral --> components[Build components<br/>Personality, ConversationManager,<br/>ToolRegistry, Indexer, Evaluator]
+    components --> backfill[Backfill conversation context<br/>from archive]
+    backfill --> sqlite{Open SQLite}
+    sqlite --> |success| agents[Init AgentRegistry +<br/>ConversationRegistry]
+    sqlite --> |failure| inmemory[In-memory fallback]
+    inmemory --> agents
+    agents --> orchestrator{use_conversations_api?}
+    orchestrator --> |yes| ensure_agent[Ensure orchestrator agent<br/>exists on Mistral]
+    orchestrator --> |no| skip[Skip]
+    ensure_agent --> grpc{grpc config?}
+    skip --> grpc
+    grpc --> |yes| grpc_server[Start gRPC server<br/>on listen_addr]
+    grpc --> |no| skip_grpc[Skip]
+    grpc_server --> reactions[Backfill reactions<br/>from Matrix timelines]
+    skip_grpc --> reactions
+    reactions --> flush[Start background<br/>index flush task]
+    flush --> sync[Start Matrix sync loop]
+    sync --> sneeze{SQLite failed?}
+    sneeze --> |yes| sneeze_rooms[Send *sneezes*<br/>to all rooms]
+    sneeze --> |no| running[Sol is running]
+    sneeze_rooms --> running
+```

 ## monitoring

@@ -195,6 +217,8 @@ Key log events:
 | Conversation created | info | `room`, `conversation_id` |
 | Agent restored/created | info | `agent_id`, `name` |
 | Backfill complete | info | `rooms`, `messages` / `reactions` |
+| gRPC session started | info | `session_id`, `project` |
+| Code reindex complete | info | `repos_indexed`, `symbols_indexed` |

 Set `RUST_LOG=sol=debug` for verbose output including tool results, evaluation prompts, and memory details.

@@ -226,3 +250,12 @@ Sol auto-joins rooms on invite (3 retries with exponential backoff). If it can't
 **Agent creation failure:**

 If the orchestrator agent can't be created, Sol falls back to model-only conversations (no agent). Check Mistral API key and quota.
+
+**gRPC connection refused:**
+
+If `sunbeam code` can't connect, verify the gRPC server is configured and listening:
+
+```sh
+sunbeam k8s get svc sol-grpc -n matrix
+sunbeam logs matrix/sol | grep grpc
+```