sol/README.md

# Sol

A virtual librarian for Matrix. Sol lives in your chat rooms, archives conversations in OpenSearch, and responds with the help of Mistral AI — with end-to-end encryption, tool use, per-user memory, and a multi-agent architecture.

Sol is built by [Sunbeam Studios](https://sunbeam.pt) as part of our self-hosted collaboration stack.

## What Sol Does

- **Matrix Presence** — Joins rooms, reads the vibe, decides when to speak. Direct messages always get a response; in group rooms, Sol evaluates relevance before jumping in.
- **Message Archive** — Every message is indexed in OpenSearch with full-text and semantic search. Sol can search its own archive via tools.
- **Tool Use** — Mistral calls tools mid-conversation: archive search, room context retrieval, room info, and a sandboxed TypeScript/JavaScript runtime (deno_core) for computation.
- **Per-User Memory** — Sol remembers things about the people it talks to. Memories are extracted automatically after conversations, injected into the system prompt before responding, and accessible from scripts via `sol.memory.get/set`.
- **User Impersonation** — Sol acts on behalf of users when calling external services. PATs are auto-provisioned via admin APIs and stored securely in OpenBao (Vault). OIDC-to-service username mappings handle identity mismatches.
- **Gitea Integration** — First domain agent (sol-devtools): list repos, search issues, create issues, list PRs, get file contents — all as the requesting user.
- **Multi-Agent Architecture** — An orchestrator agent with personality + tools + web search. Domain agent delegation is dynamic — only active agents appear in instructions. Agent state is persisted in SQLite with instructions hash for automatic recreation on prompt changes.
- **Conversations API** — Persistent conversation state per room via Mistral's Conversations API, with automatic compaction at token thresholds. Per-message context headers inject timestamps, room info, and memory notes.
- **Multimodal** — `m.image` messages are downloaded from Matrix via `mxc://`, converted to base64 data URIs, and sent as `ContentPart::ImageUrl` to Mistral vision models.
- **Reactions** — Sol can react to messages with emoji when it has something to express but not enough to say.
- **E2EE** — Full end-to-end encryption via matrix-sdk with SQLite state store.

## Architecture

```mermaid
flowchart TD
    subgraph Matrix
        sync[Matrix Sync Loop]
    end

    subgraph Engagement
        eval[Evaluator]
        rules[Rule Checks]
        llm_eval[LLM Evaluation<br/>ministral-3b]
    end

    subgraph Response
        legacy[Legacy Path<br/>Manual messages + chat completions]
        convapi[Conversations API Path<br/>ConversationRegistry + agents]
        tools[Tool Execution]
    end

    subgraph Persistence
        sqlite[(SQLite<br/>conversations + agents)]
        opensearch[(OpenSearch<br/>archive + memory)]
    end

    sync --> |message event| eval
    eval --> rules
    rules --> |MustRespond| Response
    rules --> |no rule match| llm_eval
    llm_eval --> |MaybeRespond| Response
    llm_eval --> |React| sync
    llm_eval --> |Ignore| sync
    legacy --> tools
    convapi --> tools
    tools --> opensearch
    legacy --> |response text| sync
    convapi --> |response text| sync
    sync --> |archive| opensearch
    convapi --> sqlite
    sync --> |memory extraction| opensearch
```

## Source Tree

```
src/
├── main.rs                 Entrypoint, Matrix client setup, backfill, orchestrator init
├── sync.rs                 Event loop — messages, reactions, redactions, invites
├── config.rs               TOML config with serde defaults
├── context.rs              ResponseContext — per-message sender identity threading
├── conversations.rs        ConversationRegistry — room→conversation mapping, SQLite-backed
├── persistence.rs          SQLite store (WAL mode, tables: conversations, agents, service_users)
├── agent_ux.rs             AgentProgress — reaction lifecycle (🔍→⚙️→✅) + thread posting
├── matrix_utils.rs         Message extraction, reply/edit/thread detection, image download
├── archive/
│   ├── schema.rs           ArchiveDocument, OpenSearch index mapping
│   └── indexer.rs          Batched indexing, reactions, edits, redactions
├── brain/
│   ├── conversation.rs     Sliding-window context per room (configurable group/DM windows)
│   ├── evaluator.rs        Engagement decision (MustRespond/MaybeRespond/React/Ignore)
│   ├── personality.rs      System prompt templating ({date}, {room_name}, {members}, etc.)
│   └── responder.rs        Both response paths, tool iteration loops, memory loading
├── memory/
│   ├── schema.rs           MemoryDocument, index mapping
│   ├── store.rs            Query (topical), get_recent, set — OpenSearch operations
│   └── extractor.rs        Post-response fact extraction via ministral-3b
├── agents/
│   ├── definitions.rs      Orchestrator config + domain agent definitions (dynamic delegation)
│   └── registry.rs         Agent lifecycle with instructions hash staleness detection
├── sdk/
│   ├── mod.rs              SDK module root
│   ├── vault.rs            OpenBao/Vault client (K8s auth, KV v2 read/write/delete)
│   ├── tokens.rs           TokenStore — Vault-backed secrets + SQLite username mappings
│   └── gitea.rs            GiteaClient — typed Gitea API v1 with PAT auto-provisioning
└── tools/
    ├── mod.rs              ToolRegistry — tool definitions + dispatch (core + gitea)
    ├── search.rs           Archive search (keyword + semantic via embedding pipeline)
    ├── room_history.rs     Context around a timestamp or event
    ├── room_info.rs        Room listing, member queries
    ├── script.rs           deno_core sandbox with sol.* host API, TS transpilation
    ├── devtools.rs         Gitea tool handlers (repos, issues, PRs, files)
    └── bridge.rs           ToolBridge — generic async handler map for future SDK integration
```

## Engagement Pipeline

```mermaid
sequenceDiagram
    participant M as Matrix
    participant S as Sync Handler
    participant E as Evaluator
    participant LLM as ministral-3b
    participant R as Responder

    M->>S: m.room.message
    S->>S: Archive message
    S->>S: Update conversation context
    S->>E: evaluate(sender, body, is_dm, recent)

    alt Own message
        E-->>S: Ignore
    else @mention or matrix.to link
        E-->>S: MustRespond (DirectMention)
    else DM
        E-->>S: MustRespond (DirectMessage)
    else "sol" or "hey sol"
        E-->>S: MustRespond (NameInvocation)
    else No rule match
        E->>LLM: Relevance evaluation (JSON)
        LLM-->>E: {relevance, hook, emoji}
        alt relevance >= spontaneous_threshold (0.85)
            E-->>S: MaybeRespond
        else relevance >= reaction_threshold (0.6) + emoji
            E-->>S: React (emoji)
        else Below thresholds
            E-->>S: Ignore
        end
    end

    alt MustRespond or MaybeRespond
        S->>S: Check in-flight guard
        S->>S: Check cooldown (15s default)
        S->>R: Generate response
    end
```

## Response Generation

Sol has two response paths, controlled by `agents.use_conversations_api`:

### Legacy Path (`generate_response`)

1. Apply response delay (random within configured range)
2. Send typing indicator
3. Load memory notes (topical query + recent backfill, max 5)
4. Build system prompt via `Personality` (template substitution: `{date}`, `{room_name}`, `{members}`, `{memory_notes}`, `{room_context_rules}`, `{epoch_ms}`)
5. Assemble message array: system → context messages (with timestamps) → trigger (multimodal if image)
6. Tool iteration loop (up to `max_tool_iterations`, default 5):
   - If `finish_reason == ToolCalls`: execute tools, append results, continue
   - If text response: strip "sol:" prefix, return
7. Fire-and-forget memory extraction

### Conversations API Path (`generate_response_conversations`)

1. Apply response delay
2. Send typing indicator
3. Load memory notes for the user
4. Build per-message context header (timestamps, room name, memory notes)
5. Format input: raw text for DMs, `<@user:server> text` for groups
6. Send through `ConversationRegistry.send_message()` (creates or appends to Mistral conversation)
7. Function call loop (up to `max_tool_iterations`):
   - Execute tool calls locally via `ToolRegistry`
   - Send `FunctionResultEntry` back to conversation
8. Extract assistant text, strip prefix, return

## Tool System

| Tool | Parameters | Description |
|------|-----------|-------------|
| `search_archive` | `query` (required), `room`, `sender`, `after`, `before`, `limit`, `semantic` | Search the message archive (keyword or semantic) |
| `get_room_context` | `room_id` (required), `around_timestamp`, `around_event_id`, `before_count`, `after_count` | Get messages around a point in time or event |
| `list_rooms` | *(none)* | List all rooms Sol is in with names and member counts |
| `get_room_members` | `room_id` (required) | Get members of a specific room |
| `run_script` | `code` (required) | Execute TypeScript/JavaScript in a sandboxed deno_core runtime |
| `gitea_list_repos` | `query`, `org`, `limit` | List or search repositories on Gitea |
| `gitea_get_repo` | `owner`, `repo` (required) | Get details about a specific repository |
| `gitea_list_issues` | `owner`, `repo` (required), `state`, `labels`, `limit` | List issues in a repository |
| `gitea_get_issue` | `owner`, `repo`, `number` (required) | Get full details of a specific issue |
| `gitea_create_issue` | `owner`, `repo`, `title` (required), `body`, `labels` | Create a new issue as the requesting user |
| `gitea_list_pulls` | `owner`, `repo` (required), `state`, `limit` | List pull requests in a repository |
| `gitea_get_file` | `owner`, `repo`, `path` (required), `ref` | Get file contents from a repository |

### `run_script` Sandbox

The script runtime is a fresh V8 isolate per invocation with:

- **TypeScript support** — Code is transpiled via `deno_ast` before execution
- **Timeout** — Configurable via `behavior.script_timeout_secs` (default 5s), enforced by V8 isolate termination
- **Heap limit** — Configurable via `behavior.script_max_heap_mb` (default 64MB)
- **Output** — `console.log()` + last expression value, truncated to 4096 characters
- **Temp filesystem** — Sandboxed `sol.fs.read/write/list` with path traversal protection
- **Network** — `sol.fetch(url)` restricted to `behavior.script_fetch_allowlist` domains

Host API (`sol.*`):

```typescript
sol.search(query, opts?)         // Search message archive
sol.rooms()                      // List joined rooms → [{name, id, members}]
sol.members(roomName)            // Get room members → [{name, id}]
sol.fetch(url)                   // HTTP GET (allowlisted domains only)
sol.memory.get(query?)           // Retrieve memories relevant to query
sol.memory.set(content, category?) // Save a memory note
sol.fs.read(path)                // Read file from sandbox
sol.fs.write(path, content)      // Write file to sandbox
sol.fs.list(path?)               // List sandbox directory
```

All `sol.*` methods are async — use `await`.

## Memory System

### Extraction (Post-Response, Fire-and-Forget)

After each response, a background task sends the exchange to `ministral-3b` with a structured extraction prompt. The model returns `{"memories": [{"content": "...", "category": "preference|fact|context"}]}`. Categories are normalized via `normalize_category()` — valid categories are `preference`, `fact`, `context`; anything else falls back to `general`.

### Storage (OpenSearch)

Each memory is a `MemoryDocument` with: `id`, `user_id`, `content`, `category`, `created_at`, `updated_at`, `source` (`"auto"` or `"script"`). The index name defaults to `sol_user_memory`. User isolation is enforced at the Rust level via `user_id` filtering on all queries.

### Pre-Response Loading

Before generating a response, the responder loads up to 5 memories:

1. **Topical query** — Semantic search against the trigger message
2. **Recent backfill** — If fewer than 3 topical results, fill remaining slots with most recent memories

Memory notes are injected into the system prompt (legacy path) or per-message context header (Conversations API path) as a `## notes about {display_name}` block.

## Archive

Every message event is archived as an `ArchiveDocument` in OpenSearch:

- **Batch indexing** — Messages are buffered and flushed periodically (`opensearch.batch_size` default 50, `opensearch.flush_interval_ms` default 2000)
- **Embedding pipeline** — Configurable via `opensearch.embedding_pipeline` for semantic search
- **Edit tracking** — `m.replace` events update the original document's content
- **Redaction** — `m.room.redaction` sets `redacted: true` on the original
- **Reactions** — `m.reaction` events append `{sender, emoji, timestamp}` to the document's reactions array
- **Backfill** — On startup, conversation context is backfilled from the archive; reactions are backfilled from Matrix room timelines (last 500 events per room)

## Agent Architecture

```mermaid
stateDiagram-v2
    [*] --> CheckHash: Startup
    CheckHash --> Restore: Hash matches
    CheckHash --> Recreate: Hash changed

    Restore --> CheckServer: Agent ID in SQLite
    Restore --> SearchByName: Not in SQLite

    CheckServer --> Ready: Exists on Mistral server
    CheckServer --> SearchByName: Gone from server

    SearchByName --> Recreate: Not found
    Recreate --> Ready: Agent created, conversations reset, *sneezes*

    Ready --> [*]
```

### Orchestrator

The orchestrator agent carries Sol's full personality (system prompt) plus all tool definitions converted to `AgentTool` format, including Mistral's built-in `web_search`. It's created on startup if `agents.use_conversations_api` is enabled.

When the system prompt changes, the instructions hash detects staleness and the agent is automatically recreated. All existing conversations are reset and Sol sneezes into all rooms to signal the context reset.

### Domain Agents

Domain agents are defined in `agents/definitions.rs` as `DOMAIN_AGENTS` (name, description, instructions). The delegation section in the orchestrator's instructions is built dynamically — only agents that are actually registered appear.

### User Impersonation

Sol authenticates to OpenBao via Kubernetes auth (role `sol-agent`) and stores per-user PATs at `secret/sol-tokens/{localpart}/{service}`. The `service_users` SQLite table maps Matrix localparts to service-specific usernames, handling cases where OIDC auto-registration produces different names.

For Gitea, PATs are auto-provisioned via the admin API on first use. The username is discovered by direct match or email-based search.

## Persistence

SQLite database at `/data/sol.db` (configurable via `matrix.db_path`), WAL mode.

### Tables

- **conversations** — room_id (PK), conversation_id, estimated_tokens, created_at
- **agents** — name (PK), agent_id, model, instructions_hash, created_at
- **service_users** — (localpart, service) PK, service_username, created_at

### Recovery Behavior

On startup, if the database fails to open:

1. Log error
2. Fall back to in-memory SQLite (conversations won't survive restarts)
3. After sync loop starts, send `*sneezes*` to all joined rooms to signal the hiccup

The same sneeze happens when the orchestrator agent is recreated due to prompt changes.

## Configuration Reference

Config is loaded from `SOL_CONFIG` (default: `/etc/sol/sol.toml`).

### `[matrix]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `homeserver_url` | string | *required* | Matrix homeserver URL |
| `user_id` | string | *required* | Bot's Matrix user ID |
| `state_store_path` | string | *required* | Path for Matrix SDK SQLite state |
| `db_path` | string | `/data/sol.db` | SQLite database for persistent state |

### `[opensearch]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | *required* | OpenSearch cluster URL |
| `index` | string | *required* | Archive index name |
| `batch_size` | usize | `50` | Messages per flush batch |
| `flush_interval_ms` | u64 | `2000` | Flush interval in milliseconds |
| `embedding_pipeline` | string | `tuwunel_embedding_pipeline` | Ingest pipeline for semantic embeddings |
| `memory_index` | string | `sol_user_memory` | Memory index name |

### `[mistral]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `default_model` | string | `mistral-medium-latest` | Model for response generation |
| `evaluation_model` | string | `ministral-3b-latest` | Model for engagement evaluation + memory extraction |
| `research_model` | string | `mistral-large-latest` | Model for research tasks |
| `max_tool_iterations` | usize | `5` | Max tool call rounds per response |

### `[behavior]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `response_delay_min_ms` | u64 | `100` | Min delay before direct response |
| `response_delay_max_ms` | u64 | `2300` | Max delay before direct response |
| `spontaneous_delay_min_ms` | u64 | `15000` | Min delay before spontaneous response |
| `spontaneous_delay_max_ms` | u64 | `60000` | Max delay before spontaneous response |
| `spontaneous_threshold` | f32 | `0.85` | LLM relevance score to trigger spontaneous response |
| `reaction_threshold` | f32 | `0.6` | LLM relevance score to trigger emoji reaction |
| `reaction_enabled` | bool | `true` | Enable emoji reactions |
| `room_context_window` | usize | `200` | Messages to keep in group room context |
| `dm_context_window` | usize | `200` | Messages to keep in DM context |
| `backfill_on_join` | bool | `true` | Backfill context from archive on startup |
| `backfill_limit` | usize | `10000` | Max messages to backfill |
| `instant_responses` | bool | `false` | Skip response delays (for testing) |
| `cooldown_after_response_ms` | u64 | `15000` | Cooldown before another spontaneous response |
| `evaluation_context_window` | usize | `200` | Recent messages sent to evaluation LLM |
| `detect_sol_in_conversation` | bool | `true` | Use active/passive evaluation prompts based on Sol's participation |
| `evaluation_prompt_active` | string? | *(built-in)* | Custom prompt when Sol is in conversation |
| `evaluation_prompt_passive` | string? | *(built-in)* | Custom prompt when Sol hasn't spoken |
| `script_timeout_secs` | u64 | `5` | Script execution timeout |
| `script_max_heap_mb` | usize | `64` | V8 heap limit for scripts |
| `script_fetch_allowlist` | string[] | `[]` | Domains allowed for `sol.fetch()` |
| `memory_extraction_enabled` | bool | `true` | Enable post-response memory extraction |

### `[agents]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `orchestrator_model` | string | `mistral-medium-latest` | Model for orchestrator agent |
| `domain_model` | string | `mistral-medium-latest` | Model for domain agents |
| `compaction_threshold` | u32 | `118000` | Token estimate before conversation reset (~90% of 131K context) |
| `use_conversations_api` | bool | `false` | Enable Conversations API path (vs legacy chat completions) |

### `[vault]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | `http://openbao.data.svc.cluster.local:8200` | OpenBao/Vault URL |
| `role` | string | `sol-agent` | Kubernetes auth role name |
| `mount` | string | `secret` | KV v2 mount path |

### `[services.gitea]`

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | *required if enabled* | Gitea API base URL |

## Environment Variables

| Variable | Required | Description |
|----------|----------|-------------|
| `SOL_MATRIX_ACCESS_TOKEN` | Yes | Matrix access token |
| `SOL_MATRIX_DEVICE_ID` | Yes | Matrix device ID (for E2EE state) |
| `SOL_MISTRAL_API_KEY` | Yes | Mistral API key |
| `SOL_GITEA_ADMIN_USERNAME` | No | Gitea admin username (enables devtools agent) |
| `SOL_GITEA_ADMIN_PASSWORD` | No | Gitea admin password |
| `SOL_CONFIG` | No | Config file path (default: `/etc/sol/sol.toml`) |
| `SOL_SYSTEM_PROMPT` | No | System prompt file path (default: `/etc/sol/system_prompt.md`) |

## Dependencies

Sol talks to five external services:

- **Matrix homeserver** — [Tuwunel](https://github.com/tulir/tuwunel) (or any Matrix server)
- **OpenSearch** — Message archive + user memory indices
- **Mistral AI** — Response generation, engagement evaluation, memory extraction, agents + web search
- **OpenBao** — Secure token storage for user impersonation PATs (K8s auth, KV v2)
- **Gitea** — Git hosting API for devtools agent (repos, issues, PRs)

Key crates: `matrix-sdk` 0.9 (E2EE + SQLite), `mistralai-client` 1.1.0 (private registry), `opensearch` 2, `deno_core` 0.393, `rusqlite` 0.32 (bundled), `ruma` 0.12.

## Building

```sh
cargo build --release
```

Docker (cross-compile to x86_64 Linux, vendored deps):

```sh
docker build -t sol .
```

Production build + deploy:

```sh
sunbeam build sol --push --deploy
```

The Dockerfile uses a two-stage build: deps layer (cached until Cargo.toml/vendor change) → source layer (only Sol code recompiles). Final image is `gcr.io/distroless/cc-debian12:nonroot`.

## Testing

```sh
cargo test
```

## License

Sol is dual-licensed:

- **Open Source** — [GNU Affero General Public License v3.0](LICENSE) (AGPL-3.0-or-later). You can use, modify, and distribute Sol freely under the AGPL. If you run a modified version as a network service, you must share your changes under the same license.

- **Commercial** — For organizations that want to use Sol without AGPL obligations (private modifications, proprietary integrations, no source disclosure), a commercial license is available. The commercial license grants unlimited internal use but does not permit redistribution.

For commercial licensing or other licensing questions, contact [hello@sunbeam.pt](mailto:hello@sunbeam.pt).