studio/sol

Fork 0

Files

Sienna Meridian Satterwhite cccdb7b502

update documentation for sdk, vault, gitea integration

2026-03-22 15:00:51 +00:00

21 KiB

Raw Blame History

sol

a virtual librarian for Matrix. sol lives in your chat rooms, archives conversations in OpenSearch, and responds with the help of Mistral AI — with end-to-end encryption, tool use, per-user memory, and a multi-agent architecture.

sol is built by sunbeam studios as part of our self-hosted collaboration stack for a three-person game studio.

what sol does

Matrix presence — joins rooms, reads the vibe, decides when to speak. direct messages always get a response; in group rooms, sol evaluates relevance before jumping in.
message archive — every message is indexed in OpenSearch with full-text and semantic search. sol can search its own archive via tools.
tool use — Mistral calls tools mid-conversation: archive search, room context retrieval, room info, and a sandboxed TypeScript/JavaScript runtime (deno_core) for computation.
per-user memory — sol remembers things about the people it talks to. memories are extracted automatically after conversations, injected into the system prompt before responding, and accessible from scripts via sol.memory.get/set.
user impersonation — sol acts on behalf of users when calling external services. PATs are auto-provisioned via admin APIs and stored securely in OpenBao (Vault). OIDC-to-service username mappings handle identity mismatches.
gitea integration — first domain agent (sol-devtools): list repos, search issues, create issues, list PRs, get file contents — all as the requesting user.
multi-agent architecture — an orchestrator agent with personality + tools + web search. domain agent delegation is dynamic — only active agents appear in instructions. agent state persisted in SQLite with instructions hash for automatic recreation on prompt changes.
conversations API — persistent conversation state per room via Mistral's Conversations API, with automatic compaction at token thresholds. per-message context headers inject timestamps, room info, and memory notes.
multimodal — m.image messages are downloaded from Matrix via mxc://, converted to base64 data URIs, and sent as ContentPart::ImageUrl to Mistral vision models.
reactions — sol can react to messages with emoji when it has something to express but not enough to say.
E2EE — full end-to-end encryption via matrix-sdk with sqlite state store.

architecture

flowchart TD
    subgraph Matrix
        sync[Matrix Sync Loop]
    end

    subgraph Engagement
        eval[Evaluator]
        rules[Rule Checks]
        llm_eval[LLM Evaluation<br/>ministral-3b]
    end

    subgraph Response
        legacy[Legacy Path<br/>manual messages + chat completions]
        convapi[Conversations API Path<br/>ConversationRegistry + agents]
        tools[Tool Execution]
    end

    subgraph Persistence
        sqlite[(SQLite<br/>conversations + agents)]
        opensearch[(OpenSearch<br/>archive + memory)]
    end

    sync --> |message event| eval
    eval --> rules
    rules --> |MustRespond| Response
    rules --> |no rule match| llm_eval
    llm_eval --> |MaybeRespond| Response
    llm_eval --> |React| sync
    llm_eval --> |Ignore| sync
    legacy --> tools
    convapi --> tools
    tools --> opensearch
    legacy --> |response text| sync
    convapi --> |response text| sync
    sync --> |archive| opensearch
    convapi --> sqlite
    sync --> |memory extraction| opensearch

source tree

src/
├── main.rs                 entrypoint, Matrix client setup, backfill, orchestrator init
├── sync.rs                 event loop — messages, reactions, redactions, invites
├── config.rs               TOML config (5 sections) with serde defaults
├── context.rs              ResponseContext — per-message sender identity threading
├── conversations.rs        ConversationRegistry — room→conversation mapping, SQLite-backed
├── persistence.rs          SQLite store (WAL mode, 2 tables: conversations, agents)
├── agent_ux.rs             AgentProgress — reaction lifecycle (🔍→⚙️→✅) + thread posting
├── matrix_utils.rs         message extraction, reply/edit/thread detection, image download
├── archive/
│   ├── schema.rs           ArchiveDocument, OpenSearch index mapping
│   └── indexer.rs          batched indexing, reactions, edits, redactions
├── brain/
│   ├── conversation.rs     sliding-window context per room (configurable group/DM windows)
│   ├── evaluator.rs        engagement decision (MustRespond/MaybeRespond/React/Ignore)
│   ├── personality.rs      system prompt templating ({date}, {room_name}, {members}, etc.)
│   └── responder.rs        both response paths, tool iteration loops, memory loading
├── memory/
│   ├── schema.rs           MemoryDocument, index mapping
│   ├── store.rs            query (topical), get_recent, set — OpenSearch operations
│   └── extractor.rs        post-response fact extraction via ministral-3b
├── agents/
│   ├── definitions.rs      orchestrator config + 8 domain agent definitions (dynamic delegation)
│   └── registry.rs         agent lifecycle with instructions hash staleness detection
├── sdk/
│   ├── mod.rs              SDK module root
│   ├── vault.rs            OpenBao/Vault client (K8s auth, KV v2 read/write/delete)
│   ├── tokens.rs           TokenStore — Vault-backed secrets + SQLite username mappings
│   └── gitea.rs            GiteaClient — typed Gitea API v1 with PAT auto-provisioning
└── tools/
    ├── mod.rs              ToolRegistry — 12 tool definitions + dispatch (5 core + 7 gitea)
    ├── search.rs           archive search (keyword + semantic via embedding pipeline)
    ├── room_history.rs     context around a timestamp or event
    ├── room_info.rs        room listing, member queries
    ├── script.rs           deno_core sandbox with sol.* host API, TS transpilation
    ├── devtools.rs         Gitea tool handlers (repos, issues, PRs, files)
    └── bridge.rs           ToolBridge — generic async handler map for future SDK integration

engagement pipeline

sequenceDiagram
    participant M as Matrix
    participant S as Sync Handler
    participant E as Evaluator
    participant LLM as ministral-3b
    participant R as Responder

    M->>S: m.room.message
    S->>S: archive message
    S->>S: update conversation context
    S->>E: evaluate(sender, body, is_dm, recent)

    alt own message
        E-->>S: Ignore
    else @mention or matrix.to link
        E-->>S: MustRespond (DirectMention)
    else DM
        E-->>S: MustRespond (DirectMessage)
    else "sol" or "hey sol"
        E-->>S: MustRespond (NameInvocation)
    else no rule match
        E->>LLM: relevance evaluation (JSON)
        LLM-->>E: {relevance, hook, emoji}
        alt relevance >= spontaneous_threshold (0.85)
            E-->>S: MaybeRespond
        else relevance >= reaction_threshold (0.6) + emoji
            E-->>S: React (emoji)
        else below thresholds
            E-->>S: Ignore
        end
    end

    alt MustRespond or MaybeRespond
        S->>S: check in-flight guard
        S->>S: check cooldown (15s default)
        S->>R: generate response
    end

response generation

Sol has two response paths, controlled by agents.use_conversations_api:

legacy path (`generate_response`)

Apply response delay (random within configured range)
Send typing indicator
Load memory notes (topical query + recent backfill, max 5)
Build system prompt via Personality (template substitution: {date}, {room_name}, {members}, {memory_notes}, {room_context_rules}, {epoch_ms})
Assemble message array: system → context messages (with timestamps) → trigger (multimodal if image)
Tool iteration loop (up to max_tool_iterations, default 5):
- If finish_reason == ToolCalls: execute tools, append results, continue
- If text response: strip "sol:" prefix, return
Fire-and-forget memory extraction

conversations API path (`generate_response_conversations`)

Apply response delay
Send typing indicator
Format input: raw text for DMs, <@user:server> text for groups
Send through ConversationRegistry.send_message() (creates or appends to Mistral conversation)
Function call loop (up to max_tool_iterations):
- Execute tool calls locally via ToolRegistry
- Send FunctionResultEntry back to conversation
Extract assistant text, strip prefix, return

tool system

Tool	Parameters	Description
`search_archive`	`query` (required), `room`, `sender`, `after`, `before`, `limit`, `semantic`	Search the message archive (keyword or semantic)
`get_room_context`	`room_id` (required), `around_timestamp`, `around_event_id`, `before_count`, `after_count`	Get messages around a point in time or event
`list_rooms`	(none)	List all rooms Sol is in with names and member counts
`get_room_members`	`room_id` (required)	Get members of a specific room
`run_script`	`code` (required)	Execute TypeScript/JavaScript in a sandboxed deno_core runtime

run_script sandbox

The script runtime is a fresh V8 isolate per invocation with:

TypeScript support — code is transpiled via deno_ast before execution
Timeout — configurable via behavior.script_timeout_secs (default 5s), enforced by V8 isolate termination
Heap limit — configurable via behavior.script_max_heap_mb (default 64MB)
Output — console.log() + last expression value, truncated to 4096 characters
Temp filesystem — sandboxed sol.fs.read/write/list with path traversal protection
Network — sol.fetch(url) restricted to behavior.script_fetch_allowlist domains

Host API (sol.*):

sol.search(query, opts?)         // search message archive
sol.rooms()                      // list joined rooms → [{name, id, members}]
sol.members(roomName)            // get room members → [{name, id}]
sol.fetch(url)                   // HTTP GET (allowlisted domains only)
sol.memory.get(query?)           // retrieve memories relevant to query
sol.memory.set(content, category?) // save a memory note
sol.fs.read(path)                // read file from sandbox
sol.fs.write(path, content)      // write file to sandbox
sol.fs.list(path?)               // list sandbox directory

All sol.* methods are async — use await.

memory system

extraction (post-response, fire-and-forget)

After each response, a background task sends the exchange to ministral-3b with a structured extraction prompt. The model returns {"memories": [{"content": "...", "category": "preference|fact|context"}]}. Categories are normalized via normalize_category() — valid categories are preference, fact, context; anything else falls back to general.

storage (OpenSearch)

Each memory is a MemoryDocument with: id, user_id, content, category, created_at, updated_at, source ("auto" or "script"). The index name defaults to sol_user_memory. User isolation is enforced at the Rust level via user_id filtering on all queries.

pre-response loading

Before generating a response, the responder loads up to 5 memories:

Topical query — semantic search against the trigger message
Recent backfill — if fewer than 3 topical results, fill remaining slots with most recent memories

Memory notes are injected into the system prompt as a ## notes about {display_name} block with instructions to use them naturally without mentioning their existence.

agent architecture

stateDiagram-v2
    [*] --> CheckMemory: startup
    CheckMemory --> CheckServer: agent_id in SQLite?
    CheckMemory --> SearchByName: not in SQLite

    CheckServer --> Ready: exists on Mistral server
    CheckServer --> SearchByName: gone from server

    SearchByName --> Ready: found by name
    SearchByName --> Create: not found

    Create --> Ready: agent created
    Ready --> [*]

orchestrator

The orchestrator agent carries Sol's full personality (system prompt) plus all 5 tool definitions converted to AgentTool format. It's created on startup if agents.use_conversations_api is enabled. Temperature: 0.5.

domain agents (8 definitions)

Agent	Domain
`sol-observability`	Metrics, logs, dashboards, alerts (Prometheus, Loki, Grafana)
`sol-data`	Full-text search, object storage (OpenSearch, SeaweedFS)
`sol-devtools`	Git repos, issues, PRs, kanban boards (Gitea, Planka)
`sol-infrastructure`	Kubernetes, deployments, certificates, builds
`sol-identity`	User accounts, sessions, OAuth2 (Kratos, Hydra)
`sol-collaboration`	Contacts, documents, meetings, files, email, calendars (La Suite)
`sol-communication`	Chat rooms, messages, members (Matrix)
`sol-media`	Video/audio rooms, recordings, streams (LiveKit)

Domain agents are defined in agents/definitions.rs as DOMAIN_AGENTS (name, description, instructions). Temperature: 0.3.

ToolBridge

tools/bridge.rs provides a generic async handler map (ToolBridge) for mapping Mistral tool call names to handler functions. This is scaffolding for future SDK-based tool integration where domain agents will have their own tool sets.

persistence

SQLite database at /data/sol.db (configurable via matrix.db_path), WAL mode.

tables

conversations — room_id (PK), conversation_id, estimated_tokens, created_at

agents — name (PK), agent_id, model, created_at

recovery behavior

On startup, if the database fails to open:

Log error
Fall back to in-memory SQLite (conversations won't survive restarts)
After sync loop starts, send *sneezes* to all joined rooms to signal the hiccup

multimodal

When an m.image message arrives:

Extract media source from event (MessageType::Image)
Download bytes from Matrix media API via matrix_sdk::media::get_media_content
Base64-encode as data:{mime};base64,{data} URI
Pass to Mistral as ContentPart::ImageUrl alongside any text caption

Encrypted images are not supported (the MediaSource::Encrypted variant is skipped).

configuration reference

Config is loaded from SOL_CONFIG (default: /etc/sol/sol.toml).

`[matrix]`

Field	Type	Default	Description
`homeserver_url`	string	required	Matrix homeserver URL
`user_id`	string	required	Bot's Matrix user ID
`state_store_path`	string	required	Path for Matrix SDK sqlite state
`db_path`	string	`/data/sol.db`	SQLite database for persistent state

`[opensearch]`

Field	Type	Default	Description
`url`	string	required	OpenSearch cluster URL
`index`	string	required	Archive index name
`batch_size`	usize	`50`	Messages per flush batch
`flush_interval_ms`	u64	`2000`	Flush interval in milliseconds
`embedding_pipeline`	string	`tuwunel_embedding_pipeline`	Ingest pipeline for semantic embeddings
`memory_index`	string	`sol_user_memory`	Memory index name

`[mistral]`

Field	Type	Default	Description
`default_model`	string	`mistral-medium-latest`	Model for response generation
`evaluation_model`	string	`ministral-3b-latest`	Model for engagement evaluation + memory extraction
`research_model`	string	`mistral-large-latest`	Model for research tasks
`max_tool_iterations`	usize	`5`	Max tool call rounds per response

`[behavior]`

Field	Type	Default	Description
`response_delay_min_ms`	u64	`100`	Min delay before direct response
`response_delay_max_ms`	u64	`2300`	Max delay before direct response
`spontaneous_delay_min_ms`	u64	`15000`	Min delay before spontaneous response
`spontaneous_delay_max_ms`	u64	`60000`	Max delay before spontaneous response
`spontaneous_threshold`	f32	`0.85`	LLM relevance score to trigger spontaneous response
`reaction_threshold`	f32	`0.6`	LLM relevance score to trigger emoji reaction
`reaction_enabled`	bool	`true`	Enable emoji reactions
`room_context_window`	usize	`30`	Messages to keep in group room context
`dm_context_window`	usize	`100`	Messages to keep in DM context
`backfill_on_join`	bool	`true`	Backfill context from archive on startup
`backfill_limit`	usize	`10000`	Max messages to backfill
`instant_responses`	bool	`false`	Skip response delays (for testing)
`cooldown_after_response_ms`	u64	`15000`	Cooldown before another spontaneous response
`evaluation_context_window`	usize	`25`	Recent messages sent to evaluation LLM
`detect_sol_in_conversation`	bool	`true`	Use active/passive evaluation prompts based on Sol's participation
`evaluation_prompt_active`	string?	(built-in)	Custom prompt when Sol is in conversation
`evaluation_prompt_passive`	string?	(built-in)	Custom prompt when Sol hasn't spoken
`script_timeout_secs`	u64	`5`	Script execution timeout
`script_max_heap_mb`	usize	`64`	V8 heap limit for scripts
`script_fetch_allowlist`	string[]	`[]`	Domains allowed for `sol.fetch()`
`memory_extraction_enabled`	bool	`true`	Enable post-response memory extraction

`[agents]`

Field	Type	Default	Description
`orchestrator_model`	string	`mistral-medium-latest`	Model for orchestrator agent
`domain_model`	string	`mistral-medium-latest`	Model for domain agents
`compaction_threshold`	u32	`118000`	Token estimate before conversation reset (~90% of 131K context)
`use_conversations_api`	bool	`false`	Enable Conversations API path (vs legacy chat completions)

environment variables

Variable	Required	Description
`SOL_MATRIX_ACCESS_TOKEN`	yes	Matrix access token
`SOL_MATRIX_DEVICE_ID`	yes	Matrix device ID (for E2EE state)
`SOL_MISTRAL_API_KEY`	yes	Mistral API key
`SOL_CONFIG`	no	Config file path (default: `/etc/sol/sol.toml`)
`SOL_SYSTEM_PROMPT`	no	System prompt file path (default: `/etc/sol/system_prompt.md`)

dependencies

sol talks to five external services:

Matrix homeserver — tuwunel (or any Matrix server)
OpenSearch — message archive + user memory indices
Mistral AI — response generation, engagement evaluation, memory extraction, agents + web search
OpenBao — secure token storage for user impersonation PATs (K8s auth, KV v2)
Gitea — git hosting API for devtools agent (repos, issues, PRs)

key crates: matrix-sdk 0.9 (E2EE + sqlite), mistralai-client 1.1.0 (private registry), opensearch 2, deno_core 0.393, rusqlite 0.32 (bundled), ruma 0.12.

building

cargo build --release

docker (cross-compile to x86_64 linux, vendored deps):

docker build -t sol .

production build + deploy:

sunbeam build sol --push --deploy

the Dockerfile uses a two-stage build: deps layer (cached until Cargo.toml/vendor change) → source layer (only sol code recompiles). final image is gcr.io/distroless/cc-debian12:nonroot.

testing

cargo test

unit tests covering:

config parsing (minimal, full, missing sections/fields, services, vault)
conversation windowing, context management, reset_all, delete_all
engagement rules (mention, DM, name invocation, case sensitivity, false positives)
personality template substitution (date, room, members, memory notes, timestamps, room context rules)
memory document serialization, extraction parsing, category normalization
archive search query building (filters, date ranges, wildcards, room_name keyword field)
TypeScript transpilation (basic, arrow, interface, invalid)
sandbox path isolation (traversal, symlink escape, nested dirs)
deno_core script execution (basic math, output capture)
SQLite CRUD (conversations, agents, service_users, load_all, bulk delete)
conversation message merging (DM, group, empty, single)
context derivation (@user:server → user@server, localpart extraction)
tool bridge registration and execution
agent UX formatting (tool calls, result truncation)
agent definitions (orchestrator instructions, dynamic delegation, deterministic hash)
token expiry validation (PAT, future, past, malformed, null)
Gitea API type deserialization (repos, issues, PRs, files)
PAT conflict status codes and scope constants
username mapping (OIDC → service identity)

license

MIT

21 KiB Raw Blame History

sol