studio/sol
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
261c39b424302108d923efd5f0ee61f81235120c
sol/Cargo.toml

59 lines
1.7 KiB
TOML
Raw Normal View History

feat: initial Sol virtual librarian implementation Matrix bot with E2EE (matrix-sdk 0.9) that passively archives all messages to OpenSearch and responds to queries via Mistral AI with function calling tools. Core systems: - Archive: bulk OpenSearch indexer with batch/flush, edit/redaction handling, embedding pipeline passthrough - Brain: rule-based engagement evaluator (mentions, DMs, name invocations), LLM-powered spontaneous engagement, per-room conversation context windows, response delay simulation - Tools: search_archive, get_room_context, list_rooms, get_room_members registered as Mistral function calling tools with iterative tool loop - Personality: templated system prompt with Sol's librarian persona 47 unit tests covering config, evaluator, conversation windowing, personality templates, schema serialization, and search query building.
2026-03-20 21:40:13 +00:00
[package]
name = "sol"
version = "0.1.0"
edition = "2021"
rust-version = "1.76.0"
authors = ["Sunbeam Studios <hello@sunbeam.pt>"]
description = "Sol — virtual librarian Matrix bot with E2EE, OpenSearch archive, and Mistral AI"
sol 1.0.0 — relicense to AGPL-3.0, dual-license model relicensed from MIT to AGPL-3.0-or-later. commercial license available for organizations that need private modifications (does not permit redistribution). sol 1.0.0 ships with: - multi-agent architecture with mistral conversations API - user impersonation via vault-backed PAT provisioning - gitea integration (first domain agent: devtools) - per-user memory system with automatic extraction - full-context evaluator with system prompt awareness - agent recreation on prompt changes with conversation reset - web search, sandboxed deno runtime, archive search
2026-03-22 15:13:56 +00:00
license = "AGPL-3.0-or-later"
add README, MIT license, and package metadata
2026-03-21 15:53:49 +00:00
repository = "https://src.sunbeam.pt/studio/sol"
feat: initial Sol virtual librarian implementation Matrix bot with E2EE (matrix-sdk 0.9) that passively archives all messages to OpenSearch and responds to queries via Mistral AI with function calling tools. Core systems: - Archive: bulk OpenSearch indexer with batch/flush, edit/redaction handling, embedding pipeline passthrough - Brain: rule-based engagement evaluator (mentions, DMs, name invocations), LLM-powered spontaneous engagement, per-room conversation context windows, response delay simulation - Tools: search_archive, get_room_context, list_rooms, get_room_members registered as Mistral function calling tools with iterative tool loop - Personality: templated system prompt with Sol's librarian persona 47 unit tests covering config, evaluator, conversation windowing, personality templates, schema serialization, and search query building.
2026-03-20 21:40:13 +00:00
[[bin]]
name = "sol"
path = "src/main.rs"
[dependencies]
feat: streaming conversation responses via Mistral API - Bump mistralai-client to 1.2.0 (conversation streaming support) - session.rs: add create_or_append_conversation_streaming() — calls Mistral with stream:true, emits TextDelta messages to gRPC client as SSE chunks arrive, accumulates into ConversationResponse for orchestrator tool loop. Handles corrupted conversation recovery - service.rs: session_chat_via_orchestrator uses streaming variant - Integration tests: streaming create + append against real Mistral API, SSE parsing, accumulate text + function calls
2026-03-24 21:33:01 +00:00
mistralai-client = { version = "1.2.0", registry = "sunbeam" }
feat: initial Sol virtual librarian implementation Matrix bot with E2EE (matrix-sdk 0.9) that passively archives all messages to OpenSearch and responds to queries via Mistral AI with function calling tools. Core systems: - Archive: bulk OpenSearch indexer with batch/flush, edit/redaction handling, embedding pipeline passthrough - Brain: rule-based engagement evaluator (mentions, DMs, name invocations), LLM-powered spontaneous engagement, per-room conversation context windows, response delay simulation - Tools: search_archive, get_room_context, list_rooms, get_room_members registered as Mistral function calling tools with iterative tool loop - Personality: templated system prompt with Sol's librarian persona 47 unit tests covering config, evaluator, conversation windowing, personality templates, schema serialization, and search query building.
2026-03-20 21:40:13 +00:00
matrix-sdk = { version = "0.9", features = ["e2e-encryption", "sqlite"] }
opensearch = "2"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
refactor: remove legacy chat path, fix corrupted conversation recovery - Delete CodeSession::chat() — the legacy inline tool loop that duplicated the orchestrator's conversation + tool dispatch logic - Delete wait_for_tool_result() — only used by the legacy path - Make orchestrator mandatory in run_session (no more if/else fallback) - Unify conversation creation into create_fresh_conversation() - Add corrupted conversation recovery to create_or_append_conversation: detects "function calls and responses" errors from Mistral (caused by disconnecting mid-tool-call) and auto-creates a fresh conversation - Add tracing-appender for optional rotating log file (SOL_LOG_FILE env) - Add Procfile.dev for overmind process management
2026-03-24 19:49:07 +00:00
tracing-appender = "0.2"
feat: initial Sol virtual librarian implementation Matrix bot with E2EE (matrix-sdk 0.9) that passively archives all messages to OpenSearch and responds to queries via Mistral AI with function calling tools. Core systems: - Archive: bulk OpenSearch indexer with batch/flush, edit/redaction handling, embedding pipeline passthrough - Brain: rule-based engagement evaluator (mentions, DMs, name invocations), LLM-powered spontaneous engagement, per-room conversation context windows, response delay simulation - Tools: search_archive, get_room_context, list_rooms, get_room_members registered as Mistral function calling tools with iterative tool loop - Personality: templated system prompt with Sol's librarian persona 47 unit tests covering config, evaluator, conversation windowing, personality templates, schema serialization, and search query building.
2026-03-20 21:40:13 +00:00
rand = "0.8"
regex = "1"
anyhow = "1"
chrono = { version = "0.4", features = ["serde"] }
url = "2"
feat: per-user auto-memory with ResponseContext Three memory channels: hidden tool (sol.memory.set/get in scripts), pre-response injection (relevant memories loaded into system prompt), and post-response extraction (ministral-3b extracts facts after each response). User isolation enforced at Rust level — user_id derived from Matrix sender, never from script arguments. New modules: context (ResponseContext), memory (schema, store, extractor). ResponseContext threaded through responder → tools → script runtime. OpenSearch index sol_user_memory created on startup alongside archive.
2026-03-21 15:51:31 +00:00
ruma = { version = "0.12", features = ["events", "client", "markdown"] }
libsqlite3-sys = { version = "0.30", features = ["bundled"] }
deno_core = "0.393"
deno_ast = { version = "0.53", features = ["transpiling"] }
deno_error = "0.7"
tempfile = "3"
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls"] }
uuid = { version = "1", features = ["v4"] }
feat: multi-agent architecture with Conversations API and persistent state Mistral Agents + Conversations API integration: - Orchestrator agent created on startup with Sol's personality + tools - ConversationRegistry routes messages through persistent conversations - Per-room conversation state (room_id → conversation_id + token counts) - Function call handling within conversation responses - Configurable via [agents] section in sol.toml (use_conversations_api flag) Multimodal support: - m.image detection and Matrix media download (mxc:// → base64 data URI) - ContentPart-based messages sent to Mistral vision models - Archive stores media_urls for image messages System prompt rewrite: - 687 → 150 lines — dense, few-shot examples, hard rules - {room_context_rules} placeholder for group vs DM behavior - Sender prefixing (<@user:server>) for multi-user turns in group rooms SQLite persistence (/data/sol.db): - Conversation mappings and agent IDs survive reboots - WAL mode for concurrent reads - Falls back to in-memory on failure (sneezes into all rooms to signal) - PVC already mounted at /data alongside Matrix SDK state store New modules: - src/persistence.rs — SQLite state store - src/conversations.rs — ConversationRegistry + message merging - src/agents/{mod,definitions,registry}.rs — agent lifecycle - src/agent_ux.rs — reaction + thread progress UX - src/tools/bridge.rs — tool dispatch for domain agents 102 tests passing.
2026-03-21 22:21:14 +00:00
base64 = "0.22"
rusqlite = { version = "0.32", features = ["bundled"] }
multi-agent research: parallel LLM-powered investigation new research tool spawns 3-25 micro-agents (ministral-3b) in parallel via futures::join_all. each agent gets its own Mistral conversation with full tool access. recursive spawning up to depth 4 — agents can spawn sub-agents. research sessions persisted in SQLite (survive reboots). thread UX: 🔍 reaction, per-agent progress posts, ✅ when done. cost: ~$0.03 per research task (20 micro-agents on ministral-3b).
2026-03-23 01:42:40 +00:00
futures = "0.3"
feat(code): gRPC server with JWT auth + tool routing tonic 0.14 gRPC server for sunbeam code sessions: - bidirectional streaming Session RPC - JWT interceptor validates tokens against Hydra JWKS - tool router classifies calls as client-side (file_read, bash, grep, etc.) or server-side (gitea, identity, search, etc.) - service stub with session lifecycle (start, chat, tool results, end) - coding_model config (default: devstral-small-2506) - grpc config section (listen_addr, jwks_url) - 182 tests (5 new: JWT claims, tool routing) phase 2 TODOs: Matrix room bridge, Mistral agent loop, streaming
2026-03-23 11:35:37 +00:00
tonic = "0.14"
tonic-prost = "0.14"
prost = "0.14"
tokio-stream = "0.1"
jsonwebtoken = "9"
feat(orchestrator): Phase 2 engine + tokenizer + tool dispatch Orchestrator engine: - engine.rs: unified Mistral Conversations API tool loop that emits OrchestratorEvent instead of calling Matrix/gRPC directly - tool_dispatch.rs: ToolSide routing (client vs server tools) - Memory loading stubbed (migrates in Phase 4) Server-side tokenizer: - tokenizer.rs: HuggingFace tokenizers-rs with Mistral's BPE tokenizer - count_tokens() for accurate usage metrics - Loads from local tokenizer.json or falls back to bundled vocab - Config: mistral.tokenizer_path (optional) No behavior change — engine is wired but not yet called from sync.rs or session.rs (Phase 2 continuation).
2026-03-23 17:40:25 +00:00
tokenizers = { version = "0.22", default-features = false, features = ["onig", "http"] }
feat: Gitea repo indexing via gRPC ReindexCode endpoint Gitea indexer (code_index/gitea.rs): - Walks repos via GiteaClient API (list repos → traverse dirs → fetch files) - Base64 decodes file content from Gitea API responses - Extracts symbols with tree-sitter (Rust, TypeScript, Python) - Indexes to sol_code OpenSearch index with repo/branch/source metadata - Skips hidden dirs, vendor, node_modules, files >100KB - delete_branch() for clean re-indexing Server-side tree-sitter (code_index/symbols.rs): - Full symbol extraction shared with CLI client - extract_symbols(), extract_project_symbols(), detect_language() gRPC ReindexCode RPC: - ReindexCodeRequest: org, repo, branch (all optional filters) - ReindexCodeResponse: repos_indexed, symbols_indexed, error - Uses ToolRegistry's GiteaClient (already authenticated) - Creates sol_code index if not exists ToolRegistry.gitea_client() accessor for reindex endpoint.
2026-03-24 09:36:42 +00:00
tree-sitter = "0.24"
tree-sitter-rust = "0.23"
tree-sitter-typescript = "0.23"
tree-sitter-python = "0.23"
feat(code): gRPC server with JWT auth + tool routing tonic 0.14 gRPC server for sunbeam code sessions: - bidirectional streaming Session RPC - JWT interceptor validates tokens against Hydra JWKS - tool router classifies calls as client-side (file_read, bash, grep, etc.) or server-side (gitea, identity, search, etc.) - service stub with session lifecycle (start, chat, tool results, end) - coding_model config (default: devstral-small-2506) - grpc config section (listen_addr, jwks_url) - 182 tests (5 new: JWT claims, tool routing) phase 2 TODOs: Matrix room bridge, Mistral agent loop, streaming
2026-03-23 11:35:37 +00:00
feat: 13 e2e integration tests against real Mistral API Orchestrator tests: - Simple chat roundtrip with token usage verification - Event ordering (Started → Thinking → Done) - Metadata pass-through (opaque bag appears in Started event) - Token usage accuracy (longer prompts → more tokens) - Conversation continuity (multi-turn recall) - Client-side tool dispatch + mock result submission - Failed tool result handling (is_error: true) - Server-side tool execution (search_web via conversation) gRPC tests: - Full roundtrip (StartSession → UserInput → Status → TextDone) - Client tool relay (ToolCall → ToolResult through gRPC stream) - Token counts in TextDone (non-zero verification) - Session resume (same room_id, resumed flag) - Clean disconnect (EndSession → SessionEnd) Infrastructure: - ToolRegistry::new_minimal() — no OpenSearch/Matrix needed - ToolRegistry fields now Option for testability - GrpcState.matrix now Option - grpc_bridge moved to src/grpc/bridge.rs - TestHarness loads API key from .env
2026-03-23 20:54:28 +00:00
[dev-dependencies]
dotenv = "0.15"
feat(code): gRPC server with JWT auth + tool routing tonic 0.14 gRPC server for sunbeam code sessions: - bidirectional streaming Session RPC - JWT interceptor validates tokens against Hydra JWKS - tool router classifies calls as client-side (file_read, bash, grep, etc.) or server-side (gitea, identity, search, etc.) - service stub with session lifecycle (start, chat, tool results, end) - coding_model config (default: devstral-small-2506) - grpc config section (listen_addr, jwks_url) - 182 tests (5 new: JWT claims, tool routing) phase 2 TODOs: Matrix room bridge, Mistral agent loop, streaming
2026-03-23 11:35:37 +00:00
[build-dependencies]
tonic-build = "0.14"
tonic-prost-build = "0.14"
Reference in New Issue Copy Permalink