feat: multi-agent architecture with Conversations API and persistent state
Mistral Agents + Conversations API integration:
- Orchestrator agent created on startup with Sol's personality + tools
- ConversationRegistry routes messages through persistent conversations
- Per-room conversation state (room_id → conversation_id + token counts)
- Function call handling within conversation responses
- Configurable via [agents] section in sol.toml (use_conversations_api flag)
Multimodal support:
- m.image detection and Matrix media download (mxc:// → base64 data URI)
- ContentPart-based messages sent to Mistral vision models
- Archive stores media_urls for image messages
System prompt rewrite:
- 687 → 150 lines — dense, few-shot examples, hard rules
- {room_context_rules} placeholder for group vs DM behavior
- Sender prefixing (<@user:server>) for multi-user turns in group rooms
SQLite persistence (/data/sol.db):
- Conversation mappings and agent IDs survive reboots
- WAL mode for concurrent reads
- Falls back to in-memory on failure (sneezes into all rooms to signal)
- PVC already mounted at /data alongside Matrix SDK state store
New modules:
- src/persistence.rs — SQLite state store
- src/conversations.rs — ConversationRegistry + message merging
- src/agents/{mod,definitions,registry}.rs — agent lifecycle
- src/agent_ux.rs — reaction + thread progress UX
- src/tools/bridge.rs — tool dispatch for domain agents
102 tests passing.
This commit is contained in:
@@ -1,3 +1,4 @@
|
||||
use matrix_sdk::media::{MediaFormat, MediaRequestParameters};
|
||||
use matrix_sdk::room::Room;
|
||||
use matrix_sdk::RoomMemberships;
|
||||
use ruma::events::room::message::{
|
||||
@@ -67,6 +68,61 @@ pub async fn send_reaction(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Extract image info from an m.image message event.
|
||||
/// Returns (mxc_url, mimetype, body/caption) if present.
|
||||
pub fn extract_image(event: &OriginalSyncRoomMessageEvent) -> Option<(String, String, String)> {
|
||||
if let MessageType::Image(image) = &event.content.msgtype {
|
||||
let url = match &image.source {
|
||||
ruma::events::room::MediaSource::Plain(mxc) => mxc.to_string(),
|
||||
ruma::events::room::MediaSource::Encrypted(_) => return None,
|
||||
};
|
||||
let mime = image
|
||||
.info
|
||||
.as_ref()
|
||||
.and_then(|i| i.mimetype.clone())
|
||||
.unwrap_or_else(|| "image/png".to_string());
|
||||
let caption = image.body.clone();
|
||||
Some((url, mime, caption))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Download image bytes from a Matrix mxc:// URL via the media API.
|
||||
/// Returns the raw bytes as a base64 data URI suitable for Mistral vision.
|
||||
pub async fn download_image_as_data_uri(
|
||||
client: &matrix_sdk::Client,
|
||||
event: &OriginalSyncRoomMessageEvent,
|
||||
) -> Option<String> {
|
||||
if let MessageType::Image(image) = &event.content.msgtype {
|
||||
let media_source = &image.source;
|
||||
let mime = image
|
||||
.info
|
||||
.as_ref()
|
||||
.and_then(|i| i.mimetype.clone())
|
||||
.unwrap_or_else(|| "image/png".to_string());
|
||||
|
||||
let request = MediaRequestParameters {
|
||||
source: media_source.clone(),
|
||||
format: MediaFormat::File,
|
||||
};
|
||||
|
||||
match client.media().get_media_content(&request, true).await {
|
||||
Ok(bytes) => {
|
||||
use base64::Engine;
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(&bytes);
|
||||
Some(format!("data:{};base64,{}", mime, b64))
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("Failed to download image: {e}");
|
||||
None
|
||||
}
|
||||
}
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Get the display name for a room.
|
||||
pub fn room_display_name(room: &Room) -> String {
|
||||
room.cached_display_name()
|
||||
|
||||
Reference in New Issue
Block a user