From ad11b7f55486874bacb350d9678b3fc7b808bcca Mon Sep 17 00:00:00 2001 From: Olivier Laurendeau Date: Mon, 6 Jan 2025 15:54:48 +0100 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D(docs)=20init=20architecture=20docu?= =?UTF-8?q?mentation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add docs about architecture - Add ADR about the CRDT choice --- ...-0001-20250106-use-yjs-for-docs-editing.md | 193 ++++++++++++++++++ docs/architecture.md | 19 ++ 2 files changed, 212 insertions(+) create mode 100644 docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md create mode 100644 docs/architecture.md diff --git a/docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md b/docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md new file mode 100644 index 00000000..486b8b0b --- /dev/null +++ b/docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md @@ -0,0 +1,193 @@ +## Decision TLDR; + +We will use Yjs a CRDT-based library for the collaborative editing of the documents. + +## Status + +Accepted + +## Context + +We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend. + +## Considered alternatives + +### ProseMirror + +A robust toolkit for building rich-text editors with collaboration capabilities. + +| Pros | Cons | +| --- | --- | +| Mature ecosystem | Complex integration with Django | +| Rich text editing features | Steeper learning curve | +| Used by major companies | More complex to implement offline support | +| Large community | | + +### ShareDB + +Real-time database backend based on Operational Transformation. + +| Pros | Cons | +| --- | --- | +| Battle-tested in production | Complex setup required | +| Strong consistency model | Requires specific backend architecture | +| Good documentation | Less flexible with different backends | +| | Higher latency compared to CRDTs | + +### Convergence + +Complete enterprise solution for real-time collaboration. + +| Pros | Cons | +| --- | --- | +| Full-featured solution | Commercial licensing | +| Built-in presence features | Less community support | +| Enterprise support | More expensive | +| Good offline support | Overkill for basic needs | + +### CRDT-based Solutions Comparison + +A CRDT-based library specifically designed for real-time collaboration. + +| Category | Pros | Cons | +|----------|------|------| +| Technical Implementation | • Native real-time collaboration
• No central conflict resolution needed
• Works well with Django backend
• Automatic state synchronization | • Learning curve for CRDT concepts
• More complex initial setup
• Additional metadata overhead | +| User Experience | • Instant local updates
• Works offline by default
• Low latency
• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies
• UI must handle temporary conflicts | +| Performance | • Excellent scaling with multiple users
• Reduced server load
• Efficient network usage
• Good memory optimization (especially Yjs) | • Slightly higher memory usage
• Initial state sync can be larger | +| Development | • No need to build conflict resolution
• Simple integration with text editors
• Future-proof architecture | • Team needs to learn new concepts
• Fewer ready-made solutions
• May need to build some features from scratch | +| Maintenance | • Less server infrastructure
• Simpler deployment
• Fewer points of failure | • Debugging can be more complex
• State management requires careful handling | +| Business Impact | • Better offline support for users
• Scales well as user base grows
• No licensing costs (with Yjs) | • Initial development time might be longer
• Team training required | + +#### Yjs +- **Type**: State-based CRDT +- **Implementation**: JavaScript/TypeScript +- **Features**: + - Rich text collaboration + - Shared types (Array, Map, XML) + - Binary encoding + - P2P support +- **Performance**: Excellent for text editing +- **Memory Usage**: Optimized +- **License**: MIT + +#### Automerge +- **Type**: Operation-based CRDT +- **Implementation**: JavaScript/Rust +- **Features**: + - JSON-like data structures + - Change history + - Undo/Redo + - Binary format +- **Performance**: Good, with Rust backend +- **Memory Usage**: Higher than Yjs +- **License**: MIT + +#### Legion +- **Type**: State-based CRDT +- **Implementation**: Rust with JS bindings +- **Features**: + - High performance + - Memory efficient + - Binary protocol +- **Performance**: Excellent +- **Memory Usage**: Very efficient +- **License**: Apache 2.0 + +#### Diamond Types +- **Type**: Operation-based CRDT +- **Implementation**: TypeScript +- **Features**: + - Specialized for text + - Small memory footprint + - Simple API +- **Performance**: Good for text +- **Memory Usage**: Efficient +- **License**: MIT + +Comparison Table: + +| Feature | Yjs | Automerge | Legion | Diamond Types | +|---------|-----|-----------|--------|---------------| +| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent | +| Structured Data | ✅ | ✅ | ✅ | ⚠️ | +| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High | +| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ | +| Maturity | ✅ | ✅ | ⚠️ | ⚠️ | +| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small | +| Documentation | ✅ | ✅ | ⚠️ | ⚠️ | +| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited | + +Key Differences: +1. **Implementation Approach**: + - Yjs: Optimized for text and rich-text editing + - Automerge: General-purpose JSON CRDT + - Legion: Performance-focused with Rust + - Diamond Types: Specialized for text collaboration + +2. **Performance Characteristics**: + - Yjs: Best for text editing scenarios + - Automerge: Good all-around performance + - Legion: Excellent raw performance + - Diamond Types: Optimized for text + +3. **Ecosystem Integration**: + - Yjs: Wide range of integrations + - Automerge: Good JavaScript ecosystem + - Legion: Limited but growing + - Diamond Types: Focused on text editors + +This analysis reinforces our choice of Yjs for the CRDT-based option as it provides: +- Best-in-class text editing performance +- Mature ecosystem +- Active community +- Excellent documentation +- Wide range of backend options + +## Decision + +After evaluating the alternatives, we choose Yjs for the following reasons: + +1. **Technical Fit:** +- Native CRDT support ensures reliable collaboration +- Excellent offline capabilities +- Good performance characteristics +- Flexible backend integration options + +2. **Project Requirements Match:** +- Easy integration with our Django backend +- Supports our core collaborative features +- Manageable learning curve for the team + +3. **Community & Support:** +- Active development +- Growing community +- Good documentation +- Open source with MIT license + +### Comparison of Key Features: + +| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence | +|---------|-----|-------------|----------|-------------| +| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ | +| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ | +| Django Integration | Easy | Complex | Complex | Moderate | +| Learning Curve | Medium | High | High | Medium | +| Cost | Free | Free | Free | Paid | +| Community Size | Growing | Large | Medium | Small | + +## Consequences + +### Positive +- Simplified implementation of real-time collaboration +- Good developer experience +- Future-proof technology choice +- No licensing costs + +### Negative +- Team needs to learn CRDT concepts +- Newer technology compared to alternatives +- May need to build some features available out-of-the-box in other solutions + +### Risks +- Community support might not grow as expected +- May discover limitations as we scale \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..230d3245 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,19 @@ +## Architecture + +### Global system architecture + +```mermaid +flowchart TD + User -- HTTP --> Front("Frontend (NextJS SPA)") + Front -- REST API --> Back("Backend (Django)") + Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back + Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect") + Back -- REST API --> Yserver + Back --> DB("Database (PostgreSQL)") + Back <--> Celery --> DB + Back ----> S3("Minio (S3)") +``` + +### Architecture decision records + +- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md) \ No newline at end of file