📝(docs) init architecture documentation

- Add docs about architecture
- Add ADR about the CRDT choice
This commit is contained in:
Olivier Laurendeau
2025-01-06 15:54:48 +01:00
committed by Anthony LC
parent 3d5adad227
commit ad11b7f554
2 changed files with 212 additions and 0 deletions

View File

@@ -0,0 +1,193 @@
## Decision TLDR;
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
## Status
Accepted
## Context
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
## Considered alternatives
### ProseMirror
A robust toolkit for building rich-text editors with collaboration capabilities.
| Pros | Cons |
| --- | --- |
| Mature ecosystem | Complex integration with Django |
| Rich text editing features | Steeper learning curve |
| Used by major companies | More complex to implement offline support |
| Large community | |
### ShareDB
Real-time database backend based on Operational Transformation.
| Pros | Cons |
| --- | --- |
| Battle-tested in production | Complex setup required |
| Strong consistency model | Requires specific backend architecture |
| Good documentation | Less flexible with different backends |
| | Higher latency compared to CRDTs |
### Convergence
Complete enterprise solution for real-time collaboration.
| Pros | Cons |
| --- | --- |
| Full-featured solution | Commercial licensing |
| Built-in presence features | Less community support |
| Enterprise support | More expensive |
| Good offline support | Overkill for basic needs |
### CRDT-based Solutions Comparison
A CRDT-based library specifically designed for real-time collaboration.
| Category | Pros | Cons |
|----------|------|------|
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
#### Yjs
- **Type**: State-based CRDT
- **Implementation**: JavaScript/TypeScript
- **Features**:
- Rich text collaboration
- Shared types (Array, Map, XML)
- Binary encoding
- P2P support
- **Performance**: Excellent for text editing
- **Memory Usage**: Optimized
- **License**: MIT
#### Automerge
- **Type**: Operation-based CRDT
- **Implementation**: JavaScript/Rust
- **Features**:
- JSON-like data structures
- Change history
- Undo/Redo
- Binary format
- **Performance**: Good, with Rust backend
- **Memory Usage**: Higher than Yjs
- **License**: MIT
#### Legion
- **Type**: State-based CRDT
- **Implementation**: Rust with JS bindings
- **Features**:
- High performance
- Memory efficient
- Binary protocol
- **Performance**: Excellent
- **Memory Usage**: Very efficient
- **License**: Apache 2.0
#### Diamond Types
- **Type**: Operation-based CRDT
- **Implementation**: TypeScript
- **Features**:
- Specialized for text
- Small memory footprint
- Simple API
- **Performance**: Good for text
- **Memory Usage**: Efficient
- **License**: MIT
Comparison Table:
| Feature | Yjs | Automerge | Legion | Diamond Types |
|---------|-----|-----------|--------|---------------|
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
Key Differences:
1. **Implementation Approach**:
- Yjs: Optimized for text and rich-text editing
- Automerge: General-purpose JSON CRDT
- Legion: Performance-focused with Rust
- Diamond Types: Specialized for text collaboration
2. **Performance Characteristics**:
- Yjs: Best for text editing scenarios
- Automerge: Good all-around performance
- Legion: Excellent raw performance
- Diamond Types: Optimized for text
3. **Ecosystem Integration**:
- Yjs: Wide range of integrations
- Automerge: Good JavaScript ecosystem
- Legion: Limited but growing
- Diamond Types: Focused on text editors
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
- Best-in-class text editing performance
- Mature ecosystem
- Active community
- Excellent documentation
- Wide range of backend options
## Decision
After evaluating the alternatives, we choose Yjs for the following reasons:
1. **Technical Fit:**
- Native CRDT support ensures reliable collaboration
- Excellent offline capabilities
- Good performance characteristics
- Flexible backend integration options
2. **Project Requirements Match:**
- Easy integration with our Django backend
- Supports our core collaborative features
- Manageable learning curve for the team
3. **Community & Support:**
- Active development
- Growing community
- Good documentation
- Open source with MIT license
### Comparison of Key Features:
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|---------|-----|-------------|----------|-------------|
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
| Django Integration | Easy | Complex | Complex | Moderate |
| Learning Curve | Medium | High | High | Medium |
| Cost | Free | Free | Free | Paid |
| Community Size | Growing | Large | Medium | Small |
## Consequences
### Positive
- Simplified implementation of real-time collaboration
- Good developer experience
- Future-proof technology choice
- No licensing costs
### Negative
- Team needs to learn CRDT concepts
- Newer technology compared to alternatives
- May need to build some features available out-of-the-box in other solutions
### Risks
- Community support might not grow as expected
- May discover limitations as we scale

19
docs/architecture.md Normal file
View File

@@ -0,0 +1,19 @@
## Architecture
### Global system architecture
```mermaid
flowchart TD
User -- HTTP --> Front("Frontend (NextJS SPA)")
Front -- REST API --> Back("Backend (Django)")
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
Back -- REST API --> Yserver
Back --> DB("Database (PostgreSQL)")
Back <--> Celery --> DB
Back ----> S3("Minio (S3)")
```
### Architecture decision records
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)