📝(docs) init architecture documentation
- Add docs about architecture - Add ADR about the CRDT choice
This commit is contained in:
committed by
Anthony LC
parent
3d5adad227
commit
ad11b7f554
193
docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md
Normal file
193
docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md
Normal file
@@ -0,0 +1,193 @@
|
||||
## Decision TLDR;
|
||||
|
||||
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
|
||||
|
||||
## Considered alternatives
|
||||
|
||||
### ProseMirror
|
||||
|
||||
A robust toolkit for building rich-text editors with collaboration capabilities.
|
||||
|
||||
| Pros | Cons |
|
||||
| --- | --- |
|
||||
| Mature ecosystem | Complex integration with Django |
|
||||
| Rich text editing features | Steeper learning curve |
|
||||
| Used by major companies | More complex to implement offline support |
|
||||
| Large community | |
|
||||
|
||||
### ShareDB
|
||||
|
||||
Real-time database backend based on Operational Transformation.
|
||||
|
||||
| Pros | Cons |
|
||||
| --- | --- |
|
||||
| Battle-tested in production | Complex setup required |
|
||||
| Strong consistency model | Requires specific backend architecture |
|
||||
| Good documentation | Less flexible with different backends |
|
||||
| | Higher latency compared to CRDTs |
|
||||
|
||||
### Convergence
|
||||
|
||||
Complete enterprise solution for real-time collaboration.
|
||||
|
||||
| Pros | Cons |
|
||||
| --- | --- |
|
||||
| Full-featured solution | Commercial licensing |
|
||||
| Built-in presence features | Less community support |
|
||||
| Enterprise support | More expensive |
|
||||
| Good offline support | Overkill for basic needs |
|
||||
|
||||
### CRDT-based Solutions Comparison
|
||||
|
||||
A CRDT-based library specifically designed for real-time collaboration.
|
||||
|
||||
| Category | Pros | Cons |
|
||||
|----------|------|------|
|
||||
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
|
||||
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
|
||||
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
|
||||
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
|
||||
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
|
||||
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
|
||||
|
||||
#### Yjs
|
||||
- **Type**: State-based CRDT
|
||||
- **Implementation**: JavaScript/TypeScript
|
||||
- **Features**:
|
||||
- Rich text collaboration
|
||||
- Shared types (Array, Map, XML)
|
||||
- Binary encoding
|
||||
- P2P support
|
||||
- **Performance**: Excellent for text editing
|
||||
- **Memory Usage**: Optimized
|
||||
- **License**: MIT
|
||||
|
||||
#### Automerge
|
||||
- **Type**: Operation-based CRDT
|
||||
- **Implementation**: JavaScript/Rust
|
||||
- **Features**:
|
||||
- JSON-like data structures
|
||||
- Change history
|
||||
- Undo/Redo
|
||||
- Binary format
|
||||
- **Performance**: Good, with Rust backend
|
||||
- **Memory Usage**: Higher than Yjs
|
||||
- **License**: MIT
|
||||
|
||||
#### Legion
|
||||
- **Type**: State-based CRDT
|
||||
- **Implementation**: Rust with JS bindings
|
||||
- **Features**:
|
||||
- High performance
|
||||
- Memory efficient
|
||||
- Binary protocol
|
||||
- **Performance**: Excellent
|
||||
- **Memory Usage**: Very efficient
|
||||
- **License**: Apache 2.0
|
||||
|
||||
#### Diamond Types
|
||||
- **Type**: Operation-based CRDT
|
||||
- **Implementation**: TypeScript
|
||||
- **Features**:
|
||||
- Specialized for text
|
||||
- Small memory footprint
|
||||
- Simple API
|
||||
- **Performance**: Good for text
|
||||
- **Memory Usage**: Efficient
|
||||
- **License**: MIT
|
||||
|
||||
Comparison Table:
|
||||
|
||||
| Feature | Yjs | Automerge | Legion | Diamond Types |
|
||||
|---------|-----|-----------|--------|---------------|
|
||||
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
|
||||
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
|
||||
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
|
||||
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
|
||||
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
|
||||
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
|
||||
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
|
||||
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
|
||||
|
||||
Key Differences:
|
||||
1. **Implementation Approach**:
|
||||
- Yjs: Optimized for text and rich-text editing
|
||||
- Automerge: General-purpose JSON CRDT
|
||||
- Legion: Performance-focused with Rust
|
||||
- Diamond Types: Specialized for text collaboration
|
||||
|
||||
2. **Performance Characteristics**:
|
||||
- Yjs: Best for text editing scenarios
|
||||
- Automerge: Good all-around performance
|
||||
- Legion: Excellent raw performance
|
||||
- Diamond Types: Optimized for text
|
||||
|
||||
3. **Ecosystem Integration**:
|
||||
- Yjs: Wide range of integrations
|
||||
- Automerge: Good JavaScript ecosystem
|
||||
- Legion: Limited but growing
|
||||
- Diamond Types: Focused on text editors
|
||||
|
||||
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
|
||||
- Best-in-class text editing performance
|
||||
- Mature ecosystem
|
||||
- Active community
|
||||
- Excellent documentation
|
||||
- Wide range of backend options
|
||||
|
||||
## Decision
|
||||
|
||||
After evaluating the alternatives, we choose Yjs for the following reasons:
|
||||
|
||||
1. **Technical Fit:**
|
||||
- Native CRDT support ensures reliable collaboration
|
||||
- Excellent offline capabilities
|
||||
- Good performance characteristics
|
||||
- Flexible backend integration options
|
||||
|
||||
2. **Project Requirements Match:**
|
||||
- Easy integration with our Django backend
|
||||
- Supports our core collaborative features
|
||||
- Manageable learning curve for the team
|
||||
|
||||
3. **Community & Support:**
|
||||
- Active development
|
||||
- Growing community
|
||||
- Good documentation
|
||||
- Open source with MIT license
|
||||
|
||||
### Comparison of Key Features:
|
||||
|
||||
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|
||||
|---------|-----|-------------|----------|-------------|
|
||||
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
|
||||
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
|
||||
| Django Integration | Easy | Complex | Complex | Moderate |
|
||||
| Learning Curve | Medium | High | High | Medium |
|
||||
| Cost | Free | Free | Free | Paid |
|
||||
| Community Size | Growing | Large | Medium | Small |
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Simplified implementation of real-time collaboration
|
||||
- Good developer experience
|
||||
- Future-proof technology choice
|
||||
- No licensing costs
|
||||
|
||||
### Negative
|
||||
- Team needs to learn CRDT concepts
|
||||
- Newer technology compared to alternatives
|
||||
- May need to build some features available out-of-the-box in other solutions
|
||||
|
||||
### Risks
|
||||
- Community support might not grow as expected
|
||||
- May discover limitations as we scale
|
||||
19
docs/architecture.md
Normal file
19
docs/architecture.md
Normal file
@@ -0,0 +1,19 @@
|
||||
## Architecture
|
||||
|
||||
### Global system architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
User -- HTTP --> Front("Frontend (NextJS SPA)")
|
||||
Front -- REST API --> Back("Backend (Django)")
|
||||
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
|
||||
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
|
||||
Back -- REST API --> Yserver
|
||||
Back --> DB("Database (PostgreSQL)")
|
||||
Back <--> Celery --> DB
|
||||
Back ----> S3("Minio (S3)")
|
||||
```
|
||||
|
||||
### Architecture decision records
|
||||
|
||||
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)
|
||||
Reference in New Issue
Block a user