📝(docs) init architecture documentation
- Add docs about architecture - Add ADR about the CRDT choice
This commit is contained in:
committed by
Anthony LC
parent
3d5adad227
commit
ad11b7f554
193
docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md
Normal file
193
docs/adr/ADR-0001-20250106-use-yjs-for-docs-editing.md
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
## Decision TLDR;
|
||||||
|
|
||||||
|
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
|
||||||
|
|
||||||
|
## Considered alternatives
|
||||||
|
|
||||||
|
### ProseMirror
|
||||||
|
|
||||||
|
A robust toolkit for building rich-text editors with collaboration capabilities.
|
||||||
|
|
||||||
|
| Pros | Cons |
|
||||||
|
| --- | --- |
|
||||||
|
| Mature ecosystem | Complex integration with Django |
|
||||||
|
| Rich text editing features | Steeper learning curve |
|
||||||
|
| Used by major companies | More complex to implement offline support |
|
||||||
|
| Large community | |
|
||||||
|
|
||||||
|
### ShareDB
|
||||||
|
|
||||||
|
Real-time database backend based on Operational Transformation.
|
||||||
|
|
||||||
|
| Pros | Cons |
|
||||||
|
| --- | --- |
|
||||||
|
| Battle-tested in production | Complex setup required |
|
||||||
|
| Strong consistency model | Requires specific backend architecture |
|
||||||
|
| Good documentation | Less flexible with different backends |
|
||||||
|
| | Higher latency compared to CRDTs |
|
||||||
|
|
||||||
|
### Convergence
|
||||||
|
|
||||||
|
Complete enterprise solution for real-time collaboration.
|
||||||
|
|
||||||
|
| Pros | Cons |
|
||||||
|
| --- | --- |
|
||||||
|
| Full-featured solution | Commercial licensing |
|
||||||
|
| Built-in presence features | Less community support |
|
||||||
|
| Enterprise support | More expensive |
|
||||||
|
| Good offline support | Overkill for basic needs |
|
||||||
|
|
||||||
|
### CRDT-based Solutions Comparison
|
||||||
|
|
||||||
|
A CRDT-based library specifically designed for real-time collaboration.
|
||||||
|
|
||||||
|
| Category | Pros | Cons |
|
||||||
|
|----------|------|------|
|
||||||
|
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
|
||||||
|
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
|
||||||
|
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
|
||||||
|
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
|
||||||
|
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
|
||||||
|
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
|
||||||
|
|
||||||
|
#### Yjs
|
||||||
|
- **Type**: State-based CRDT
|
||||||
|
- **Implementation**: JavaScript/TypeScript
|
||||||
|
- **Features**:
|
||||||
|
- Rich text collaboration
|
||||||
|
- Shared types (Array, Map, XML)
|
||||||
|
- Binary encoding
|
||||||
|
- P2P support
|
||||||
|
- **Performance**: Excellent for text editing
|
||||||
|
- **Memory Usage**: Optimized
|
||||||
|
- **License**: MIT
|
||||||
|
|
||||||
|
#### Automerge
|
||||||
|
- **Type**: Operation-based CRDT
|
||||||
|
- **Implementation**: JavaScript/Rust
|
||||||
|
- **Features**:
|
||||||
|
- JSON-like data structures
|
||||||
|
- Change history
|
||||||
|
- Undo/Redo
|
||||||
|
- Binary format
|
||||||
|
- **Performance**: Good, with Rust backend
|
||||||
|
- **Memory Usage**: Higher than Yjs
|
||||||
|
- **License**: MIT
|
||||||
|
|
||||||
|
#### Legion
|
||||||
|
- **Type**: State-based CRDT
|
||||||
|
- **Implementation**: Rust with JS bindings
|
||||||
|
- **Features**:
|
||||||
|
- High performance
|
||||||
|
- Memory efficient
|
||||||
|
- Binary protocol
|
||||||
|
- **Performance**: Excellent
|
||||||
|
- **Memory Usage**: Very efficient
|
||||||
|
- **License**: Apache 2.0
|
||||||
|
|
||||||
|
#### Diamond Types
|
||||||
|
- **Type**: Operation-based CRDT
|
||||||
|
- **Implementation**: TypeScript
|
||||||
|
- **Features**:
|
||||||
|
- Specialized for text
|
||||||
|
- Small memory footprint
|
||||||
|
- Simple API
|
||||||
|
- **Performance**: Good for text
|
||||||
|
- **Memory Usage**: Efficient
|
||||||
|
- **License**: MIT
|
||||||
|
|
||||||
|
Comparison Table:
|
||||||
|
|
||||||
|
| Feature | Yjs | Automerge | Legion | Diamond Types |
|
||||||
|
|---------|-----|-----------|--------|---------------|
|
||||||
|
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
|
||||||
|
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
|
||||||
|
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
|
||||||
|
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
|
||||||
|
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
|
||||||
|
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
|
||||||
|
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
|
||||||
|
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
|
||||||
|
|
||||||
|
Key Differences:
|
||||||
|
1. **Implementation Approach**:
|
||||||
|
- Yjs: Optimized for text and rich-text editing
|
||||||
|
- Automerge: General-purpose JSON CRDT
|
||||||
|
- Legion: Performance-focused with Rust
|
||||||
|
- Diamond Types: Specialized for text collaboration
|
||||||
|
|
||||||
|
2. **Performance Characteristics**:
|
||||||
|
- Yjs: Best for text editing scenarios
|
||||||
|
- Automerge: Good all-around performance
|
||||||
|
- Legion: Excellent raw performance
|
||||||
|
- Diamond Types: Optimized for text
|
||||||
|
|
||||||
|
3. **Ecosystem Integration**:
|
||||||
|
- Yjs: Wide range of integrations
|
||||||
|
- Automerge: Good JavaScript ecosystem
|
||||||
|
- Legion: Limited but growing
|
||||||
|
- Diamond Types: Focused on text editors
|
||||||
|
|
||||||
|
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
|
||||||
|
- Best-in-class text editing performance
|
||||||
|
- Mature ecosystem
|
||||||
|
- Active community
|
||||||
|
- Excellent documentation
|
||||||
|
- Wide range of backend options
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
After evaluating the alternatives, we choose Yjs for the following reasons:
|
||||||
|
|
||||||
|
1. **Technical Fit:**
|
||||||
|
- Native CRDT support ensures reliable collaboration
|
||||||
|
- Excellent offline capabilities
|
||||||
|
- Good performance characteristics
|
||||||
|
- Flexible backend integration options
|
||||||
|
|
||||||
|
2. **Project Requirements Match:**
|
||||||
|
- Easy integration with our Django backend
|
||||||
|
- Supports our core collaborative features
|
||||||
|
- Manageable learning curve for the team
|
||||||
|
|
||||||
|
3. **Community & Support:**
|
||||||
|
- Active development
|
||||||
|
- Growing community
|
||||||
|
- Good documentation
|
||||||
|
- Open source with MIT license
|
||||||
|
|
||||||
|
### Comparison of Key Features:
|
||||||
|
|
||||||
|
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|
||||||
|
|---------|-----|-------------|----------|-------------|
|
||||||
|
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
|
||||||
|
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
|
||||||
|
| Django Integration | Easy | Complex | Complex | Moderate |
|
||||||
|
| Learning Curve | Medium | High | High | Medium |
|
||||||
|
| Cost | Free | Free | Free | Paid |
|
||||||
|
| Community Size | Growing | Large | Medium | Small |
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
- Simplified implementation of real-time collaboration
|
||||||
|
- Good developer experience
|
||||||
|
- Future-proof technology choice
|
||||||
|
- No licensing costs
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
- Team needs to learn CRDT concepts
|
||||||
|
- Newer technology compared to alternatives
|
||||||
|
- May need to build some features available out-of-the-box in other solutions
|
||||||
|
|
||||||
|
### Risks
|
||||||
|
- Community support might not grow as expected
|
||||||
|
- May discover limitations as we scale
|
||||||
19
docs/architecture.md
Normal file
19
docs/architecture.md
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Global system architecture
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
User -- HTTP --> Front("Frontend (NextJS SPA)")
|
||||||
|
Front -- REST API --> Back("Backend (Django)")
|
||||||
|
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
|
||||||
|
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
|
||||||
|
Back -- REST API --> Yserver
|
||||||
|
Back --> DB("Database (PostgreSQL)")
|
||||||
|
Back <--> Celery --> DB
|
||||||
|
Back ----> S3("Minio (S3)")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Architecture decision records
|
||||||
|
|
||||||
|
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)
|
||||||
Reference in New Issue
Block a user