We upgraded ESLint to version 9 in the y-provider server,
which includes several improvements and fixes.
This change also involves updating the ESLint
configuration files to the new format and ensuring
compatibility with the latest ESLint features.
This allows API users to process document content, enabling the
use of Docs as a headless CMS for instance, or any kind of document
processing. Fixes#1206.
We need a new endpoint in the y-provider server allowing the backend to
retrieve the number of active connections on a document and if a session
key exists.
Accept raw payload on convert-endpoint and respond with raw Yjs payload
This change replaces Base64-encoded I/O with direct binary streaming,
yielding several benefits:
- **Network efficiency**: Eliminates the ~33% size inflation of Base64,
cutting bandwidth and latency.
- **Memory savings**: Enables piping DOCX (already compressed) buffers
straight to DocSpec API without holding, encoding and decoding multi-MB
payload in RAM.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Return 401 Unauthorized for missing/invalid API keys (per RFC 7235);
403 is reserved for valid-but-forbidden credentials.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Migrated from jest to vitest for server/y-provider, gaining faster runs,
esm-native support and cleaner mocking.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Renamed the `convert-markdown` endpoint to `convert` as a
general-purpose conversion endpoint for integration with DocSpec
conversion (DOCX import), without altering its existing functionality.
In a future contribution, this endpoint will not only support conversion
from Markdown -> BlockNote -> Yjs but also directly BlockNote -> Yjs.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Problem:
- Default Express JSON parser limit (100kb) is insufficient for larger
transcription files
- 2-hour audio transcriptions slightly exceed the 100kb limit, causing request
failures
Solution:
- Implemented custom middleware to apply different JSON parser configurations
based on route
- Applied 500kb limit specifically for transcription conversion endpoints
- Maintained default limits for all other routes to preserve security
Technical notes:
- Could not find a built-in Express solution to specify parser config per route
- Custom middleware conditionally applies the appropriate parser configuration
The way to connect to the hocuspocus server needs to be proxified in
nginx to query a dedicated route in the django application and then
follow the request to the express server with the additionnal headers.
The auth can be done in the express server by querying the backend on
the document retrieve endpoint. If the response status code is 200, the
user has access to the document, otherwise it is not the case. Then we
can check the abilities to determine what the user can do or not.
Many routes were in the server.ts file, now they
are in their own files in the handlers folder.
The server.ts file is now AppServer that handles
the routes.
We split as well the tests.
Support for two API keys has been added to the YProvider microservice to
decouple responsibilities between the collaboration server and other
endpoints. This improves security by scoping keys to specific purposes and
ensures a clearer separation of concerns for easier management and debugging.
This code is quite poor. Sorry, I don't have much time working
on this feature. However, it should be functional.
I've reused the code we created for the Demo with Kasbarian.
I've not tested it yet with all corner case. Error handling
might be improved for sure, same for logging.
This endpoint is not modular. We could easily introduce options
to modify its behavior based on some options. YAGNI
I've added bearer token authentification, because it's unclear
how this micro service would be exposed. It's totally not required
if the microservice is not exposed through an Ingress.
