studio/docs
Archived
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

♻️(frontend) raw payloads on convert endpoint

Browse Source 
Accept raw payload on convert-endpoint and respond with raw Yjs payload

This change replaces Base64-encoded I/O with direct binary streaming,
yielding several benefits:
- **Network efficiency**: Eliminates the ~33% size inflation of Base64,
cutting bandwidth and latency.
- **Memory savings**: Enables piping DOCX (already compressed) buffers
straight to DocSpec API without holding, encoding and decoding multi-MB
payload in RAM.

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
This commit is contained in:
Stephan Meijer
2025-07-04 12:41:32 +02:00
parent cef2d274fc
commit fde520a6f3
4 changed files with 43 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
src/frontend/servers/y-provider/__tests__/convert.test.ts
View File

@@ -26,8 +26,9 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', 'wrong-api-key');
.set('origin', origin)
.set('authorization', 'wrong-api-key')
.set('content-type', 'application/json');
expect(response.status).toBe(401);
expect(response.body).toStrictEqual({
@@ -40,8 +41,9 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', 'Bearer test-secret-api-key');
.set('origin', origin)
.set('authorization', 'Bearer test-secret-api-key')
.set('content-type', 'application/json');
// Warning: Changing the authorization header to Bearer token format will break backend compatibility with this microservice.
expect(response.status).toBe(401);
@@ -55,8 +57,9 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', apiKey);
.set('origin', origin)
.set('authorization', apiKey)
.set('content-type', 'application/json');
expect(response.status).toBe(400);
expect(response.body).toStrictEqual({
@@ -69,11 +72,10 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', apiKey)
.send({
content: '',
});
.set('origin', origin)
.set('authorization', apiKey)
.set('content-type', 'application/json')
.send('');
expect(response.status).toBe(400);
expect(response.body).toStrictEqual({
@@ -93,20 +95,17 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', apiKey)
.send({
content: document,
});
.set('origin', origin)
.set('authorization', apiKey)
.set('content-type', 'application/json')
.send(document);
expect(response.status).toBe(200);
expect(response.body).toStrictEqual({
content: expect.any(String),
});
expect(response.body).toBeInstanceOf(Buffer);
const editor = ServerBlockNoteEditor.create();
const doc = new Y.Doc();
Y.applyUpdate(doc, Buffer.from(response.body.content, 'base64'));
Y.applyUpdate(doc, response.body);
const blocks = editor.yDocToBlocks(doc, 'document-store');
expect(blocks).toStrictEqual([

20
src/frontend/servers/y-provider/__tests__/server.test.ts
View File

@@ -40,30 +40,30 @@ describe('Server Tests', () => {
});
});
it('allows JSON payloads up to 500kb for the CONVERT route', async () => {
it('allows payloads up to 500kb for the CONVERT route', async () => {
const app = initApp();
const largePayload = 'a'.repeat(400 * 1024); // 400kb payload
const response = await request(app)
.post(routes.CONVERT)
.set('Origin', origin)
.set('Authorization', apiKey)
.set('Content-Type', 'application/json')
.send({ data: largePayload });
.set('origin', origin)
.set('authorization', apiKey)
.set('content-type', 'text/markdown')
.send(largePayload);
expect(response.status).not.toBe(413);
});
it('rejects JSON payloads larger than 500kb for the CONVERT route', async () => {
it('rejects payloads larger than 500kb for the CONVERT route', async () => {
const app = initApp();
const oversizedPayload = 'a'.repeat(501 * 1024); // 501kb payload
const response = await request(app)
.post(routes.CONVERT)
.set('Origin', origin)
.set('Authorization', apiKey)
.set('Content-Type', 'application/json')
.send({ data: oversizedPayload });
.set('origin', origin)
.set('authorization', apiKey)
.set('content-type', 'text/markdown')
.send(oversizedPayload);
expect(response.status).toBe(413);
});