Added Helm templates for docspec deployment and service to enable
document specification conversion in the Kubernetes environment.
Updated Tiltfile, compose.yml, and Helm values to
configure docspec integration alongside the
backend converter service for document import functionality.
Refactored converter services based on PR #1609 review comments:
- Renamed parameter to `data` across all convert methods for consistency
- Replaced recursive call with explicit sequential calls for readability
- Hardcoded CONVERSION_API_SECURE=True in Production class for security
- Removed unused YdocConverter import from viewsets.py
- Updated tests to match new error message wording
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
Added comprehensive tests covering DocSpec converter service,
converter orchestration, and document creation with file uploads.
Tests validate DOCX and Markdown conversion workflows, error
handling, service availability, and edge cases including empty
files and Unicode filenames.
Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
We can now import documents in formats .docx and .md.
To do so we added a new container "docspec", which
uses the docspec service to convert
these formats to Blocknote format.
More here: #1567#1569.
The template feature is removed.
Migration created to drop related tables.
Files modified:
- viewsets
- serializers
- models
- admin
- factories
- urls
- tests
- demo data
## Purpose
The email template is made with the idea that they link to a document.
This change allows to customize the label of the button (currently,
"Open") to allow for a different action verb. Additionally, the
'document_title' parameter is renamed to 'link_label' to reflect that it
can link to other things than documents.
## Proposal
- [x] Email template `template.mjml` updated as proposed
- [x] Method `send_email()` updated
- [x] Translations updated
Since celery version 5.6.0 we have trouble with retrying tasks and it is
impactig the malware_detection workflow. We have to use version 5.5.3
while we found the issue.
Added:
- ✨(backend) add documents/all endpoint with descendants
- ✅(export) add PDF regression tests
- 📝(docs) Add language configuration documentation
- 🔒(helm) Set default security context
- ✨(backend) use langfuse to monitor AI actions
Changed:
- ♿(frontend) improve accessibility:
- ♿(frontend) make html export accessible to screen reader users
- ♿(frontend) add missing label and fix Axes errors to improve a11y
Fixed:
- ✅(backend) reduce flakiness on backend test
- 🐛(frontend) fix clickable main content regression
- 🐛(backend) fix TRASHBIN_CUTOFF_DAYS type error
- 💄(frontend) fix icon position in callout block
Security:
- 🔒️(backend) validate more strictly url used by cors-proxy endpoint
- 🔒️(frontend) fix props vulnerability in Interlinking
We want to monitor AI actions. For this we choose to use langfuse. As
this usage is optional, we load langfuse sdk only if settings are
configured. Also, the openai client from langfuse is a dropin
replacement of openai client, so we only have to change how openai is
imported.
The cors-proxy endpoint was allowing redirect when fetching the target
url. This can be usefull if an image url has changed but also dangerous
if an attacker wants to hide a SSRF behind a redirect.
When the content-type return by the targeted url is not an image, the
endpoint was returning a 415 status code. We don't want to provide this
info anymore avoid disclosing information an attacker can use.
The cors-proxy endpoint allow to download images host externally without
being blocked by cors headers. The response is filter on the return
content-type to avoid disclosure and the usage of this endpoint as the
proxy used by attacker. We want to restrict the usage of this endpoint
by filtering on non legit ips used. This filter avoid exploitation of
Server Side Request Forgery (SSRF).
When all the backend authentication has been moved in the django-lasuite
library, we kept the tests to ensure that the mirgration was successful
and we didn't miss something during the transition. Now this tests are
managed in the django-lasuite library and should be maintained in it,
not in docs.
In one test related to the Document::restore function, one more query is
made. Probably a cache issue fixed in django-treebeard 4.8.0. When
updating the numchild parent, one more query is made to fetch in
database the parent document, this was not made before.
Make sure the full is never John for the first user in order to make
sure we always have only 2 users (as the search is performed on both the
email and the full name).
Fixes#1765
Signed-off-by: Anto59290 <antonin59290@hotmail.com>
External dashboards need to find the latest updated documents across
the entire hierarchy. Currently this requires many API calls to
/documents/ and /documents/{id}/children for each level.
This endpoint allows retrieving all accessible documents in a single
request, enabling dashboards to efficiently display recently changed
documents regardless of their position in the hierarchy.
Signed-off-by: ChristopherSpelt <christopherspelt@icloud.com>
Added:
- ✨(helm) redirecting system
- 📱(frontend) add comments for smaller device
- ✨(project) add custom js support via config
Changed:
- 🥅(frontend) intercept 401 error on GET threads
- 🦺(frontend) check content type pdf on PdfBlock
- ✈️(frontend) pause Posthog when offline
Fixed:
- 🐛(frontend) fix tables deletion
- 🐛(frontend) fix children not display when first resize
Added:
- ✨(backend) allow to create a new user in a marketing system
- ✨(backend) add async indexation of documents on save
(or access save)
- ✨(backend) add debounce mechanism to limit indexation jobs
- ✨(api) add API route to search for indexed documents in Find
- 🥅(frontend) add boundary error page
Changed:
- 🛂(backend) stop throttling collaboration servers
- 🚸(backend) use unaccented full name for user search
- 🌐(backend) internationalize demo
- ♿(frontend) improve accessibility:
- ♿️Improve keyboard accessibility for the document tree
Fixed:
- 🐛(frontend) paste content with comments from another document
- 🐛(frontend) Select text + Go back one page crash the app
As we filter the empty documents from the batch during indexing some batches
can be empty and cause an error. Now they are ignored.
Add --batch-size argument to the index command.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Reduce the number of Find API calls by grouping all the latest changes
for indexation : send all the documents updated or deleted since the
triggering of the task.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Keep ordering by score from Find API on search/ results and
fallback search still uses "-update_at" ordering as default
Refactor pagination to work with a list instead of a queryset
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Replace indexer_debounce_lock|release functions by indexer_throttle_acquire()
Instead of mutex-like mechanism, simply set a flag in cache for an amount of
time that prevents any other task creation.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Rename FindDocumentIndexer as SearchIndexer
Rename FindDocumentSerializer as SearchDocumentSerializer
Rename package core.tasks.find as core.task.search
Remove logs on http errors in SearchIndexer
Factorise some code in search API view.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Filter deleted documents from visited ones.
Set default ordering to the Find API search call (-updated_at)
BaseDocumentIndexer.search now returns a list of document ids instead of models.
Do not call the indexer in signals when SEARCH_INDEXER_CLASS is not defined
or properly configured.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
New SEARCH_INDEXER_CLASS setting to define the indexer service class.
Raise ImpoperlyConfigured errors instead of RuntimeError in index service.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
On document content or permission changes, start a celery job that will call the
indexation API of the app "Find".
Signed-off-by: Fabre Florian <ffabre@hybird.org>
We observe some throttling pick here and there.
We observed that when the collaboration has a
problem, it is retrying to connect, leading to more
requests to the django backend. At one point, the
throttling is reached and the user would not
be able to use the application anymore.
Now when the request comes from a collaboration
server, we do not throttle it anymore.
This allows the demo to generate user and filenames with other locales
than English, for all languages defined in the project's settings.
In particular, it allows the generation of accented names, which were
previously missing.
We have the user full name through OIDC in the database, but the search only
used the email field.
This change allows to search for a user by their first and/or
last name (fix#929).
Given that user names are more likely than emails to include diacritics, it
unaccents both the query and the database entry for search (fix#1091).
It also unaccents for email so that internationalized domain names are
managed whether or not the accent is included in the search.
An unaccented gin index is added on users full_name an email fields.
Using a manual migration because a wrapper around unaccent is necessary
to make it IMMUTABLE (cf.
https://stackoverflow.com/questions/9063402/ )
We want to create a new user in a marketing system to create a dedicated
onboarding for each of them. The marketing service is implemented in the
django-lasuite library and it is possible to pick the backend we want
or implement a new one following the documentation on this library.
Added:
- ✨ Add comments feature to the editor
- ✨(backend) Comments on text editor
- ✨(frontend) link to create new doc
Changed:
- ⚡️(sw) stop to cache external resources likes videos
- 💥(frontend) upgrade to ui-kit v2
- ⚡️(frontend) improve perf on upload and table of contents
- ♿(frontend) improve accessibility:
- ♿(frontend) improve share modal button accessibility
- ♿(frontend) improve screen reader support in DocShare modal
Fixed:
- 🐛(frontend) fix toolbar not activated when reader
- 🐛(frontend) preserve left panel width on window resize
- 🐛(frontend) prevent duplicate as first character in title
