As we filter the empty documents from the batch during indexing some batches
can be empty and cause an error. Now they are ignored.
Add --batch-size argument to the index command.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Reduce the number of Find API calls by grouping all the latest changes
for indexation : send all the documents updated or deleted since the
triggering of the task.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Keep ordering by score from Find API on search/ results and
fallback search still uses "-update_at" ordering as default
Refactor pagination to work with a list instead of a queryset
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Replace indexer_debounce_lock|release functions by indexer_throttle_acquire()
Instead of mutex-like mechanism, simply set a flag in cache for an amount of
time that prevents any other task creation.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Rename FindDocumentIndexer as SearchIndexer
Rename FindDocumentSerializer as SearchDocumentSerializer
Rename package core.tasks.find as core.task.search
Remove logs on http errors in SearchIndexer
Factorise some code in search API view.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Filter deleted documents from visited ones.
Set default ordering to the Find API search call (-updated_at)
BaseDocumentIndexer.search now returns a list of document ids instead of models.
Do not call the indexer in signals when SEARCH_INDEXER_CLASS is not defined
or properly configured.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
New SEARCH_INDEXER_CLASS setting to define the indexer service class.
Raise ImpoperlyConfigured errors instead of RuntimeError in index service.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
On document content or permission changes, start a celery job that will call the
indexation API of the app "Find".
Signed-off-by: Fabre Florian <ffabre@hybird.org>
We observe some throttling pick here and there.
We observed that when the collaboration has a
problem, it is retrying to connect, leading to more
requests to the django backend. At one point, the
throttling is reached and the user would not
be able to use the application anymore.
Now when the request comes from a collaboration
server, we do not throttle it anymore.
This allows the demo to generate user and filenames with other locales
than English, for all languages defined in the project's settings.
In particular, it allows the generation of accented names, which were
previously missing.
We have the user full name through OIDC in the database, but the search only
used the email field.
This change allows to search for a user by their first and/or
last name (fix#929).
Given that user names are more likely than emails to include diacritics, it
unaccents both the query and the database entry for search (fix#1091).
It also unaccents for email so that internationalized domain names are
managed whether or not the accent is included in the search.
An unaccented gin index is added on users full_name an email fields.
Using a manual migration because a wrapper around unaccent is necessary
to make it IMMUTABLE (cf.
https://stackoverflow.com/questions/9063402/ )
We want to create a new user in a marketing system to create a dedicated
onboarding for each of them. The marketing service is implemented in the
django-lasuite library and it is possible to pick the backend we want
or implement a new one following the documentation on this library.
Added:
- ✨ Add comments feature to the editor
- ✨(backend) Comments on text editor
- ✨(frontend) link to create new doc
Changed:
- ⚡️(sw) stop to cache external resources likes videos
- 💥(frontend) upgrade to ui-kit v2
- ⚡️(frontend) improve perf on upload and table of contents
- ♿(frontend) improve accessibility:
- ♿(frontend) improve share modal button accessibility
- ♿(frontend) improve screen reader support in DocShare modal
Fixed:
- 🐛(frontend) fix toolbar not activated when reader
- 🐛(frontend) preserve left panel width on window resize
- 🐛(frontend) prevent duplicate as first character in title
In the UserLightSerializer we were fallbacking on a strategy to never
have a full_name or short_name empty. We use the part of the email
befire the @. We are doing the same thing now in the main
UserSerializer.
This commit add the CRUD part to manage comment lifeycle. Permissions
are relying on the Document and Comment abilities. Comment viewset
depends on the Document route and is added to the
document_related_router. Dedicated serializer and permission are
created.
In order to store the comments on a document, we created a new model
Comment. User is nullable because anonymous users can comment a Document
is this one is public with a link_role commentator.
Added:
- ✨(frontend) enable ODT export for documents
- ✨(frontend) improve mobile UX by showing subdocs count
Changed:
- ♻️(frontend) preserve @ character when esc is pressed
after typing it
- ♻️(frontend) make summary button fixed to remain visible
during scroll
- ♻️(frontend) pdf embed use full width
Fixed:
- ♿(frontend) improve accessibility:
- ♿(frontend) improve ARIA in doc grid and editor
for a11y
- ♿(frontend) improve accessibility and styling
of summary table
- ♿(frontend) add focus trap and enter key support
to remove doc modal
- 🐛(docx) fix image overflow by limiting width to
600px during export
- 🐛(frontend) fix fallback translations with Trans
- 🐛(pdf) fix table cell alignment issue in exported
documents
- 🐛(pdf) preserve image aspect ratio in PDF export
A complete API was able to manage templates lifecycle, from the creation
to the deletion and managing accesses on them. This API is not used by
the frontend application, is not finished. A connected user can interact
with this API and lead to unwanted behavior in the interface. Refering
ot issue #1222 templates can maybe totaly remove in the future. While
it's here and used, we only keep list and retrive endpoints. The
template management can still be done in the admin interface.
Like in other abilities, we compute a set_role_to property on the
abilities. This set_role_to contains all the roles lower or equal than
the current user role. We rely on this propoerty to validate the accept
endpoint and it will be used by the front allpication to built the role
select list.
We check that the role set in a ask_for_access is not higher than the
user's role accepting the request. We prevent case where ad min will
grant a user owner in order to take control of the document. Only owner
can accept an owner role.
Added
- ✨(frontend) create skeleton component for DocEditor
- ✨(frontend) add an EmojiPicker in the document
tree and title
- ✨(frontend) ajustable left panel
Changed:
- ♻️(frontend) adapt custom blocks to new implementation
- ♻️(backend) increase user short_name field length
- 🚸(frontend) separate viewers from editors
Fixed:
- 🐛(frontend) fix duplicate document entries in grid
- 🐛(backend) fix trashbin list
- ♿(frontend) improve accessibility:
- ♿(frontend) remove empty alt on logo due to Axe
a11y error
- 🐛(backend) fix s3 version_id validation
- 🐛(frontend) retry check media status after page reload
- 🐛(frontend) fix Interlinking memory leak
- 🐛(frontend) button new doc UI fix
- 🐛(frontend) interlinking UI fix
The regex used on the version_detail endpoint path is not fully
compatible with the S3 spec. In the S3 specs, Version IDs are Unicode,
UTF-8 encoded, URL-ready, opaque strings that are no more than 1,024
bytes long. We don't accept all unicode characters but enough to be
compliant.
The user's short_name field length was set to 20. This is not enought
and we have some users who cannot register because of that. We changed
this length to a higher one, 100, like the full_name.
The document change admin page is unusable. The django treebeard library
can change the form used by one provided but this one is really slow.
And it is collapsing the configuration made with the other fields and
readonly fields declared on the DocumentAdmin class. In a first time we
remove the form usage, it seems useless. Later we have to provide more
information on this admin page.
