On the media upload endpoint, we want to set the content-disposition
header. Its value is based on the uploaded file mime-type and if flagged
as unsafe. If the file is not an image or is unsafe then the
contentDisposition is set to attachment to force its download.
Otherwise, we set it to inline.
The frontend cannot access custom headers of a file,
so we need to add a flag in the filename.
We add the `unsafe` flag in the filename to
indicate that the file is unsafe.
Previous filename: "/{UUID4}.{extension}"
New filename: "/{UUID4}-unsafe.{extension}"
When creating a new document/template via the API, we add the
logged-in user as owner of the created object. This should be
done atomically with the object creation to make sure we don't
end-up with an orphan object that the creator can't access
anymore.
Only owners can see and restore deleted documents. They can only do
it during the grace period before the document is considered hard
deleted and hidden from everybody on the API.
Now that we have introduced a document tree structure, it is not
possible to allow deleting documents anymore as it impacts the whole
subtree below the deleted document and the consequences are too big.
We introduce soft delete in order to give a second thought to the
document's owner (who is the only one to be allowed to delete a
document). After a document is soft deleted, the owner can still
see it in the trashbin (/api/v1.0/documents/trashbin).
After a grace period (30 days be default) the document disappears
from the trashbin and can't be restored anymore. Note that even
then it is still kept in database. Cleaning the database to erase
deleted documents after the grace period can be done as a maintenance
script.
Only administrators or owners of a document can move it to a target
document for which they are also administrator or owner.
We allow different moving modes:
- first-child: move the document as the first child of the target
- last-child: move the document as the last child of the target
- first-sibling: move the document as the first sibling of the target
- last-sibling: move the document as the last sibling of the target
- left: move the document as sibling ordered just before the target
- right: move the document as sibling ordered just after the target
The whole subtree below the document that is being moved, moves as
well and remains below the document after it is moved.
We choose to use Django-treebeard for its quality, performance and
stability. Adding tree structure to documents is as simple as
inheriting from the MP_Node class.
When the query looks like an email (includes @) we search by
Levenstein distance because we are just trying to prevent typing
errors, not searching anymore.
It is important to still propose results with a short Levenstein
distance because it is frequent to forget a double letter in
someone's name for example "Pacoud" or even "pacou" instead of
"Paccoud" and we want to prevent duplicates or failing on
invitation.
We consider the query string to be an email as soon as it contains
a "@" character. Trying harder to identify a string that is really
an email would lead to weird behaviors like toto@example.gouv looking
like and email but if we continue typing toto@example.gouv.f not
looking like an email... before toto@example.gouv.fr finally looking
like an email. The result would be jumping from one type of search
to the other. As soon as there is a "@" in the query, we can be
sure that the user is not looking for a name anymore and we can
switch to matching by Levenstein distance.
All the uploaded files had the content-type set
to `application/octet-stream`. It create issues
when the file is downloaded from the frontend
because the browser doesn't know how to handle
the file.
We now determine the content-type of the file
and set it to the file object.
3 requests we able to create a document:
- POST document request
- GET collaboration-auth
- GET media-auth
If the 2 last were faster than the first, a
document was created without the necessary
informations.
We want trusted external applications to be able to create documents
via the API on behalf of any user. The user may or may not pre-exist
in our database and should be notified of the document creation by
email.
When an access is updated or removed, the
collaboration server is notified to reset the
access connection; by being disconnected, the
accesses will automatically reconnect by passing
by the ngnix subrequest, and so get the good
rights.
We do the same system when the document link is
updated, except here we reset every access
connection.
We need to improve security on the access to The collaboration server
We can use the same pattern as for media files leveraging the nginx
subrequest feature.
We want to use the same pattern for the websocket collaboration service
authorization as what we use for media files.
This addition comes in the next commit but doing it efficiently
required factorizing some code with the media auth view.
This is the minimal and fast search feature, while we are working on
a full text search based on opensearch. For the moment we only search
on the title of the document.
We recently allowed authenticated users to mark a document as favorite.
We were lacking the possibility for users to see only the documents
they marked as favorite.
We want to be able to limit the documents displayed on a logged-in user's
list view by the documents they created or by the documents that other
users created.
This is different from having the "owner" role on a document because this
can be acquired and even lost. What we want here is to be able to
identify documents by the user who created them so we add a new field.
On the user search API by similarity, we had a flaky test because
2 users had the same similarity score. Adding a secondary ordering
field makes ordering deterministic between users who share the same
similarity score.
The new UI will display the number of accesses on each document.
/!\ Once team accesses will be used, this will not represent the number
of people with access anymore and will have to be improved by
computing the number of people in each team.
A user can now mark/unmark documents as favorite.
This is done via a new action of the document API endpoint:
/api/v1.0/documents/{document_id}/favorite
POST to mark as favorite / DELETE to unmark
I realized most of the database queries made when getting a document
list view were to include nested accesses. This detailed information
about accesses in only necessary for the document detail view.
I introduced a specific serializer for the document list view with
less fields. For a list of 20 documents with 5 accesses, we go down
from 3x5x20= 300 queries to just 3 queries.
Add setting CRISP_WEBSITE_ID. This setting is
used to configure the Crisp chat widget.
It will be available to the conf endpoint, to
be used by the frontend.
The frontend need to know the collab server url,
so we need to add a new setting to the backend,
in order to expose this value to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the ws requests,
but in development we need to define it to target
the collaboration server.
The frontend need to know the base url for the
media files, so we need to add a new setting
to the backend, in order to expose this value
to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the media requests,
but in development we need to define it to target
the nginx server.
In some edge cases, the domain part the email addresse is
longer than the name part. Users searches by email similarity
then return a lot of unsorted results.
We can improve this by being more demanding on similarity when
the query looks like an email. Sorting results by the similarity
score is also an obvious improvement.
At the moment, we still think it is good to propose results with
a weak similarity on the name part because we want to avoid
as much as possible creating duplicate users by inviting one of
is many emails, a user who is already in our database.
Fixes 399
Only users who have the rights to manage accesses on the document should
be allowed to see and manipulate invitations. Other users can see access
rights on the document but only when the corresponding user/team has
actually been granted access.
We added a parameter in document abilities so the frontend knows when
the logged-in user can invite another user with the owner role or not.
We created 2 new action endpoints on the document
to perform AI operations:
- POST /api/v1.0/documents/{uuid}/ai-transform
- POST /api/v1.0/documents/{uuid}/ai-translate
We want to allow users to upload files to a document, not just images.
We try to enforce coherence between the file extension and the real
mime type of its content. If a file is deemed unsafe, it is still accepted
during upload and the information is stored as metadata on the object
for display to readers.
Getting versions was not working properly. Some versions returned
were not accessible by the user requesting the list of available
versions.
We refactor the code to make it simpler and let the frontend handle
pagination (load more style).
Change the email invitation content. More
document related variables are added.
To benefit of the document inheritance, we moved
the function email_invitation to the document model.
We open a specific endpoint to update documents link configuration
because it makes it more secure and simple to limit access rights
to administrators/owners whereas other document fields like title
and content can be edited by anonymous or authenticated users with
much less access rights.
Link access was either public or private and was only allowing readers.
This commit makes link access more powerful:
- link reach can be private (users need to obtain specific access by
document's administrators), restricted (any authenticated user) or
public (anybody including anonymous users)
- link role can be reader or editor.
It is thus now possible to give editor access to an anonymous user or
any authenticated user.
We make use of nginx subrequests to block media file downloads while
we check for access rights. The request is then proxied to the object
storage engine and authorization is added via the "Authorization"
header. This way the media urls are static and can be stored in the
document's json content without compromising on security: access
control is done on all requests based on the user cookie session.
We only rely on S3 to store attachments for a document. Nothing
is persisted in the database as the image media urls will be
stored in the document json.
