We can't prevent document editors from copy/pasting content to from one
document to another. The problem is that copying content, will copy the
urls pointing to attachments but if we don't do anything, the reader of
the document to which the content is being pasted, may not be allowed to
access the attachment files from the original document.
Using the work from the previous commit, we can grant access to the readers
of the target document by extracting the attachment keys from the content and
adding themto the target document's "attachments" field. Before doing this,
we check that the current user can indeed access the attachment files extracted
from the content and that they are allowed to edit the current document.
We took this opportunity to refactor the way access is controlled on
media attachments. We now add the media key to a list on the document
instance each time a media is uploaded to a document. This list is
passed along when a document is duplicated, allowing us to grant
access to readers on the new document, even if they don't have or
lost access to the original document.
We also propose an option to reproduce the same access rights on the
duplicate document as what was in place on the original document.
This can be requested by passing the "with_accesses=true" option in
the query string.
The tricky point is that we need to extract attachment keys from the
existing documents and set them on the new "attachments" field that is
now used to track access rights on media files.
Migration tests should not import and use factories or models
directly from the code because they would not be in sync with
the database in the state that each state needs to test it.
Instead the migrator object passed as argument allows us to
retrieve a minimal version of the models in sync with the state
of the database that we are testing. What we get is a minimal
model and we need to simulate all the methods that we could have
on the real model and that are needed for testing.
Tests were forgotten. While writing the tests, I fixed
a few edge cases like the possibility to connect to the
collaboration server for an anonymous user.
These 2 actions had factorized code but a few iterations lead to
spaghetti code where factorized code includes "if" clauses.
Refactor abstractions so that code factorization really works.
Some providers with S3-compatible APIs have slightly different
implementations. In this case, Scaleway didn't accept version_id=""
and has a different version ID scheme. This was tested successfully
and should remain compatible with any other provider.
The user list endpoint is throttle to avoid users discovery. The
throttle is set to 500 requests per day. This can be changed using the
settings API_USERS_LIST_THROTTLE_RATE.
The user list endpoint does not use anymore a pagination, the results is
directly return in a list and the max results returned is limited to 5.
In order to modify this limit the settings API_USERS_LIST_LIMIT is
used.
The cors-proxy endpoint allowed to use every type of files and to
execute it in the browser. We limit the scope only to images and
Content-Security-Policy and Content-Disposition headers are also added
to not allow script execution that can be present in a SVG file.
Swaggers urls where not using the same base route path /api/v1.0, we
prepend it to have the same path everywhere. Moreover, a double slash
was used for swagger and redoc dashboard.
The ai translation were quite lossy about formatting.
Colors, background, breaklines, table sizes were
lost in the translation.
We improve the AI translation request to keep
the formatting as close as possible by using
html instead of markdown.
When exporting a document in PDF and if the doc contains external
resources, we want to fetch them using a proxy bypassing CORS
restrictions. To ensure this endpoint is not used for something else
than fetching urls contains in the doc, we use access control and check
if the url really exists in the document.
The numchild attribute must be incremented/decremented manually
when we soft delete a document if we want it to remain accurate,
which is important to display the tree structure in the frontend.
The "nb_accesses" field was displaying the number of access instances
related to a document or any of its ancestors. Some features on the
frontend require to know how many of these access instances are related
to the document directly.
If a document already gets a link reach/role inheriting from one of its
ancestors, we should not propose setting link reach/role on the
document that would be more restrictive than what we inherited from
ancestors.
We want to be able to make a search query inside a hierchical document.
It's elegant to do it as a document detail action so that we benefit
from access control.
the "filter_queryset" method is called in the middle of the
"get_object" method. We use the "get_object" in actions like
"children", "tree", etc. which start by calling "get_object"
but return lists of documents.
We would like to apply filters to these views but the it didn't
work because the "get_object" method was also impacted by the
filters...
In a future PR, we should take control of the "get_object" method
and decouple all this. We need a quick solution to allow releasing
the hierchical documents feature in the frontend.
We want to display the tree structure to which a document belongs
on the left side panel of its detail view. For this, we need an
endpoint to retrieve the list view of the document's ancestors
opened.
By opened, we mean that when display the document, we also need to
display its siblings. When displaying the parent of the current
document, we also need to display the siblings of the parent...
- allow the language attribute on the user to be updated via API
- add frontend function to update the user language via API
- extend defaults on the test users, to have fixed language in E2E tests
- extend types and variables using the types with the new field
- allow the language on the user to be unset
- set the default language to be unset
- helps us determine that the user has yet to set a language preference
- language for invitation emails => language saved on the invited user
- if invited user does not exist yet => language of the sending user
- if for some reason no sending user => system default language
To determine the descendant to restore or not, we were looking building
a complex exclude clause. This can be simplify focusing only on data we
already have without making an extra query to fetch the list of
descendant to exclude.
On the media upload endpoint, we want to set the content-disposition
header. Its value is based on the uploaded file mime-type and if flagged
as unsafe. If the file is not an image or is unsafe then the
contentDisposition is set to attachment to force its download.
Otherwise, we set it to inline.
The frontend cannot access custom headers of a file,
so we need to add a flag in the filename.
We add the `unsafe` flag in the filename to
indicate that the file is unsafe.
Previous filename: "/{UUID4}.{extension}"
New filename: "/{UUID4}-unsafe.{extension}"
In the restore method, all the ancestors with a deleted_at date set are
extracted from the database and then the oldest value is extracted using
the min python function. This usage of min can be removed by sorting
directly the deleted_at at the databse level and then fetching the first
one. It's faster and easier to maintain.
Previous commit add "core/tests/migrations".
The linter could not pass on it because all the
migration folders were excluded from the linter.
We remove this exclusion, tests and migrations can
now be linted and formatted automatically.
The frontend was setting a default titles for
documents with empty titles.
This migration updates the document table to set
the title to null instead of the default title.
We add a test to ensure that the migration
works as expected.
We want to be able to define whether AI features are available to
anonymous users who gained editor access on a document, or if we
demand that they be authenticated or even if we demand that they
gained their editor access via a specific document access.
Being authenticated is now the default value. This will change the
default behavior on your existing instance (see UPGRADE.md)
We changed the way we upload the translations to
Crowdin, some translations were missing for the
email templates. We add them back and improve
the tests to make sure we don't forget them again.
When creating a new document/template via the API, we add the
logged-in user as owner of the created object. This should be
done atomically with the object creation to make sure we don't
end-up with an orphan object that the creator can't access
anymore.
Only owners can see and restore deleted documents. They can only do
it during the grace period before the document is considered hard
deleted and hidden from everybody on the API.
Now that we have introduced a document tree structure, it is not
possible to allow deleting documents anymore as it impacts the whole
subtree below the deleted document and the consequences are too big.
We introduce soft delete in order to give a second thought to the
document's owner (who is the only one to be allowed to delete a
document). After a document is soft deleted, the owner can still
see it in the trashbin (/api/v1.0/documents/trashbin).
After a grace period (30 days be default) the document disappears
from the trashbin and can't be restored anymore. Note that even
then it is still kept in database. Cleaning the database to erase
deleted documents after the grace period can be done as a maintenance
script.
Only administrators or owners of a document can move it to a target
document for which they are also administrator or owner.
We allow different moving modes:
- first-child: move the document as the first child of the target
- last-child: move the document as the last child of the target
- first-sibling: move the document as the first sibling of the target
- last-sibling: move the document as the last sibling of the target
- left: move the document as sibling ordered just before the target
- right: move the document as sibling ordered just after the target
The whole subtree below the document that is being moved, moves as
well and remains below the document after it is moved.
This test was missing the status code check. Without this check
the error that follows does not make sense because the content
returned is not at all what we expect in the following assert
statement.
