Commit Graph

160 Commits

Author SHA1 Message Date
renovate[bot]
65fdf115be ⬆️(dependencies) update django to v5.1.4 [SECURITY] 2024-12-13 21:28:11 +01:00
Anthony LC
ecb2b35ec8 (email) white brand email
The email was branded "La Suite Numérique",
we updated the template to make it generic, we
will use settings env variables to customize the
email for each brand.
2024-12-13 17:58:43 +01:00
renovate[bot]
2d13e0985e ⬆️(dependencies) update python dependencies 2024-12-12 18:56:25 +01:00
Samuel Paccoud - DINUM
5cdd06d432 (backend) add server-to-server API endpoint to create documents
We want trusted external applications to be able to create documents
via the API on behalf of any user. The user may or may not pre-exist
in our database and should be notified of the document creation by
email.
2024-12-12 14:01:46 +01:00
Anthony LC
7dfc62b2c5 🔖(minor) release 1.9.0
Added:
- (backend) annotate number of accesses
on documents in list view
- (backend) allow users to mark/unmark
documents as favorite

Changed:
- 🔒️(collaboration) increase collaboration access security
- 🔨(frontend) encapsulated title to its own component
- ️(backend) optimize number of queries on
document list view
- ♻️(frontend) stop to use provider with version
- 🚚(collaboration) change the websocket key name

Fixed:
- 🐛(frontend) fix initial content with collaboration
- 🐛(frontend) Fix hidden menu on Firefox
- 🐛(backend) fix sanitize problem IA
2024-12-12 08:37:10 +01:00
Anthony LC
57c5c394f5 🐛(backend) improve sanitizer ai_services
The json response of the AI service is badly formatted.
This commit improves the sanitizer to try
to handle the response correctly.
2024-12-11 21:18:40 +01:00
Anthony LC
ed90769081 🐛(backend) fix sanitize problem IA
Albert send us back a malformed IA json, the
sanitize function was not able to handle it correctly.
We add a try catch on it, to not use the sanitizer if
the json.loads fails.
2024-12-11 15:21:56 +01:00
Anthony LC
94a1ba7989 (backend) notify collaboration server
When an access is updated or removed, the
collaboration server is notified to reset the
access connection; by being disconnected, the
accesses will automatically reconnect by passing
by the ngnix subrequest, and so get the good
rights.
We do the same system when the document link is
updated, except here we reset every access
connection.
2024-12-11 14:54:41 +01:00
Samuel Paccoud - DINUM
66553ee236 (backend) add subrequest auth view for collaboration server
We need to improve security on the access to The collaboration server
We can use the same pattern as for media files leveraging the nginx
subrequest feature.
2024-12-11 14:54:41 +01:00
Samuel Paccoud - DINUM
64674b6a73 ♻️(backend) rename, factorize and improve the subrequest media auth view
We want to use the same pattern for the websocket collaboration service
authorization as what we use for media files.

This addition comes in the next commit but doing it efficiently
required factorizing some code with the media auth view.
2024-12-11 14:54:41 +01:00
Anthony LC
aff036d9fb 🚚(collaboration) change the websocket url name
We will have 2 urls targeting the server, better
to improve the naming to avoid confusion.
2024-12-06 15:23:16 +01:00
Samuel Paccoud - DINUM
ba962af914 ⬆️(backend) bump openai library version as it breaks tests
This looks like an instability in the openai library's definition
of dependencies.
2024-12-05 23:34:06 +01:00
Samuel Paccoud - DINUM
1af7b797bc (backend) test interference btw documents permissions and filtering
We want to make sure that applying filters on the document view list
does not interfere with permissions.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
b5c159bf63 (backend) allow filtering on document titles
This is the minimal and fast search feature, while we are working on
a full text search based on opensearch. For the moment we only search
on the title of the document.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
bfbdfb2b5c (backend) allow filtering documents by link reach
We want to be able to limit document list views to only public documents,
or only restricted or authenticated documents.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
08bb64ddc1 (backend) allow filtering by documents marked as favorite
We recently allowed authenticated users to mark a document as favorite.
We were lacking the possibility for users to see only the documents
they marked as favorite.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
23f90156bf (backend) add creator field on document and allow filtering on it
We want to be able to limit the documents displayed on a logged-in user's
list view by the documents they created or by the documents that other
users created.

This is different from having the "owner" role on a document because this
can be acquired and even lost. What we want here is to be able to
identify documents by the user who created them so we add a new field.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
1899cff572 🐛(backend) fix flaky test by clarifying user ordering
On the user search API by similarity, we had a flaky test because
2 users had the same similarity score. Adding a secondary ordering
field makes ordering deterministic between users who share the same
similarity score.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
774c2ce248 (backend) annotate number of accesses on documents
The new UI will display the number of accesses on each document.

/!\ Once team accesses will be used, this will not represent the number
    of people with access anymore and will have to be improved by
    computing the number of people in each team.
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
89d9075850 (backend) allow users to mark/unmark documents as favorite
A user can now mark/unmark documents as favorite.
This is done via a new action of the document API endpoint:
/api/v1.0/documents/{document_id}/favorite
POST to mark as favorite / DELETE to unmark
2024-11-28 16:02:27 +01:00
Samuel Paccoud - DINUM
2c915d53f4 ️(backend) optimize number of queries on document list view
I realized most of the database queries made when getting a document
list view were to include nested accesses. This detailed information
about accesses in only necessary for the document detail view.

I introduced a specific serializer for the document list view with
less fields. For a list of 20 documents with 5 accesses, we go down
from 3x5x20= 300 queries to just 3 queries.
2024-11-28 16:02:27 +01:00
Anthony LC
797d9442ac 🔖(patch) release 1.8.2
Changed:

- ♻️(SW) change strategy html caching
2024-11-28 15:31:03 +01:00
Anthony LC
2035a256f5 🔖(patch) release 1.8.1
Fixed:
🐛(frontend) link not clickable and flickering firefox
2024-11-27 17:17:35 +01:00
Anthony LC
2194301716 🔖(minor) release 1.8.0
Added:
- 🌐(backend) add german translation
- 🌐(frontend) Add German translation
- (frontend) Add a broadcast store
- (backend) whitelist pod's IP address
- (backend) config endpoint
- (frontend) config endpoint
- (frontend) add sentry
- (frontend) add crisp chatbot

Changed:
- 🚸(backend) improve users similarity search and
sort results
- ♻️(frontend) simplify stores
- (frontend) update $css Box props type to add
styled components RuleSet
- (CI) trivy continue on error

Fixed:
- 🔧(backend) fix logging for docker and make it
configurable by envar
- 🦺(backend) add comma to sub regex
- 🐛(editor) collaborative user tag hidden when
read only
- 🐛(frontend) users have view access when revoked
- 🐛(frontend) fix placeholder editable when double clicks
2024-11-27 09:47:42 +01:00
Anthony LC
af039d045d 🔧(backend) add CRISP_WEBSITE_ID setting
Add setting CRISP_WEBSITE_ID. This setting is
used to configure the Crisp chat widget.
It will be available to the conf endpoint, to
be used by the frontend.
2024-11-25 17:06:02 +01:00
Anthony LC
0e55bf5c43 🔒️(helm) allow server host and whitelist pod IP for health checks
In a Kubernetes environment, we need to whitelist the pod's IP address
to allow health checks to pass. This ensures that Kubernetes liveness and
readiness probes can access the application to verify its health.
2024-11-22 13:01:55 +01:00
Samuel Paccoud - DINUM
9f66f73501 🔧(backend) fix logging for docker and make it configurable by envar
Logs were not made to the console so it was hard to debug in k8s.
We propose a ready made logging configuration that sends everything
to the console and allow adjusting log levels with environment
variables.
2024-11-20 11:51:20 +01:00
Anthony LC
9623ac4141 🩹(backend) get current release from pyproject.toml
"get_release" was returning NA, we fixed it by
getting the version from pyproject.toml, to do so we
use tomllib
Since tomllib is a native library from Python 3.11,
we bump the required version to 3.11 on the pyproject.toml.
2024-11-20 09:51:08 +01:00
Anthony LC
c8edbd285b 🔧(backend) add FRONTEND_THEME setting
The frontend need to know the theme to be used,
so we need to add a new setting to the backend,
in order to expose this value to the frontend.
2024-11-20 09:51:08 +01:00
Anthony LC
016597d5a2 🔧(backend) add COLLABORATION_SERVER_URL setting
The frontend need to know the collab server url,
so we need to add a new setting to the backend,
in order to expose this value to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the ws requests,
but in development we need to define it to target
the collaboration server.
2024-11-20 09:51:08 +01:00
Anthony LC
52dea8fa2f 🔧(backend) add MEDIA_BASE_URL setting
The frontend need to know the base url for the
media files, so we need to add a new setting
to the backend, in order to expose this value
to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the media requests,
but in development we need to define it to target
the nginx server.
2024-11-20 09:51:08 +01:00
Anthony LC
0a37a8ea6d (backend) add public endpoint /api/v1.0/config/
Add public endpoint /api/v1.0/config/ to
share some public configuration values.
2024-11-20 09:51:08 +01:00
Anthony LC
526e649f06 🦺(backend) add comma to sub regex
Some sub have comma, the regex was a bit too strict
and didn't allow it, this commit fixes that.
2024-11-08 10:53:53 +01:00
lindenb1
c750cf10a8 🌐(backend) adding de_DE translation for the backend
This adds German translation to the backend and
adjusts the .po file sequence by priority.

Signed-off-by: lindenb1 <linden@b1-systems.de>
2024-11-07 11:49:41 +01:00
Samuel Paccoud - DINUM
4f4951cdcd 🚸(backend) improve users similarity search and sort results
In some edge cases, the domain part the email addresse is
longer than the name part. Users searches by email similarity
then return a lot of unsorted results.

We can improve this by being more demanding on similarity when
the query looks like an email. Sorting results by the similarity
score is also an obvious improvement.

At the moment, we still think it is good to propose results with
a weak similarity on the name part because we want to avoid
as much as possible creating duplicate users by inviting one of
is many emails, a user who is already in our database.

Fixes 399
2024-11-06 08:27:18 +01:00
Anthony LC
50891afd05 🔖(minor) release 1.7.0
Added:
- 📝Contributing.md
- 🌐(frontend) add localization to editor
- Public and restricted doc editable
- (frontend) Add full name if available
- (backend) Add view accesses ability

Changed:
- ♻️(frontend) avoid documents indexing in search engine
- ♻️(frontend) list accesses if user has abilities
- 👔(backend) doc restricted by default

Fixed:
- 🐛(backend) require right to manage document
  accesses to see invitations
- 🐛(i18n) same frontend and backend language using
  shared cookies
- 🐛(frontend) add default toolbar buttons
- 🐛(frontend) throttle error correctly display

Removed:
- 🔥(helm) remove infra related codes
2024-10-25 14:41:48 +02:00
Anthony LC
cbb6fc740a 👔(backend) doc restricted by default
By default a created document was in "authenticated"
mode, we switch to "restricted" by default.
2024-10-25 14:25:48 +02:00
Samuel Paccoud - DINUM
15700ddd8d (backend) add new ability on document "accesses_view"
We need this ability in the frontend to know whether we should try
to display the list of users who have document accesses. If this
ability is False (e.g for anonymous users), we should only show
the link reach and link role when clicking on the "Share" button.
2024-10-24 17:31:34 +02:00
rvveber
3c374e3cc7 🐛(i18n) same frontend and backend language using shared cookies
frontend: switch to cookie-based language selection
backend: use cookie for language
2024-10-23 12:35:48 +02:00
Samuel Paccoud - DINUM
0f0f812059 🐛(backend) fix invitations API endpoint access rights
Only users who have the rights to manage accesses on the document should
be allowed to see and manipulate invitations. Other users can see access
rights on the document but only when the corresponding user/team has
actually been granted access.

We added a parameter in document abilities so the frontend knows when
the logged-in user can invite another user with the owner role or not.
2024-10-22 19:39:59 +02:00
renovate[bot]
305359ae15 ⬆️(dependencies) update python dependencies 2024-10-21 09:20:33 +02:00
Anthony LC
15235a9bc2 🔖(minor) release 1.6.0
Added:
- AI to doc editor
- (backend) allow uploading more types of attachments
- (frontend) add buttons to copy document to clipboard as HTML/Markdown

Changed:
- ♻️(frontend) More multi theme friendly
- ♻️ Bootstrap frontend
- ♻️ Add username in email

Fixed:
- 🛂(backend) do not duplicate user when disabled
- 🐛(frontend) invalidate queries after removing user
- 🐛(backend) Fix dysfunctional permissions on document create
- 🐛(backend) fix nginx docker container
- 🐛(frontend) fix copy paste firefox
2024-10-17 17:50:57 +02:00
Samuel Paccoud - DINUM
6a95d24441 🛂(backend) do not duplicate user when disabled
When a user is disabled and tries to login, we
don't want the user to be duplicated,
the user should not be able to login.

Fixes #324

Work initially contributed by @qbey on:
https://github.com/numerique-gouv/people/pull/456
2024-10-17 16:54:40 +02:00
Anthony LC
aff3b43c9d (backend) create ai endpoint
We created 2 new action endpoints on the document
to perform AI operations:
- POST /api/v1.0/documents/{uuid}/ai-transform
- POST /api/v1.0/documents/{uuid}/ai-translate
2024-10-16 22:58:52 +02:00
Samuel Paccoud - DINUM
e8d95facdf (backend) allow uploading more types of attachments
We want to allow users to upload files to a document, not just images.
We try to enforce coherence between the file extension and the real
mime type of its content. If a file is deemed unsafe, it is still accepted
during upload and the information is stored as metadata on the object
for display to readers.
2024-10-16 19:40:28 +02:00
Samuel Paccoud - DINUM
a9f08df566 (backend) move freezegun to dev dependencies
Freezegun is for testing and should not be installed in the
production image.
2024-10-16 19:40:28 +02:00
Samuel Paccoud - DINUM
2fecbc1162 🚚(backend) split test file for api template accesses
The number of lines in this file had exceeded 1000 lines.
2024-10-16 19:16:50 +02:00
Samuel Paccoud - DINUM
1fc3029d12 🐛(backend) fix dysfunctional permissions on document create
When creating a document access, users were benefitting on the targeted
document from the highest access right they have among all documents.
This is because we forgot to filter on the document ID when retrieving
the role of the user. We improved all tests to secure this issue.
2024-10-16 19:16:50 +02:00
renovate[bot]
e4a7ac0f3c ⬆️(dependencies) update python dependencies 2024-10-16 10:41:25 +02:00
Anthony LC
24630791d8 ♻️(email) use full name instead of email
If the full name is available,
we will use it to identify the user in the email
instead of the email address.
2024-10-16 09:36:33 +02:00