We want to persist the session during development. Otherwise the session
is reset everytime the server is restart. This behavior make developing
bot a front and back feature a nigthmare, we spend our time login again
and again
To ease filtering issues on sentry, we want to use tags instead of extra
scope. Tags are indexed and searchable, it's not the case with extra
scope. Moreover using setEtra to add additional data is deprecated.
We added content-security-policy on nginx.
It should be safe to allow svg files now.
We remove the svg file from the unsafe
attachments list. We adapt the tests accordingly.
We want to be able to define whether AI features are available to
anonymous users who gained editor access on a document, or if we
demand that they be authenticated or even if we demand that they
gained their editor access via a specific document access.
Being authenticated is now the default value. This will change the
default behavior on your existing instance (see UPGRADE.md)
Now that we have introduced a document tree structure, it is not
possible to allow deleting documents anymore as it impacts the whole
subtree below the deleted document and the consequences are too big.
We introduce soft delete in order to give a second thought to the
document's owner (who is the only one to be allowed to delete a
document). After a document is soft deleted, the owner can still
see it in the trashbin (/api/v1.0/documents/trashbin).
After a grace period (30 days be default) the document disappears
from the trashbin and can't be restored anymore. Note that even
then it is still kept in database. Cleaning the database to erase
deleted documents after the grace period can be done as a maintenance
script.
We choose to use Django-treebeard for its quality, performance and
stability. Adding tree structure to documents is as simple as
inheriting from the MP_Node class.
When creating a document on behalf of a user via the server-to-server
API, a special edge case was broken that should should never happen
but happens in our OIDC federation because one of the provider modifies
the users "sub" each time they login.
We end-up with existing users for who the email matches but not the sub.
They were not correctly handled.
I made a few additional fixes and improvements to the endpoint.
It was pointed by @lebaudantoine that the OIDC specification uses
the term "essential claims" for what we called required claims.
Further more, the Mozilla OIDC library that we use, validates claims
in a method called "verify_claims". Let's override this method.
Support for two API keys has been added to the YProvider microservice to
decouple responsibilities between the collaboration server and other
endpoints. This improves security by scoping keys to specific purposes and
ensures a clearer separation of concerns for easier management and debugging.
Abstracted base URL and API key under 'y-provider' for
reuse in future endpoints, aligning with microservice naming.
Please note the YProvider API here is internal to the cluster.
In facts, we don't want these endpoints to be exposed by any ingress
Minor adjustments were needed after working in parallel on two PRs.
The microservice now accepts an API key without requiring it as a Bearer token.
A mistake in reading the microservice response was corrected after refactoring
the serializer to delegate logic to the converter microservice.
The email was branded "La Suite Numérique",
we updated the template to make it generic, we
will use settings env variables to customize the
email for each brand.
We want trusted external applications to be able to create documents
via the API on behalf of any user. The user may or may not pre-exist
in our database and should be notified of the document creation by
email.
When an access is updated or removed, the
collaboration server is notified to reset the
access connection; by being disconnected, the
accesses will automatically reconnect by passing
by the ngnix subrequest, and so get the good
rights.
We do the same system when the document link is
updated, except here we reset every access
connection.
We need to improve security on the access to The collaboration server
We can use the same pattern as for media files leveraging the nginx
subrequest feature.
Add setting CRISP_WEBSITE_ID. This setting is
used to configure the Crisp chat widget.
It will be available to the conf endpoint, to
be used by the frontend.
In a Kubernetes environment, we need to whitelist the pod's IP address
to allow health checks to pass. This ensures that Kubernetes liveness and
readiness probes can access the application to verify its health.
Logs were not made to the console so it was hard to debug in k8s.
We propose a ready made logging configuration that sends everything
to the console and allow adjusting log levels with environment
variables.
"get_release" was returning NA, we fixed it by
getting the version from pyproject.toml, to do so we
use tomllib
Since tomllib is a native library from Python 3.11,
we bump the required version to 3.11 on the pyproject.toml.
The frontend need to know the collab server url,
so we need to add a new setting to the backend,
in order to expose this value to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the ws requests,
but in development we need to define it to target
the collaboration server.
The frontend need to know the base url for the
media files, so we need to add a new setting
to the backend, in order to expose this value
to the frontend.
If the setting is not defined, the frontend current
domain will be used as the base url.
In production this setting do not need to be defined
since we have nginx capturing the media requests,
but in development we need to define it to target
the nginx server.
We created 2 new action endpoints on the document
to perform AI operations:
- POST /api/v1.0/documents/{uuid}/ai-transform
- POST /api/v1.0/documents/{uuid}/ai-translate
We want to allow users to upload files to a document, not just images.
We try to enforce coherence between the file extension and the real
mime type of its content. If a file is deemed unsafe, it is still accepted
during upload and the information is stored as metadata on the object
for display to readers.
Some OIDC identity providers may provide a random value in the "sub"
field instead of an identifying ID. In this case, it may be a good
idea to fallback to matching the user on its email field.
Fix following warning messages :
- You have not set a value for the SECURE_HSTS_SECONDS setting.
- Your SECURE_SSL_REDIRECT setting is not set to True.
We only rely on S3 to store attachments for a document. Nothing
is persisted in the database as the image media urls will be
stored in the document json.
Email settings were wrongly configured. It leed to
unsent email and timeout response from the backend
server. This commit fixes the issue by enabling SSL
when sending email.
We want to be able to share a document with a person even if this person
does not have an account in impress yet.
This code is ported from https://github.com/numerique-gouv/people.
The default Logout view provided by Mozilla Django OIDC is not suitable
for the Agent Connect Logout flow.
Previously, when a user was logging-out, only its Django session was ended.
However, its session in the OIDC provider was still active.
Agent Connect implements a 'session/end' endpoint, that allows services to
end user session when they logout.
Agent Connect logout triggers cannot work with the default views implemented
by the dependency Mozilla Django OIDC. In their implementation, they decided
to end Django Session before redirecting to the OIDC provider.
The Django session needs to be retained during the logout process.
An OIDC state is saved to the request session, pass to Agent Connect Logout
endpoint, and verified when the backend receives the Logout callback from Agent
Connect. It seems to follow OIDC specifications.
If for any reason, the Logout flow cannot be initiated with Agent Connect,
(missing ID token in cache, unauthenticated user, etc), the user is redirected
to the final URL, without interacting with Agent Connect.
Prepare adding advanced authentication features. Create a dedicated
authentication Python package within the core app.
This code organization will be more extensible.
Versions are retrieved directly from object storage and served on API
endpoints. We make sure a user who is given access to a document will
only see versions that were created after s.he gained access.
The content field is a writable property on the model which is persisted
in object storage. We take advantage of the versioning, robustness and
scalability of S3.
mozilla-django-oidc didn't add the https://
prefix to the redirect_uri.
We set the option SECURE_PROXY_SSL_HEADER to
('HTTP_X_FORWARDED_PROTO', 'https') in the
settings.py file to force the https prefix.
Enabled Dockerflow Django app by activating liveness probes. The previously
unavailable routes such as `__heartbeat__` and `__lbheartbeat__` are now
accessible. New endpoints include:
* GET /__version__
* GET /__heartbeat__
* GET /__lbheartbeat__
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.
With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.
I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.
I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.
Two new dependencies were added, redis and django-redis.
I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
