Integrate multiselect search users in the
modal add members.
We are using react-select to implement the
multiselect search users. We are using this
library in waiting for Cunningham to implement
the multiselect asynch component.
The CI was controlling if the traduction was made
in every PR. It makes the workflow quite grueling
when we have to change the literal, plus the synch
is complicating when we have multiple PR opened.
We remove the CI control on the traduction, we
will do dedicated PR to update the traduction.
We will add the CI control on the traduction in
the future, before a release by example.
When generating a batch of users using Faker, there's a possibility of
generating multiple users with the same email address. This breaches
the uniqueness constraint set on the email field, leading to flaky
tests that may fail when random behavior coincides unfavorably.
Implemented a method inspired by Identity's model to ensure unique
email addresses when creating user batches with Faker.
Updated relevant tests for improved stability.
Important ordering fields for TeamAccess depend on user's
identities data. User and identities has a one-to-many relationship,
which forced us to prefetch the user-related data when listing
team's accesses.
Prefetch get data from the database using two SQL queries, and
join data in Python. User's data were not available in the first
SQL query.
Without annotating the query set with user main identities data,
we could not use default OrderingFilter backend code, which order_by()
the queryset.
Enhance list capabilities, by adding the OrderingFilter as filter backend,
to the TeamAccess viewset.
API response can be ordered by TeamAccess role. More supported ordering
fields will be supported later on.
We created a custom Pagination class, were max_page_size is overriden.
I decided to add bare minimum tests to make sure we can set the maximum
page size using the 'page_size' query parameter.
Our Pagination class inherits from the PageNumberPagination Django class.
However, this base class as not ordering attribute. Thus, setting a
default value wont have any effect on the code.
Why did we end up passing a value to this non-existing attribute? Becasue
we copy/pasted some code sources from Joanie, and Joanie also has this
attribute set to a default value.
If you take a look at DRF pagination style documentation, the only three
attributes they set on the child class are 'page_size', 'max_page_size'
'page_size_query_param'. 'ordering' is not mentionned in the attributes
you may override. However, the CursorPagination class offers the latter
attribute, which may explain why we did end up setting this non-existing
attribute in Joanie.
djangorestframework released version 3.15.0, which includes modifications of
details upon returning 404 errors (see related issue
https://github.com/encode/django-rest-framework/pull/8051).
This commit changes the expected details of 404 responses in our tests,
to match DRF 3.15.0.
Based on @rouja reco, I added a dedicated ingress to serve Django Admin
pages and Django statics. The admin route will be secured by the oauth proxy.
I simply copy/pasted the first ingress template, and adapted it.
This commit is working in progress. I have added an extra chart to take
benefits of the Redis operator developed by Indie hoster.
When using the dev environment, I used bitnami redis chart to deploy
a Redis service with authentication disable.
CSRF trusted origins are set using an environment variable. The env
value was wrongly name to CORS_ALLOWED_ORIGINS, which doesn't exist
in our Django configurations. I fixed this minor issue.
Set OIDC secrets for the dev environment. Please note that we use different
secrets between dev and staging. Why? Benoit created two client id, thus we
could easily tests Agent Connect feature from the local host and the staging
one.
The local host is desk.127.0.0.1.nip.io. If this value change at any time,
please consider asking Benoit to update the host value linked to the dev
client id.
Thx @rouja for your help on deploying Desk. This commit slightly modifies
helm charts and helmfile to prepare the initial project deployment in a
staging environment.
@rouja updates:
- added secrets files for dev and staging environments (dev's one is empty)
- disable ingress by default, to avoid any security issue
- added an extra chart to benefit from Indie hoster Postgres operator
Thx to this commit we deployed a first draft version figured out
that the Django session were broken. We are using a cache session engine,
and wrongly configure cache backend to local memory. Thus, Django server
is not able to resolve the session, and enters in an infinite loop to
log-in the user.
Please note that this Helmfile is uncomplete, it lacks services as
redis, celery, mail ... which are declared in the Docker Compose file
but not yet used in development and production images.
Thus, to run the Desk Helm chart, we only add a postgres database to run the
Django backend server, and apply migrations.
For now, this Helmfile is quite hard to test in dev environment, because the
frontend redirects automatically to the SSO login page. We cannot really
assess if backend and frontend are working properly. We might adjust some
configurations after the first deployment in stagging.
(We are a bit in rush, to respect the current sprint deadline.)
Development values points https://desk.127.0.0.1.nip.io URL. Please note that
the frontend image for now has been built with this URL for the backend address.
Meaning that we either need to rebuild and publish a frontend image with the
staging URL when deploying the project, or enhance our frontend image, to pass
the backend URL at runtime.
First, thanks a LOT @rouja for your help along the way.
This commit propose a first draft of Helm chart to prepare deployment.
It follows Plane's Helm Chart, hosted on the shared team repo,
please https://github.com/numerique-gouv/helm-charts, PR #11
It offers advanced templating function under _helpers.tpl, an auto-generated
README file when running ./generate-readme.sh, and a clear files structure.
The chart itself is quite simple. We have two deployments, one for the
frontend and one for the backend. Both need a dedicated service, which are
exposed using a common ingress. Frontend is accessible from the / path and
backend's from /api path.
Please note, we added a backend job to migrate the database when deploying
backend's pods. This job should be auto-cleaning itself 100s after it completes
to avoid any error when syncing helm.
values.yaml file is quite pristine, all common env variables will be set
in helmfile configuration.
Deploying frontend static files through kubernetes is temporary, we plan to
either remplace it by an external CDN or use minio to host static output in
a S3 bucket within the cluster.
When we pull the translations from crowdin we
get lot of git diff noise with the json file.
We order the keys in the json file to make the
diffs more readable.
`backend-development` and `backend-production` CMD syntaxes were
using a Shell Form. Shell form prevented Unix signals from reaching
our container correctly, such as SIGTERM. Also, the shell process
ends up being the PID 1, instead of our Python scripts.
Docker recommends to use the exec form whenever possible.
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.
With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.
I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.
I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.
Two new dependencies were added, redis and django-redis.
I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
It was quite confusing having development, production and
frontend images' names in the same Docker file. New comers
to the project would have some difficuluties when
differentiating frontend from backend images.
Try to make these naming more explicit and consistent.
Thanks @rouja for your recommendation.
This is a major issue. Docker Images were built and published with a
root user in the CI.
if a user manages to break out of the application running as root in the
container, he may gain root user access on host. In addition, configuring
container to user unprivileged is the best way yo prevent privilege
escalation attacks.
We mitigated this issue by creating a new environment variable DOCKER_USER.
DOCKER_USER is set with id -u and id -g outputs. Then, it is passed as a
build-args when running docker/build-push-action steps.
Ignore frontend dependencies when coping frontend sources to build
the frontend Docker image. It would improve a bit performances locally,
when building the frontend image.
Build and push the frontend image to DockerHub. Backend an Frontend
images will be stored in separate repos: people-backend and people-frontend.
It will be cleaner than managing all images in a single repo and creating
tags to discriminate frontend and backend images.
CI code is not factorized between jobs. Frontend and backend jobs could be
a bit factorized. Hovewer it might be a bit premature, and I prefer having
them decoupled for now. @rouja suggested to introduce a custom github actions
to avoid maintaining the same logic accross different repo.
Please not as the images are built from the same Dockerfile, it's important
to precise the right target.
Updated to Node Image version 20 to align with the frontend image. It will
save us having two different Node versions in the same docker file, and
should not impact mail-builder.
To facilitate deployment on Kubernetes, we've introduced a Docker image for the
frontend. The Next.js project is built, and its static output is served using an
Nginx reverse proxy.
Since DevOps lacks a certified cold storage solution (e.g., S3) for serving
static files, we've decided to containerize the frontend as a quick workaround
for deploying staging environments.
Please note this Docker Image is WIP. One of the main issue still not resolved
concerns environment variables, which are only available when building the
Docker Image. Thus, having different environment variables values between
environment (dev, pre-prod, prod) will require us to build several frontend
images, and tag them with the appropriate target environment.
The `.env.production` values are not the final ones. For now, they were set to
dev values. It allows us to test the frontend image with the development setup.
Important: The frontend image is built-on top of an unprivileged Nginx image,
which exposes by default port 8080 instead of 80 for classic Nginx image.
You can find more info https://github.com/nginxinc/docker-nginx-unprivileged.
The Docker Compose Nginx service is used to proxy OIDC requests to keycloak,
in order to share the same host when initiating an OIDC flow, from outside and
inside docker virtual network.
All Nginx configurations related to serve frontend static build were moved to a
newly created conf file under src/frontend/apps/desk. When starting the frontend
image, we desire to start the minimum Nignx config required to serve frontend
statics.
The current implementation of our product demo via the make command lacks
user identity for a significant portion of generated users, limiting the
realism of the showcased scenarios. As it stands, users created by the make
command lack complete information, such as full names and email addresses,
because they don't have any identity.
I tried to come up with the simplest solution:
We now generate a very small portion of our users with 0 identities. The
probability for users to have only 1 identity is the highest but they
can have up to 4 with decreasing probabilities. I removed the possibility
to set a maximum number of identities as it doesn't bring any value.
3% percent of the identities created will have no email and 3% no name.
Fixes https://github.com/numerique-gouv/people/issues/90
Some tokens were not available from the hook.
We only had the tokens of the currentTheme available
but actually the theme is an augmentation of the
default theme, so we should use the default theme
tokens as a base and then override them with the
currentTheme tokens.
It is what this commit does.
We often need unstyled button to wrap around some content,
we were using Cunningham's button for this purpose,
but it is not the best choice as lot of style is applied
to their buttons.
This component is a simple wrapper around the button
element with all the Box functionalities. Usefull
for wrapping icons by example.
Compute Trigram similarity on user's name, and sum it up
with existing one based on user's email.
This approach is inspired by Contact search feature, which
computes a Trigram similarity score on first name and last
name, to sum up their scores.
With a similarity score influenced by both email and name,
API results would reflect both email and name user's attributes.
As we sum up similarities, I increased the similarity threshold.
Its value is empirical, and was finetuned to avoid breaking
existing tests. Please note, the updated value is closer to the
threshold used to search contacts.
Email or Name can be None. Summing two similarity scores with
one of them None, results in a None total score. To mitigate
this issue, I added a default empty string value, to replace
None values. Thus, the similarity score on this default empty
string value is equal to 0 and not to None anymore.
When testing user search, we generated few identities
with mocked emails.
Name attribute was introduced on Identity model. Currently
names are freely and randomly generated by the factory.
To make this mocked data more realist, mock also identities'
names to match their email.
It should not break existing tests, and will make them more
predictable when introducing advanced search features.
Few minor layout issues were fixed.
First display label and dates inline, so they wrap nicely
when screen's size decreases. It also fixes the text overflow
when the screen's size is tiny.
Then, align screen with the Figma design, where items are
justified on the left of the Team info component.
