This commit is working in progress. I have added an extra chart to take
benefits of the Redis operator developed by Indie hoster.
When using the dev environment, I used bitnami redis chart to deploy
a Redis service with authentication disable.
CSRF trusted origins are set using an environment variable. The env
value was wrongly name to CORS_ALLOWED_ORIGINS, which doesn't exist
in our Django configurations. I fixed this minor issue.
Set OIDC secrets for the dev environment. Please note that we use different
secrets between dev and staging. Why? Benoit created two client id, thus we
could easily tests Agent Connect feature from the local host and the staging
one.
The local host is desk.127.0.0.1.nip.io. If this value change at any time,
please consider asking Benoit to update the host value linked to the dev
client id.
Thx @rouja for your help on deploying Desk. This commit slightly modifies
helm charts and helmfile to prepare the initial project deployment in a
staging environment.
@rouja updates:
- added secrets files for dev and staging environments (dev's one is empty)
- disable ingress by default, to avoid any security issue
- added an extra chart to benefit from Indie hoster Postgres operator
Thx to this commit we deployed a first draft version figured out
that the Django session were broken. We are using a cache session engine,
and wrongly configure cache backend to local memory. Thus, Django server
is not able to resolve the session, and enters in an infinite loop to
log-in the user.
Please note that this Helmfile is uncomplete, it lacks services as
redis, celery, mail ... which are declared in the Docker Compose file
but not yet used in development and production images.
Thus, to run the Desk Helm chart, we only add a postgres database to run the
Django backend server, and apply migrations.
For now, this Helmfile is quite hard to test in dev environment, because the
frontend redirects automatically to the SSO login page. We cannot really
assess if backend and frontend are working properly. We might adjust some
configurations after the first deployment in stagging.
(We are a bit in rush, to respect the current sprint deadline.)
Development values points https://desk.127.0.0.1.nip.io URL. Please note that
the frontend image for now has been built with this URL for the backend address.
Meaning that we either need to rebuild and publish a frontend image with the
staging URL when deploying the project, or enhance our frontend image, to pass
the backend URL at runtime.
First, thanks a LOT @rouja for your help along the way.
This commit propose a first draft of Helm chart to prepare deployment.
It follows Plane's Helm Chart, hosted on the shared team repo,
please https://github.com/numerique-gouv/helm-charts, PR #11
It offers advanced templating function under _helpers.tpl, an auto-generated
README file when running ./generate-readme.sh, and a clear files structure.
The chart itself is quite simple. We have two deployments, one for the
frontend and one for the backend. Both need a dedicated service, which are
exposed using a common ingress. Frontend is accessible from the / path and
backend's from /api path.
Please note, we added a backend job to migrate the database when deploying
backend's pods. This job should be auto-cleaning itself 100s after it completes
to avoid any error when syncing helm.
values.yaml file is quite pristine, all common env variables will be set
in helmfile configuration.
Deploying frontend static files through kubernetes is temporary, we plan to
either remplace it by an external CDN or use minio to host static output in
a S3 bucket within the cluster.
When we pull the translations from crowdin we
get lot of git diff noise with the json file.
We order the keys in the json file to make the
diffs more readable.
`backend-development` and `backend-production` CMD syntaxes were
using a Shell Form. Shell form prevented Unix signals from reaching
our container correctly, such as SIGTERM. Also, the shell process
ends up being the PID 1, instead of our Python scripts.
Docker recommends to use the exec form whenever possible.
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.
With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.
I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.
I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.
Two new dependencies were added, redis and django-redis.
I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
It was quite confusing having development, production and
frontend images' names in the same Docker file. New comers
to the project would have some difficuluties when
differentiating frontend from backend images.
Try to make these naming more explicit and consistent.
Thanks @rouja for your recommendation.
This is a major issue. Docker Images were built and published with a
root user in the CI.
if a user manages to break out of the application running as root in the
container, he may gain root user access on host. In addition, configuring
container to user unprivileged is the best way yo prevent privilege
escalation attacks.
We mitigated this issue by creating a new environment variable DOCKER_USER.
DOCKER_USER is set with id -u and id -g outputs. Then, it is passed as a
build-args when running docker/build-push-action steps.
Ignore frontend dependencies when coping frontend sources to build
the frontend Docker image. It would improve a bit performances locally,
when building the frontend image.
Build and push the frontend image to DockerHub. Backend an Frontend
images will be stored in separate repos: people-backend and people-frontend.
It will be cleaner than managing all images in a single repo and creating
tags to discriminate frontend and backend images.
CI code is not factorized between jobs. Frontend and backend jobs could be
a bit factorized. Hovewer it might be a bit premature, and I prefer having
them decoupled for now. @rouja suggested to introduce a custom github actions
to avoid maintaining the same logic accross different repo.
Please not as the images are built from the same Dockerfile, it's important
to precise the right target.
Updated to Node Image version 20 to align with the frontend image. It will
save us having two different Node versions in the same docker file, and
should not impact mail-builder.
To facilitate deployment on Kubernetes, we've introduced a Docker image for the
frontend. The Next.js project is built, and its static output is served using an
Nginx reverse proxy.
Since DevOps lacks a certified cold storage solution (e.g., S3) for serving
static files, we've decided to containerize the frontend as a quick workaround
for deploying staging environments.
Please note this Docker Image is WIP. One of the main issue still not resolved
concerns environment variables, which are only available when building the
Docker Image. Thus, having different environment variables values between
environment (dev, pre-prod, prod) will require us to build several frontend
images, and tag them with the appropriate target environment.
The `.env.production` values are not the final ones. For now, they were set to
dev values. It allows us to test the frontend image with the development setup.
Important: The frontend image is built-on top of an unprivileged Nginx image,
which exposes by default port 8080 instead of 80 for classic Nginx image.
You can find more info https://github.com/nginxinc/docker-nginx-unprivileged.
The Docker Compose Nginx service is used to proxy OIDC requests to keycloak,
in order to share the same host when initiating an OIDC flow, from outside and
inside docker virtual network.
All Nginx configurations related to serve frontend static build were moved to a
newly created conf file under src/frontend/apps/desk. When starting the frontend
image, we desire to start the minimum Nignx config required to serve frontend
statics.
The current implementation of our product demo via the make command lacks
user identity for a significant portion of generated users, limiting the
realism of the showcased scenarios. As it stands, users created by the make
command lack complete information, such as full names and email addresses,
because they don't have any identity.
I tried to come up with the simplest solution:
We now generate a very small portion of our users with 0 identities. The
probability for users to have only 1 identity is the highest but they
can have up to 4 with decreasing probabilities. I removed the possibility
to set a maximum number of identities as it doesn't bring any value.
3% percent of the identities created will have no email and 3% no name.
Fixes https://github.com/numerique-gouv/people/issues/90
Some tokens were not available from the hook.
We only had the tokens of the currentTheme available
but actually the theme is an augmentation of the
default theme, so we should use the default theme
tokens as a base and then override them with the
currentTheme tokens.
It is what this commit does.
We often need unstyled button to wrap around some content,
we were using Cunningham's button for this purpose,
but it is not the best choice as lot of style is applied
to their buttons.
This component is a simple wrapper around the button
element with all the Box functionalities. Usefull
for wrapping icons by example.
Compute Trigram similarity on user's name, and sum it up
with existing one based on user's email.
This approach is inspired by Contact search feature, which
computes a Trigram similarity score on first name and last
name, to sum up their scores.
With a similarity score influenced by both email and name,
API results would reflect both email and name user's attributes.
As we sum up similarities, I increased the similarity threshold.
Its value is empirical, and was finetuned to avoid breaking
existing tests. Please note, the updated value is closer to the
threshold used to search contacts.
Email or Name can be None. Summing two similarity scores with
one of them None, results in a None total score. To mitigate
this issue, I added a default empty string value, to replace
None values. Thus, the similarity score on this default empty
string value is equal to 0 and not to None anymore.
When testing user search, we generated few identities
with mocked emails.
Name attribute was introduced on Identity model. Currently
names are freely and randomly generated by the factory.
To make this mocked data more realist, mock also identities'
names to match their email.
It should not break existing tests, and will make them more
predictable when introducing advanced search features.
Few minor layout issues were fixed.
First display label and dates inline, so they wrap nicely
when screen's size decreases. It also fixes the text overflow
when the screen's size is tiny.
Then, align screen with the Figma design, where items are
justified on the left of the Team info component.
Nest invitation router below team router and add create endpoints for
authenticated administrators/owners to invite new members to their team,
list valid and expired invitations or delete invite altogether.
Update will not be handled for now. Delete and recreate if needed.
Integrate the design and functionality
for updating a member's role.
Managed use cases:
- when the user is an admin
- when the user is the last owner
- when the user want to update other orner
Integrate the action button dropdown in
the member grid. For the moment it will be
used to update the role of a member.
Manage use cases:
- Does not display when member's role
- Does not display when member is an admin
that wants to update owner role.
Break copy/pasted comment from Joanie in several inline
comments, that are more specific and easy to read.
Hopefully, it will help future myself understanding this
queryset and explaining it.
To compute accesses's abilities, we need to determine
which is the user's role in the team.
We opted for a subquery, which retrieves the user's role
within the team and annotate queryset's results.
The current subquery was broken, and retrieved other
users than the request's user. It led to compute accesses'
abilities based on a randomly picked user.
Abilities on team accesses are computed based on request user role.
Thus, members' roles in relation with user's role matters a lot, to
ensure the abilities were correctly computed.
Complexified the test that lists team accesses while being authenticated.
More members are added to the team with privileged roles. The user
is added last to the less with the less privileged role, "member".
Order matters, because when computing the sub query to determine
user's role within the team, code use the first result value to set the
role to compute abilities.
When running make ruff-check, a warning informs the user that
some config are deprecated, and gives her the step to migrate.
This warning appears after Ruff released its v0.2.0.
Fix it, by keeping our pyproject.toml up to date.
We recently updated Ruff from 0.2.2 to v0.3, which introduced
Ruff 2024.2 style. This new style updated Ruff formatter's behavior,
making our make lint command fails.
Ruff 2024.2 style add a blank line after the module docstring.
Please take a look at Ruff ChangeLog to get more info.
