studio/meet
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

✨(project) Django boilerplate

Browse Source 
This commit introduces a boilerplate inspired by https://github.com/numerique-gouv/impress.
The code has been cleaned to remove unnecessary Impress logic and dependencies.

Changes made:
- Removed Minio, WebRTC, and create bucket from the stack.
- Removed the Next.js frontend (it will be replaced by Vite).
- Cleaned up impress-specific backend logics.

The whole stack remains functional:
- All tests pass.
- Linter checks pass.
- Agent Connexion sources are already set-up.

Why clear out the code?

To adhere to the KISS principle, we aim to maintain a minimalist codebase. Cloning Impress
allowed us to quickly inherit its code quality tools and deployment configurations for staging,
pre-production, and production environments.

What’s broken?
- The tsclient is not functional anymore.
- Some make commands need to be fixed.
- Helm sources are outdated.
- Naming across the project sources are inconsistent (impress, visio, etc.)
- CI is not configured properly.

This list might be incomplete. Let's grind it.
This commit is contained in:
Samuel Paccoud - DINUM
2024-01-09 15:30:36 +01:00
committed by lebaudantoine
parent 2d81979b0a
commit 5b1a2b20de
146 changed files with 12668 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
.github/workflows/deploy.yml vendored Normal file
View File

@@ -0,0 +1,52 @@
name: Deploy
on:
push:
tags:
- 'preprod'
- 'production'
jobs:
notify-argocd:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "impress,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_PRODUCTION_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_PRODUCTION_WEBHOOK_URL
start-test-on-preprod:
needs:
- notify-argocd
runs-on: ubuntu-latest
if: startsWith(github.event.ref, 'refs/tags/preprod')
steps:
-
name: Debug
run: |
echo "Start test when preprod is ready"

185
.github/workflows/docker-hub.yml vendored Normal file
View File

@@ -0,0 +1,185 @@
name: Docker Hub Workflow
on:
workflow_dispatch:
push:
branches:
- 'main'
tags:
- 'v*'
pull_request:
branches:
- 'main'
env:
DOCKER_USER: 1001:127
jobs:
build-and-push-backend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "impress,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-backend
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
target: backend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-frontend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "impress,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-frontend
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./src/frontend/Dockerfile
target: frontend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-y-webrtc-signaling:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "impress,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-y-webrtc-signaling
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./src/frontend/Dockerfile
target: y-webrtc-signaling
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
notify-argocd:
needs:
- build-and-push-frontend
- build-and-push-backend
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "impress,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL

171
.github/workflows/impress-frontend.yml vendored Normal file
View File

@@ -0,0 +1,171 @@
name: impress Workflow
on:
push:
branches:
- main
pull_request:
branches:
- "*"
jobs:
install-front:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "18.x"
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Build CI App
run: cd src/frontend/ && yarn ci:build
- name: Cache build frontend
uses: actions/cache@v4
with:
path: src/frontend/apps/impress/out/
key: build-front-${{ github.run_id }}
test-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Test App
run: cd src/frontend/ && yarn app:test
lint-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e:
runs-on: ubuntu-latest
needs: build-front
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set services env variables
run: |
make data/media
make create-env-files
cat env.d/development/common.e2e.dist >> env.d/development/common
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Restore the build cache
uses: actions/cache@v4
id: cache-build
with:
path: src/frontend/apps/impress/out/
key: build-front-${{ github.run_id }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build the Docker images
uses: docker/bake-action@v4
with:
targets: |
app-dev
y-webrtc-signaling
load: true
set: |
*.cache-from=type=gha,scope=cached-stage
*.cache-to=type=gha,scope=cached-stage,mode=max
- name: Start Docker services
run: |
make run
- name: Apply DRF migrations
run: |
make migrate
- name: Add dummy data
run: |
make demo FLUSH_ARGS='--no-input'
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test
- uses: actions/upload-artifact@v3
if: always()
with:
name: playwright-report
path: src/frontend/apps/e2e/report/
retention-days: 7

272
.github/workflows/impress.yml vendored Normal file
View File

@@ -0,0 +1,272 @@
name: impress Workflow
on:
push:
branches:
- main
pull_request:
branches:
- "*"
jobs:
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/impress.yml' | grep "print("
- name: Check absence of fixup commits
run: |
! git log | grep 'fixup!'
- name: Install gitlint
run: pip install --user requests gitlint
- name: Lint commit messages added to main
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
runs-on: ubuntu-latest
if: |
contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 50
- name: Check that the CHANGELOG has been modified in the current branch
run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.after }} | grep 'CHANGELOG.md'
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
if [ $max_line_length -ge 80 ]; then
echo "ERROR: CHANGELOG has lines longer than 80 characters."
exit 1
fi
build-mails:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: "18"
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
lint-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format . --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
- name: Lint code with pylint
run: ~/.local/bin/pylint .
test-back:
runs-on: ubuntu-latest
needs: build-mails
defaults:
run:
working-directory: src/backend
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: impress
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
DJANGO_CONFIGURATION: Test
DJANGO_SETTINGS_MODULE: impress.settings
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
DB_HOST: localhost
DB_NAME: impress
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
AWS_S3_ENDPOINT_URL: http://localhost:9000
AWS_S3_ACCESS_KEY_ID: impress
AWS_S3_SECRET_ACCESS_KEY: password
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Start Minio
run: |
docker pull minio/minio
docker run -d --name minio \
-p 9000:9000 \
-e "MINIO_ACCESS_KEY=impress" \
-e "MINIO_SECRET_KEY=password" \
-v /data/media:/data \
minio/minio server --console-address :9001 /data
- name: Configure MinIO
run: |
MINIO=$(docker ps | grep minio/minio | sed -E 's/.*\s+([a-zA-Z0-9_-]+)$/\1/')
docker exec ${MINIO} sh -c \
"mc alias set impress http://localhost:9000 impress password && \
mc alias ls && \
mc mb impress/impress-media-storage && \
mc version enable impress/impress-media-storage"
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Install gettext (required to compile messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages
- name: Run tests
run: ~/.local/bin/pytest -n 2
i18n-crowdin:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "infrastructure,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/impress/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
- name: Install gettext (required to make messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install development dependencies
working-directory: src/backend
run: pip install --user .[dev]
- name: Generate the translation base file
run: ~/.local/bin/django-admin makemessages --keep-pot --all
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "18.x"
cache: "yarn"
cache-dependency-path: src/frontend/yarn.lock
- name: Install dependencies
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Extract the frontend translation
run: make frontend-i18n-extract
- name: Upload files to Crowdin
run: |
docker run \
--rm \
-e CROWDIN_API_TOKEN=$CROWDIN_API_TOKEN \
-e CROWDIN_PROJECT_ID=$CROWDIN_PROJECT_ID \
-e CROWDIN_BASE_PATH=$CROWDIN_BASE_PATH \
-v "${{ github.workspace }}:/app" \
crowdin/cli:3.16.0 \
crowdin upload sources -c /app/crowdin/config.yml