I've protected this endpoint with a feature flag, and an authentication
class, as it will be exposed on the public internet.
I've tried to keep the viewset logic as minimal as possible, I've
to ship smth and will continue iterating on this piece of code.
At some point, abstracting webhook endpoint and authentication class
might be beneficial for the project. YAGNI as of today.
A recording is savable only if it's active or stopped. In other
status (error, already saved, etc.) it won't be possible. I might
iterate on this piece of code. Let's ship it.
When a new file is uploaded to a Minio Bucket, a webhook can be
configured to notify third parties about the event. Basically,
it's a POST call with a payload providing informations on the
event that just happened.
When a recording worker will stop, it will upload its data to a Minio
bucket, which will trigger the webhook.
Try to introduce the minimalest code to parse these events, discard
them whener it's relevant, and extract the recording ID, thus we
know which recording was successfully saved to the Minio bucket.
In the longer runner, it will trigger a callback.
Addressed a `DeprecationWarning` in `RecordingFactory` related to the
`_after_postgeneration` method, which will stop saving the instance after
postgeneration hooks in the next major release. To resolve this,
`skip_postgeneration_save=True` was added to `RecordingFactory.Meta` to
avoid extraneous save calls. Alternatively, if instance saving is needed,
the save call can be moved to postgeneration hooks or by overriding
`after_postgeneration`.
Avoided unnecessary manipulation of the room name to prevent issues when
starting an egress worker. Previously, hyphens were stripped from the room
name, likely inherited from the legacy setup with Jitsi in Magnify, though
the purpose of this change is unclear and might be an undesired legacy
feature.
To ensure accurate room matching during egress worker requests, this update
removes any manipulation of the room name. This approach minimizes the risk
of errors when initiating recordings and maintains the integrity of the
original room name throughout the process.
The LiveKit egress worker interactions are proxied through the backend for
security reasons. Allowing clients to directly use tokens with sufficient
grants to start recordings could lead to misuse, enabling users to spam the
egress worker API and potentially initiate a DDOS attack on the egress
service. To prevent this, only users with room-specific privileges can
initiate recordings.
We make sure only one recording at the time can be made on a room.
The requested recording mode is stored so it can be referenced later when
the recording is saved, triggering a callback action as needed.
A feature flag was also introduced for this capability; while this is a simple
approach, a more robust system for managing feature flags could be valuable
long-term. For now, KISS (Keep It Simple, Stupid) applies.
The viewset endpoints were designed to be as straightforward as possible—
let me know if anything can be improved.
Introducing a new worker service architecture. Sorry for the long commit.
This design adheres to several key principles, primarily the Single
Responsibility Principle. Dependency Injection and composition are
prioritized over inheritance, enhancing modularity and maintainability.
Interactions between the backend and external workers are encapsulated in
classes implementing a common `WorkerService` interface. I chose Protocol
over an abstract class for agility, aligning closely with static typing
without requiring inheritance. Each `WorkerService` implementation can
independently manage recordings according to its specific requirements.
This flexibility ensures that adding a new worker service, such as for
LiveKit, can be done without any change to existing components.
Configuration management is centralized in a single `WorkerServiceConfig`
class, which loads and provides all settings for different worker
implementations, keeping configurations organized and extensible. The worker
service class itself handles accessing relevant configurations as needed,
simplifying the configuration process.
A basic dictionary in Django settings acts as a factory, responsible for
instantiating the correct worker service based on the client's request mode.
This approach aligns with Django development conventions, emphasizing
simplicity. While a full factory class with a builder pattern could provide
future flexibility, YAGNI (You Aren't Gonna Need It) suggests deferring
such complexity until it’s necessary.
At the core of this design is the worker mediator, which decouples worker
service implementations from the Django ORM and manages database state
according to worker state. The mediator is purposefully limited in
responsibility, handling only what’s essential. It doesn’t instantiate
worker services directly; instead, services are injected via composition,
allowing the mediator to manage any object conforming to the `WorkerService`
interface. This setup preserves flexibility and maintains a clear
separation of responsibilities. The factory create worker services,
the mediator runs it.
(sorry for this long commit)
Write the proper ORM code to sanitize the rows in db and avoid
existing users lose access to our app.
Existing duplicate user accounts are merged, and resource accesses
are transferred.
Add test cases for email-based user matching fallback logic:
- String comparison edge cases
- Multiple users with matching email addresses
- Invalid email format handling
Fix will follow in subsequent commit.
When OIDC providers return random values in the "sub" field instead of stable
identifiers, implement email-based user matching as fallback.
Note: Current implementation needs improvement. Tests forthcoming.
Original: @sampaccoud (ff7914f) on Impress
Handle case where user is inactive.
Previously this edge case would cause unexpected behavior.
Related to previous commit that added the test coverage.
Remove redundant wrapper method that duplicates SUB validation logic.
Already validated in parent class, following the pattern established by
@sampaccoud in People and Impress modules.
Handle case where sub value is an empty string instead of None.
Previously this edge case would cause unexpected behavior.
Related to previous commit that added the test coverage.
Add failing test for corner case when sub value is an empty string.
This edge case was discovered by @sampaccoud and was previously untested.
Fix will follow in subsequent commit.
Manage Recording in the Django Admin. As of today, I have not enforced
a strict policy to avoid edit on recording rows or even creating new
data point directly from the admin. Will do in the future.
Implements routes to manage recordings within rooms, following the patterns
established in Impress. The viewset exposes targeted endpoints rather than
full CRUD operations, with recordings being created (soon) through
room-specific routes (e.g. room/123/start-recording).
The implementation draws from @sampaccoud's initial work and advices.
Review focus areas:
- Permission implementation choices
- Serializer design and structure
Credit: Initial work by @sampaccoud
The Recording model is introduced to track recording lifecycle within rooms,
while maintaining strict separation of access controls between rooms and
recordings.
Recordings follow the BaseAccess pattern (similar to Documents in Impress),
providing independent access control from room permissions. This ensures that
joining a room doesn't automatically grant access to previous recordings,
allowing for more flexible permission management.
The implementation was driven by TDD, particularly for the get_abilities
function, resulting in reduced nesting levels and improved readability.
The Recording model is deliberately kept minimal to serve as a foundation for
upcoming AI features while maintaining flexibility for future extensions.
I have avoided LiveKit-specific terminology for better abstraction.
Note: Room access control remains unchanged in this commit, pending future
refactor to use BaseAccess pattern (discussed IRL with @sampaccoud).
Renames a Django setting in the Resource base class to use resource-agnostic
terminology. While rooms are currently the only resource type, keeping base
class naming generic improves clarity.
Inspired by Joanie.
In Frontend context, env variables are only available at build time,
not runtime. This is one of the easiest way to pass frontend dynamic
configurations while running.
This commit only exposes the view already existing.
'hash' built-in function is randomly seed by Python process.
In staging or production, our backend runs over 3 pods, thus 3
Python processes. For a given identity, it was not prompting
the same hash across all pods.
Why 'hash' is randomly seed? For security reasons, there was
a vulnerability disclosure exploiting key collision. Since Python 3.2,
'hash' is by default randomly seed.
Fixed it! Thx @jonathanperret for your help.
Participants need to be room admin to interact with LiveKit server
api. Until we offer private room, with moderation, all participants
will be considered as room admin.
note: room admin doesn't offer permission to record a room. please,
refer to LiveKit documentation to learn more.
Color is stored in participant's metadata.
Using a hash of some participant's info allow to persist the color
for logged-in participant without storing the color in db.
Please feel free to tune the saturation and lightness range.
Code is inspired by a tutorial found online.
Switched to using query parameters instead of GET requests, enabling the
inclusion of additional parameters when calling the create endpoint via POST.
Simplified the access token generation process by removing redundant calls to
`with_identity` and consolidating token generation steps. This streamlines the
method flow by preparing all necessary data before generating the access token.
In this commit, we'll integrate a third-party service to track user events.
We start by using the `identify` method to track sign-ins and sign-ups.
Additionally, we use the `track` method to monitor custom events such as room
creation, access token generation, and logouts. This will provide us with
valuable data on current usage patterns.
The analytics library operates by opening a queue in a separate thread for
posting events, ensuring it remains non-blocking for the API. Let's test
this in a real-world scenario.
Add a new property 'email_anonymized' to the User model,
to allow tracking a user's email without any personal data.
In fact, we're dealing with professional data, thus it shouldn't
be subject to the GDPR, however I prefer taking extra care
when working with potentially first and last names.
Recent updates of dev/ruff and dev/pylint dependencies led
to new linting warnings.
Pylint 3.2.0 introduced a new check `possibly-used-before-assignment`,
which ensures variables are defined regardless of conditional statements.
Some if/else branches were missing defaults. These have been fixed.
Silent login attempts to re-authenticate the user without interaction,
provided they have an active session, improving UX by reducing manual auth.
It's an essential feature to really feel the SSO in La Suite.
A new query parameter, 'silent', allows the client to initiate a silent login.
In this flow, an extra parameter, 'prompt=none', is passed to the OIDC provider.
The requested flow is persisted in session data to adapt the authentication
callback behavior.
In a silent login flow, an authentication failure should not be considered as a
real failure. Instead, users should be redirected back to the originating view.
A silent login fails when user has no active session.
Why return the 'success_url'? The 'success_url' will redirect the user agent to
the 'returnTo' parameter provided when requesting authentication.
It's necessary to enable a silent login on any URL.
Minimal test coverage has been added for these two custom views to ensure
correct behavior.
The Pylint job was failing due to those TODO items. In our make lint
command sequence, Pylint runs first. If it fails, Ruff won't run,
which is quite inconvenient.
I've extracted those TODOs into an issue for further review.
Quick and dirty approach. It works, that's essential.
Frontend can pass a desired username for the user. This would
be the name displayed in the room to other participants.
Usernames don't need to be unique, but user identities do
If no username is passed, API will fall back to a default username.
Why? This serves as a security mechanism. If the API is called
incorrectly by a client, it maintains the previous behavior.
While refactoring 'Impress' to introduce features from 'Magnify',
few unnecessary changes were traced in the database migrations.
Do some clean up before releasing a first version in production.
It seems appropriate that backend owns the responsability of knowing any
information/configurations of the LiveKit server. Then, it shares those
with the frontend.
Please see my previous commit to understand why environment variables are
not appropriate for deployment in several remove environments.
As of today, the LiveKit server URL is the only configuration exposed
dynamically to the frontend. Thus, it doesn't justify adding a new route
to the API, responsible for exposing configurations (e.g. /configuration).
As the frontend needs to call the backend when it wants to initiate a new
webconference room, let's pass the server URL when retrieving the room's token.
It is relevant, to get both the room location and the keys to open the room in
the same call.
I prefered to be pragmatic, if the need appears any soon, I would refactor
these parts.
I have updated all references of "Impress" to "Meet".
Migrations were manually updated and not regenerated. Never-mind,
they all will be squashed before the first release.
I have also searched for reference to "Magnify", and replaced them
by "Meet".
While updating the backend sources, I have also fixed other parts of
the project, namely:
- Compose file
- Github documentation and CI
- Makefile commands
Introduce a utility function to issue a basic LiveKit access token with the minimal
required video grants for videoconferencing.
/!\ This function is naive, and doesn’t handle properly all cases. It’s under construction.
Testing was conducted using the LiveKit connection test tool https://livekit.io/connection-test,
which allows users to input the address of their local LiveKit server and an access token.
** Upcoming improvements? **
- Unit tests should be added.
- User display name should be their full name instead of their email address.
- Anonymous users should be allowed to provide a full name when requesting access to the room.
- Video grants should be adapted based on the room configuration and the user's role.
These improvements will be addressed in future commits.
Nevertheless, with this draft, we should be able to address various situations, including
public rooms, permanent rooms, temporary rooms, logged-in users, and anonymous users.
Introduce CRUD API endpoints for the Rooms and ResourceAccess models.
The code follows the Magnify logic, with the exception that token generation
has been removed and replaced by a TODO item with a mocked value.
Proper integration of LiveKit will be added in future commits.
With the removal of group logic, some complex query sets can be simplified.
Previously, we checked for both direct and indirect access to a room.
Indirect access meant a room was shared with a group, and the user was a
member of that group. I haven’t simplified those query set, as I preferred
isolate changes in dedicated commits.
Additionally, all previous tests are still passing, although tests related
to groups have been removed.
I picked few models from Magnify to build our MVP:
- Resource:
A generic model representing any type of resource. Though currently used only by Room,
it encapsulates a meaningful business logic as an abstract model.
- Room:
The primary object we manipulate, representing a meeting room with access
and permission controls.
- ResourceAccess
Ensures relevant users have the appropriate permissions for a given room.
** What’s different from Magnify ? **
Removed group logic; it will be added later. For now, we rely on the user model's
property to get its groups via desk.
Removed any logic or method related to Jitsi or LiveKit. These servers will be integrated
in the upcomming commits.
Focus on Room-related models to maintain a minimal and functional product (KISS principle)
until we achieve product-market fit (PMF).
Creating simple public and private, permanent and temporary rooms
is sufficient for building our MVP.
The Meeting model in Magnify, which supports recurrence, should be handled by
the collaborative calendar instead.
Adapted the unit test to use Pytest, and linted all the sources using Ruff linter.
(Migrations will be squashed before releasing the MVP)
This commit introduces a boilerplate inspired by https://github.com/numerique-gouv/impress.
The code has been cleaned to remove unnecessary Impress logic and dependencies.
Changes made:
- Removed Minio, WebRTC, and create bucket from the stack.
- Removed the Next.js frontend (it will be replaced by Vite).
- Cleaned up impress-specific backend logics.
The whole stack remains functional:
- All tests pass.
- Linter checks pass.
- Agent Connexion sources are already set-up.
Why clear out the code?
To adhere to the KISS principle, we aim to maintain a minimalist codebase. Cloning Impress
allowed us to quickly inherit its code quality tools and deployment configurations for staging,
pre-production, and production environments.
What’s broken?
- The tsclient is not functional anymore.
- Some make commands need to be fixed.
- Helm sources are outdated.
- Naming across the project sources are inconsistent (impress, visio, etc.)
- CI is not configured properly.
This list might be incomplete. Let's grind it.
