Timo and I agreed previously that we should ditch the class pattern for view models and instead have them be interfaces which are simply created by functions. They're more straightforward to write, mock, and instantiate this way.
The code for media view models and media items is pretty much the last remaining instance of the class pattern. Since I was about to introduce a new media view model for ringing, I wanted to get this refactor out of the way first rather than add to the technical debt.
This refactor also makes things a little easier for https://github.com/element-hq/element-call/pull/3747 by extracting volume controls into their own module.
On second glance, the way that we determined a media tile to be 'waiting for media' was too implicit for my taste. It would appear on a surface reading to depend on whether a participant was currently publishing any video. But in reality, the 'video' object was always defined as long as a LiveKit participant existed, so in reality it depended on just the participant. We should show this relationship more explicitly by moving the computation into the view model, where it can depend on the participant directly.
* Fix the interactivity of buttons while reconnecting or in earpiece mode
When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles.
* Write a Playwright test for reconnecting
* fix lints
Signed-off-by: Timo K <toger5@hotmail.de>
* fix test
Signed-off-by: Timo K <toger5@hotmail.de>
* enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU
* remove rate limit for delayed events
* more time to connect to livekit SFU
* Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button
* adapt to most recent Element Web version
* Use the "End call" button as proofe for a started call
* Currrenty disabled due to recent Element Web
- not indicating the number of participants
- bypassing Lobby
* linting
* disable 'can only interact with header and footer while reconnecting' for firefox
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Timo K <toger5@hotmail.de>
Co-authored-by: fkwp <github-fkwp@w4ve.de>
* Add a fullscreen button that uses the element request Fullscreen browser api
Signed-off-by: Timo K <toger5@hotmail.de>
* use body instead of root node
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
This hook is simpler in its implementation (therefore hopefully more correct & performant) and enforces a type-level distinction between raw Observables and Behaviors.
* Disambigute displaynames
* Add test
* fixup test functions
* prettier
* lint
* Split displayname utils into own file and add tests.
* Split out fixtures
* Add more testcases for displayname calculation.
* lint
* Also listen for displayname changes. (I stand corrected!)
* fix missing media tiles on missing member
* Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
Revert "Enable @typescript-eslint/consistent-type-imports lint rule"
This reverts commit ba385fa00b7e410cc508fd5fb9fe972233ae114f.
Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
.
* Format
* make tiles based on rtc member
* display missing lk participant + fix tile multiplier
* add show_non_member_participants config option
* per member tiles
* merge fixes
* linter
* linter and tests
* tests
* adapt tests (wip)
* Remove unused keys
* Fix optionality of nonMemberItemCount
* video is optional
* Mock RTC members
* Lint
* Merge fixes
* Fix user id
* Add explicit types for public fields
* isRTCParticipantAvailable => isLiveKitParticipantAvailable
* isLiveKitParticipantAvailable
* Readonly
* More keys removal
* Make local field based on view model class not observable
* Wording
* Fix RTC members in tes
* Tests again
* Lint
* Disable showing non-member tiles by default
* Duplicate screen sharing tiles like we used to
* Lint
* Revert function reordering
* Remove throttleTime from bad merge
* Cleanup
* Tidy config of show non-member settings
* tidy up handling of local rtc member in tests
* tidy up test init
* Fix mocks
* Cleanup
* Apply local override where participant not yet known
* Handle no visible media id
* Assertions for one-on-one view
* Remove isLiveKitParticipantAvailable and show via encryption status
* Handle no local media (yet)
* Remove unused effect for setting
* Tidy settings
* Avoid case of one-to-one layout with missing local or remote
* Iterate
* Remove option to show non-member tiles to simplify code review
* Remove unused code
* Remove more remnants of show-non-member-tiles
* iterate
* back
* Fix unit test
* Refactor
* Expose TestScheduler as global
* Fix incorrect type assertion
* Simplify speaking observer
* Fix
* Whitespace
* Make it clear that we are mocking MatrixRTC memberships
* Test case for only showing tiles for MatrixRTC session members
* Simplify diff
* Simplify diff
These changes are in https://github.com/element-hq/element-call/pull/2809
* .
* Whitespaces
* Use asObservable when exposing subject
* Show "waiting for media..." when no participant
* Additional test case
* Don't show "waiting for media..." in case of local participant
* Make the loading state more subtle
- instead of a label we show a animated gradient
* Use correct key for matrix rtc foci in code comment. (#2838)
* Update src/tile/SpotlightTile.tsx
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Make the purpose of BaseMediaViewModel.local explicit
* Use named object instead of unnamed array for spotlightAndPip
* Refactor spotlightAndPip into spotlight and pip
* Use if statement instead of ternary for readability in spotlight and pip logic
* Review feedback
* Fix tests for CallEventAudioRenderer
* Lint
* Revert "Make the loading state more subtle"
This reverts commit 765f7b4f319b86839fcb4fde28d1e0604e542577.
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Fix spelling
* Remove a non-null assertion that failed at runtime
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
* Refactor to make encryption system available in view models
* WIP show encryption errors from LiveKit
* Missing CSS
* Show encryption status based on LK and RTC
* Lint
* Lint
* Fix tests
* Update wording
* Refactor
* Lint
The observable-hooks package provides hooks that do exactly the same thing as these custom React hooks I had written a while back. (even the names are the same, wow)
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
* Keep tiles in a stable order
This introduces a new layer of abstraction on top of MediaViewModel: TileViewModel, which gives us a place to store data relating to tiles rather than their media, and also generally makes it easier to reason about tiles as they move about the call layout. I have created a class called TileStore to keep track of these tiles.
This allows us to swap out the media shown on a tile as the spotlight speaker changes, and avoid moving tiles around unless they really need to jump between the visible/invisible regions of the layout.
* Don't throttle spotlight updates
Since we now assume that the spotlight and grid will be in sync (i.e. an active speaker in one will behave as an active speaker in the other), we don't want the spotlight to ever lag behind due to throttling. If this causes usability issues we should maybe look into making LiveKit's 'speaking' indicators less erratic first.
* Make layout shifts due to a change in speaker less surprising
Although we try now to avoid layout shifts due to the spotlight speaker changing wherever possible, a spotlight speaker coming from off screen can still trigger one. Let's shift the layout a bit more gracefully in this case.
* Improve the tile ordering tests
* Maximize the spotlight tile in portrait layout
* Tell tiles whether they're actually visible in a more timely manner
* Fix test
* Fix speaking indicators logic
* Improve readability of marbles
* Fix test case
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Catching two accessibility issues along the way: we were putting the wrong accessible labels on the 'expand' button, and even the off-screen pages of the spotlight tile were being exposed to accessibility technologies rather than hidden.
We need to be consistent about whether we import matrix-js-sdk from `src` or
`lib`, otherwise we get two copies of matrix-js-sdk, and everything explodes.
The buttons were scrolling with the view instead of always being visible in a fixed location on the tile, and the indicators were not adopting the correct width.
The Compound design tokens package is now set up to generate React components for every icon, so we no longer need to use our more error-prone method of importing the SVGs.
Includes the mobile UX optimizations and the tweaks we've made to cut down on wasted space, but does not yet include the change to embed the spotlight tile within the grid.
react-rxjs is the library we've been using to connect our React components to view models and consume observables. However, after spending some time with react-rxjs, I feel that it's a very heavy-handed solution. It requires us to sprinkle <Subscribe /> and <RemoveSubscribe /> components all throughout the code, and makes React go through an extra render cycle whenever we mount a component that binds to a view model. What I really want is a lightweight React hook that just gets the current value out of a plain observable, without any extra setup. Luckily the observable-hooks library with its useObservableEagerState hook seems to do just that—and it's more actively maintained, too!
Here I've implemented an MVP for the new unified grid layout, which scales smoothly up to arbitrarily many participants. It doesn't yet have a special 1:1 layout, so in spotlight mode and 1:1s, we will still fall back to the legacy grid systems.
Things that happened along the way:
- The part of VideoTile that is common to both spotlight and grid tiles, I refactored into MediaView
- VideoTile renamed to GridTile
- Added SpotlightTile for the new, glassy spotlight designs
- NewVideoGrid renamed to Grid, and refactored to be even more generic
- I extracted the media name logic into a custom React hook
- Deleted the BigGrid experiment
