Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
# Global JS controls
2025-04-29 22:12:07 +02:00
A few aspects of Element Call's interface can be controlled through a global API on the `window` .
## Picture-in-picture
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
- `controls.canEnterPip(): boolean` Determines whether it's possible to enter picture-in-picture mode.
- `controls.enablePip(): void` Puts the call interface into picture-in-picture mode. Throws if not in a call.
- `controls.disablePip(): void` Takes the call interface out of picture-in-picture mode, restoring it to its natural display mode. Throws if not in a call.
2025-04-29 22:12:07 +02:00
2025-05-22 19:10:09 +02:00
## Audio devices
2025-04-29 22:12:07 +02:00
2025-05-22 18:58:18 +02:00
On mobile platforms (iOS, Android), web views do not reliably support selecting audio output devices such as the main speaker, earpiece, or headset. To address this limitation, the following functions allow the hosting application (e.g., Element Web, Element X) to manage audio devices via exposed JavaScript interfaces. These functions must be enabled using the URL parameter `controlledAudioDevices` to take effect.
2025-04-29 22:12:07 +02:00
2025-05-22 18:58:18 +02:00
- `controls.setAvailableAudioDevices(devices: { id: string, name: string, forEarpiece?: boolean, isEarpiece?: boolean isSpeaker?: boolean, isExternalHeadset?, boolean; }[]): void` Sets the list of available audio outputs. `forEarpiece` is used on iOS only.
2025-05-16 11:32:32 +02:00
It flags the device that should be used if the user selects earpiece mode. This should be the main stereo loudspeaker of the device.
2025-05-22 18:58:18 +02:00
- `controls.onAudioDeviceSelect: ((id: string) => void) | undefined` Callback called whenever the user or application selects a new audio output.
- `controls.setAudioDevice(id: string): void` Sets the selected audio device in Element Call's menu. This should be used if the OS decides to automatically switch to Bluetooth, for example.
2025-05-22 19:07:47 +02:00
- `controls.setAudioEnabled(enabled: boolean)` Enables/disables all audio output from the application. Output is enabled by default.
2025-06-10 12:35:04 +02:00
- `controls.onAudioPlaybackStarted: ((id: string) => void) | undefined` : This will be called the first time we start
playing audio in the webview. It can be helpful to do device setup on the native app when the webviews audio is ready.
In particular android is using it to setup the output channel so that the call volume can
be controlled by the hardware volume rocker.
2025-06-24 10:46:04 +02:00
## Element Call button delegation
Callbacks for buttons in EC that are handled by the native application
- `showNativeAudioDevicePicker: (() => void) | undefined` . Callback called whenever the user presses the output button in the settings menu.
This button is only shown on iOS. (`userAgent.includes("iPhone")` )
- `onBackButtonPressed: (() => void) | undefined` . Callback when the webview detects a tab on the header's back button.
