Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
/*
|
2025-06-26 05:08:57 -04:00
|
|
|
Copyright 2024-2025 New Vector Ltd.
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
|
2025-02-18 17:59:58 +00:00
|
|
|
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
|
2024-09-06 10:22:13 +02:00
|
|
|
Please see LICENSE in the repository root for full details.
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
*/
|
|
|
|
|
|
2025-06-20 12:37:25 -04:00
|
|
|
import { Subject } from "rxjs";
|
2025-08-04 16:43:08 +02:00
|
|
|
import { logger } from "matrix-js-sdk/lib/logger";
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
|
|
|
|
|
export interface Controls {
|
2025-04-29 22:12:07 +02:00
|
|
|
canEnterPip(): boolean;
|
|
|
|
|
enablePip(): void;
|
|
|
|
|
disablePip(): void;
|
2025-06-26 05:08:57 -04:00
|
|
|
|
|
|
|
|
setAvailableAudioDevices(devices: OutputDevice[]): void;
|
|
|
|
|
setAudioDevice(id: string): void;
|
|
|
|
|
onAudioDeviceSelect?: (id: string) => void;
|
|
|
|
|
onAudioPlaybackStarted?: () => void;
|
|
|
|
|
setAudioEnabled(enabled: boolean): void;
|
|
|
|
|
showNativeAudioDevicePicker?: () => void;
|
|
|
|
|
onBackButtonPressed?: () => void;
|
|
|
|
|
|
2025-05-22 18:58:18 +02:00
|
|
|
/** @deprecated use setAvailableAudioDevices instead*/
|
2025-05-15 17:20:12 +02:00
|
|
|
setAvailableOutputDevices(devices: OutputDevice[]): void;
|
2025-05-22 18:58:18 +02:00
|
|
|
/** @deprecated use setAudioDevice instead*/
|
2025-05-15 17:20:12 +02:00
|
|
|
setOutputDevice(id: string): void;
|
2025-05-22 18:58:18 +02:00
|
|
|
/** @deprecated use onAudioDeviceSelect instead*/
|
2025-04-29 22:12:07 +02:00
|
|
|
onOutputDeviceSelect?: (id: string) => void;
|
2025-05-22 18:58:18 +02:00
|
|
|
/** @deprecated use setAudioEnabled instead*/
|
2025-04-29 22:12:07 +02:00
|
|
|
setOutputEnabled(enabled: boolean): void;
|
2025-05-22 18:58:18 +02:00
|
|
|
/** @deprecated use showNativeAudioDevicePicker instead*/
|
2025-05-16 12:28:49 +02:00
|
|
|
showNativeOutputDevicePicker?: () => void;
|
2025-04-29 22:12:07 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
export interface OutputDevice {
|
|
|
|
|
id: string;
|
|
|
|
|
name: string;
|
2025-05-14 19:55:08 +02:00
|
|
|
forEarpiece?: boolean;
|
2025-05-16 15:50:19 +02:00
|
|
|
isEarpiece?: boolean;
|
|
|
|
|
isSpeaker?: boolean;
|
2025-05-16 17:06:54 +02:00
|
|
|
isExternalHeadset?: boolean;
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
}
|
|
|
|
|
|
2025-05-19 14:14:08 +02:00
|
|
|
/**
|
|
|
|
|
* If pipMode is enabled, EC will render a adapted call view layout.
|
|
|
|
|
*/
|
2024-12-17 04:01:56 +00:00
|
|
|
export const setPipEnabled$ = new Subject<boolean>();
|
2025-06-20 12:37:25 -04:00
|
|
|
|
|
|
|
|
export const availableOutputDevices$ = new Subject<OutputDevice[]>();
|
|
|
|
|
|
2025-06-25 12:14:05 +02:00
|
|
|
export const outputDevice$ = new Subject<string>();
|
2025-06-20 12:37:25 -04:00
|
|
|
|
2025-05-19 14:14:08 +02:00
|
|
|
/**
|
2025-05-21 12:51:00 +02:00
|
|
|
* This allows the os to mute the call if the user
|
2025-05-19 14:14:08 +02:00
|
|
|
* presses the volume down button when it is at the minimum volume.
|
|
|
|
|
*
|
|
|
|
|
* This should also be used to display a darkened overlay screen letting the user know that audio is muted.
|
|
|
|
|
*/
|
2025-05-22 18:58:18 +02:00
|
|
|
export const setAudioEnabled$ = new Subject<boolean>();
|
2025-06-26 05:08:57 -04:00
|
|
|
|
2025-06-10 12:35:04 +02:00
|
|
|
let playbackStartedEmitted = false;
|
|
|
|
|
export const setPlaybackStarted = (): void => {
|
|
|
|
|
if (!playbackStartedEmitted) {
|
|
|
|
|
playbackStartedEmitted = true;
|
|
|
|
|
window.controls.onAudioPlaybackStarted?.();
|
|
|
|
|
}
|
|
|
|
|
};
|
2025-06-26 05:08:57 -04:00
|
|
|
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
window.controls = {
|
|
|
|
|
canEnterPip(): boolean {
|
2024-12-17 04:01:56 +00:00
|
|
|
return setPipEnabled$.observed;
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
},
|
|
|
|
|
enablePip(): void {
|
2024-12-17 04:01:56 +00:00
|
|
|
if (!setPipEnabled$.observed) throw new Error("No call is running");
|
|
|
|
|
setPipEnabled$.next(true);
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
},
|
|
|
|
|
disablePip(): void {
|
2024-12-17 04:01:56 +00:00
|
|
|
if (!setPipEnabled$.observed) throw new Error("No call is running");
|
|
|
|
|
setPipEnabled$.next(false);
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
},
|
2025-06-26 05:08:57 -04:00
|
|
|
|
2025-05-22 18:58:18 +02:00
|
|
|
setAvailableAudioDevices(devices: OutputDevice[]): void {
|
2025-06-20 12:37:25 -04:00
|
|
|
logger.info("setAvailableAudioDevices called from native:", devices);
|
2025-05-21 12:51:00 +02:00
|
|
|
availableOutputDevices$.next(devices);
|
2025-05-15 17:20:12 +02:00
|
|
|
},
|
2025-05-22 18:58:18 +02:00
|
|
|
setAudioDevice(id: string): void {
|
2025-06-20 12:37:25 -04:00
|
|
|
logger.info("setAudioDevice called from native", id);
|
2025-05-21 12:51:00 +02:00
|
|
|
outputDevice$.next(id);
|
2025-04-29 22:12:07 +02:00
|
|
|
},
|
2025-05-22 18:58:18 +02:00
|
|
|
setAudioEnabled(enabled: boolean): void {
|
2025-06-20 12:37:25 -04:00
|
|
|
logger.info("setAudioEnabled called from native:", enabled);
|
2025-05-22 18:58:18 +02:00
|
|
|
if (!setAudioEnabled$.observed)
|
2025-05-19 14:14:08 +02:00
|
|
|
throw new Error(
|
2025-05-22 18:58:18 +02:00
|
|
|
"Output controls are disabled. No setAudioEnabled$ observer",
|
2025-05-19 14:14:08 +02:00
|
|
|
);
|
2025-05-22 18:58:18 +02:00
|
|
|
setAudioEnabled$.next(enabled);
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
// wrappers for the deprecated controls fields
|
|
|
|
|
setOutputEnabled(enabled: boolean): void {
|
|
|
|
|
this.setAudioEnabled(enabled);
|
|
|
|
|
},
|
|
|
|
|
setAvailableOutputDevices(devices: OutputDevice[]): void {
|
|
|
|
|
this.setAvailableAudioDevices(devices);
|
|
|
|
|
},
|
|
|
|
|
setOutputDevice(id: string): void {
|
|
|
|
|
this.setAudioDevice(id);
|
2025-04-29 22:12:07 +02:00
|
|
|
},
|
Add simple global controls to put the call in picture-in-picture mode (#2573)
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
2024-08-27 07:47:20 -04:00
|
|
|
};
|