Files
element-call/src/tile/SpotlightTile.tsx

379 lines
11 KiB
TypeScript
Raw Normal View History

/*
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type ComponentProps,
type FC,
type Ref,
type RefAttributes,
useCallback,
useEffect,
useRef,
useState,
} from "react";
import {
ExpandIcon,
CollapseIcon,
ChevronLeftIcon,
ChevronRightIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons";
import { animated } from "@react-spring/web";
import { type Observable, map } from "rxjs";
import { useObservableRef } from "observable-hooks";
import { useTranslation } from "react-i18next";
import classNames from "classnames";
import { type TrackReferenceOrPlaceholder } from "@livekit/components-core";
2025-03-13 13:58:43 +01:00
import { type RoomMember } from "matrix-js-sdk";
import FullScreenMaximiseIcon from "../icons/FullScreenMaximise.svg?react";
import FullScreenMinimiseIcon from "../icons/FullScreenMinimise.svg?react";
import { MediaView } from "./MediaView";
import styles from "./SpotlightTile.module.css";
import {
type EncryptionStatus,
LocalUserMediaViewModel,
type MediaViewModel,
2024-05-28 13:57:23 -04:00
ScreenShareViewModel,
type UserMediaViewModel,
} from "../state/MediaViewModel";
import { useInitial } from "../useInitial";
import { useMergedRefs } from "../useMergedRefs";
import { useReactiveState } from "../useReactiveState";
import { useLatest } from "../useLatest";
import { type SpotlightTileViewModel } from "../state/TileViewModel";
import { useBehavior } from "../useBehavior";
2024-05-28 13:57:23 -04:00
interface SpotlightItemBaseProps {
ref?: Ref<HTMLDivElement>;
2024-05-28 13:57:23 -04:00
className?: string;
"data-id": string;
targetWidth: number;
targetHeight: number;
Make video tiles be based on MatrixRTC member not LiveKit participants (#2701) * make tiles based on rtc member * display missing lk participant + fix tile multiplier * add show_non_member_participants config option * per member tiles * merge fixes * linter * linter and tests * tests * adapt tests (wip) * Remove unused keys * Fix optionality of nonMemberItemCount * video is optional * Mock RTC members * Lint * Merge fixes * Fix user id * Add explicit types for public fields * isRTCParticipantAvailable => isLiveKitParticipantAvailable * isLiveKitParticipantAvailable * Readonly * More keys removal * Make local field based on view model class not observable * Wording * Fix RTC members in tes * Tests again * Lint * Disable showing non-member tiles by default * Duplicate screen sharing tiles like we used to * Lint * Revert function reordering * Remove throttleTime from bad merge * Cleanup * Tidy config of show non-member settings * tidy up handling of local rtc member in tests * tidy up test init * Fix mocks * Cleanup * Apply local override where participant not yet known * Handle no visible media id * Assertions for one-on-one view * Remove isLiveKitParticipantAvailable and show via encryption status * Handle no local media (yet) * Remove unused effect for setting * Tidy settings * Avoid case of one-to-one layout with missing local or remote * Iterate * Remove option to show non-member tiles to simplify code review * Remove unused code * Remove more remnants of show-non-member-tiles * iterate * back * Fix unit test * Refactor * Expose TestScheduler as global * Fix incorrect type assertion * Simplify speaking observer * Fix * Whitespace * Make it clear that we are mocking MatrixRTC memberships * Test case for only showing tiles for MatrixRTC session members * Simplify diff * Simplify diff These changes are in https://github.com/element-hq/element-call/pull/2809 * . * Whitespaces * Use asObservable when exposing subject * Show "waiting for media..." when no participant * Additional test case * Don't show "waiting for media..." in case of local participant * Make the loading state more subtle - instead of a label we show a animated gradient * Use correct key for matrix rtc foci in code comment. (#2838) * Update src/tile/SpotlightTile.tsx Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Update src/state/CallViewModel.ts Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Make the purpose of BaseMediaViewModel.local explicit * Use named object instead of unnamed array for spotlightAndPip * Refactor spotlightAndPip into spotlight and pip * Use if statement instead of ternary for readability in spotlight and pip logic * Review feedback * Fix tests for CallEventAudioRenderer * Lint * Revert "Make the loading state more subtle" This reverts commit 765f7b4f319b86839fcb4fde28d1e0604e542577. * Update src/state/CallViewModel.ts Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Fix spelling * Remove a non-null assertion that failed at runtime --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io> Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
2024-12-06 12:28:37 +01:00
video: TrackReferenceOrPlaceholder | undefined;
videoEnabled: boolean;
member: RoomMember;
2024-05-28 13:57:23 -04:00
unencryptedWarning: boolean;
encryptionStatus: EncryptionStatus;
2024-05-28 13:57:23 -04:00
displayName: string;
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable: boolean;
"aria-hidden"?: boolean;
Make video tiles be based on MatrixRTC member not LiveKit participants (#2701) * make tiles based on rtc member * display missing lk participant + fix tile multiplier * add show_non_member_participants config option * per member tiles * merge fixes * linter * linter and tests * tests * adapt tests (wip) * Remove unused keys * Fix optionality of nonMemberItemCount * video is optional * Mock RTC members * Lint * Merge fixes * Fix user id * Add explicit types for public fields * isRTCParticipantAvailable => isLiveKitParticipantAvailable * isLiveKitParticipantAvailable * Readonly * More keys removal * Make local field based on view model class not observable * Wording * Fix RTC members in tes * Tests again * Lint * Disable showing non-member tiles by default * Duplicate screen sharing tiles like we used to * Lint * Revert function reordering * Remove throttleTime from bad merge * Cleanup * Tidy config of show non-member settings * tidy up handling of local rtc member in tests * tidy up test init * Fix mocks * Cleanup * Apply local override where participant not yet known * Handle no visible media id * Assertions for one-on-one view * Remove isLiveKitParticipantAvailable and show via encryption status * Handle no local media (yet) * Remove unused effect for setting * Tidy settings * Avoid case of one-to-one layout with missing local or remote * Iterate * Remove option to show non-member tiles to simplify code review * Remove unused code * Remove more remnants of show-non-member-tiles * iterate * back * Fix unit test * Refactor * Expose TestScheduler as global * Fix incorrect type assertion * Simplify speaking observer * Fix * Whitespace * Make it clear that we are mocking MatrixRTC memberships * Test case for only showing tiles for MatrixRTC session members * Simplify diff * Simplify diff These changes are in https://github.com/element-hq/element-call/pull/2809 * . * Whitespaces * Use asObservable when exposing subject * Show "waiting for media..." when no participant * Additional test case * Don't show "waiting for media..." in case of local participant * Make the loading state more subtle - instead of a label we show a animated gradient * Use correct key for matrix rtc foci in code comment. (#2838) * Update src/tile/SpotlightTile.tsx Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Update src/state/CallViewModel.ts Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Make the purpose of BaseMediaViewModel.local explicit * Use named object instead of unnamed array for spotlightAndPip * Refactor spotlightAndPip into spotlight and pip * Use if statement instead of ternary for readability in spotlight and pip logic * Review feedback * Fix tests for CallEventAudioRenderer * Lint * Revert "Make the loading state more subtle" This reverts commit 765f7b4f319b86839fcb4fde28d1e0604e542577. * Update src/state/CallViewModel.ts Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> * Fix spelling * Remove a non-null assertion that failed at runtime --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io> Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
2024-12-06 12:28:37 +01:00
localParticipant: boolean;
2024-05-28 13:57:23 -04:00
}
interface SpotlightUserMediaItemBaseProps extends SpotlightItemBaseProps {
videoFit: "contain" | "cover";
}
interface SpotlightLocalUserMediaItemProps
extends SpotlightUserMediaItemBaseProps {
vm: LocalUserMediaViewModel;
}
const SpotlightLocalUserMediaItem: FC<SpotlightLocalUserMediaItemProps> = ({
vm,
...props
}) => {
const mirror = useBehavior(vm.mirror$);
return <MediaView mirror={mirror} {...props} />;
};
2024-05-28 13:57:23 -04:00
SpotlightLocalUserMediaItem.displayName = "SpotlightLocalUserMediaItem";
interface SpotlightUserMediaItemProps extends SpotlightItemBaseProps {
vm: UserMediaViewModel;
}
const SpotlightUserMediaItem: FC<SpotlightUserMediaItemProps> = ({
vm,
...props
}) => {
const cropVideo = useBehavior(vm.cropVideo$);
2024-05-28 13:57:23 -04:00
Keep tiles in a stable order (#2670) * Keep tiles in a stable order This introduces a new layer of abstraction on top of MediaViewModel: TileViewModel, which gives us a place to store data relating to tiles rather than their media, and also generally makes it easier to reason about tiles as they move about the call layout. I have created a class called TileStore to keep track of these tiles. This allows us to swap out the media shown on a tile as the spotlight speaker changes, and avoid moving tiles around unless they really need to jump between the visible/invisible regions of the layout. * Don't throttle spotlight updates Since we now assume that the spotlight and grid will be in sync (i.e. an active speaker in one will behave as an active speaker in the other), we don't want the spotlight to ever lag behind due to throttling. If this causes usability issues we should maybe look into making LiveKit's 'speaking' indicators less erratic first. * Make layout shifts due to a change in speaker less surprising Although we try now to avoid layout shifts due to the spotlight speaker changing wherever possible, a spotlight speaker coming from off screen can still trigger one. Let's shift the layout a bit more gracefully in this case. * Improve the tile ordering tests * Maximize the spotlight tile in portrait layout * Tell tiles whether they're actually visible in a more timely manner * Fix test * Fix speaking indicators logic * Improve readability of marbles * Fix test case --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-11-06 04:36:48 -05:00
const baseProps: SpotlightUserMediaItemBaseProps &
RefAttributes<HTMLDivElement> = {
2024-05-28 13:57:23 -04:00
videoFit: cropVideo ? "cover" : "contain",
...props,
};
return vm instanceof LocalUserMediaViewModel ? (
Keep tiles in a stable order (#2670) * Keep tiles in a stable order This introduces a new layer of abstraction on top of MediaViewModel: TileViewModel, which gives us a place to store data relating to tiles rather than their media, and also generally makes it easier to reason about tiles as they move about the call layout. I have created a class called TileStore to keep track of these tiles. This allows us to swap out the media shown on a tile as the spotlight speaker changes, and avoid moving tiles around unless they really need to jump between the visible/invisible regions of the layout. * Don't throttle spotlight updates Since we now assume that the spotlight and grid will be in sync (i.e. an active speaker in one will behave as an active speaker in the other), we don't want the spotlight to ever lag behind due to throttling. If this causes usability issues we should maybe look into making LiveKit's 'speaking' indicators less erratic first. * Make layout shifts due to a change in speaker less surprising Although we try now to avoid layout shifts due to the spotlight speaker changing wherever possible, a spotlight speaker coming from off screen can still trigger one. Let's shift the layout a bit more gracefully in this case. * Improve the tile ordering tests * Maximize the spotlight tile in portrait layout * Tell tiles whether they're actually visible in a more timely manner * Fix test * Fix speaking indicators logic * Improve readability of marbles * Fix test case --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-11-06 04:36:48 -05:00
<SpotlightLocalUserMediaItem vm={vm} {...baseProps} />
2024-05-28 13:57:23 -04:00
) : (
<MediaView mirror={false} {...baseProps} />
);
};
2024-05-28 13:57:23 -04:00
SpotlightUserMediaItem.displayName = "SpotlightUserMediaItem";
interface SpotlightItemProps {
ref?: Ref<HTMLDivElement>;
vm: MediaViewModel;
targetWidth: number;
targetHeight: number;
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable: boolean;
intersectionObserver$: Observable<IntersectionObserver>;
/**
* Whether this item should act as a scroll snapping point.
*/
snap: boolean;
"aria-hidden"?: boolean;
}
const SpotlightItem: FC<SpotlightItemProps> = ({
ref: theirRef,
vm,
targetWidth,
targetHeight,
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable,
intersectionObserver$,
snap,
"aria-hidden": ariaHidden,
}) => {
const ourRef = useRef<HTMLDivElement | null>(null);
const ref = useMergedRefs(ourRef, theirRef);
const displayName = useBehavior(vm.displayName$);
const video = useBehavior(vm.video$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const unencryptedWarning = useBehavior(vm.unencryptedWarning$);
const encryptionStatus = useBehavior(vm.encryptionStatus$);
// Hook this item up to the intersection observer
useEffect(() => {
const element = ourRef.current!;
let prevIo: IntersectionObserver | null = null;
const subscription = intersectionObserver$.subscribe((io) => {
prevIo?.unobserve(element);
io.observe(element);
prevIo = io;
});
return (): void => {
subscription.unsubscribe();
prevIo?.unobserve(element);
2024-05-28 13:57:23 -04:00
};
}, [intersectionObserver$]);
2024-05-28 13:57:23 -04:00
const baseProps: SpotlightItemBaseProps & RefAttributes<HTMLDivElement> = {
ref,
"data-id": vm.id,
className: classNames(styles.item, { [styles.snap]: snap }),
targetWidth,
targetHeight,
video,
videoEnabled,
member: vm.member,
unencryptedWarning,
displayName,
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable,
encryptionStatus,
"aria-hidden": ariaHidden,
localParticipant: vm.local,
};
return vm instanceof ScreenShareViewModel ? (
<MediaView videoFit="contain" mirror={false} {...baseProps} />
) : (
<SpotlightUserMediaItem vm={vm} {...baseProps} />
);
};
SpotlightItem.displayName = "SpotlightItem";
interface Props {
ref?: Ref<HTMLDivElement>;
Keep tiles in a stable order (#2670) * Keep tiles in a stable order This introduces a new layer of abstraction on top of MediaViewModel: TileViewModel, which gives us a place to store data relating to tiles rather than their media, and also generally makes it easier to reason about tiles as they move about the call layout. I have created a class called TileStore to keep track of these tiles. This allows us to swap out the media shown on a tile as the spotlight speaker changes, and avoid moving tiles around unless they really need to jump between the visible/invisible regions of the layout. * Don't throttle spotlight updates Since we now assume that the spotlight and grid will be in sync (i.e. an active speaker in one will behave as an active speaker in the other), we don't want the spotlight to ever lag behind due to throttling. If this causes usability issues we should maybe look into making LiveKit's 'speaking' indicators less erratic first. * Make layout shifts due to a change in speaker less surprising Although we try now to avoid layout shifts due to the spotlight speaker changing wherever possible, a spotlight speaker coming from off screen can still trigger one. Let's shift the layout a bit more gracefully in this case. * Improve the tile ordering tests * Maximize the spotlight tile in portrait layout * Tell tiles whether they're actually visible in a more timely manner * Fix test * Fix speaking indicators logic * Improve readability of marbles * Fix test case --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-11-06 04:36:48 -05:00
vm: SpotlightTileViewModel;
expanded: boolean;
onToggleExpanded: (() => void) | null;
targetWidth: number;
targetHeight: number;
showIndicators: boolean;
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable: boolean;
className?: string;
style?: ComponentProps<typeof animated.div>["style"];
}
export const SpotlightTile: FC<Props> = ({
ref: theirRef,
vm,
expanded,
onToggleExpanded,
targetWidth,
targetHeight,
showIndicators,
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable = true,
className,
style,
}) => {
const { t } = useTranslation();
const [ourRef, root$] = useObservableRef<HTMLDivElement | null>(null);
const ref = useMergedRefs(ourRef, theirRef);
const maximised = useBehavior(vm.maximised$);
const media = useBehavior(vm.media$);
const [visibleId, setVisibleId] = useState<string | undefined>(media[0]?.id);
const latestMedia = useLatest(media);
const latestVisibleId = useLatest(visibleId);
const visibleIndex = media.findIndex((vm) => vm.id === visibleId);
const canGoBack = visibleIndex > 0;
const canGoToNext = visibleIndex !== -1 && visibleIndex < media.length - 1;
const isFullscreen = useCallback((): boolean => {
const rootElement = document.body;
if (rootElement && document.fullscreenElement) return true;
return false;
}, []);
const FullScreenIcon = isFullscreen()
? FullScreenMinimiseIcon
: FullScreenMaximiseIcon;
const onToggleFullscreen = useCallback(() => {
const rootElement = document.body;
if (!rootElement) return;
if (isFullscreen()) {
void document?.exitFullscreen();
} else {
void rootElement.requestFullscreen();
}
}, [isFullscreen]);
// To keep track of which item is visible, we need an intersection observer
// hooked up to the root element and the items. Because the items will run
// their effects before their parent does, we need to do this dance with an
// Observable to actually give them the intersection observer.
const intersectionObserver$ = useInitial<Observable<IntersectionObserver>>(
() =>
root$.pipe(
map(
(r) =>
new IntersectionObserver(
(entries) => {
const visible = entries.find((e) => e.isIntersecting);
if (visible !== undefined)
setVisibleId(visible.target.getAttribute("data-id")!);
},
{ root: r, threshold: 0.5 },
),
),
),
);
const [scrollToId, setScrollToId] = useReactiveState<string | null>(
(prev) =>
prev == null || prev === visibleId || media.every((vm) => vm.id !== prev)
? null
: prev,
[visibleId],
);
const onBackClick = useCallback(() => {
const media = latestMedia.current;
const visibleIndex = media.findIndex(
(vm) => vm.id === latestVisibleId.current,
);
if (visibleIndex > 0) setScrollToId(media[visibleIndex - 1].id);
}, [latestVisibleId, latestMedia, setScrollToId]);
const onNextClick = useCallback(() => {
const media = latestMedia.current;
const visibleIndex = media.findIndex(
(vm) => vm.id === latestVisibleId.current,
);
if (visibleIndex !== -1 && visibleIndex !== media.length - 1)
setScrollToId(media[visibleIndex + 1].id);
}, [latestVisibleId, latestMedia, setScrollToId]);
const ToggleExpandIcon = expanded ? CollapseIcon : ExpandIcon;
return (
<animated.div
ref={ref}
className={classNames(className, styles.tile, {
[styles.maximised]: maximised,
})}
style={style}
>
{canGoBack && (
<button
className={classNames(styles.advance, styles.back)}
aria-label={t("common.back")}
onClick={onBackClick}
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
tabIndex={focusable ? undefined : -1}
>
<ChevronLeftIcon aria-hidden width={24} height={24} />
</button>
)}
<div className={styles.contents}>
{media.map((vm) => (
<SpotlightItem
key={vm.id}
vm={vm}
targetWidth={targetWidth}
targetHeight={targetHeight}
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
focusable={focusable}
intersectionObserver$={intersectionObserver$}
// This is how we get the container to scroll to the right media
// when the previous/next buttons are clicked: we temporarily
// remove all scroll snap points except for just the one media
// that we want to bring into view
snap={scrollToId === null || scrollToId === vm.id}
aria-hidden={(scrollToId ?? visibleId) !== vm.id}
/>
))}
</div>
<div className={styles.bottomRightButtons}>
<button
className={classNames(styles.expand)}
aria-label={"maximise"}
onClick={onToggleFullscreen}
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
tabIndex={focusable ? undefined : -1}
>
<FullScreenIcon aria-hidden width={20} height={20} />
</button>
{onToggleExpanded && (
<button
className={classNames(styles.expand)}
aria-label={
expanded ? t("video_tile.collapse") : t("video_tile.expand")
}
onClick={onToggleExpanded}
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
tabIndex={focusable ? undefined : -1}
>
<ToggleExpandIcon aria-hidden width={20} height={20} />
</button>
)}
</div>
{canGoToNext && (
<button
className={classNames(styles.advance, styles.next)}
aria-label={t("common.next")}
onClick={onNextClick}
Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486) * Fix the interactivity of buttons while reconnecting or in earpiece mode When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles. * Write a Playwright test for reconnecting * fix lints Signed-off-by: Timo K <toger5@hotmail.de> * fix test Signed-off-by: Timo K <toger5@hotmail.de> * enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU * remove rate limit for delayed events * more time to connect to livekit SFU * Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button * adapt to most recent Element Web version * Use the "End call" button as proofe for a started call * Currrenty disabled due to recent Element Web - not indicating the number of participants - bypassing Lobby * linting * disable 'can only interact with header and footer while reconnecting' for firefox --------- Signed-off-by: Timo K <toger5@hotmail.de> Co-authored-by: Timo <16718859+toger5@users.noreply.github.com> Co-authored-by: Timo K <toger5@hotmail.de> Co-authored-by: fkwp <github-fkwp@w4ve.de>
2025-09-18 12:58:47 +02:00
tabIndex={focusable ? undefined : -1}
>
<ChevronRightIcon aria-hidden width={24} height={24} />
</button>
)}
{!expanded && (
<div
className={classNames(styles.indicators, {
[styles.show]: showIndicators && media.length > 1,
})}
>
Keep tiles in a stable order (#2670) * Keep tiles in a stable order This introduces a new layer of abstraction on top of MediaViewModel: TileViewModel, which gives us a place to store data relating to tiles rather than their media, and also generally makes it easier to reason about tiles as they move about the call layout. I have created a class called TileStore to keep track of these tiles. This allows us to swap out the media shown on a tile as the spotlight speaker changes, and avoid moving tiles around unless they really need to jump between the visible/invisible regions of the layout. * Don't throttle spotlight updates Since we now assume that the spotlight and grid will be in sync (i.e. an active speaker in one will behave as an active speaker in the other), we don't want the spotlight to ever lag behind due to throttling. If this causes usability issues we should maybe look into making LiveKit's 'speaking' indicators less erratic first. * Make layout shifts due to a change in speaker less surprising Although we try now to avoid layout shifts due to the spotlight speaker changing wherever possible, a spotlight speaker coming from off screen can still trigger one. Let's shift the layout a bit more gracefully in this case. * Improve the tile ordering tests * Maximize the spotlight tile in portrait layout * Tell tiles whether they're actually visible in a more timely manner * Fix test * Fix speaking indicators logic * Improve readability of marbles * Fix test case --------- Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-11-06 04:36:48 -05:00
{media.map((vm) => (
<div
key={vm.id}
className={styles.item}
data-visible={vm.id === visibleId}
/>
))}
</div>
)}
</animated.div>
);
};
SpotlightTile.displayName = "SpotlightTile";