Multi-Stream and Auto-framing with Multi-focus framing features

Last updated July 22, 2022

1. Introducing ‘Multi-Stream’

In Neat software release version 20210817, the Neat team brings a new experience called ‘Multi-Stream’ in partnership with Zoom. Multi-Stream takes the Zoom meeting experience to the next level, while at the same time allowing our customers to continue to use the Neat’s own Auto-framing/Neat Symmetry features.

In this article, we aim to explain the changes that occurred in our 20210817 software release and how you can configure/use these features.

IMPORTANT: Multi-Stream is only available in licensed Zoom Rooms. Zoom for Home users will have the option to use Manual, Auto-framing and Multi-focus framing features.

2. Neat’s Auto-framing Settings have moved to Zoom portal

In addition to the Multi-Stream features, our 20210817 release also allows administrators to define the default camera position for one or more Neat rooms via Zoom’s administrative portal. This makes it much easier to manage all the Neat rooms centrally, without having to go into individual meeting rooms to set the default behavior.

To facilitate this (and also to avoid confusing our users), we have removed the Auto-framing features from Neat’s System Settings and moved these to Zoom’s portal (Zoom portal -> Room profile -> Default Camera Control Mode). If you now go to Neat’s System Settings -> Audio & Video -> you will see a message that will advise you to configure auto framing options from the Zoom portal (see Figure 1).

IMPORTANT: If you have been using Neat’s Auto-framing (group or individual) features until now, once you upgrade, you may find these features no longer work (as the options have now moved to Zoom). You will need to enable these features for your Neat room by going into the Zoom portal -> Room profile -> Default Camera Control Mode and settings either ‘Auto-framing’ (group auto framing) or ‘Multi-focus framing’ (individual framing/Neat symmetry).

Note: Framing Boundary feature will still be part of Neat’s System Settings and not part of Zoom. (See section 5 of this article for more information).

3. Zoom Room Profile: Default Camera Control Mode

As explained in the previous section, the configuration for Auto Framing has moved to the Zoom portal. Please login as the Zoom administrator and navigate to Zoom portal -> Room profile -> Default Camera Control Mode and you can configure the default behavior of your Neat Device when it first joins the meeting.

By default, the Default Camera Control Mode on Zoom portal is set to ‘Manual’ (see Figure 2). This simply means that the camera will be in a fixed position and join a meeting without any Auto framing options enabled. While in the meeting, you can bring up the local camera control menu and choose different options – but it will not persists over the next call. If you want your Neat Devices to join meetings with Auto framing enabled by default, you’ll need to change the Default Camera Control Mode on the Zoom portal to any of the following options:

• Auto-Framing–Auto-frame a group of people at once: this setting will allow you to have the same experience as Neat’s ‘Auto framing: Group’ feature. See section 3.1 for more information.
• Auto-Framing–Auto-frame a group of people at once -> Enable Multi-Focus Framing: this setting will allow you to have the same experience as Neat’s ‘Auto framing: Individual’ feature, aka ‘Neat Symmetry’. See section 3.2 for more information.
• Multi-Stream: This setting allows you to enable the ‘Multi-Stream’ feature. See section 3.3 for more information.

Finally, it is also possible to choose a different layout while in a Zoom meeting, using the Camera Control button. In this release, the Camera Control feature on your Neat Pad (or on the Neat Board) has been updated to reflect these new options on Zoom portal. This is explained in section 4 of this article.

Note: Speaker Switching is not available for Neat devices.

3.1 Auto-Framing: Auto-frame a group of people at once

The ‘Auto-framing’ feature on Zoom is the same behavior as Neat’s Group Framing.

When this option is chosen, the camera will automatically focus on the meeting room participants and group them together. If participants move around or leave/enter the meeting room, the framing will be adjusted to accommodate those changes (see Figures 3 and 4).

3.2 Auto-Framing: Multi-focus framing

The ‘Multi-focus framing’ on Zoom is the same feature as Neat Symmetry, also known as, Auto framing: Individual (originally introduced in Neat’s February 2021 release).

When Multi-focus framing (Neat Symmetry/Individual framing) is enabled, Neat devices can frame each participant and place them side-by-side, as seen in Figure 5. Compared to the image in Figure 4, the table is no longer taking up large amount of space on the foreground and instead, a closer, equally-sized view of the participants is achieved.

This feature allows meeting room participants to be viewed better in gallery mode by remote users as seen in Figure 6 below.

3.3 Multi-Stream

The Multi-Stream feature brings about a set of new changes in how we view and frame participants in the meeting room.

3.3.1 Voice localization on Neat Bar Pro

If you are using a Neat Bar Pro, the active speaker in a room can always be highlighted while using Multi-Stream in Zoom. In “Gallery View” there will be a green rectangle around the frame that displays the camera stream that contains the active speaker. In Speaker View and Thumbnail View the camera stream that contains the active speaker will be shown instead of always showing the overview camera.

Note: Only the overview camera is transmitted as 720p/1080p. The additional streams are transmitted as 360p, thus the Speaker and Thumbnail View may show lower resolution than expected. We’re working together with Zoom to address this in a future release.

3.3.2 Multi-Stream with 2 or more participants in the meeting room:

If there are two or more participants in the meeting room, the Multi-Stream feature provides a brand new experience for the remote participants in the meeting room. The meeting room is split over three separate streams: the first stream providing a full-view of the meeting room; the second and third streams providing individually framed views of the participants in the meeting room (e.g. with 2 people, 1 in each frame; with 4 people, 2 in each frame and so on).

Figure 7 and 8 provide examples of how this is viewed by others in Gallery mode.

In Figure 7, the Multi-Stream feature is enabled in the room called ‘Sunrise’ and there are three participants in the meeting room. In the Gallery view, the Neat device (Sunrise) sends three streams to Zoom:

• Sunrise: first stream showing the full view of the meeting room
• Sunrise 1: second stream showing two of the meeting room participants in multi-focus/individually framed view
• Sunrise 2: third stream showing the third meeting room participant.

In Figure 8, there are 6 participants in the meeting room. Similar to Figure 7, there are three streams sent from Neat to Zoom: a) first stream shows full view of the meeting room; b) second stream shows multi-focus/individually-framed view of three participants and c) third stream shows multi-focus/individually-framed participants of the remaining three participants.

3.3.3 Limitation with Multi-Stream in Speaker view:

When Multi-Stream mode is chosen, the experience can vary depending on how the remote users are viewing the particular meeting room.

The Multi-Stream feature is best viewed in the Gallery mode as the viewers are able to see all three streams from the Neat device (as seen in Figures 7 and 8).

However, if the meeting room is viewed in Active Speaker mode, Zoom will only select the full-screen room view as the loudest speaker, and not the individual stream that consists of a close-up view of the speaker.

IMPORTANT: This is only a limitation for the current iteration of the Multi-Stream feature. Zoom and Neat plan to improve this experience in future releases. If your meetings are usually viewed in Active Speaker mode (as opposed to Gallery mode), the ‘Multi-focus framing’ (aka Neat Symmetry) will provide a better experience over Multi-Stream.

In Figure 9, the meeting room has 5 participants and Multi-Stream mode is chosen. As you can observe, when the viewer chooses ‘Speaker’ mode, the full room-view is shown as the Active Speaker, rather than the close-up individually-framed view.

4. The updated in-meeting ‘Camera Control’

Zoom Room Controller on Neat Pad or Neat Board also provides an in-meeting Camera Control option to change camera view during an on-going meeting. Until the June release, these options were only limited to setting presets in Manual mode or enable/disable Auto-framing and Mirror-effect.

With the new changes in this release, the in-meeting Camera Control on Zoom has also changed to reflect these new options. Now, despite what you have configured as your default camera control position (under Zoom portal -> Rooms profile), you can further change the camera settings on the fly using the Camera Control on your Neat Pad/Neat Board during an ongoing meeting.

The new options under local Camera Control now allows you to choose between Manual, Auto-framing (with or without Multi-focus framing) and Multi-Stream as shown in Figures 10 and 11:

5. Framing Boundary:

Finally a small note on Framing Boundary feature that was introduced in our June release. For rooms with glass partitions or open plan environments, you can set a framing boundary for auto framing to avoid focusing on people beyond your meeting space. This feature can be used in conjunction with Zoom’s Auto-framing/Multi-focus/Multi-Stream settings. However, Framing Boundary is a Neat feature and can be configured only under Neat’s System Settings. For more information, please see our June release notes here.