Zoom’s Latest Feature Means There’s No Chance You Can Hide Your Face in Meetings

Zoom’s Intelligent Director beta will use machine learning algorithms to track participants’ heads through multiple cameras stationed in a room.

Zoom isn’t the same as it used to be. Without a wide-ranging pandemic keeping everybody at home, and with more companies demanding their workers slog to the office, the video conference company is trying to angle its software to increasingly hybridized workplaces. Now, the company is adding features to ensure everybody in a meeting is seen up close and personal, no matter how hard they try and avoid eye contact during regular company meet-and-greets.

On Tuesday, Zoom announced its new Intelligent Director feature, meant to allow more people in mid-to-large sized board rooms to appear present in a meeting, even if they’re trying to squirm far away from the evil eye of a camera within a giant screen at the head of the room. Zoom’s software will focus on each participant’s face, display their face in a box on screen, then track them as they move. The company claimed this new system avoids the “bowling alley effect” seen in a lot of large board room video meetings where most participants sit as far away from the camera as possible like all the cool kids hiding in the back of the class. Intelligent Director is available now but only available on a case-by-case basis, and companies need to individually contact Zoom through the website to sign up.

The system can currently track up to 16 meeting members, the company said. The machine learning that powers the software can also decide what best camera to use to track participants head-on—as long as there’s multiple cameras set up in the space. The feature is essentially an extension of the existing Smart Gallery video feeds, though now it can use multiple cameras to make sure there’s be no chance of avoiding eye contact in a meeting.

In a demonstration, the company showed that the cameras will automatically pick up on any figure that has a face or shoulders, even cardboard cutouts of people like Michael Scott from The Office. Zoom Chief Product Officer Smita Hashim claimed its system doesn’t collect any kind of biometric data, but simply detects if the people in front of the camera have the requisite proportions.

Zoom stressed this lack of data gathering to Gizmodo. Earlier this year, rights groups criticized the company after an executive mentioned the company was interested in potentially incorporating emotion-tracking AI into the Zoom experience. While Zoom described the new Director as AI, its more like a highly focused machine learning-based head-tracking algorithm. Earlier this year, Zoom announced it was adding other AI features from OpenAI’s API that can summarize chats, whiteboard sessions, and creating meeting agendas.

There are limits to this system, however. The AI cannot differentiate between who is in a room and who remains outside. A person walking by a meeting room window will briefly appear as a talking head in the meeting until after they disappear out of view. The company said it is working on iterating with this feature in the future to fine tune its ability to only recognize figures meant to be in the meeting.

This new system won’t work with any off the rack webcam. Certain cameras like the AVer Cam550, the Poly e70, and the Yealink UVC 86 are designed to work with Intelligent Director. The minimum specs require a computer running an Apple M1 chip or an Intel i5 processor. It’s also only supported on Windows and Mac platforms, though Zoom promised wider availability in the future.

Source