Meta wants to track people’s facial expressions in metaverse

Meta Platforms Inc. wants to monitor people’s facial expressions, eye movements, and body poses as well as create “virtual stores” that sell ad-sponsored digital goods in an effort to monetize the metaverse, dozens of recently filed patents suggest.

The company that began as Facebook and which has recently rebranded itself as Meta filed hundreds of patents recently with the US Patent and Trademark Office. These filings shed light on the company’s plans for the internet’s new frontier.

The patents, which were first reported by the Financial Times, indicate that Meta wants to use biometric data in order to create a more realistic digital avatar using virtual reality or augmented reality.

Meta also plans to collect data on users so that it could tailor specific ad content while they immerse themselves in the metaverse, the patent filings suggest.

One patent filing shows a sketch of a “wearable magnetic sensor system” that the user places around their torso for “body pose tracking.”

Meta’s patent filings indicate that it wants to collect user data points such as eye movements in order to create realistic, virtual avatars and sell ad-sponsored digital goods.
Bloomberg via Getty Images

Another patent can detect a user’s facial expressions through a headset. The system would then compute the expressions and “adapt media content” based on its findings.

The company is essentially following the same business model that translated into tens of billions of dollars in profit through its social network.

Meta also filed a patent for a headset that is equipped with cameras and sensors that ensure the user sees brighter graphics depending on where they’re looking.

The firm is also said to be developing an “avatar personalization engine” that can create three-dimensional avatars based on a user’s photos.

“For us, the business model in the metaverse is commerce-led,” Nick Clegg, Meta’s head of global affairs, told FT.

“Clearly ads play a part in that.”

Yaser Sheikh, director of Facebook Reality Labs, Pittsburgh, speaks during the virtual Facebook Connect event, where the company announced its rebranding as Meta, in New York, U.S., on Thursday, Oct. 28, 2021. A major theme at the annual conference will be the company's ambitions for the so-called metaverse, a new digital space that it believes will supplant smartphone apps as the primary form of online interaction. Photographer: Michael Nagle/Bloomberg via Getty Images
Meta is planning to invest $100 billion over the next decade to develop metaverse technology.
Bloomberg via Getty Images

Some of the patents appear to have originated in a science fiction movie. In December, Meta was granted a patent for the “mechanical eyeball 100,” which is said to be a “robotic eye designed to resemble a [human] eye.”

The eye is designed to “mate” with an animatronic robot head in an effort to make the eye “appear authentic to an observer,” according to Insider, which first reported the patent filing.

Meta CEO Mark Zuckerberg has said his company plans to invest $10 billion per year over the next decade to develop metaverse technology.

The race to metaverse supremacy has set up a clash of tech titans as Apple and Microsoft have also indicated they plan to spend considerable sums of money to grab a share of the market.

Meta’s push to collect more data could trigger a backlash, particularly in light of claims that the company disregarded its own engineers’ advice and bombarded Facebook and Instagram users with targeted content that was known to be harmful.

Shares of Meta rose by more than 1.2% in the first hour of trading on Wall Street on Wednesday.