Facebook's Prototype VR Face Tracking Got Even Better
Why Hacker News recommends it
Pointed to in metaverse and VR threads as proof that realtime photorealistic face tracking from headset cameras is coming.
What it is
UploadVR covers a Facebook Reality Labs prototype that tracks facial expressions using cameras mounted in a VR headset. The updated version captures subtle micro-expressions for realistic avatars.
Recommended by 1 HN commenters across 1 different months, 2020 to 2022.
What Hacker News said
Strong rec josephg · 2021-09 · on “What it's like to spend 40-50 hours in VR every week”
> The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. The oculus team put together a seriously jawdropping demo of facial tracking using a few cameras in a headset, combined with a lot of AI. The prototype was from a couple years ago. I can't wait until it makes it into commercial headsets. I wonder what the hold up is? Well worth a look if you haven't seen it: [this video]
Cites jp555 · 2020-08 · on “Transparent OLEDs”
It will ( [this video] ), and VR already conveys MUCH more of the micro-interactions of being with people that we are unknowingly blind to in Zoom meetings, which is one reason Zoom meetings are so draining.
Cites baby · 2021-08 · on “‘Horizon Workrooms’: remote collaboration reimagined”
I don't think they don't understand it, they understand incremental improvements. Whatever you're talking about is coming: [this video]
Cites roveo · 2021-11 · on “Ask HN: How is the “metaverse” concept different from the Second Life boom?”
There are algos that reconstruct photorealistic 3D facial expressions from headset cameras in realtime. [this video]
Cites microjim · 2022-03 · on “Face-to-face interaction enhances learning, innovation”
Title should be amended to something like 'Face-to-face, rather than shoulder-to-shoulder, enhances learning, innovation.' For a high quality face-to-face remote experience, I expect we'll start to see really impressive consumer ready technology announced soon. See what Facebook have been doing with cameras mounted in VR headsets [1] or Google's Project Starline [2]. That said, when we're doing this much transformation and synthesis of the raw data to achieve high-fidelity telepresence, are we going to have issues trusting it? [1] [this video] [2] [LINK]