Facebook Is Working On Photorealistic Avatars That No Longer Require Face Tracking Cameras

Back in March of 2019, Facebook showed off its Codec Avatars, which were driven by a VR headset containing dedicated eye and mouth cameras once generated. Setting aside the possible tribulations of having to generate these avatars in the first place, adding those three cameras to headsets would be costly, and wouldn’t be the most ergonomic. All of this together is why Facebook’s news about a new prototype is so exciting.

The new Codec Avatars don’t require dedicated eye and mouth cameras, instead using a neural network to use eye-tracking data and microphone audio to infer the user’s facial expression. While Oculus headsets do not yet support eye tracking, it is in the works for more reasons than just avatars (check out this blog post about eye tracking). While these avatars could still be years away from reaching public use, we at Acadicus look forward to seeing their development. They will bring a whole new level of realism, which could have major implications for the medical simulation industry.