Face2Face: Real-time Face Capture and Reenactment of RGB Videos (CVPR 2016 Oral)
Why Hacker News recommends it
HN uses it as a reference point in deepfake and fake-news discussions, showing that convincing real-time video manipulation was already possible in 2016.
What it is
The CVPR 2016 oral demo of Face2Face, a system that captures a person's face and reenacts another video's face in real time using only RGB video. The authors frame it as research on modern vision and graphics.
Recommended by 0 HN commenters across 0 different months, 2016 to 2021.
What Hacker News said
Cites sigmar · 2016-03 · on “Microsoft demos 'holoportation' 3D presence tech with HoloLens”
Yea, perhaps. Like in the recent "face2face: realtime face capture and re-enactment video"[1]. and your reference to "your avatar" reminds me of the scene from the matrix where Morpheus describes "residual self-image"[2] [1] [this video] [2] [LINK]
Cites dgacmu · 2016-06 · on “Former NASA chief unveils $100M neural chip maker KnuEdge”
I find it amusing that they're touting it for an application where I find it quite likely that similar NN algorithms will excel at generating speech in someone else's voice. (c.f., "neural style transfer" in images, but applied to speech.) We're already getting pretty decent at this for video -- see, for example, [this video] (Face2Face: Real-time Face Capture and Reenactment)
Cites jefe_ · 2016-08 · on “Interactive Dynamic Video”
The consumer use cases are interesting but the propagandist use cases are terrifying. Along with this (Real-time Face Capture and Reenactment): [this video]
Cites dredmorbius · 2016-10 · on “Deepgram – Find Damning Soundbites”
OK, throw this into the mix: many of you reading this have smartphones which are voice controlled, and for which voice control is activated at all times. In the case of Google, that processing must take place on Google's centralised servers. Siri may or may not do centralised processing (and can operate in standalone modes). Microsoft's Cortana, Facebook's "M" (IIRC) and Amazon's Aero are all various stylings of "Stasi in a Glade form factor", as Maciej Czeglowski so memorably put it.[1] Voice stores distressingly cheaply in terms of space, and with the Internet of (broken) Things (that spy on you), odds of finding yourself surrounded by microphones in the most unexpected locations,[2] contr
Cites RGamma · 2016-10 · on “Deepgram – Find Damning Soundbites”
It gets even better when combined with Face2Face, a live-editing software that can transfer facial expressions including lip movements from actors to e.g. politicians on TV: [this video]
Cites rawnlq · 2016-12 · on “Carrie Fisher has died”
Maybe CG will be good enough to keep casting dead actors? The face2face[1] tech looks pretty convincing to me and it already works in real-time (which isn't necessary for films) and you have plenty of old footage of them. [1] [this video]
Cites omegaworks · 2017-02 · on “Combating “Fake News” with a Smartphone “Proof Mode””
I still think that it is incredibly important that we make tools for validation of primary source material easy to use and friendly for non-technical people. Tech for faking video is getting more powerful day by day. [LINK] [this video]
Cites mbrookes · 2017-03 · on “Deep Voice: Real-Time Neural Text-To-Speech”
> "Fake News" is about to get a lot more compelling hen you can make anyone say anything as long as you have some previous recordings of their voice. Adobe has already developed that technology: [LINK] Now imagine combining it with this: Face2Face: Real-time Face Capture and Reenactment of RGB Videos [this video] Perhaps using the intonation from the face-actor's voice to guide the speech synthesis.