The Future of AI Avatars and DeepFake Technology w/ Joshua Xu & Steve Brown | EP #161
Recorded live at the Abundance Summit's Google TechHub stage, Peter Diamandis talks with HeyGen co-founder Joshua Xu about the rise of photorealistic AI avatars and digital twins, and how HeyGen's human-centric video model works from just two minutes of training footage. Diamandis demos his own 'Peterbot' clone and shares his three personal longevity protocols (Fountain Life, Viome, OneSkin). The second half is a presentation by Peter's chief AI officer Steve Brown on training personal AI agents, featuring a live demo of 'Hypocrates,' a medical AI agent that correctly flagged Brown's multiple myeloma diagnosis from his chart data after human specialists missed it. The episode closes with Diamandis's safety advice on verbal passwords against voice-clone scams and recording elderly relatives before it's too late.
Joshua Xu explains HeyGen trains a personalized avatar from about two minutes of footage, modeling voice, facial muscle movement, and body language, then adds an LLM knowledge layer on top.
Xu describes HeyGen's founding mission as 'replacing the camera' so businesses without studios or on-camera talent can produce unlimited video content.
Diamandis and Xu discuss sending personal AI avatars to attend Zoom meetings, run parallel conversations with many customers at once, and negotiate deals in different languages.
Steve Brown frames it simply: an avatar is the audiovisual layer ('AV agent'); an agent is the trained brain behind it that needs to know and be trusted by its owner.
Xu says every avatar on HeyGen requires first-party video consent plus AI and human moderation to block hate speech, fraud, and political campaign misuse.
Diamandis urges everyone to set a family verbal password to verify identity on suspicious calls or FaceTime videos asking for money, citing scams targeting aging parents.
Diamandis urges recording parents' or grandparents' voice, video, and life story now, either as a full avatar or a simple phone-recorded interview, as a gift to future grandkids.
Steve Brown describes building agents whose sole job is interviewing you about your life story, values, and purpose to generate the corpus that makes a personal clone agent meaningful.
Brown demos 'Hypocrates,' an AI agent connected to his medical record, which analyzed his chart and correctly flagged a probable plasma cell disorder that two human specialists had missed.
Brown recounts losing his home in the LA fires, then being hospitalized with symptoms specialists couldn't explain, until a young ER doctor pieced together a multiple myeloma diagnosis that AI later confirmed was visible in his historical records all along.
Diamandis's personal longevity stackLongevity▶ 17:27
Diamandis details his three-part health routine: annual Fountain Life full-body diagnostic uploads, Viome microbiome testing to guide diet, and daily OneSkin peptide use for skin senescence.
Multiple AI doctors with different viewpointsHealth▶ 33:00
Brown proposes running 100 differently-trained medical agents in parallel as different 'search strategies' into the same medical knowledge base, since human specialists get siloed by narrow expertise.
Predictions made
openJoshua Xu: Every business will create roughly 100x more video content for business communication than it does today, replacing email and static images as the dominant format.
“video editing is going to solve with AI very soon as well”
Your call:
openPeter Diamandis: People will hire teams of AI marketing agents that show up individually on Slack, email, and video calls, each with a distinct name and appearance.
HeyGen needs only about 2 minutes of training footage per avatarUsed to model a person's voice, facial muscle movement, and body language for a personalized avatar.
HeyGen avatar videos are produced 10 times faster than traditional productionFrom HeyGen's marketing video, claiming 10x speed advantage across 175 languages in 4K.
HeyGen avatars support 175 languages and dialectsCited repeatedly for avatar voice/translation capability.
A live demo avatar for an audience member took about 4 minutes to createAudience member 'Reichard' got an avatar made at the conference booth overnight in roughly 4 minutes of his time.
Joshua Xu has personally created more than 300 different avatars over 4 yearsTesting HeyGen's own technology on himself.
Fountain Life digitizes about 200 gigabytes of diagnostic data per person per yearPeter Diamandis describes his annual full-body diagnostic upload used to catch disease at inception.
Steve Brown's light chain levels dropped over 50% after 2 weeks of chemotherapy plus DaratumumabEarly response indicator cited by his Hypocrates AI agent as a positive prognostic sign for his multiple myeloma treatment.
Less than 1% of people are able to create great videos todayJoshua Xu's framing of the camera/production barrier HeyGen aims to remove.
Worth digging into
🕳️ HeyGen's human-centric video model architecture
The two-minute-footage training pipeline that captures voice, facial muscle motion, and body language is the technical core of the whole avatar wave and is barely explained beyond a sketch here.
🕳️ Hypocrates medical AI agent diagnosing multiple myeloma from chart data
A concrete, documented case of an AI catching a diagnosis that human specialists missed is exactly the kind of checkable claim worth verifying against the real chart data and timeline.
🕳️ Voice-clone and deepfake scams targeting elderly relatives
Diamandis references 'lots of friends' whose aging parents were targeted by voice-clone scams; the scale and mechanics of this are worth quantifying.
🕳️ Running many differently-trained medical agents in parallel
Brown's idea of 100 doctor-agents as different 'search strategies' into one knowledge base is a novel framing for AI diagnostics ensembling that could be checked against existing medical-AI ensemble research.