2025-07-11

AI Experts React: Elon’s Grok 4 Is Now #1 in AI —This Changes Everything w/ Emad, Salim & Dave #182

Peter Diamandis convenes Dave Blundin, Salim Ismail, and Emad Mostaque for a special reaction episode the day after xAI launched Grok 4, which topped independent benchmarks (AIME 100%, Humanity's Last Exam 44.4% on the Heavy tier) just 28 months after xAI's founding. The panel debates whether Grok 4 is reasoning without yet planning, dissects the economics behind its $300/month 'Heavy' tier and the 340,000-GPU Colossus cluster, and argues that token costs will keep falling 5-10x a year toward roughly a dollar per million words. They cover AI's near-term impact on medicine (augmentation before replacement), coding (context engineers replacing coders), and entertainment (AI games and movies challenging Hollywood), before closing on GPU and chip-supply constraints as the real bottleneck to scaling toward Grok 5 and beyond.

▶ Watch on YouTube

Topics

Grok 4 benchmark dominance AI ▶ 0:03

Grok 4 scored 100% on the AIME math benchmark and leapt from mid-pack to #1 on the Artificial Analysis Intelligence Index, just 28 months after xAI was founded.

Reasoning vs. planning in frontier models AI ▶ 3:05

Dave and Emad agree Grok 4 reasons well enough to solve PhD-level problems but still can't autonomously plan or decide what to build — that judgment stays with the human operator.

xAI's Colossus cluster economics Compute ▶ 12:55

The panel breaks down how Elon built a 340,000-GPU cluster (worth roughly $10B) from a cold start in 28 months, something every AI expert said was impossible at that scale.

Token pricing and the cost-collapse curve Economy ▶ 22:36

Grok 4's ~$20/million-token cost is compared to Claude Sonnet and o3, with Emad projecting 5-10x annual cost declines driven by next-gen chips like Vera Rubin.

AI in medicine: augmentation before replacement Health ▶ 29:50

Citing a Google study, the panel discusses doctor-alone, doctor+AI, and AI-alone diagnostic accuracy, concluding regulation (not capability) is the real blocker to AI replacing clinicians.

AI-generated games, TV, and movies vs. Hollywood Other ▶ 33:26

A clip of Elon describing a 4-hour AI-built FPS game leads into a debate on whether Hollywood will be disrupted; Emad argues attention, not production cost, is the real constraint.

The end of coding as a craft AI ▶ 42:59

Emad revisits his 'no more coders in five years' prediction, arguing code is just an intermediate language step that's being replaced by direct, context-engineered instruction to AI.

xAI's upcoming video/world model AI ▶ 45:40

Elon describes training a from-scratch video model on 100,000+ GB200 GPUs; the panel discusses how video models double as world models useful for robotics and self-driving.

GPU and chip supply as the real AGI bottleneck Compute ▶ 55:00

With OpenAI, Google, Meta, and xAI all racing to a million-plus GPUs, the panel concludes capital isn't the constraint — chip packaging and supply chains are, with only ~20M GPUs in the world.

Edge AI and model commoditization AI ▶ 59:10

Emad predicts frontier models become a commodity utility like electricity, while efficient edge models (e.g., Liquid AI running ~100x more efficiently on devices like Apple's M-series) become the real differentiator.

Predictions made

open Emad Mostaque: Grok 4 or its near-term successors will discover genuinely new technology, and almost certainly new physics, within the next one to two years.

· due: 2026 · ▶ watch

“I would be shocked if it has not done so next year.”

Your call:

open Emad Mostaque: A frontier model will reach a 100% score on Humanity's Last Exam within about two years, likely as soon as next year.

· due: 2027 · ▶ watch

“Uh two years max I would say probably next year.”

Your call:

open Emad Mostaque: Equivalent-intelligence token costs will keep falling 5-10x per year — helped by next-gen chips like Nvidia's Vera Rubin making inference 3-4x cheaper — until a million words of top-tier AI output costs about a dollar.

· due: 2026-2027 · ▶ watch

“So it'll be a buck for a million amazing words.”

Your call:

open Elon Musk (clip): The first genuinely good AI-generated video game will ship next year, with a watchable half-hour of AI-generated TV this year and a full watchable AI-generated movie the year after.

· due: 2026 · ▶ watch

“I would expect the first really good AI video game to be next year.”

Your call:

open Elon Musk (clip): xAI will begin training its from-scratch video/world model on over 100,000 Nvidia GB200 chips within three to four weeks.

· due: August 2025 · ▶ watch

“We expect to be training our video model with uh over 100,000 GB200s... within the next three or four weeks.”

Your call:

open Alex Wissner-Gross: If AI solves fundamental physics within the next one to three years, it may reveal that quantum teleportation allows real-time communication, meaning we could instantly discover other intelligences in the universe.

· due: 2028 · ▶ watch

“If we can solve physics in the next year or two or three... we instantly discover all these other intelligences around the universe.”

Your call:

open Dave Blundin: Within about two years, robots will be capable enough to perform 90-95% of human labor, with mass economic adoption limited only by manufacturing supply chains, not by the technology itself.

· due: 2027 · ▶ watch

“In two years robots will be good enough to do what 90, 95% of human labor.”

Your call:

Numbers that matter

Grok 4 scored 100% on the AIME math benchmark Cited as evidence frontier models are running out of benchmarks to saturate.
Humanity's Last Exam: o3 21%, Grok 4 25.4%, Gemini 2.5 26.9%, Grok 4 Heavy 44.4%, best humans ~5% Scale AI's exam designed to stay unsolved for years; Grok 4 Heavy's jump is called a qualitative leap.
Grok 4 API pricing: $3/million input tokens, $15/million output tokens, up to 256k context Compared favorably to Claude Sonnet and o3 pricing.
xAI's Colossus cluster has 340,000 GPUs at roughly $30,000+ each (~$10B in hardware) Built from a cold start in 28 months.
Jensen Huang projects AI infrastructure spend will hit $1 trillion/year by 2030 Framed as context for why ~$1B/day is currently flowing into AI.
Compute cost to train a 10^27-flop ('ronoflop') model is about $312 million Dave asked Gemini for the current market rate live on the show; compared to a single OpenAI signing bonus.
Video game industry ~$450B vs. movie industry ~$70B Cited to argue Hollywood's real competitor is gaming, not other studios.
Movie industry revenue grew from ~$50B to $60B in 10 years (avg. IMDb 6.3); video game industry grew from $170B to ~$500B, avg. Metacritic score rising from 69% to 74% Used to show gaming is both bigger and improving in quality faster than film.
Only about 20 million GPUs exist in the world Cited as the hard ceiling on how many 'million-GPU' training runs can happen concurrently.
Liquid AI's edge models run roughly 100x more efficiently than a brute-force transformer Cited as why edge inference on devices like Apple's M-series chips matters more than people realize.

Worth digging into

🕳️ Humanity's Last Exam question set

The panel reads out sample questions neither they nor most humans could parse, which made the 44.4% score feel genuinely superhuman rather than benchmark-gaming.

🕳️ Google's AI-vs-doctor diagnostic study

The ~70%/87%/90% doctor/doctor+AI/AI-alone figures are repeated as near-fact but given as rough memory, not citation.

🕳️ Liquid AI's edge-efficiency claim

Emad calls Liquid AI's ~100x efficiency gain a bigger deal than people appreciate, with Dave having personally funded Liquid's early compute.

🕳️ Grok 4 Heavy's $300/month unit economics

Dave guesses xAI is losing money on the tier as a loss leader; no company discloses true per-query compute cost.

🕳️ xAI's 100,000-GB200 video model training run

Elon claims training starts within 3-4 weeks on a from-scratch world model that could also improve robotics and self-driving.