The Attention Mechanism with Andrew Mayne

Andrew and Justin examine the compute crunch behind always-on AI assistants, from Grokbot’s tightening limits to the challenge of choosing the right model for each task. They also discuss Siri’s persistent shortcomings, Anthropic’s finances ahead of its IPO, the widening gap they see between frontier labs and Chinese open models, and what happens when AI starts improving other AI.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro
01:50 - Compute Crunch
13:55 - Muse, Dots, and Assistants
28:44 - Open Source Frontier Models
37:26 - Models Making Models
43:30 - Wrap-up
Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin unpack Anthropic’s S-1, from founder control to compute costs, and consider why OpenAI might take its time going public. They also share first impressions of the new Terra model, examine a reported Qwen “phone home” incident, and discuss OpenAI’s latest claims of mathematical and scientific breakthroughs — and its approach to collaborating with researchers.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro
01:55 - Anthropic's S-1
21:59 - 6.1 Terra
24:46 - Chinese Models
28:38 - OpenAI and Science
35:47 - Wrap-up
Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin examine Meta’s bid for frontier status, mathematicians’ divided response to OpenAI’s wave of new proofs, Anthropic’s growing momentum with Claude, and how Google’s AI push has obscured the search experience that once defined its brand.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro
00:54 - Meta
29:11 - Math
44:24 - State of the Race
59:12 - Wrap-up
Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss their wishlist for OpenAI's upcoming Dev Day event, the delay of GPT-6.1, Meta's success with Muse, and more.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00 - Intro and Dev Day
00:37 - GPT-6.1 Delayed
14:27- $500 Plans?
21:54 - Sora and Video Generation
28:32 - Open Source Potential
30:28 - Dev Day Wishlist
35:51 - Meta and Muse
44:48 - Wrap-up
Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss the swirling rumors surrounding OpenAI's launch plans for this week ahead of Dev Day, a new committee designed to reveal Millennium Prize math problems being solved by ChatGPT, whether an IPO is coming in 2026, and whatever the heck Jev is.

The Attention Mechanism is now available in video! Check it out here.

Chapters

00:00:00 - Intro
00:00:58 - What's Jev?
00:03:24 - OpenAI Pre-Dev Day Rumors
00:23:09 - Millennium Prize Math Committee
00:38:44 - Are Hacking Incidents Overstated?
00:54:07 - IPO
00:59:58 - Wrap-up
Hosted on Acast. See acast.com/privacy for more information.