Memory Cost and Token Economics

Elad analyzes HBM and SRAM costs, showing faster token rates can drastically increase per-token dollar costs.

Play episode from 25:11

chevron_right

Transcript

chevron_right

Transcript

Episode notes

On this episode of Invested, Michael Eisenberg sits down with Elad Raz, Founder and CEO of NextSilicon, for a deep dive into the future of compute - and why the AI revolution may be bottlenecked by the very chips powering it.

In this technical and provocative conversation, Michael and Elad unpack what it actually means to “reinvent compute,” why today’s dominant architectures from Nvidia, Google, and Amazon may be fundamentally limited, and how NextSilicon is taking a radically different approach—one that replaces traditional CPUs and GPUs with a dynamically reconfigurable chip architecture.

Drawing on Elad’s background in elite Unit 8200 and decades of experience in high-performance computing, he explains why most AI chips are solving for the present—not the future—and why the real breakthroughs may come from entirely new paradigms like dataflow computing. He also shares bold claims about performance, including running workloads up to 10x faster at a fraction of the power, and discusses NextSilicon’s role in building next-generation supercomputers for U.S. national labs.

They discuss:

- Why the world is entering the “age of compute”

- What Nvidia—and the rest of the industry—may be getting wrong

- The real bottleneck in AI: memory vs compute

- Why faster AI often means dramatically higher cost

- The difference between training, prefill, and inference architectures

- Why future AI models may look nothing like transformers

- The economics of chips, data centers, and trillion-parameter models

- How NextSilicon’s architecture could reshape both HPC and AI

Elad Raz is the Founder and CEO of NextSilicon, a deep-tech company redefining high-performance computing and AI infrastructure. A graduate of the IDF’s elite Unit 8200, Elad brings more than 20 years of experience in technology, entrepreneurship, and executive leadership. Before founding NextSilicon, he founded Integrity Project, which was acquired by Mellanox Technologies in 2014. Elad resides in Ramat Gan, Israel, with his wife and three sons.

If you want to understand where AI infrastructure is really heading—and why the next wave may not belong to today’s incumbents—this episode is essential viewing.

Please rate this episode 5 stars wherever you stream your podcasts!

KEY TOPICS

00:00 - Intro - “10x Faster, Quarter the Power”

00:35 - Secret Supercomputer in U.S. Labs

03:27 - “Reinventing Compute” vs Nvidia

05:27 - Why the World Needs Infinite Compute

08:23 - Throwing Away CPUs and GPUs

11:25 - 30x–100x Chip Throughput Explained

12:47 - “What Jensen Missed”

13:00 - Why Dataflow Failed—Until Now

17:23 - Top 500 Supercomputer Reveal Tease

25:22 - Why Faster AI Gets More Expensive