The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

20VC: Chips, Models or Applications; Where is the Value in AI | Is Compute the Answer to All Model Performance Questions | Why Open AI Shelved AGI & Is There Any Value in Models with OpenAI Price Dumping with Aidan, Gomez, Co-Founder @ Cohere

Aug 19, 2024

Aidan Gomez, CEO and Co-founder of Cohere and co-author of the influential Transformer paper, dives deep into the complexities of AI. He discusses whether compute power truly drives model performance and shares his evolving views on data's role. Aidan tackles the current demand for AI models amidst OpenAI's pricing strategies and challenges the sustainability of being an independent model provider. He emphasizes the importance of data quality over sheer scale and explores the necessity of embracing adaptive business models in today's AI landscape.

58:35

Creator website

Episode guests

Aidan Gomez

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The podcast highlights that increasing AI model size can enhance performance, but innovative approaches may yield better efficiency without excessive compute costs.

Aidan Gomez discusses the shift towards prioritizing high-quality data over sheer scale, emphasizing its critical role in improving model performance and competitive advantage.

The urgency for enterprises to adopt AI has intensified, with key use cases emerging that augment employee capabilities and drive productivity enhancements.

Deep dives

The Value of Scaling AI Models

Increasing the size of AI models typically results in improved performance, making it a common strategy for those with ample financial resources. While this scaling approach is effective, it is seen as inefficient and overly simplistic, with a strong belief that more innovative methods could yield better outcomes. Recently, models with fewer parameters have been observed to outperform massively scaled counterparts, illustrating that efficiency can be achieved without excessive compute costs. This ongoing tension between scaling budgets and strategic innovation will likely shape the future of AI development.

Other than scaling, Data Drives Innovation

01:27

RAG: retrieval augmented generation

01:23

Embrace Failure as a Path to Progress

04:02

Great Vision is not enough to drive Execution

01:34

Intro

4min

Gaming as a Catalyst for Entrepreneurial Success

2min

Curriculum Learning in AI: Striking a Balance

8min

Future of AI: Data, Chips, and Interfaces

15min

Navigating AI Trends and Challenges

20min

Navigating Data and AI in Tech Development

5min

The Impact of Productivity on Healthcare and Economic Stagnation

4min

Aidan Gomez is the Co-founder & CEO at Cohere, the leading AI platform for enterprise, having raised over $1BN from some of the best with their last round pricing the company at a whopping $5.5BN. Prior to Cohere, Aidan co-authored the paper “Attention is All You Need,” which introduced the groundbreaking Transformer architecture. He also collaborated with a number of AI luminaries, including Geoffrey Hinton and Jeff Dean, during his time at Google Brain, where the team focused their efforts on large-scale machine learning.

In Today's Episode with Aidan Gomez We Discuss:

1. Compute vs Data: What is the Bottleneck:

Does Aidan believe that more compute will result in an equal increase in performance?
How much longer do we have before it becomes a case of diminishing returns?
What does Aidan mean when he says "he has changed his mind massively on the role of data"? What did he believe? How has it changed?

2. The Value of the Model:

Given the demand for chips, the consumer need for applications, how does Aidan think about the inherent value of models today? Will any value accrue at the model layer?
How does Aidan analyze the price dumping that OpenAI are doing? Is it a race to the bottom on price?
Why does Aidan believe that "there is no value in last year's model"?
Given all of this, is it possible to be an independent model provider without being owned by an incumbent who has a cloud business that acts as a cash cow for the model business?

3. Enterprise AI: It is Changing So Fast:

What are the biggest concerns for the world's largest enterprises on adopting AI?
Are we still in the experimental budget phase for enterprises? What is causing them to move from experimental budget to core budget today?
Are we going to see a mass transition back from Cloud to On Prem with the largest enterprises not willing to let independent companies train with their data in the cloud?
What does AI not do today that will be a gamechanger for the enterprise in 3-5 years?

4. The Wider World: Remote Work, Downfall of Europe and Relationships:

Given humans spending more and more time talking to models, how does Aidan reflect on the idea of his children spending more time with models than people? Does he want that world?
Why does Aidan believe that Europe is challenged immensely? How does the UK differ to Europe?
Why does Aidan believe that remote work is just not nearly as productive as in person?

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more