Investing in AI: Trends and Challenges

https://www.listennotes.com/podcasts/chit-chat-stocks/the-future-of-artificial-3ylaqngtMR7/

[00:00:00]

intro: Welcome to Chitchat Stocks on this show host Ryan Henderson and Brett Schaefer analyze businesses and riff on the world of investing. As a quick reminder, Chitchat Stocks is a CCM media group podcast. Anything discussed on Chitchat Stocks by Ryan, Brett, or any other podcast guest is not formal advice or recommendation.

intro: Now, please enjoy this episode.

Brett: Welcome in. We have another episode of the Chitchat Stocks podcast for you this week. We have a fantastic interview coming up with Shawn Wang from LatentSpace. It is a, I'll say covering anything AI. A lot of the stuff, you know, for me and Ryan, it might be going over our heads and it might be a little too hard technically for us, but that's what we brought on Shawn to the show today.

Brett: For a lot of public market investors, this new [00:01:00] AI stuff, it's hard to You know, see what is, what is working, what's just a narrative, what all the stuff that's getting thrown at us during this boom times. So Shawn, we wanted to bring you back on the show. You came on, I think almost exactly two years ago now to talk more cloud stuff.

Brett: Now we're really going to talk about cloud AI and how it is impacting the startup ecosystem. So Shawn, as we kick off the show. What is your relevant expertise in this booming AI field? 

swyx: Oh God, that's the million dollar question here. So I am so for, for listeners who haven't heard back the previous episode that I was on I was finance and public markets guy.

swyx: And I was, I was in a hedge fund for my first career. And then I changed careers to tech where I worked at AWS and three unicorn Sort of developer tooling cloud startups. My relevant expertise is you know, on, on some [00:02:00] level, I, I'm just a software engineer that is building with AI now. And then on another level, I had, I actually, when I was an options trader back in the sales side, I actually did a lot of natural language processing of the Bloomberg chats.

swyx: So I fed all of the Bloomberg chats into a pricing mechanism. Then built our global pricer. So our entire options desk was running off of that thing. This was about 13 years ago. So so you know, I, I've always had some involvement with like AI, but like, you know, it was never a big part of my identity and I think.

swyx: The more foundation models came into focus, and foundation models is a very special term as opposed to traditional, maybe machine learning finance that a lot of your listeners might be familiar with then you start to build differently, and there the traditional software engineering skills become a lot more relevant.

swyx: So relevant expertise now is that I, I guess I've sort of popularized and created the term of AI engineer, which you can talk about and created the industry such that Gartner now considers, considers [00:03:00] it like the peak of its hype right now. And I, I consider that both a point of success and also a challenge because I have to prove Gartner wrong that it has not peaked, but you know, they put us at the top of the hype cycle, which is kind of funny.

swyx: Because I started it, so. 

Ryan: Yeah, it's it's a unique challenge but yeah, funny anecdote. Okay, so a lot has changed since we last spoke. Yeah. Pretty much this whole world of AI that everyone's talking about now or at least has become mainstream has, I believe that kind of kicked off right after the discussion or our last discussion.

Ryan: So I guess the last discussion was really focused on the cloud computing industry broadly. And that was actually right around the time when AWS. Azure, GCP all the revenue growth rates were coming down and actually now with the hindsight bottoming. So my question for you is what has, I guess, [00:04:00] what has changed over the last two years and why has revenue growth at the big cloud providers re accelerated?

swyx: Yeah, again, like, revenue growth at big cloud providers is due to factors that, you know, probably I don't have a full appreciation of. I also challenge the fact, the idea that everything has changed. You know, I think in some ways, this is just like the next wave of something that was just a broader, maybe like 20, 30 year long trend anyway.

swyx: You know, we, we needed more cloud compute. Now we need even more cloud compute. Now we need more GPUs in the cloud instead of CPUs, right? Like, what's really changed? I don't know. Like, you know, people still want serverless everything. People still want orchestration. People still want you know, unlimited storage and bandwidth and all the sort of core components of cloud.

swyx: In that sense, it hasn't really changed. I do think that if you see there are plots over time of the amount of money and flops invested in machine learning models, that actually used to follow a pretty log linear Moore's law type growth chart for the last like 40 years. [00:05:00] And then, You had 2022 happen and now everyone's like, oh, you can train foundation models now.

swyx: And actually you've seen a big inflection upwards in the amounts that people are throwing in throwing the money in there just because they see the money now. So like every, it's like obvious to everyone, including us, including me in a way that it wasn't obvious to basically everyone, but Sam Altman and Satya Nadella circa 2019.

swyx: Like they knew this. Four years five years ahead of everyone else. And that's why they went big on OpenAI. But now that we see this, obviously everyone's throwing money into NVIDIA, basically. 

Brett: I had, why, why are, and this is maybe a question I think I know, but I'd like the answer again, and it feels like it's maybe a basic question, but a lot of, I think listeners are going to want to kind of understand this connection.

Brett: Why do these new AI companies require so much upfront spending? On NVIDIA chips, cloud computing costs. All that stuff. 

swyx: Yeah. I mean, so [00:06:00] you have to split it by whether you're a foundation model lab or you're basically everyone else that consumes foundation models. So the rough estimate for, let's say GPT 3 was like 50 million to a hundred million in compute for one run.

swyx: And for every one successful final training run, maybe you have between a hundred to a thousand. Prior runs before that, right? So just pure R& D. The estimate for GPT 4 was 500 million. We've actually had two generations of frontier models since then, just for OpenAI. So that would be GPT 4. 0 and GPT those are the models that, those are only, only the models they've released.

swyx: And also not, those are only the text models, we haven't counted the video models and all the other stuff. So it's just a lot of upfront investment, right? Like, I think it's, it's like the classic capital fixed costs upfront thing, where, you know, you have a pre training phase where you're just consuming all of the internet.

swyx: [00:07:00] Data that's, you know, there's nuances to that, but we won't go into that. And, and, Alka, Alka comes the other end, you know, 3 to 6 months later, Alka comes a model that you then spend another 6 months fine tuning and red teaming, and post training, and then it's ready for release. So, like, so there...