Reinforcement Learning From Human Feedback

ollow the Curiosity Podcast on:

Instagram: https://www.instagram.com/curiositypod101/ 

TikTok: https://www.tiktok.com/@curiosity_podcast101?is_from_webapp=1&sender_device=pc 

YouTube: https://www.youtube.com/@CuriosityPodcast 

Substack: https://curiositypodcast.substack.com/ Follow the Curiosity Podcast on:

Instagram: https://www.instagram.com/curiositypod101/ 

TikTok: https://www.tiktok.com/@curiosity_podcast101?is_from_webapp=1&sender_device=pc 

YouTube: https://www.youtube.com/@CuriosityPodcast 

Substack: https://curiositypodcast.substack.com/

Transcription:

Amjad Masad (00:00:00):

Pain and pleasure are like core features of consciousness and they seem important for humans operating in the world. Can you actually construct that in a machine?

Immad Akhund (00:00:27):

Hi everyone. Welcome to the first ever podcast recording of curiosity podcast where we go deep with an expert in their field. The tagline is Delivering 10,000 hours of learning in one hour. So that's a <laugh> ambitious tag line. I'm ial, I'm the co-founder and CEO of Mercury. I've been doing kindness startups and investing in startups since 2006.

Raj Suri (00:00:49):

I'm Raj Suri. I'm the founder and CEO of Presto Automation, which delivers AI type applications for for traditional industries like restaurants. Also co-founded Lyft and yeah, very excited to be cotting this with you. Mad. This is an opportunity to go really deep into some really interesting areas with some of the smartest people and you know, most thoughtful people on the planet. So excited to be able to explore, you know, in depth.

Immad Akhund (00:01:14):

Yeah. And today we have Amjad Masad with us. He's the co-founder CEO and I believe now head of engineering at Relet. What's the one line of relet? Mjad.

Amjad Masad (00:01:27):

It's the fastest way to make software. We have a platform that provides an online programming environment that's collaborative and we have large community of developers making things for each other, for other people. And we're getting into supporting teams of developers in the same way that say, you know, Figma is a collaborative design program replica is kind of that for programming.

Immad Akhund (00:01:54):

What's kind of interesting is it started relatively small, right? This was like an ID online and I think you were like compiling whatever programming to JavaScript and running it on the browser, right? But now it's this kinda hosted package combined with the teams combined with like learning and template. Like was that always your vision that you're gonna like progressively get to this level of like product or was it kinda incremental?

Amjad Masad (00:02:21):

I actually sort of saw a lot of it in my mind's eye pretty early on and a lot of it is like pretty obvious stuff. I'm actually surprised that no one had built it because I started working on it in college in like 2009, something like that. And then when I came to the US based on an Oprah source project that's related, I worked at Code Academy as a founding engineer and used this sort of the same technology that I built to like make browser coding possible and then left that and went to work at Facebook, worked on React and React Native was founding engineer on React native. React native is like the best way to make cross-platform mobile apps. And in 2016 revisited the idea and had found that basically nobody had built it, which was really surprising. It is the opposite or efficient market hypothesis, like why is in the market not producing this?

(00:03:20):

And I actually did not wanna start a startup, but I was so compelled by this thing and I knew that this idea had to exist. Well it turns out I think the answer that why nobody had built it is because it's incredibly hard. But yeah, in terms of execution it was started very simple. But in terms of vision it was always, you know, I was an early GitHub user, like perhaps in their beta. I think I was so excited by GitHub. I was also so disappointed by how little they evolved beyond the initial sort of kernel of an idea. So I always thought about making collaborative developer community that's like more exciting that you can ma do more things in it. And yeah, you know, I've always imagined a lot of the features that we're building. Of course with time just what became available in terms of technology, we started adding a lot of these things that I didn't really think about at the start. But overall, like even like now we have this tipping mechanism where you can tip developers. That was like, I've thought about that like fairly early on. It was actually kind of frustrating cuz like as a founder developer, you're sort of like, everything feels like it's like one weekend of hacking away and it's like turns out, no, it's actually more than a decade <laugh> of hacking <laugh>.

Raj Suri (00:04:36):

Mj it's a really interesting description. You talked about the fastest way to make software, right? Is the primary user base people who don't normally make software or are not really comfortable setting up their own environment cuz that takes time Or is it more the power users who are probably already, you know, they already have a CS degree or something like that, they have a lot of knowledge. What's the primary user base for this type of application?

Amjad Masad (00:05:00):

One of the exciting things about running Relet is the user base is so diverse. It also makes it really hard to run the company because people like crave these simple personas. I'm sure you guys at, at your companies like the design team or the product team want to talk about one or two personas. But I always push back on sort of persona building because ultimately I tell them that we have one persona and that persona is the developer, the software creator. Because like you start segmenting people into students or professionals or hobbyists or, and all these segmentations are true to some extent ultimately, like if you're familiar with Clay Christensen's jobs to be done, I think that inversed the question. It's not like a customer segmentation thing, which is totally arbitrary kind of based on user characteristics. It might be actually interesting to go into jobs to be done a little bit.

(00:05:58):

But basically the idea is that most companies, the way they think about bringing a product to market as they think about a customer persona and that really confuses people about what they should be building and what the actual need that they're doing it for. And so Clay Christensen switched that question, like his primary insight is that people hire products in the same way that you hire people. When you go and you want some accounting done, you don't go look for like a middle-aged man in New York. You know, you don't do that, right? You go and you say, I want some accounting done. And you find someone who's competent at accounting in the same way that you want to get software done or you want to make software, you go to a place that makes it really easy to make software. And from that lens, I just think about our community as as sort of really the main jobs to be done here is that you wanna make something.

(00:07:03):

And it turns out students wanna make something. They wanna learn to make something hobbyists wanna make something, they wanna crunch some data, they wanna make a fun app, they want to make a game. Professional developers wanna make software to bring it to their customers. And so that's the sort of shared goal of our community in terms of like the makeup of our community. It's cues younger and there's partly cultural, partly some product limitations. The cultural aspect is that developers tend to be very conservative people. They don't want to change their tools and they're actually quite haughty and quite, you know, I love them. My best friends are developers, I'm a developer, but we're actually like difficult people. And you know, you see it in Stack Overflow comments, it can be a little, people kind of look down on things and people kind of tend to not change too much with time. They stick to one language or one technology. And

Immad Akhund (00:08:05):

I was just thinking why someone didn't like make this innovation between like when you started in 2016. And I think it's partly because of this, right? Like developers are like, they're often build like little incremental tools for themselves to make their life a little easier, but they don't think about like transforming their trade. Like you don't go like, oh I want to like do it in this new way. Completely.

Amjad Masad (00:08:26):

I think that's right. Which is surprising because developers are the agent of change for the larger economy. Or maybe it's not developers that are the agent of change. It's crazy founders that recruit developers to, to make changes <laugh> it's a tough market because the habits really change really slow. And like there's common saying that, you know, programming changes one generation at a time and that is sort of true. But today, rep is is has some product limitations that we're working through mostly on the technology side, like making an immediately accessible institute, producible cloud development environment. Turns out it's like a very tough like distributor systems problem, like one Yep. Problem that we've like dealt with that's sort of introduced like a big product limitation is that if you're introducing this multiplayer programming environment, there's a a potential for what we call race conditions, which is like people like trying to edit the project at the same time and introduces data corruption and all of that.

(00:09:33):

So we use the file system, basically the system that manages your files, that was what we call atomics. So basically every time you do a right, you create a snapshot of your entire system and that is really good for distributed systems to protect against data races and all of that. It also turns out it just introduced like a really hard limitations on scale. Like we can't have like a one terabyte desk per project. So some of the trade offs we made early on created limitations to how much power the platform has. But all of these are solvable. It just requires a ton of innovations and so we're, we're working through those right now. Yeah,

Raj Suri (00:10:15):

I mean that's really interesting. So you talk about again, the fastest way to build software is on rep. What do you envision will be the fastest? Give us some examples of like, you know, how fast someone could build something, how much faster they could build something in the future versus now. And I know you know you're thinking about AI or probably already working on it, you know, in terms of now we can support coding. It'd be really interesting to hear you as the expert kind of what the future looks like in terms of how fast someone can build something.

Amjad Masad (00:10:41):

The fastest story I've heard on Relet is someone coming into Relet with an idea for a product and getting it 30 minutes later. And the way they got it is without writing a single line of code is by relying on a human machine. Centar of sorts, <laugh> basically we have this bounties program where you can like post, you can pay some money Wow. And post like a description of what you want to build. Someone posted like a Figma. I actually tweeted about it. I was like hey, I want to build this. And literally someone got them a prototype in like 30 minutes and that person who's getting them a prototype is not just a person. They're also powered by ai cuz we have a Ghost Rider product similar to GitHub co-pilot. It's actually a little more advanced than GitHub co-pilot because not only has the auto complete thing, you can also chat, you can talk with AI that's writing code with you.

(00:11:37):

And so it's like, feels more like a chat G P T that understands your code and that sits right there with your editor. And so we're really trying to be un ideological about how people make software and the idea is like really trying to reduce the distance between an idea and a, and a piece of product. Like that's been the trajectory of sort of a human history where if you're sort of hunter gatherer and you have an idea for a product that's sort of impossible to do it right. Like you, let's say you wanna build a spear, that in itself is like really, really hard. You have an idea for a hammer for example. It's like impossible to find the rock the right size and do all the right. These things go into, you know, agrarian societies and it's still pretty difficult to make things probably a little easier. Industrial society got way easier, you now had factories, you know, had accessible labor. Capitalism becomes a thing where you can organize people in groups to create things, but it's still fairly difficult to get a product on the market. And I think the information age is reduction of this idea of like having an idea in your head and getting a product in the hands of people. And I think we can get it down to like on the order minutes, <laugh> like to get something on the market.

Immad Akhund (00:12:56):

Like right now you said like there was a human that was like potentially using an AI to like generate this thing, this kind of prototype. Is there eventual state that there isn't a human involved and like the AI is just generating something good enough? Or do you think there'll always be a human in the loop with like our current AI at least?

Amjad Masad (00:13:12):

So the way I think about automation is that it sort of like takes the, takes each of the jobs from the bottom up. That's like not always true, but for the most part true. I believe it's gonna be true for software, meaning the low skill software creation I think will get fully automated. So I think pretty soon probably this year there's gonna be...