And especially when the volume of feedback is very high, this feels like something that might have taken you even just to calculate those Excel statistics at the top 15, 20 minutes. and then to read and summarize all these, maybe another 30, 45 minutes. We're talking about a one hour work session that's

essentially taken us two to three minutes here. Right. And then what you could do, right, if the alternative, let's say you're a very diligent PM and you're like, well, I read every single app store of you. Great, right? You can now use this to zoom in and say, okay, hey, can you make me a CSV with everything in this category or all the one-star reviews or everything that people are complaining about split screen? Let's say this is an initiative we have coming up, and I want to include those in the kickoff. Great, you can still read every single App Store review. You don't have to give that up.

And in terms of the project that you placed this in, we saw the prompt. What type of context did you put in to make this project so effective?

Yeah. So the cool thing about projects, this is in Gemini, it's called Gems. In Chagipiti, it's also called projects. The idea is that it's a bunch of context that gets started in every thread that you have. So if you were just to use, you know, open up ChatGPT one morning or Cloud one morning and just start typing in a chat, it starts from zero context. Maybe it has a little bit of memory about you, but nothing serious. If you've created a project and you've put into that for a product manager, it might be, hey, what's the vision of the company? Who's your target customer? You can get personal and say, who's on my team? You can say, what were my last performance reviews? And so on and so forth. Basically, you could start with asking yourself, what would you give a new product manager that just joined your company? Like what, before they even have a team, what document would you have them sit down and read? Right. Just start with that. Like that should be, put that in the context because using LOMs without a lot of context, you've probably heard the expression, you're just going to get the average of the internet. And so there's all this like latent power you're not getting. But the moment you give a little bit of context, right, a little bit more context, you are basically turning, you're like focusing all the power of the LM to this like laser through that context. And then you get way more than the average of the internet. That's where it gets really powerful.

So what's our next use case of AI for PMs? So one thing that I constantly wish I had more

of my manager's time to help me with is I just need another pair of eyes on my crazy calendar to just like look at it from the side and just give me an audit just give me like a fresh view of it so what I'm going to do is again I'm inside the project so it has all this context on me including things like feedback I've gotten in the past from my manager my performance reviews 360 reviews things that I have told my co-pilot that I need to work on, right? Things like that. Things like, oh, you don't make enough time for just spending one-on with a designer in that phase. And I'm going to upload a bunch of screenshots of my calendar. So they look like this let's see without the arrows there we go so we're just going to upload a bunch of these just like random samples and i'm going to say hey i have attached screenshots on my calendar um what observation insights do you notice what you know what do you what do here? And of course, it's always a really good tip when you use AI. Like, give me exact quotes. Give me specific examples. Illustrate what you're saying. Because otherwise, LMs tend to just kind of stay really high level, which feels good, but then backfires long term. All right. So these are some cool vanity stats um there's like half my work week okay frequent topic switching um multiple interactions in a single day uh limited deep work time oh shit um right and you can also if you really want to be really rigorous here you can tell the ai hey before you analyze my calendar, I want you to go like ask me about anything that you don't know what it means. So some meeting titles might not be clear to people who don't have context on the company. So ask me what they are and then go ahead and do this. You can go even crazier and say, I want you to visualize it. So I'm going to go back and just change this prompt. And say, here's screenshots of my calendar and go over these weeks and tell me about every single calendar event. Right. Categorize each event. If it's unclear, ask me what they are and just you know summarize it for me um here are some examples just to help understand riverside and the context there um so at the end what we're going to do is we're going to say great now you like created this like raw data you categorize everything it's kind of like quickbooks for my time um create a pie chart right tell me about trends then like yeah i can have a field day with visualizing like yeah just making it like html css the charts and then you can really just have this like very visceral understanding of like oh holy crap i'm spending a lot of time here and i should be spending time here but that's like a really small slice of the pie chart. You can take that to your manager and say, hey, we need to go over these stats of my time, right? Some of you have like mixed panel and amplitude

for how you spend your days. So that wasn't possible before I launched. I wonder if there's a way to give it like the description of the events and the attendees and those types of data points as well. That's a really cool idea. I bet with something like

Zapier or similar kind of tools, you can even connect it to your Google Calendar. I haven't tried that. That's a really cool idea. Or

that's a startup idea for somebody. Or a startup idea. Yeah. Integrate with your Google Calendar. ask for your google calendar permission and then download more than just the title to optimize people's calendars yeah

i mean anybody hearing this i think anytime a workflow with ai gets a little complicated but still very similar for everybody you know involved in a target market like that's a startup yeah

And everybody's always worried about building these wrappers, but I like what Aravind Srinivas said, which is like, the LLM models are wrappers on top of NVIDIA. NVIDIA is a wrapper on top of TSMC. Everything is a wrapper at some point. Yeah.

I mean, anytime I hear somebody use the term, oh, it's just a wrapper, I'm like, great. One less person that's going to compete with me. Right. Cool. You keep saying that. i mean just think of how many um sas products we use that could be done in google sheets right or probably a lot of people still use google sheets for instead of jira but jira is a company yeah um you can think of the same craigslist for a lot of consumer things as well. And so here's like some insights. Nice. And then I'll say create a pie chart. So again, nothing special here, just like natural language. Create a pie chart. Like give me some like inference and insights. When

do you prefer to edit the prompt versus just continue the conversation below? That's a good question. If

you don't like the response and don't want to keep working with that, go edit the prompt, right? Like it's basically an opportunity to rewind and redo. If you're like, that's cool, let's use that and let's build on top of that and continue the chat. That's the best way to think. I used to

rarely edit prompts, but I think I'm going to start editing them more.

I think that's the most important feature of ChatGPT, Claw, Gemini, the edit pencil icon. That is, I think that's like all of prompt engineering, that pencil icon, because you don't really need to know the best practices or the latest like academic papers on what prompts work you just need to be able to type something in look at the result apply your taste say did they do what i wanted if not you know like a person what context was it missing let me go back edit the prompt add another sentence of context uh did it do something i didn't like right then i'll go back edit the prompt add another sentence don't this thing right do that a few times you're going to get really close to what you wanted so here we go a little messy i think it's because we're really zoomed in um for screen share but this is there we go here's like there's a concern recommendations almost a slide and it made interactive. So, um, a little out of order, a little weird, but pretty impressive. Right. This is something that I can then export as a CSV. I can like say, Hey, take, give me that data back. I can say, save this for my next model with my manager and go over it. And we still have the raw data. We still have the calendar screenshots as well to really understand what is it talking about. All right, let's move on to our next use case. All right. So I want to give credit to Alan and my community for this really cool idea. I think one of the challenges as a product manager is just keeping up with new technologies, being able to intelligently hold your own, at least to know what questions to ask. You don't have to be an expert, but some of these concepts are, you know, they're popping up all the time. So for me, one thing that I really need help with is learning the concept of RAG, right? Retrieval Augmented Generation. That's like something I think is more art than science, but very technical. And so I want AI.

For people who don't know, why should they care about this RAG? Oh, okay. So this is a concept that's used often

in a lot of AI products and AI applications. It's when you have way more data that can fit in a context window. How do you still let the AI work? So for example, something like custom GPT is actually, this is how they work. That's where you can upload so many files to a custom GPT. Or for example, like Notion AI is another example, right? Like they don't want you to have to limit how many documents you can traverse but they also can't fit you know a company's entire knowledge base into one context so rag is a technical concept that solves that but right now i don't know what that means i've heard that thrown around in the hallway but i need to understand so i figured out, okay, I know what, I know what the acronym means. And you can basically adapt this to your learning style. So for me, I want to teach me the concept like I am five year old and ask, ask me questions. So like interactive. Now, the cool thing is it's inside, again, the Riverside PM co-pilot. So it's going to use Riverside when it can and like relate it to my job. So this is cool, right? It's like explaining it super simply. This is like, and it's adapting it to Riverside's vision, right? Think of this as like you found that like perfect blog post that explains this concept just for you and your style but without having to google so if i have to go retrieve some data without a helper i'd have to answer from memory um with rag i get a special helper, kind of like an expert librarian or a paralegal. Anyway, so it's them saying, okay, well, it's not over. It's not just like read this as a blog post. It's asking me to engage. So why am I answering a question using only your memory, send this on the top? I might forget details. So this is actually the answer. Let's see what it has to say. All right, so I've created like a little teacher for something that's pretty complicated. But finally, you know, oftentimes engineers on your team, when you ask them to explain something, they might mean really well, but they might make you more confused. So it's really helpful to like do this ahead of time and study up. All

right, let's move on to our next use case. Should I run an A-B test?

One of the biggest questions as a PM is like really high stakes for your team, but also like a really difficult thing to explain to stakeholders is why you are or are not running an A-B test for something, right? And it always sounds like really good for somebody marketing or sales or CEO to be like, well, why don't you just run it as an A-B test? And it's never a good look to explain like, hey, rationally, this logically is a bad decision to run this with an A-B test. You can do this, by the way, for any framework, any decision framework you find online. It really helps sometimes to like say, hey, I did this AI and it like gives it a little bit more rigor, just like inviting an external, you know, third party consult. So a long time ago, I wrote a blog post on like what is the decision tree for running an A-B test. I took that blog post. I, at the time, I put it into AI. And I was like, hey, can you convert this into like Markdown? Can you convert this into like a text version decision tree? And then I said, great. I'm trying to decide whether a change that I want to make in the product should be run as an A-B test. So let's come up with a feature. Akash, what would be like a feature you'd really want to see in Riverside? Thumbnails for

podcasts on YouTube. So like automatically generated thumbnails

for podcasts on YouTube. Yep. Cool. Let's throw in the term AI in there. So the rest here is, I didn't write this. I mean, I wrote the blog post, but I used AI to just like convert the blog post to a decision tree. So let's just run it. So this is like a really cool way to apply critical thinking, right? And it also really saves a lot of time in articulating a really difficult, really complex decision that can be testing. What's cool is it's first of all, answering, asking me a few questions. And let's see. Okay. Do we need precise quantification of the change we don't there's we don't and two there's not a lot of downside the country do agree like even if the feature wasn't super effective it wouldn't um set you back as a creator on Riverside.

Yeah, mainly just, like, a little bit of brand downside, right? Like, if you generate something for somebody and they don't like it, they may just abandon the platform, especially as a new user. Okay, so there is a downside.

It could make bad, or say like bad outputs could make bad impression on the entire platform. I mean, like wonder if other features were working out. Right. Don't necessarily revenue. So on one hand, we don't need to know exactly how much money this made or it affected conversion, but we do need to make sure it's not like ruining people's impression on the product. I ignored the other questions. It'll be just fine. Okay. So now it's walking me through its thinking. I answered yes here. Let's do the secondary requirements. It's cool. It's kind of facilitating me, like, Socratically. Oftentimes, if you are doing this in a thread where you've been talking about this feature for a while, like, hey, we, this user research, help you write a PRD, things like that. It won't ask you this many questions. And let me just try something like take a guess at all questions. So it's kind of doing a PRD. It's all PRD here. And then it'll make a recommendation, which is don't do an A-B test, but make it an opt-in feature. And you can keep going, right? One thing you can do is like, okay, that's cool, but play devil's advocate. And I want you to argue the opposite. You can keep going with this and make it a conversation. At the end of the day, as a PM, you're kind of like observing your team debate each other and all like getting informed by all sides of the debate and you can make a judgment call. But this really helps save a lot of time, a lot of like brain cells, right?

Like you have a whole team discussing this really difficult topic with you. Awesome. Let's move on to our next use case, reverse simulating a hard conversation? Everybody listening to this has

had to have a conversation that they were not excited about. Could be with somebody on their team, right, about some friction or missed expectations, the stakeholder on how to better work together or what's not working. And it's really nerve-wracking. I know for me, right, it's something I try to put off, but I know it needs to happen. And AI is a really good way to practice for it. hard conversation. But even better, right? You could use AI to simulate what it's like to talk to the other person. But another thing you can do is flip it. You can have the AI be you and you be, let's say, the stakeholder. So it's a matter of giving context. So just tell the AI what happened the way if you were like really stressed about it and sitting with a friend over a beer. So let's see. I'm really nervous about having a conversation with my product marketing manager about how we're not really. Let's see. We're really dropping the ball on rollout plans and just clarity on who's getting what uh feature when and what collateral is going with that and the clarity around that to the rest of the company so i merged a bunch of hard conversations into like one so i'm going to say i'm really nervous about a conversation with a marketing partner about how you need to better communicate the role of the company. been clear in the past about the workflow, but it doesn't seem to happen. Right. And I'm leaving this ambiguous could be the problem is me here, but, um, and I will play, uh, I'll play Joe. I probably. Okay, so we're asking the AI to play me. And all right, so this is, I'm already feeling this like empathy. This is me and I'm Joe. So I'm reading this. Hey, Joe, thanks for making the time. I know we've been juggling a lot. What's your take on how it's been going? Where do you think we're getting stuck? I'm already feeling feelings reading this. I already can like, I'm like, oh, okay. That's what it's probably like to hear this and i'm also noticing um you know some really good sentences that are really helping me i guess be more open to this kind of conversation and it's talking about impact right so i'm noticing what i could be doing and if the AI doesn't do a good job, you can notice what doesn't work either. But this is like a really cool way to understand what it's like to be a person you're going to be talking with. I can say, I think it's going great. I'm going to be really extreme here. So you can keep going on this. You can do this for, you know, really tough. I've done this for like really tough stakeholder situations. You can do this for personal life. You can do this for product interviews. So Ben Arias, another instructor on Maven, I think also on your list. He does a really cool job with this and a really specialized, really focused for fan interviews. And yeah, it's hard to explain the feeling you get. Even in make-believe land, you have to put the other person's head on. But it's really, really powerful. And I don't think this was something that was possible before LLM's.