Reverse Engineering Character Animation for Consistent Characters

Note: This episode is best experienced as a video: https://www.youtube.com/watch?v=KDD4c5__qxc

Hey Hitchhikers!

MidJourney V5 was just released yesterday so it felt like the perfect opportunity to do a deep dive on prompting with a fellow AI newsletter . Linus creates amazing MidJourney creations every day ranging from retro rally cars to interior design photography that looks like it came straight out of a magazine. You wouldn’t believe that some of Linus’s images are made with AI when you see them.

But what I love most about Linus is his focus on educating and sharing his prompting techniques with his followers. In fact, if you follow Linus on Twitter you will see that every image he creates includes the prompt in the “Alt” text description!

In this episode, we cover how Linus shares how he went from designer to AI influencer, what generative AI means for the design industry, and we go through a few examples of prompting in MidJourney live. One thing we cover that is beneficial for anyone using MidJourney for creating character-driven stories is how to create consistent characters in every image.

Using the tips I learned from Linus, I was able to create some pretty cool Midjourney images of my own, including this series where I took 90s movies and turned them into Lego!

I also want to thank Linus for recommending my newsletter on his substack, which has helped me grow my subscribers to over a thousand now! Linus has an awesome AI newsletter that you can subscribe to here:

I hope you enjoy the episode and don’t forget to subscribe to this newsletter at http://HitchhikersGuideToAI.com.

Show Notes

Links:

- Watch on Youtube: https://bit.ly/3mWrE5e

- The Hitchhikers Guide to AI newsletter: http://hitchhikersguidetoai.com

- Linus's twitter: http://twitter.com/linusekenstam

- Linus's newsletter: http://linusekenstam.substack.com

- Bedtime stories: http://bedtimestory.ai

- MidJourney: http://midjourney.com

Episode Contents:

00:00 Intro

02:39 Linus's journey into AI

05:09 Generative AI and Designers

08:49 Prompting and the future of knowledge work

15:06 Midjourney prompting

16:20 Consistent Characters

28:36 Imagination to image generation

30:30 Bonzi Trees

31:32 Star Wars Lego Spaceships

37:57 Creating a scene in Lego

43:03 What Linus is most excited about in AI 46:10 Linus's Newsletter

Transcript

Intro

aj_asver: Hey everyone. And welcome to the Hitchhiker's guide to AI. I am so excited for you to join me on this episode, where we are going to do a deep dive on mid journey.

aj_asver: MidJourney V5, just launched. So it felt like the perfect time for me to jump in with my guests, Linus Ekenstam. And learn how to be a prompting pro.

aj_asver: Linus is a designer turned AI influencer. Not only does he have an AI newsletter called inside my mind, but he's also created a really cool website where you can generate bedtimestories for your kids. Complete with illustrations. And he is a mid journey prompting pro. I am constantly amazed by the photos and images that Linus has created using mid journey. It totally blows my mind.

aj_asver: From rally cars with retro vibes to bonsai trees that have candy growing on them. And most recently hyper-realistic photographs of interior design that looked like they came straight out of a magazine. Linus is someone I cannot wait to learn from. And he's also going to share his perspective on what all this generative AI means for the design industry, which he has been a part of for over a decade. By the way it's worth noting that a lot of the stuff we cover in this episode is very visual. So if you're listening to this. As an audio only podcast. You may want to click on the YouTube link in the show notes and jump straight to the video when you have time.

aj_asver: So if you're excited about I'm one to learn how you can take the ideas in your head and turn them into awesome images. Then join me for this episode of the Hitchhiker's guide to AI.

aj_asver: Thank you so much for joining me on the Hitch Hiker's Guide to ai. Really glad to have you on the podcast. I feel like I'm gonna learn so much in this episode.

Linus Ekenstam: Yeah. Thank you for having me.

Linus Ekenstam: I mean, I'm not sure about the prompt, you know, prompt guru, but let's try

aj_asver: Well, I mean, you tweet about your prompts every day.

aj_asver: on Twitter, and they seem to be getting better every time. So You are my source of truth when it comes to becoming a great prompter. And I also, by the way, love the one thing you do when you tweet your mid journey kind of pictures that you built, um, that you've created, that you always add in the alt text on Twitter. Um, exactly what the prompt was. And I found that really helpful. Cause when I'm trying to work out how to use Mid Journey, I look at a lot of your alt texts. So, um, also include a link to your Twitter handle so everyone

Linus Ekenstam: Nice

aj_asver: it out. But I guess

Linus Ekenstam: I guess I'll stop.

aj_asver: you know, you've been in the tech industry for a while as both a designer and a founder as well

Linus Ekenstam: Yeah. Yep.

aj_asver: love to hear your story on what made you, um, kind of get excited about AI and starting an AI newsletter and then, you know, sharing everything you've been learning as, as you go.

Linus's journey into AI

Linus Ekenstam: Yeah. I mean, if we rewind a bit and, and we start from the beginning, um, I got into the tech industry a little bit on a banana, like a bananas ski. I, I started working in, like, the agency world when I was 17. I'm 36 now, so 19 years ago, time flies. Um, and after like working with, um, customers, clients, and big ones as well, through like, through my initial years there, I kind of got fed up with it.

Linus Ekenstam: And. . I went into my first SaaS business as an employee and it was email like way, way, way, way, way before this day and age, right, where you had to like code everything using tables and transparent GIFs. It was just a different world.

Linus Ekenstam: And 2012 was like, that's when I started my first own business. And that was like my first foray into like the, the startup world or like building something that was used by people outside of the vicinity of, of, of Sweden or Nordics. Um, it was very interesting times. Um, and I, I've always been kind of like early when it comes to New tech, I consider myself being a super early adopter. I got Facebook as like one of the first people in. By hacking or like social hacking a friend's edu email address. And I got an MIT email address just so I could sign up on Facebook.

Linus Ekenstam: Um, so now that we are here, it's like I've been touching all of these steps, like all the early tech, every single time, but I never really capitalized on it or I, I never really pushed myself into a position. I would contribute, but this time around I just, you know, I felt like I had a bit more under my belt.

Linus Ekenstam: I've seen these cycles come and go, uh, and I just get really excited about like, oh s**t. Like this is the first time ever that I might get automated out by a machine. So my response or flight and fight response to this was just like, learn as much as possible as quickly as possible, and share as much of my learnings as possible to help others.

Linus Ekenstam: Cannot not end up. In the same position where they fear for their lives.

aj_asver: Yeah, it's, it's interesting you talk about that because I think that's a huge motivator for me as well. It's just help people understand that this AI technology is coming and it's not like it's gonna replace everyone's job, but it certainly is gonna change the way we work. And make the way we work very different. And as -you've been doing and sharing, you know, how to prompt and what it means to use ai, one of the things I've noticed is you've also received a little bit of backlash, you know, from other designers in the space

Generative AI and Designers

aj_asver: That maybe as embracing of AI as you have. And I, I know recently there were probably two or three different startups that announced text to UX products where you can basically type in the kind of, uh, user experience you want and it generates, mockups right which I thought was amazing and I thought, You know, that would take years to get to, but we've got that now.

Linus Ekenstam: yeah, you

Linus Ekenstam: post.

aj_asver: and I think one of the things you said was designers need to have less damn ego and lose the God complex.

aj_asver: Tell

aj_asver: me a little,

aj_asver: what the feedback has been like in the AI space around kind of how it's impacting design, especially your field.

Linus Ekenstam: So I think, um, there, there is this like weird thing going on where. They're a lot of nice tooling coming out and engineers and, and, and developers. You kind of embrace it. They just like have a really open mindset and go, yeah, if this can help me, you know, I'll, I'll, I'll use it.

Linus Ekenstam: Like, take Github Copilot is a good example. People are just raving about it and, and there is some people that are like, oh, it's, it's not good enough yet, or whatever. But like the general consensus is that this is a great tool, it's saving me a lot of time and I can focus on like more heavy lifting or thinking about deeper problems.

Linus Ekenstam: But then enter the designer , like turtleneck, you know, black, all dressed in black. I mean, I'm, I, I'm one of those, right? So I'm, I'm, I'm making fun of myself as well. I'm not just pointing fingers at others here. I just think it's like weird that. Here's a tool that comes along and it's a tool, it won't replace you.

Linus Ekenstam: Like I'm being slightly sarcastic and using like marketing hooks to get people really drawn in, in my content on Twitter. So I'm not really, meaning, it's not literal. I'm not saying, Hey, you're gonna be out of a job. It's more like, You better embrace this because like the change is happening and the longer you stay on the sidelines, the, the, the more of a, a leap that your peers will have that are starting to embrace this technology.

Linus Ekenstam: And I, it's so weird to see like people being so anti and it's like, it's just a tool. It's not, it's not like the tool itself is dangerous. It's like people with the tool will become dangerous and they will threaten your position. Right. So I just find it very interesting to this whole kind of landscape where people, on one hand, it's just embracing it and people on the other hand are just like, no, I'm not, I'm not gonna touch it cuz he can't do X or he can't do Y.

Linus Ekenstam: It's like, bear with you. It's like we're in the very, very early days of ai. , we might be seeing half a percent or 1% of what's possible. And these tools are here today, like you said. Um, so I think my kind of like vantage point is like I'm not looking at the next six months or the next 12 months. I'm just like drawing out an arc and going like, where are we 20, 30?

Linus Ekenstam: My whole game here is to get as many people as possible.

Linus Ekenstam: Ve well versed into these tools as fast as possible. Like, I want to make sure that the divide between the people that haven't got experience and the people that haven't yet played with these tools kind of make sure that divide doesn't grow too big. I think that's my mission really.

aj_asver: Yeah, You pointed out there there about how, you know these advancements are happening really quickly and you want people to be able to adopt the tools is I think a really important one. And I think a lot of people don't really understand conceptually how exponential advancements kind of work. And I think Sam Altman recently had this good quote where he said something along along the lines of, when you're standing on an exponential qu curve, it's flat behind you, but it's like vertical in front of you, right?

aj_asver: And we're like like climbing this exponential curve, and I think some of us probably see the writing on the wall of how quickly this is all gonna happen. But for other people, know there's always gonna be this resistance. You mentioned like how this tool is gonna help people and people should embrace it, and you of course share a lot of what you are learning and especially when it comes to prompting and you and the art of kind of prompting in mid journey to create interesting images.

aj_asver: And I think you've done some prompting on ChatGPT as well

Linus Ekenstam: Yeah,

Prompting and the future of knowledge work

aj_asver: One thing I'm curious about is, do you think that's the future of the, like knowledge work for us? Is it gonna be like we all just become really good prompt engineers and we're just prompting away when it comes to like writing, you know, writing documents or when it comes to creating design in ux or when it comes to, you know, making images or, or do you think there's more to it than that?

Linus Ekenstam: I think prompting the way that it's done now is gonna be very short-lived. Um, if we're doing an analogy and compare, like prompting with ai, with what we're people that are doing really root level programming, let's say assembly type code for computers.

Linus Ekenstam: Um, so we're in the age now where everything is new, uh, and the way to interact with these models, whether. ChatGPT or Mid Journey or Dolly or Stable Diffusion. It's a very, very root level. Uh, I think I'm, I'm already starting to see like products popping up that are precursors or like tools that put, put themselves like a layer on top.

Linus Ekenstam: So instead of writing like a 200 keyword, Um, prompt to mid journey. You're essentially writing like five words or 10 words that's very descriptive of what you want. And then the precursor takes care of like generating the necessary keywords for you.

Linus Ekenstam: I don't think we'll see these like prompt tags where people figure out me included, figuring out like ways to, you know, if you do this in this sequence or this order, you will able to do. This with a, with a, you know, with a model. Right. Um, I think we'll see less of that potentially, um, and, and move more towards like really natural language and, and less kind of like prompt engineering around it.

Linus Ekenstam: But I think very, um, important here to note is that the kind of like precursors will happen, like we will kind of move away from talking assembly line situation with AI models and that's also like, that's a barrier to entry right now. If you look at mid journey, you want to get. A lot of the things you need to just overcome is kind of how do you write what you want to get?

Linus Ekenstam: Because like if you just write, I want this image that does X, Y, and Z, you're probably not gonna get the thing that you have in your mind. So it's gonna be like you trying out different things and then getting to like slowly speak ai.

Linus Ekenstam: Don't focus too much on becoming like a, a super good prompter, right? To try to, to learn more like principles or techniques or, or, or like think more, um, holistically about this whole thing. How do you interact with ai? I think that's, uh, yeah, something I would recommend

aj_asver: I liked your analogy of, you know, assembly language and how we're all kind of writing assembly code right now. Um, another analogy I've also heard is like, it's, it's like DOS before Windows came and we're all kind of at the command line trying to get what we want by typing it into a computer before someone, you know, made like a user interface around it.

aj_asver: You know, very famously, you know, Xerox Park did it and then Apple was the one who kind of released the version of it. And I, I think that's gonna be something. We'll, all welcome. But in the meantime, I'm curious, how did you kind of learn that steep learning curve of becoming really great at prompting?

aj_asver: Because we all kind of start from zero here, and I think the example you gave where you said like, you know, you think you can just come in and describe what you want. That's exactly how I started. I was

Linus Ekenstam: I was just like

aj_asver: I'm just gonna the the scene and then it didn't end up anything. like wanted

aj_asver: What was your journey of becoming good at prompting and, and learning how to use the journey effectively?

Linus Ekenstam: I think the, the community's really well set up, uh, with my journey that you can. You can go onto midjourney.com and you can start kind of exploring what everyone else is doing. Um, initially I didn't really understand that they actually had a website. Um, but, but then after a few, I guess a few months, then I'm like, oh, there's actually a website here.

Linus Ekenstam: And maybe it wasn't there in the beginning either, so it might be that I just missed it completely. So I think like dissecting other people's work and trying to figure out like, oh, this was a nice cinematic shot. How did I get that look? Um, and, and mind you, like, I've been doing this now for a few months, or yeah, almost half a year.

Linus Ekenstam: Um, and it, it was really different. There was less people using the tool like six months ago than there is now. So I think. It's easier to jump into the tool now and see what others are doing and kind of learning by do, like learning by dissecting essentially. I think it's the same with design. Like if you're learning design today, like the best way is just to try to replicate as much work from everyone else as possible. And I'm not saying replicate in the sense of like, oh, that's a nice prompt, command C, command V, oh, now I did it. Um, that's k that's, that's not learning to prompt . All that it's very easy if you, if you find something you like, you wanna make your own derivative of it, go ahead.

Linus Ekenstam: I mean, that's the beauty of these tools as well. But if you really want to like learn the skill or like, you know, I have this idea I want to do, then I can go do this. But then I must say there's also some really talent. People, uh, on Twitter and elsewhere that are sharing their journeys as well, and, you know, figuring out ways to, to structure their prompts or, yeah, th there there's a bunch of people that we could potentially look at later or, or that we could recommend in the show notes.

Linus Ekenstam: Um, for sure.

aj_asver: Yeah, that would be, that would be great. And I think for people that aren't familiar, the way Mid Journey works is you, you have the website where you kind of explore existing images, but all of the work of creating Images is done through their Discord, which

aj_asver: Unintuitive to anyone that's not familiar with online communities and with Discord, which Discord itself is a fairly new phenomenon from the last like three or four years right? It was originally used in gaming, but now it's used a lot in communities across ai, across crypto and other and other places. And so was another thing that got, that took a while to get used to is like interacting with AI via Discord. But there's one cool advantage of it that I didn't really fully grasp until now, which is that I can pick up my phone and jump in the Discord any time when I have an idea for an image and just start making images.

aj_asver: One of the things I'm curious about, you've been doing this for about six months, ballpark, how many images have you created

Linus Ekenstam: I think I just passed, like the 10 K club. I'm not sure. I'm gonna have to look later.

aj_asver: I mean, one thing I would love to do in this episode, um, is learn from you some of the skills of like, you know, being a pro prompter in, in, uh, MidJourney. So I was wondering if we could kind of jump into it and maybe one, take a look at some of the creations you've done in the past. Walk us through a little bit, um, how you, how you came up with them and then I have a few ideas of things I wanna do in Midge. Maybe you can help me, um, make that happen.

Linus Ekenstam: Let's jump into it.

Midjourney prompting

aj_asver: Awesome. So we're gonna jump into Mid Journey now, and Linus is gonna show us some of the images he's created, give us a bit of a sense of kind of his approach to prompting and then we're gonna jump in and do a few examples too

aj_asver: So one area, for example, that I would love to learn more about is consistent characters in Mid journey. After we did the podcast, uh, with Ammar, where we built the, where we created the, um, children's book, a lot of people asked, oh, how do you get consistent characters across all the pages of the children's book in the illustrations? So I'm really curious about how you achieve that, cuz that's something I've struggled with as well.

Linus Ekenstam: This is interesting. So, um, it, it started with, A lot of people trying to achieve the same thing using mid journey, which is essentially, you know, you have a character, you want the character to be in different poses or in different photos or in different, you know, could be a car cartoon, it could be a real person.

Linus Ekenstam: Uh, and I saw different ways of doing it, and mainly they were for cartoons. Then I'm like, this doesn't not work well with a human. Uh, cuz I tried and it didn't work. So I'm like, there must be some other way to do this. Uh, and obviously this is like brute forcing a password really cuz like mid journey is not supposed to be this tool.

Linus Ekenstam: Uh, the best way to do consistent characters is to use stable diffusion or something else that you can pre-train on a set of images.

Consistent Characters

Linus Ekenstam: the, the way that I went about doing it is essentially, , uh, going to how illustrators work and when, when they create like a character for, for a movie or for an animated, whatever it might be that they're doing, they need reference materials so that other artists can work on the same characters.

Linus Ekenstam: You might have hundreds of artists working on the same character. Um, so then, you know, looking at how they are doing these, I'm like, maybe if I simplify this, what if, you know, I take left and right and up and down, and. and I used those images as inspiration cuz that's something you can do in my journey.

Linus Ekenstam: You can like image prompt, essentially just like putting an array of images and then adding your prompt. So I, I, I went about like starting up making a character, um, just using like a very simple prompt here. I didn't really have any intent of, of, of the output. I just like, let's make journey, do its thing.

Linus Ekenstam: Um, and then when I, I, I found one that I kind of like, oh, this could be nice. Let's work with this. Um, I started using something called a seed. and use that image. So a seat is essentially the noise number or the random noise that an image gets started from. So if. For anyone that doesn't know, you know, mid journey is a diffusion model, which essentially starts from noise and it takes a string of text and it uses that text to take the noise and transform it into an end result.

Linus Ekenstam: So if you want to know the pattern, the exact pattern of the noise that you're starting from, you can include a seed number and it's like randomly generated every time you do an image. So if you have an image and you use the seed and you prompt against that seed again, uh, the likelihood of getting something very similar is quite.

Linus Ekenstam: So I, I kind of went away and, and started doing different angles of this woman. And then once I had more angles, um, I put the angles together. So I'm just scrolling through here, but essentially just finding those up, down, left, right, and forward. And when I was happy with all of them, I just put them together in a long prompt.

Linus Ekenstam: Um, and then just having the same prompt again as the first time, you know, uh, a style, a, a, a portrait shot of a woman. Um, street photo of a woman shot on Kodak, which is essentially just like the, the type of film I wanted to emulate. And then I get the e exact woman out and, hi, this is like, you know, okay, now here we go.

Linus Ekenstam: Uh, what can we do with this? Right? Uh, and there is a bunch of things, like a bunch of learnings, um, from this, which is essentially like you can. Very specific images, if you have a bunch of, of images that you're using as the inspiration images,

Linus Ekenstam: but also when you do these technique. My kind of the, the culprit here is that I use street style photos.

Linus Ekenstam: So every time I'm trying to get her to do other things, like we can go over here. Uh, I, I wanted to try to get her in a, in a space suit, right? We can see that she's kind of in a space. , but she's still standing on a street. So

aj_asver: It's like a very, like a urban chic space suit,

Linus Ekenstam: yes, . It's an urban cheek spacesuit. And we can even see here, like try some different, um, ar like some different aspect ratios. She's in a spacesuit, but we still have the background of, of the street, right? So, . One way to combat this and that, you know, figure this out after the fact that I made this tutorial is like the, the, the source material, the source MAs that you're using, they should be isolated.

Linus Ekenstam: They should be like either against a transparent background or a white background. And that way all of a sudden you can start placing this woman in different areas. So what's neat about this, that you don't need to train a model. You only need to have a set of image. So let's say you have six or nine images that are your inspiration images, and they don't have to be AI generated either.

Linus Ekenstam: You could use like yourself, you can take photos of you from the different angles and put them together. Um, and I think a lot of people, it resonated with a lot of people because this is one of these things that are inherently hard to do in my journey, and there is quite a big use case for it. So I, I, I personally hope that, you know, my journey goes into the direction of kind of a little bit.

Linus Ekenstam: Stable effusion or Leonardo, where they're giving you tools to do these kind of things like fine tuning, but not maybe to the extent of like training your entire, your own model completely. Right. And we can look at this example. I think this is very nice as well. Like we can get her smiling cuz that was one of the things that, you know, people, oh, you used all these photos, which she's not smiling, you're never gonna get her to smile.

Linus Ekenstam: Uh, and basically you can like, there's a lot of like things you can do, even though mid journey is very. Has very opinionated. So there are ways to work around this. And if we're like diving a little bit into prompting here, um, we can just, let's, let's grab this full command here. Um, and we can,

Linus Ekenstam: yeah. Sorry,

Linus Ekenstam: Yeah.

aj_asver: Was you basically reverse engineered how, you know, a character animator would approach this idea of consistent characters. And the way they do it is they have different poses of a character that they kind of create first. So you kind of have a base, kind of almost like a sculpture that hasn't been fully molded yet. So to get an understanding of the character, you generated those using ai, but you could also have a. you know, photos you already have of a person or maybe you take photos of yourself at different angles. Then you inputted that as actually with the prompt, you inputted the images too, that was what allowed you to kind of then create these consistent characters cuz you now have this base image to work from. one of the things you said was, if you want to be able to change the background, so move them from like street photos for example, to be in space, you kind of need to remove the background from the original base images because that's that background. If you keep it in there, like the street photos has a street background is gonna influence what mid journey creates as well.

Linus Ekenstam: Yeah, correct. Um, let's , I just pushed this in here. Let's see if my journey does something with it. Sometimes when you're using a lot of high resolution, um, inspiration images, it actually crashes the, the bot. So it doesn't work. Oh, we're actually getting something that's good. So, uh, it's not entirely sure we're gonna get a smiling woman this time, but the way to kind of like force smiling for example, is give smiling a very high weight.

Linus Ekenstam: So when you're using, um, Let's see if I can scroll in here. Yeah. So when you are adding, uh, is it colon? Yeah. Is it colon, semicolon? Um, colon? Yeah, colon. Colon five, for example. Then you give smiling. The word smiling, uh, a weight of five. Uh, I think standard weight is like zero or one, I think one. Um, so we're really emphasizing here that we want her to be smiling and now I think we actually got something.

Linus Ekenstam: And it might not be that she's smiling in all of. , but she's, uh, kind of smiling . Forced,

aj_asver: got like a little bit

Linus Ekenstam: yeah. a bit of a forced smile. Uh, but

aj_asver: smile. Yeah.

Linus Ekenstam: yeah, and this is the thing. I mean, mid journey is opinionated and you, you might have to do re-roll, you might have to do things like over and over. And because it's not really trained on her smiling or being neutral is actually trained on her being.

Linus Ekenstam: Angry or just very like,

Linus Ekenstam: uh, so re-roll is like, essentially press a button here in, in, in this cord and it takes the exact same, uh, prompt, the same parameters, and a different seed so it won't use the same noise again. So it will start from the beginning One more time. Uh, if you wanted to get like more diverse outputs, we could use chaos, which is essentially how chaotic the difference is between the four image.

Linus Ekenstam: that we're going for. So we could add, uh, dash C and then let's say a hundred. So this value goes between zero and a hundred, and it will d dictate the difference between the four different images. So we can see up in, yeah.

aj_asver: I noticed, um, by the way, that there's a few different kind of arguments you can add to the end of your. Mid journey prompt, and I think one that you use often is aspect ratio. Um, and then chaos is one. You just mentioned hph and hph, and C. Where did you learn about these and how does someone kind of work their way around trying all these different ones?

Linus Ekenstam: um, i, I mid journey.com, they have like documentation I think people are a bit afraid of, of the documentation because they might not know what they're looking for or like, um, yeah, it, it's not that hard. Like when, when you're prompting in mid Journey and then you go like, okay, there is like, I think, uh, 6, 7, 8, 9.

Linus Ekenstam: Nine different arguments that you can use. So it's like aspect ratio, chaos. Quality seed stylized tile, which not many people know, and version and quality. So version, you don't need it if you're not. Like now it comes preloaded with the latest model. So if you just add dash dash V four four, it actually uses an older model.

Linus Ekenstam: Um, so a lot of prompts you'll see will have dash, dash, v4, uh, not necessary. So essentially now the model that's running is V4 C, which is like the third iteration of v4, uh, and quality two. You can go quality one, two, and up to five, I think. But they've done a lot of testing internally. people can't tell the difference between Q1 and q2.

Linus Ekenstam: Like, so it's just a waste of GQ 10 because essentially when you're doing quality two, you're gonna use twice as much GPU render time. And GPU render time is essentially how long, um, of like, how much of your credits get used to render an image? Um,

aj_asver: it. So high quality means if you're paying for mid

Linus Ekenstam: yeah.

aj_asver: actually to use the bot directly versus the

Linus Ekenstam: Yes. Yeah.

aj_asver: it's gonna cost more per image basically.

Linus Ekenstam: Yeah.

aj_asver: I notice as well is that you also include some details around how the shot is taken, right? The actual camera. Um, is, does that make a lot of difference kind of picking the, the camera? Because I noticed that was a pretty cool thing that I didn't, I wasn't aware of actually until I saw your, your images.

Linus Ekenstam: Yeah, I think we're, there's a bunch of people that I've, like, I, I saw this like in December, the first time. I think like people using camera. Like it's shot on a canon or it's shot on a hassle blood, or it's shot on an icon, or it's shot on this type of film, you know, emulating black and white film or emulating sepia tone film.

Linus Ekenstam: Um, and then now I think it's, there's a lot more people that are kind of dissecting it and like really going nitty gritty on it. Um, and, and trying to just be like, what are the things that we can do with this? Like, how. , how much can we describe with this? And it's, as it turns out quite a lot, especially like camera angles type of shots.

Linus Ekenstam: Like, you know, using wide, ultra wide narrow, you can go and use like lens parameters. So like for those that are interested in photography, um, you, you could use 50 millimeters, so 50 mm. Um, to, to decide kind of the, the, the, the framing of your shot and kind of what. Output should look like because it has a very distinct look.

Linus Ekenstam: You could go 80 millimeter, 120 millimeter tele lens. All these things matter. Actually, it matters quite a lot cuz if we go here and, and check some of the photos I did the other day about, so I did some, uh, national Euro graphic shots, right? So these are quite interesting where we have like, um, shot on the telephoto lens as one of the key things.

Linus Ekenstam: So what it does, it really gives you this super consumed in

Linus Ekenstam: photo with like bulky in the background. So you have a really blurred background and we can see that, like my journey is really good with hair. Um, and the compression might blow it, blow it down a bit, but Maur is fantastic with hair and feathers and fibers.

Linus Ekenstam: I'm not sure what they've done there, but it's, it's absolutely fantastic. Um, so yeah, and lens matters quite a lot.

aj_asver: you are, what you're doing there is really kind of imagining the camera you would take this photo with if it was a real photo, and using those, um, those properties of the camera as parts of the prompt. And one of the things I also noticed with your prompts is you are not necessarily describing the scene.

aj_asver: You are often described lots of, characteristics of the scene, right. What is your approach when it comes to, you have this idea in your head, uh, you, you, you kind of, ima have this idea in your imagination of what you wanna create and then getting that down into a prompt. How do you, how do you approach that?

Imagination to image generation

Linus Ekenstam: um, I mean, in the beginning I, I did write a really interesting one. Threads on this as well. Cause I was sitting in a restaurant you mentioned earlier that like, that, you know, it runs in Discord. You can bring up your phone, you can start prompting. I was, um, we're, I got two kids, right? And me and my partner, we actually had like the first weekend without kids, um, since pre pandemic.

Linus Ekenstam: So we basically haven't been out alone. And, and you, what, what I do, I sit with my phone in mid journey. That came out bad anyhow, we were sitting there and we're actually using it together. So we were like, we're talking, we're talking about like what we're building with bedtimestory and then we saw this like really nice geisha on the world cuz we were eating at an Asian fusion restaurant.

Linus Ekenstam: And I'm like, I wonder if I can do that mid journey. And then we're just like, you know, open up discord on the phone and we're sitting there chatting, drinking a little bit and just like, oh, okay, we, you know, let's try this. I think I ended up doing like 50, maybe 50 or 60 generations where like the initial.

Linus Ekenstam: Was a geisha, but it didn't look anything like the thing we saw on the wall, right? Because the, the geisha on the wall was like on a wooden plaque,

Linus Ekenstam: uh, just like a really nice white geisha face mask and some red, really tiny red, uh, pieces in it. So we basically just went like, iterated removed, you know, added, redacted.

Linus Ekenstam: It's just like added words, removing words, try different things, went completely crazy and go, what if we just take away all of this and write something completely different? Um, so it it, it's easy if you have an idea, right? That to just like continue to, to plow through. And then once you hit what you want, then you have that kind of like base prompt.

Linus Ekenstam: Then you can start altering that you. Exchange an a subject or an object or exchange a post or, but, but you have kind of your, your, your base prompt figured out.

Linus Ekenstam: So getting to the base prompt could be tricky. Sometimes you hit gold after just a few tries. Um, it really depends, uh, on what it is that you're looking to create.

Bonzi Trees

Linus Ekenstam: Right?

Linus Ekenstam: I had luck with like bon's eyes, for example. I just, what can I do bons eyes with, with ma journey and how does that work? You know? And I just type like, uh, pine tree bonsai. Why wait a minute. Like this is fabulous. You know, I, I, I think I made some bonis again yesterday just for fun. Um, so like raspberry bonai, that's, that's, this is the prompt.

Linus Ekenstam: This is it, you know, it's not harder than that. And like, you could do

Linus Ekenstam: raspberry. That's it. , right? And, and, and you can, you can, you can, you can imagine, you can do thousands of thousands of these, right? Uh, and you can be crazy about it. You can do, like, I think I did Candy Bon. Yeah. Here we go. Ken Bonk. Who, who knew

aj_asver: it's got like lollipops.

Linus Ekenstam: Yeah.

aj_asver: that bonai tree is something my kids would absolutely adore. So I, I had this idea, um, Linus, um, I would love to learn kind of how to do this. Um, and I have a concept in my head and I was wondering if we could try it out,

Linus Ekenstam: Yeah. Let's, let's,

aj_asver: if we, we can kind of bring it to life.

Linus Ekenstam: yeah, let's try

Star Wars Lego Spaceships

aj_asver: So the concept is I also have two kids and they're four and they are absolutely obsessed with Star Wars right

aj_asver: got their first two Star Wars Lego sets and now they want like everything in the collection.

aj_asver: And they went to the library recently and got a book, and the book just has all these Star Wars, um, Lego ships in it. And it made me think like that would be a cool, fun thing to do in Mid Journey is like create imaginary Star Wars kind of spaceships. And so I was just wondering how would I approach that?

aj_asver: Um, do I just type in Star Wars spaceships made out of lego. Do I need to kind of think about how it's shot? Do I need to think about kind of the features? And so that's my idea. Star Wars Lego spaceships. How do we turn that into a cool, mid journey image?

Linus Ekenstam: Okay, let, let's just start straight off with, with what you just said, like Lego , Lego Star Wars spaceship. We we're probably just gonna get something that's very similar to what what's already in, um, in Star Wars, but with some kind of reim to Lego. Mid journey is relatively good at like, creating Lego. So we're gonna have to figure that one out. Star Wars. Um, spaceship. actually we're gonna put,

aj_asver: It's already starting to generate some images and, and the cool thing about mid gen is it kind of shows you bit by bit as it's evolving, right?

aj_asver: They already look pretty, pretty cool from the, from the outset. Okay. So we got some Star Wars Lego images.

Linus Ekenstam: it doesn't re does it look Star Wars, though?

aj_asver: It, it kind of looks like, um, yeah, it, it doesn't look like a Star Wars ship, I would imagine existing in the Star Wars world, but it has some kind of Lego vibes about it.

Linus Ekenstam: Let's try to see if we can get an imperial. Maybe in a pure cruiser or something that's also a known set. Maybe there is something that we could go crazy about instead. So when we started with Spaceship and we wanted to be Nubian fighter, um, maybe we want it to be like silver with, um, loose stripes.

Linus Ekenstam: Want the side shots. See if we can get something there.

Linus Ekenstam: So

aj_asver: you just typed

Linus Ekenstam: So,

aj_asver: in Star Wars spaceship and then Nubian fighters. You're trying to be a bit more specific about it. And then you also added some color hints as well, right? Silver with with white stripes and then Lego. That was an important part

Linus Ekenstam: Yeah, and I've also added side shot here to make sure that we get the, the, the, the model from the side. I'm not sure we will actually get it, uh, the way we want here. And I'm also not sure a Nubian fighter is, well this, this was the first one, like the imperial, some imperial ship here still.

aj_asver: to look a bit more, a bit more like a

Linus Ekenstam: Yeah,

aj_asver: I think.

Linus Ekenstam: it looks like Star Wars Lego, but it's still, I don't know, mid Journey is doing some weird things here with like, I think it's trying, oh, okay. Now, now let's,

aj_asver: ones, when you described a specific type of, um, ship, it looks a bit closer. So, I mean, another one we could try is like, you know, a tie fighter or an or an X-wing. Oh,

Linus Ekenstam: yeah.

aj_asver: looks a lot more like Lego right now. So

Linus Ekenstam: Yeah.

aj_asver: see something come to shape. It looks a lot more like a Lego we might have.

aj_asver: So maybe we could try like X X-wing fighter or tie fighter.

Linus Ekenstam: Let's try with X-wing. X-wing, and we want it silver with orange stripes. Maybe we want it like in space. See, I think the, the thing thing that I, the, the, what I'm kind of doing when I'm promming is like, I'm really kind of playful with it. I don't really mind if it takes me a hundred shots to get something.

Linus Ekenstam: Uh, sometimes it's a bit frustrating because like you think you, you kind of get down into this rabbit hole and you just like, I'm, I'm doing the right thing, you know, I'm writing these things, why it's not giving me what I want. Um, but then either just like remove completely and you start over and you do something different.

Linus Ekenstam: you just like try a different image and then come back to it because like maybe you have some other things that you want to try out. Um, but I think this, this might turn out great actually. Then, then again, the, the X the X-wing is a known object, right? So I would be surprised if, if we wouldn't get anything here.

Linus Ekenstam: I think where mid journey might China is like trying to combine, um, things. But if we want something that looks outta Star Wars, um, especially in Lego, there we go. This doesn't look.

aj_asver: This looks like it could be a real Lego set now. So we've got like a couple of different riffs on X-Wing. They have like really big engines, which is, which is really cool, and lots of lasers, which the kids absolutely

Linus Ekenstam: Yeah.

aj_asver: And you just clicked U2 there. Now what that does is that upscales, the second image, right when you hit U2 and

Linus Ekenstam: Yeah, so I, I just wanted to see kind of like what we would get if we, if we kind of like tried to get this in a slightly larger, uh, resolution. So these images are relatively low rest, I think when you're doing 69. Let, I'm doing now aspect ratio 69. We're gonna see, like, the image is 600, um, 640 pixels white.

Linus Ekenstam: so, And what I did here now is like when, when you have a shot, uh, a co, a collection of images that you get back, if you react with the envelope emoji, you get back the, the job id, um, the seed number and the four individual images.

Linus Ekenstam: So singular images, if you. If you want to save them low rest, and if you're doing like image prompting where, where you kind of put images into the prompt as well to give some, like, to give the prompt some inspiration. This is a neat way to like make sure you're using low risk images instead of like pushing the highest definition images into, into the image prompt.

Linus Ekenstam: Cuz that could slow down my journey quite a lot.

Linus Ekenstam: Um,

aj_asver: that gave you four individual images. Instead of

Linus Ekenstam: yeah.

aj_asver: one image, which

Linus Ekenstam: Yeah.

aj_asver: it's kind of four, the default is one image of four things in a grid. Right. And instead it generated four different images. And to do that, you clicked on the envelope emoji. So that is like a super, um, that's like a really great Easter egg for people to know.

aj_asver: Cause I would not have known that if, if you hadn't told me. So, envelope emoji. Gives you the original images, um, in low res, which you can then use to feed back into mid journey.

Linus Ekenstam: And, and again, this, yeah, this is impressive cuz if we consider what's happening here, it's like the, the model is interpreting our Lego as like actually building something in Lego. It it, it kind of tries to do that and emulate that. And if we look at the lighting here, the reflection in the cockpits, we can see that it's like shot with some kind of studio lighting over overhead lighting.

Linus Ekenstam: Um, ah, look at this. This turned out great. Wow. I'm, I'm surprised myself, so

Creating a scene in Lego

aj_asver: is so cool. And, um, I have one more challenge for you. This one's a little bit harder. We'll see if we can make it happen. So I

aj_asver: I was lego.

aj_asver: okay, this is cool, but it's even more fun if you can create a scene, right. With a few different characters in it and the Lego, uh, the, and the Lego, um, figurines too.

aj_asver: So I was wondering if we could give that a try. I have this, um, I have this fun idea for a Twitter thread where you recreate scenes from famous TV shows in movies in Lego in Mid journey. So maybe we can start with like a Star Wars one or we can, we, we can do a different one. Um, but I think that could be an interesting one to try too, because one of the things that I think is a little bit harder is when you have characters or multiple characters and you're trying to get, get, get a scene going.

aj_asver: So how would you approach that?

Linus Ekenstam: Actually I, I think this is an interesting one cuz like, okay, let's do, we have a scene in mind from Star Wars that we would like to try to reenact? Um.

Linus Ekenstam: So what I would, I would, what I would then do is like, I would pro, I probably do something like this where I would go and look for, For source material, like what is it that I'm trying to create, like to get an idea for how it looks, right? So, uh, I think this one is relatively cool. Um, I'm not sure we could actually do this, but this could be an interesting experiment.

Linus Ekenstam: So let's copy this image address and let's try to use this as an image prompt. So we go imagine, and then we paste the image url and then we say,

aj_asver: so you're pasting URL of a Star Wars scene that you found on Google Image search. Right.

Linus Ekenstam: Yeah, so I don't know her name here, but this is Finn, right? And what's her name? This is from the later Star Wars. So we go Finn

aj_asver: should know this cause we talk about Star Wars characters all the time.

Linus Ekenstam: star Wars Finn Running, and then it's BBB eight rolling, uh, sand Dunes and uh, Lego. We want to do aspect ratio 69. So we have the image and what I'm doing now, I'm basically just describing the image that we're looking at, um, and. Adding that as the prompt and adding Lego. We could actually just make sure we weight this, so we go Lego important.

Linus Ekenstam: Uh, and let's see. So here is a bit of like, if you listen in on the mid journey, um, all hands or in the way they call 'em community chat or, or town halls. Um, you'll hear, uh, the founder speaking a lot and the team speaking a lot about how the next evolution of of prompt. Probably gonna be image to image or like, people will do a lot of stuff using an image, um, or images, multiple images.

Linus Ekenstam: And I kind of agree because like, um, if I have a, if I have the ability to kite, either just take a, a snapshot of something or I grab something on the internet and I can like, take that and mesh it with something else and then put my prompt on it, the, the likelihood of me getting what I want is like 10 times higher

Linus Ekenstam: than if I'm just like, uh, writing.

Linus Ekenstam: Prompts

Linus Ekenstam: So this did not turn into Lego at all. So let's skip the idea of using the image prompt. And let's try something else. So Lego Star Wars is known, right? There is,

aj_asver: Mm-hmm.

Linus Ekenstam: uh, Lego Star Wars. So what are we trying to do? We're trying to do a cinematic shot, maybe, um, what's gonna happen in there? Who we're gonna have, we're gonna have some new. Storm troopers maybe Darth. Sorry, I'm going a bit slow here. I'm just thinking, um, we want to, one, we want to talk about lighting perhaps.

Linus Ekenstam: If we try, try this, sorry. Cinematic Darth Vader, indoors, --ar 6:9.

aj_asver: I noticed by the way, that you put Lego Star Wars Co on at the beginning. Is that something you can do to kind of set the, the scene

aj_asver: of

Linus Ekenstam: Yeah, I act, Actually I should have done this. Select do a multi prompt. So we're deciding that like the first part of the prompt is Lego Star Wars. Like that's like just pure definition. And then we want the cinematic shot with Stormtrooper Star Vader indoors.

Linus Ekenstam: Actually, this is probably gonna give us something that's relatively good,

Linus Ekenstam: So it interpreted this, well, I don't, I don't think it turned it into a multi promptt, but it worked anyway, so yeah. Here we go. We have Lego

aj_asver: start to see an actual cinematic scene.

aj_asver: Okay. So now we, you know, did a few different iterations and by adding Lego Star Wars at the beginning of the prompt, we actually got to a scene or a few different scenes where we have storm troopers, we have Darth Vader, we even have a lightsaber.

aj_asver: And the background actually looks like it's Lego too. So this is really, really cool. I feel like, um, we made a lot of progress on here and this is giving me a ton of inspiration. I'm gonna go off and make my own Lego scenes after I, after I saw this now, and I feel like I've learned a few tips and tricks from it too.

aj_asver: Thanks, lightness. This is really, really cool.

linus_ekenstam-1: Yeah, you're welcome. Uh, I, I'm, I'm excited too. I'm a big Lego fan actually, so, uh, yeah. I should, I should, I should actually get, I should actually do some things for this and post to some of the Lego

What Linus is most excited about in AI

aj_asver: Yeah, we have to do

aj_asver: it. Alright.

aj_asver: well, I feel like this is setting me off on a really awesome path, and I'm really excited to explore this further. Um, I've learned a ton from just going through this with you, so thank you so much, Linus. Um, before we wrap up, I'm curious, like what are some of the things you're most excited about, um, in AI and especially in general AI right now?

Linus Ekenstam: I, I think in general, I'm just like really excited about the possibilities of all these tools, to be honest. Like these tools are, are accessible to pretty much anyone. Anyone that has a smartphone or anyone that has like a a a computer with a web browser doesn't really have to be a good computer.

Linus Ekenstam: Cuz all these tools are running in the browser, right? Like the gps are in the cloud. Uh, it doesn't matter if it's ChatGPT or if it's MidJourney or something else. Like it's enabling anyone, uh, to, to be creative. Like I don't really, I don't need to know anything about like drawing or, or, or, or being an artist.

Linus Ekenstam: Right. I can just, if I have a good enough imagination or if I, and everyone is imaginative, so.

Linus Ekenstam: Pretty much everyone fits that. Um, so I, I think I'm, I'm most excited about that, that like, there is no real barrier here. Like we, we could like go out and just tell anyone to go try this and, and anyone could.

Linus Ekenstam: Right? I think the biggest boundary or the biggest barrier has been, uh, Actually the interface. So the fact that Mid Journey decided to be like, we're gonna do this in Discord, um, a few months back on, on, you know, the, the all hands, like there was a lot of people complaining like, oh, you know, I love Mid Journey, but it was so difficult to learn Discord.

Linus Ekenstam: And I'm like, wait a minute, why don't you just have a wonderful website? Why don't you just put the UI there, like the generative part. There could, could be easily done. Like look@lexi.art, amazing platform, super simple, prompt books on the website. Um, yeah, so I mean, there, there are few things that we could, that need solving, but like the fact that this is available for everyone, I think I'm most excited about.

Linus Ekenstam: And then when it comes to new tools, like it's really hard to keep up. Um, there's basically like a ton of new tools every day getting released. It feels like, you know, we're kind of in a. Hype cycle. A lot of people are getting their hands dirty. There is a lot of really good ideas and a lot of people are executing really fast.

Linus Ekenstam: I think ElevenLabs is sitting on some gold. They kind of like made it super simple to clone a voice and then used that voice by just inputting text.

Linus Ekenstam: So for example, if you know me, I'm building a, a storybook, um, generator for, for kids stories and, and we want to be able to cl clone parents' voices. So it's super easy for us to just like, Hey, record 60 seconds of your voice and then you can have any of the stories that you created written, like read back by you to your kids.

Linus Ekenstam: So I think that's, I'm really excited about that. And then I'm really excited about. A lot of these platforms opening up their APIs to developers. So, uh, we saw yesterday, the day before yesterday, like, you know, open AI released chat, G P T API and whisper api, um, through the world for everyone to use. And I think super excited about that as well.

Linus Ekenstam: So, yeah. Um, there there's not much underground, well, there is a few, you know, things that are popping up on the radar, but I don't think any, anything that's as exciting as the big things the macro.

Linus's Newsletter

aj_asver: Yeah, there is, so much happening in the space and folks can there is actually um, newsletter to, to get updates on, on kind of your, your take on the

Linus Ekenstam: Yeah. Yeah.

aj_asver: what's the name of the newsletter?

Linus Ekenstam: So the name is my name. So it's linusekenstam.substack.com. Uh, and actually the name of the name of the newsletter is inside my head. Um, so maybe I should change the URL to be inside my head. Uh, but yeah, it's linus eam.ck.com and, uh,

Linus Ekenstam: Yeah.

aj_asver: linusekenstam.substack.com Thank you so much Linus, for joining me, um, and helping me become a pro at Mid Journey. And also I feel like I learned a lot about kind of your, your perspectives on AI and generative AI and how it's gonna impact the design, industry too. So I'm really appreciate it.

aj_asver: Thank you so much. And until the next episode of Hitchhiker's Guide to AI. Thank you everyone for joining us.

Linus Ekenstam: Thank you. Thanks for having.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit guidetoai.parcha.com