Developer Relations and SQL Mesh in Handling Data Assets

Join Brian and Marisa Smith as they dive into the world of developer advocacy, the challenges of agile methodologies in data engineering, and the vital role of open-source communities. Discover how to better support and communicate with your developers in this insightful episode!

Overview

In this episode, Brian Milner interviews developer relations expert Marisa Smith to explore the vital role of developer advocates in bridging the gap between companies and their users.

Marisa shares her insights on the challenges of communicating with developers, emphasizing the need to create a welcoming environment for questions and feedback. She also discusses the unique difficulties developers face when implementing agile methodologies, particularly in the realm of data engineering.

They highlight the significance of open-source communities in fostering innovation and collaboration and provide a preview of Marisa's upcoming talk at Agile 2024 on enhancing data pipelines with SQLMesh.

Listen Now to Discover:

[1:08] - Join Brian in an engaging conversation with Dr. Marisa Smith, PhD, Developer Relations Expert, Developer Advocate, and Speaker.
[2:43] - Marisa Smith sheds light on the crucial role of a developer advocate, explaining how they bridge the gap between developers and the wider community.
[3:49] - Brian digs into common mistakes in how we communicate with developers and poses the question: what are we getting wrong in our interactions?
[5:57] - Marisa outlines the hurdles developers face in a Scrum team environment, shedding light on common obstacles.
[12:00] - Marisa explores the hurdles in developer communication, offering insights into improving dialogue and understanding.
[12:55] - Mountain Goat Software offers Working on a Scrum Team, a private class to help Scrum teams foster a team dynamic that supports the whole team, including bridging the gap in communicating with developer teams.
[15:00] - Marisa discusses how SQLMesh has empowered data engineers to streamline their tasks, sparking a sense of 'Marie Kondoing' their work.
[24:11] - Marisa emphasizes the vital importance of open-source developer communities for fostering innovation and teamwork.
[26:51] - Brian shares a big thank you to Marisa for joining him on the show.
[27:50] - We invite you to subscribe to the Agile Mentors Podcast. Do you have feedback or a great idea for an episode of the show? Great! Just send us an email.
[27:54] - If you’d like to continue this discussion, join the Agile Mentors Community. You get a year of free membership into that site by taking any class with Mountain Goat Software, such as CSM or CSPO. We'd love to see you in one of Mountain Goat Software's classes, you can find the schedule here.

References and resources mentioned in the show:

Dr. Marisa Smith, PhD
Join the SQLMesh Community
Agile 2024
SQLMesh
Working on a Scrum Team
Subscribe to the Agile Mentors Podcast
Mountain Goat Software’s Private Training
Certified ScrumMaster® Training and Scrum Certification
Certified Scrum Product Owner® Training
Mountain Goat Software Certified Scrum and Agile Training Schedule
Join the Agile Mentors Community

Want to get involved?

This show is designed for you, and we’d love your input. 

• Enjoyed what you heard today? Please leave a rating and a review. It really helps, and we read every single one.
• Got an Agile subject you’d like us to discuss or a question that needs an answer? Share your thoughts with us at podcast@mountaingoatsoftware.com

This episode’s presenters are:

Brian Milner is SVP of coaching and training at Mountain Goat Software. He's passionate about making a difference in people's day-to-day work, influenced by his own experience of transitioning to Scrum and seeing improvements in work/life balance, honesty, respect, and the quality of work.

Marisa Smith is a Developer Relations expert who bridges the gap between the community and development teams, addressing problems and promoting open-source software. With a Ph.D. in Computational & Theoretical Physical Chemistry, she has a background in simulating radiation effects in water.

Auto-generated Transcript:

Brian (00:00)
Welcome in Agile Mentors. We're here for another episode of the Agile Mentors podcast. I'm with you as always, Brian Milner. And today I have the one, the only Marisa Smith with us. Welcome in Marisa.

Marisa (00:13)
Hi, thank you so much for having me.

Brian (00:15)
Very excited to have Marisa with us. If you're not familiar with Marisa, her title is Developer Relations Expert. So right there, that's an episode, right? We could talk just about that. And we'll get into that a little bit more, but there's a lot of really interesting stuff here about Marisa. She has her PhD in theoretical and computational physical chemistry. So...

Marisa (00:41)
Yeah.

Brian (00:42)
Again, wow, right? I mean, this is amazing stuff. She's worked at Streamlet. She was their very first developer advocate there. And she has since, Streamlet's been acquired by Snowflake. And you founded Tobacco Data, is that right?

Marisa (01:07)
Uh, no, I, um, I am their first developer advocate at Tupiqium data. Yeah. No words.

Brian (01:11)
OK, gotcha. Sorry about that. Messed that up. So very, very interesting background. And one of the things that caught our notice, Marisa spoke last year at Agile 2023 and is speaking again this year at Agile 2024. So again, if you're going to come out, I highly recommend you attend her talk. Her talk is called Marie Kondo. your data pipelines with SQLMesh, which I think is really, really interesting. But I'm talking too much, and I want to turn it over to Marisa here. Help us understand developer relations expert and developer advocate. What does that mean?

Marisa (01:59)
Yeah, so I am, what I always say is that I am the person that connects your company to the people who use your product. And it just so happens that the companies that I work for are companies that work in the tech industry. They're building some sort of piece of the tech stack. So the people that use it, their customers are other developer, developers essentially, or technical people.

Brian (02:22)
Yeah, so you're an expert in the...

Marisa (02:27)
in the art of, in the art of like, how do we communicate with other developers? How do we pass that information back and forth between the developers that are making a product and the developers that use a product. And how do we make sure that, you know, we're getting, we're, we're getting the best out of our, out of our users and that they're getting the best out of the technology that we're trying to build for them.

Brian (02:49)
That is so, so interesting. And so I'm sure product owners are listening going, yeah, help me. Help me. I want to understand. How do I talk to developers? So gosh, there's so many directions we can go with this. What do you think people misunderstand most when they try to communicate with developers? What do we get wrong?

Marisa (02:55)
hahahaha Oh, wow, that's a great question. Let me think about this for a second. I think, I think from, from, from my perspective, as somebody who spends a lot of time, like running different communities, especially open source communities, I think that people get the wrong idea in that. Yes, these developers are your customer, but a lot of the time they have very limited time. They come in, they look at, you know, maybe your open source product or, uh, you know, your free version or something that they're trying to see if they can integrate this in their own stack. And I think people can. or companies can come at them a little bit too quickly, a little bit too salesy, right? And then that ends up driving them away. They're like, no, no, no, I'm really not interested in any of this. I'm just trying to figure out if this is the right technology. A lot of developers like to iterate. They try things out, right? And so I think if you come at them too early with, oh, here's our sales process. Here's this, this is how much this costs. It's like, no, no, no, I'm like... way early, I'm MVP POC type of thing, trying to see if I can understand, or if this works with my team, my workflow, my current pipeline, the other technologies that we use, you know, that, that type of thing. I think that's one of the biggest mistakes that you can make, especially when you're talking about open source, which is kind of like my bread and butter.

Brian (04:28)
Right, right. Yeah. And you know, the communication, especially, I mean, because we talk about Scrum and Agile here on the podcast, you know, that relationship between the business side of the house and the developer side of the house, it's almost like, you know, Romeo and Juliet and the two houses, you know, they speak different languages. They want different things. They see the world in a very different way.

Marisa (04:34)
Mm -hmm. Mm -hmm.

Brian (04:58)
And yet somehow these two groups have to figure out how are we going to work together to really deliver something that is valuable, right? So you work with a lot of developers. You talk with a lot of developers. And I know there's lots of different kind of practices and things that are out there that developers are using these days.

Marisa (05:09)
Yes. Yeah.

Brian (05:26)
When you talk to them about something like Scrum, or when that kind of process comes up, what are some of the chief complaints that you hear from developers when they talk about working on a Scrum team? What are they not like about it?

Marisa (05:44)
Ooh, interesting. Yeah, I would say. I would say, I mean, in the area that I'm working in right now, I'm working pretty deep in the data pipelines. So Tobico data runs these two open source projects, SQLMesh and SQL glot. And it's essentially your T of your ETL pipeline, right? We're using SQL, we understand SQL and we're transforming your SQL queries into these tables and we're helping you manage these pipelines as they evolve over time. And so, um, and so I think, you know, in this space, what happens is when you're talking about, you know, working with scrum teams and stuff like that, one of the pieces of agile is trying out new things and iterating. And that can actually be super difficult for a lot of like these data engineers and developers to do, because you have accountability at the end of the day, right? If you're changing up your data, you're mutating some, some SQL query that changes some model that changes your data pipeline downstream. And then all of a sudden. you know, you've pushed it to production and, uh -oh, this data is not what you expected. It's not what you had like originally tested for. And that's because, you know, teams have to work across many different, um, they have many different like iterations that they're working on and many different teams are, you know, making changes potentially simultaneously. And so I think that for, for developers, when you talk about scrum and you talk about agile, particularly if you're talking about like, adjusting tooling or trying out something new again, coming back to this fact that like, you know, they're just there to test things out. They have very limited time and that's because they get stressed out. It's like, well, I don't want to break production. We have to protect production, your production environment as much as possible. And, you know, part of agile and part of doing all these things is trying, trying these new tools, trying these new companies, trying these new methods. And you can get a lot of resistance from that because. they know this method works, they know that they have accountability and they know that production will be fine if they use this methodology that they've set up. Sometimes it can be a little bit of a matchstick thing in the back end.

Brian (07:47)
Yeah. Well, and I mean, just hearing you talk about that and thinking about it, I know one of the big friction points between the business side and the developer side is take SQLMesh. If that's something that my team has never worked with and I have someone on the team who is interested in that and wants to, thinks it might give us some benefits, There's friction there with the product side to say, product has all these things they want, and they want to push these things forward. But I think this bit of adding this to our stack is going to improve things. But how do I communicate that to the business to give us the time to do this? Because it's not directly leading to a feature, but it will improve things moving on. How do you? How do you balance it? How do you have that conversation with a product person or the business side of the house to really say, Hey, this is worthwhile.

Marisa (08:50)
Yeah. Yeah. Honestly, it's super difficult. And I think it's really a balance of, you know, having to have these engineers dedicate some time to new tooling and testing things out. And then once you have done that in their schedule, you know, I don't know how frequently you may want to do it like once a month or something like that, where they, you know, take a day and just review what new is out there. What else should we be looking at? What other tools, what other... You know, with, especially with the emergence of AI and all of that recently, that changes a mile a minute, I think. So, you know, keeping on top of that is, is, is a huge burden on these engineering teams. And so time needs to be dedicated for actually getting that kind of stuff done and the freedom to actually try these things out and do like just a minimum viable product, right? Just a tiny POC with like a little Tinkit data set. And then on, I think there's some.

Brian (09:23)
Ha ha.

Marisa (09:45)
there's some weight of this on the company that's developing these open source products or new tooling, that they have to start communicating and figuring out their business value as well. Because those developers cannot, with this little example, show all of the benefits that you would get. What cost savings do you get? What efficiency savings? What increase in productivity? All of that has to be done by the company and you need to have that ready. so that you can back up your developers that are trying out your product and your open source projects so that they have something to go off of. They're like, hey, look, I made this. It took one week or something like that. And I got it to a place where it's really good and look at all these cool features. And this will make us so much faster in our pipeline. And then here is the company's documentation or case study or whatever it is that says this should increase our productivity approximately this much. And... that these other big companies that are similar to us have this sort of success with it. And, you know, it, I think those two combined can really help alleviate these pressure that the developers feel and give them the time to actually try out new tools, which in many cases they love to do. They love to try new things. They love open source, right. And they will be your best advocates. If you spend the time talking to them, communicating with them and giving them the things that they need to be successful.

Brian (11:12)
Yeah, I would think that's kind of a, there's a duality to, I would imagine your role in speaking with them about your product, because in one case you have to sell them on, hey, look how beneficial this is. This is, you should add this to your stack. But on the other hand, you've got to equip them with the, the, uh, the sales pitch, I guess, that they can then make to their business to say, hey, you should allow us to do this. You should give us the, whether it's finances to do this or the time or the resources to do this, that, you know, it does benefit you as well. So that's gotta be a really difficult part of that communication is kind of, you know, getting people who are not really salespeople, you know, having been a developer, I know I kind of get, you know, the personality type. Uh, and you know, we, we don't want to have to talk to people. We want to be able to put our headphones on and get our work done. Uh, but now, now I'm in the position where if I want to do this thing, I know it is the right thing to do. I've got to convince others and that's not really my strong point. So how do you, how do you help them with that?

Marisa (12:24)
Mm -hmm. Yeah. Yeah, it's definitely a long road. I would say, you know, it's not done in a split second, especially when you're talking about larger companies, like any sort of fan company. They will take a lot of time to make this decision. So you have to be really committed, I think, to each person that you're talking to and each person that you're trying to help get moving with your product to really make them successful. And... For us, what we've been doing recently is we go in and we help them with that communication point, right? Like our developers know our tool the best. And so if there needs be, like we'll stop and we will actually go in and do a presentation to the wider team and be like, okay, you guys have set up this POC. You've tried it out. You really like it. Here are the benefits. Here are, you know, here's how we describe it. Here's how, you know, We have seen other companies succeed with this and we have some decks that are basically ready to go. So we can go in and actually help them with that communication stage as well.

Brian (13:34)
Yeah, that's awesome. Well, then let's, because this is fascinating, and I really enjoy kind of the idea of trying to be an advocate for the developers. But I'm curious as well, with your upcoming talk at Agile 2024, by the way, just I don't think I've said the date, but July 22nd through 26th in Dallas, Texas, go to. AgileAlliance .org. You can find out more information about it. I think I've told everyone here on the podcast, I'm speaking there as well. So come and see both of us in my hometown in Dallas. So Marie Kondo, your data pipelines with SQLMesh. Tell us what was the idea behind this, where you got the idea for this talk, and what it is you're trying to get across in it.

Marisa (14:18)
Thank you. Yeah, absolutely. Well, here's the story. One of our users, I was doing case studies for our, for us, because we need to understand our business value. We need to show people that like they can get this, these, you know, these cost savings, these productivity savings and things like that. So I've been doing interviews with some of the companies that currently use SQLMesh. And one of the, one of the interviewees, we were just chatting and he was like, oh, You know, he brought Marie Kondo up and he was like, yeah, like SQLMesh just brings me joy. It brings joy back to my data engineering. And I'm like, well, wait a minute. What, wait a minute. What do you mean? mean? like, oh yeah. Well, I mean, you know, we spend all of this time. Fretting Fretting about these data pipelines, getting the correct data down the pipeline, getting the business needs on the timeline that they need them, you know, updating your production value or your production environments with, with anything new that's been requested. And.

Brian (15:01)
you

Marisa (15:21)
there isn't really a proper process for data deployments, right? Like for code, you know, we have GitHub, we have Git, we have all these things, right? You make some changes, you save it, you test it out, you make some more changes, you save that, you test that out, right? Like all of these iterations, they create all these versions and these checkpoints. But on the data side of that, there is nothing. It's just like match disks and toothpicks that hold up some of these.

Brian (15:30)
Yeah. Hahaha.

Marisa (15:49)
some of these pipelines, right? And that's become the standard. I'm not saying that this is particular companies that are doing this or something like that. It's pretty much everybody. And all of this falls on top of your DevOps engineers or your data engineers that are a very small percentage of your company. And they're treading through this escalating technical debt, right? Every time you add a new table, every time you add a new dashboard, that's a new backend that they are managing, right? It becomes very stressful for them. And... This individual was saying that he had lost a lot of the joy out of his work and you spend how many hours a day working, right? This is a huge chunk of your life that it's just like, oh my God, I don't want to do this anymore. I don't want to make that change because the pipeline currently works. It's not broken. It's not broken. Don't change that type of thing, right? But this isn't what business is. This isn't what agile is. You're supposed to iterate. You're supposed to make these changes. You're supposed to investigate.

Brian (16:24)
Yeah. Right.

Marisa (16:42)
find new things and find new insights. And you can't do that unless you start changing things. And so he had said that once they started implementing SQLMesh with the different features that it has with this virtual data environments and these data versionings, and we have these data contracts that essentially allow you to turn, have checkpoints for your data and have essentially unit tests for your data. He was like, oh my gosh, now I can not have to worry about it. I can just try something, see if it works. And then if it doesn't, it doesn't matter because I can roll it back super easily and things like that. So that's really where the inspiration came from.

Brian (17:21)
That's awesome. Yeah, that's such a huge hole and it's such a needed kind of component to the stack, as you said, because, you know, I mean, you're right in kind of a programmer world. And, you know, if you're outside of the database world, there's all these tools you can use and put in place and test and see how things are going. But that fear that you have when you work in the database world where if I make the wrong error here, It could mess up all our data. It's not just that it's going to present it in the wrong way, but it could actually damage, which is a valuable asset. It's a hugely valuable asset, the data that the company has, maybe one of their most valuable assets. So yeah, that's an amazing tool. And...

Marisa (18:10)
Yeah. Cool. And these days, yeah.

Brian (18:16)
So this is also an open source project, right? So tell me how that's been interacting with the community on this and working in a corporate environment, but also it's an open source environment on this product that you're a developer advocate for.

Marisa (18:19)
Mm -hmm. Yep. Yeah. Well, I love it. I love open source. As I mentioned before, open source is kind of my bread and butter because Streamlit, you know, was essentially the other end. I've gone from the dashboard side, which Streamlit is basically a free open source dashboarding tool that's built just in Python, to the side of the tables that make that, and the SQL that makes those dashboards possible in the backend. And so this, it's something that I really love about my job because... I've been lucky enough to work with a bunch of tools that are super useful and that people really love, uh, and that they have that they, you know, people who co -founded it, that there are co -founders all came from huge fang companies, right? SQLMesh was basically born out of these data problems specifically to solve them because they had been literally experiencing them at these companies themselves. And they, you know, went out and were like, okay, what's the solution out there? And, you know, uh,

Brian (19:21)
Yeah.

Marisa (19:31)
There's this, there's another company called dbt. They were pretty much the first one on the scene. They've been around for like, I want to say like 10 or 15 years and they changed the space. Like they really did make some huge advancements. But the thing is, is they came at it from a completely different angle than we're trying to come at it because they came at it from the almost like the data science and data analytics side. And they weren't necessarily thinking about future, how big these data sets were going to get with.

Brian (19:52)
Yeah.

Marisa (19:58)
these different like Netflix, Airbnb and stuff like that, right? Their data sets are huge. They're parsing huge amounts of data. And, you know, in the current tools, the systems that you have, you have to refresh your entire data warehouse every time you want to make a change to production. If you have a terabyte worth of data that you're trying to refresh every single time you make a change, that literally, you're just, you're twiddling your thumbs. Your analysts are just like sitting around waiting anxiously to find out.

Brian (20:03)
Yeah.

Marisa (20:27)
you know, if those changes they just made to the sequel is actually viable and good, right? So, sorry, I think I lost. I think they lost the train of the question I got all passionate about.

Brian (20:32)
Yeah. No, no, no, that's fine. That's fine. No, I mean, I was just, I was asking about, you know, kind of the interactivity with the community on that and working with, you know, these open source projects, you know, you have volunteers, you have people who are giving up their time for free, basically to improve your product. That's got to come with a whole just mess of other considerations and concerns and how has that been?

Marisa (21:07)
Yeah, yeah, yeah. So that's been good. And where I had been going with that point was that I've been lucky enough to work with companies and tools that are super useful, that were developed out of pain points that other people have experienced. So that side of things has been really great. When you're trying to develop a community, I just try to make the most welcoming space so that people feel comfortable in asking any sorts of questions. Right? Because that is the only way that you kind of surface some of these things that might've been otherwise hiding. Right? If people are nervous to ask questions, then you're not going to really have a proper conversation around, Oh, well, wait a minute. How did you get to that point? Like what led you to use it this way or to do it this way or something like that. Um, and so, yeah, there's lots of considerations, but we've been lucky enough that the, you know, a lot of our users are very happy with it, but also are.

Brian (21:38)
Yeah.

Marisa (22:01)
because of that, they're very vocal and they are very happy to, you know, take that five, 10 minutes, fill that survey in or meet with me and talk through, you know, a case study or like how they found SQLMesh and how it's going and stuff like that. So I think that the real balance comes in as to how much time do you want to dedicate my time? Do you want to dedicate to doing these interviews and getting this feedback versus Like, oh, we've already got like a pretty large signal that the next feature they need is this. Like, let's just run and do that feature and get that out for them, as opposed to continuing to bog up their time with interview questions or survey questions and bog up my time with it as well. So I think that's more the balancing point is when do you start acting on the signals that you're starting to see from your community versus like just constantly collecting data?

Brian (22:56)
Yeah. And I love your point there about kind of creating that welcoming environment. It's very similar to what we talk about with just teams of the whole psychological safety kind of aspect of it. And if I feel like I can't say something without being made fun of or feel like I'm made to look stupid for my question, then you're right. I don't surface the things that I should, even if it's, you know, just, hey, I'm not sure how to do this and getting help for that kind of thing.

Marisa (23:22)
Mm -hmm. Yeah. And I mean, I feel like with open source communities as well, they're all online. And this is a, I like that psychological safety and I like to try and promote this friendly environment because there's no guarantee that the person on the other end even speaks English. They might be running it through a translator, right? Like you have no idea. So they need to have like that safe place that they can just be like, oh, Hey, I tried this, but I couldn't get it to work. Right. And then that's when you can start to expand and be like, Oh, okay. Well, actually this person is from.

Brian (23:39)
Yep.

Marisa (23:54)
I don't know, like France, they speak French, right? So you're trying to translate kind of back and forth, either in your head or with a tool. And it's kind of like, okay, well, what else can I do to help them? Right? It's like, oh, well, what they need is a more in -depth, like getting started guide, right? We need to add these steps in or put this little note in there that's like, hey, if you get tripped up and you get this error, that's because of this and this is how you fix it type of thing, right? So that you just kind of like fill in those gaps of knowledge.

Brian (24:23)
Yeah. I mean, there's probably so much that we could extract just from the remoteness of the team that you work with, because open source is an area where there is no office. It's not like we all come into the same place. And yet, it works. So for people who think it can't work if you're remote, well, open source is proof that it can. Yeah.

Marisa (24:37)
Yeah. Yeah, absolutely. And not because of that, our team is a hundred percent remote as well. Right. Like our entire Tobacco team works async, right. And we're only able to do that because of different tools and processes that we have put in place and, and that we do have this community and our community is a, is a Slack community. We actually, we could put a link in or something so that people can check it out if they were interested. But, but yeah. And, and that community isn't just about.

Brian (25:11)
Sure, sure, sure.

Marisa (25:16)
that specific tool. And I think that's also another thing that you want to surface when you're talking to developers is that this isn't just a space to ask questions about SQLMesh or SQLGlot, it's a space to talk in general. What are some of the best practices around evolving your data pipelines? What, you know, do you have any questions about your DevOps? Like, you know, what, what are people using for their cloud service provider? Why? Like, you know, did you switch from this to this and, you know, What was your thoughts around that? And, you know, what do people do with a dataset that is, you know, 300 rows and, and like 700 columns, like, how do you deal with this size of data? How do you want to, like, how do others mutate it? Like, do you incrementally load it? So just like try and load in the newest data. Do you only re when do you refresh these, like all these questions and all of this conversation needs to happen because we need to start deciding on a proper process and. DBT had started doing that and we're trying to continue this conversation.

Brian (26:16)
Yeah, I would imagine that's working with the product as well, that that's very beneficial to even product development. Because you can hear from them, even if it's not part of your product, but if you hear those questions about, hey, well, how have you handled this, or what do you do in this kind of scenario, I can imagine that would lead you to maybe new product ideas based off of some of those conversations. Yeah.

Marisa (26:43)
Oh, absolutely. 100%. I don't have like a specific example at the top of my head, but I definitely...

Brian (26:48)
Right. No, no, no. Yeah. Yeah, it could come from this. So it's community, right? I mean, it's just the importance of having that community and not just being, no, we can only talk about this one product here, but no, we're a supportive community and we help each other here. Yeah. Awesome. Well, this is fascinating. And I could talk with you about this for a long, long time. I'm looking forward to your talk.

Marisa (26:55)
Mm -hmm. Mm -hmm. Yeah. Yeah.

Brian (27:16)
So I haven't checked the schedule yet, but I'm gonna, hopefully it's not on one of those times when like, normally, I don't know if you find this as well, but it's like the ones that I start and think, oh, that's the one I really wanna go to, turns out to be the one where you're speaking or somebody who's a lifelong friend is talking at that time. You're like, I can't miss that person's. So I'm hoping that's not the case, cause I really wanna come and hear this talk and hear more about it.

Marisa (27:17)
Great, thank you. Yeah.

Brian (27:45)
Thank you for giving us your time and helping us to understand this, Marisa. I really appreciate you coming on.

Marisa (27:49)
What? Yeah. Thank you so much for having me.