Speaker 3
I mean, I think that's part of why we sort of restated data engineering in terms of the state engineering lifecycle. It's what everyone has been doing for a long, long time, regardless of if it was like Oracle or DBI, BMDV2 or Hadoop, it's like you're going to ingest data, you're going to transform it, you're interested in applications downstream, you're interested in serving data. And so that there's such we expend so much energy talking about the latest and greatest tools in engineering disciplines that we often miss the point that the fundamentals actually haven't changed that rapidly. Like the fundamentals have really been this way since probably the 1980s, would you say Joe since or maybe 1990 or so since the Data Warehouse concept is really codified? It depends on how far you want to go back. I mean, it might have been earlier depending on what aspects of the fundamentals you want to talk about. I mean, I'm reading a lot of old books right now, not
Speaker 1
like Shakespeare or something, but like old data books. And it's interesting, a lot of the things that people like Cod and Peter Chan and other people talk about are it's like, it's still relevant today. They'll probably be relevant 20 years from now, right? And so it was interesting to kind of bring this home. I was talking to a person who's done the undergrad and information systems yesterday, and that person wants to become a data engineer now. They found our book and I thought it was very inspiring and had a good chat. And this person asked, okay, so what do you think I should focus on? Do you try to learn DBT and Snowflake? I was like, well, I mean, you could, I'm not going to stop you from doing that. But how are your coding chops, right? Like, you know, how good of a software engineer are you? Do you understand data modeling, both relational and dimensional modeling? Do you understand how databases work at a fundamental level? Do you understand like how networking works and security and all these other things? I said, you know, if I'm a near shoes, I would spend my time. You know, I probably figured these things out first, the fundamentals, right? And I also encourage them to become an awesome software engineer. Like if you can, if you can, like do software engineering really well, you got your pick of jobs coming out of school, right? Because you can, you know, you're really, if you really get it right in code, if you really got solving problems with code, like this is like the skill I went over index on because if you under sand data, what she does, it came from an analytics background, but if you also can like be a kick ass software engineer, you kind of have a lot of options at this point, right? And so, you know, that's what I encourage them to do. But notice I didn't tell them, oh, yeah, you should definitely go learn like DBT, Snowflake, like all these tools. I think that's this, once you understand Hall, kind of all the innards work, then you can start moving up. But Andy asked me about AWS too. And I was like, look, you can certainly go learn Redshift. If I am your shoes, I'm going to learn like how S3 works at like a very fundamental level, how VPC networking works, how IM works, you know, security permissions. I'm going to get into EC2 and like dork with that for a really long time and like break lots of stuff. Because if you can understand the building blocks, you know, especially these cloud infrastructures, you can kind of do whatever you want after that. If you're starting at the top and sort of working way down, especially in something like AWS, there's a lot of, I would say hidden complexity that you don't really realize until you start using it, you know, you're going to be kind of hosed, frankly. So, to learn the building blocks, you know, that's this people, people are too enamored by technologies, you know, and then I talked about a lot about this, but it's like, focus on the people in process parts. People part is like, you know, just learn how to learn how to figure out requirements for people, learn how to ask good questions. The process part is like learning stuff like modeling, you know, how do the underlying tools work, not the tools themselves, but what's the building blocks from there. You know, you'll have a lot easier time figuring out technologies and how to use them in the data engineering life cycle like Matt pointed
Speaker 2
that's a great point. So the fundamentals being, you know, starting with software engineering, but you know, one layer further going to higher level systems, like networking, EC2, you know, dimensional modeling versus relational modeling. And you know, once you do that, then the abstractions can carry over between any type of new shiny tool that enters the market with like their take on how you should do it. Like, because ultimately, you know, every unicorn data company, they're really good at making things seem shiny and new and late and great. But you know, you have to cut through the noise, right? And if you have the type of training that you mentioned in your book, then you're better armed to scrutinize which ones are really the best tool for your specific job, right? It's interesting
Speaker 1
because you know, the other great book in our field is designing data intensive applications by Martin Klezmann. I feel like that's sort of a sequel to Fundamentals of Data Engineering. So read our book first and read Martin's after that, and you'll just be a good, you'll be a rock star ninja data engineer.
Speaker 2
Exactly. And subscribe to what's new in data and not me. You know, I do really recommend, I've recommended the fundamentals of data engineering and a multiple of my posts. And I think it is one of those books that's super inspiring, you know, like I like to, you know, ever since I was a CS student, I always like reading stuff that actually made me inspired to learn and inspired to build more and apply certain frameworks. And I really think your book, the Fundamentals of Data Engineering fits into that category. The other passage, or I guess it was a chapter that I really liked was talking about building loosely coupled systems. Can we chat about that one a bit?