TWiGS host Chris Adams is joined once again by Gaël Duez to discuss the latest news in green software around AI. They discuss insights from recent reports by Google, Meta, and Amazon, as well as looking at the implementation of the GSF’s Software Carbon Intensity metric. Similarly, the conversation touches on the distribution of renewable energies and the use of different means of measuring carbon in reporting, and how this can affect the behavior of consumers and organizations alike. Tune in for an enlightening discussion on the latest in green software.

Learn more about our people:

Find out more about the GSF:

The Green Software Foundation Website
Sign up to the Green Software Foundation Newsletter

News:

Events:

Green IO London 2024 [1:01:15]
CNCF Cloud Native Sustainability Week 2024 [1:05:48]

Resources:

If you enjoyed this episode then please either:

Follow, rate, and review on Apple Podcasts
Follow and rate on Spotify
Watch our videos on The Green Software Foundation YouTube Channel!

Connect with us on Twitter, Github and LinkedIn!

TRANSCRIPT BELOW:

Gaël Duez: We cannot fully rely on companies agreeing on how they should measure their own environmental impact, even if they are well meaning, with tons of great people trying to do the right things, etc. It's not a black and white world out there, but there is a question at some point of financial pressure, shareholder pressures in many of these companies.

They're just stronger than the entire stakeholder's pressure.

Chris Adams: Hello, and welcome to Environment Variables, brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding green software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software. I'm your host, Chris Adams. Hello, and welcome to Environment Variables, where we bring you the latest news and updates from the world of sustainable software development. I'm your host, Chris Adams. Round about this time last year, Gaël Duez, the voice of the Green IO podcast, came on to Environment Variables, This Week in Green Software to talk tech, sustainability, and moving from France to live on La Reunion Island.

And the ups and downs of consulting remotely on digital sustainability from a small island off the coast of East Africa. It was a fun chat. And when I asked him if he'd be up for coming on again to review a few of the latest stories around green software, he basically said, "yes, Chris, but I can't do Friday because I promised my daughter we'd pop out to go whale watching." As one does on a tropical island, I guess. So here we are, recording on a Monday morning instead. Gaël, thanks so much for coming back on. Can I give you a bit of space to introduce yourself and what you're up to these days? Also, how are the whales?

Gaël Duez: Oh Thanks, Chris. It's great to be back again on Environment Variables. And the whales were there, like several made quite a show, including a mother and her little newborn. Well, little meaning four tons. So it's always impressive to see there's a 20 tons mammals jump in the air like this and cherry on top of the cake, actually, we saw a lot of dolphins and two turtles.

So it was a perfect trip.

Chris Adams: Wow, I'm jealous. Okay. Yeah?

Gaël Duez: Actually, to connect a bit more on our sustainability topic, I'm also relieved that the best practices for whale watching are more and more enforced, such as, minimum distance to approach them, turning off the engine when they come in, or direction, et cetera.

And, I also say, on top of the pleasure of, watching them and being on the boat, etc. It's a very positive sign that we can enjoy nature without destroying it. And that's pretty cool.

Chris Adams: Cool. So it sounds like there might be, hopefully, when your daughter's taking, maybe some of her kids for whale watching, there's a chance to continue that, by the sounds of things.

Gaël Duez: Yeah, I'd like to focus on whale and dolphins rather than coral reefs, which are something that really is puzzling me, but yeah, we might, hopefully. And to answer your other question, when I'm not whale watching in Reunion, I try to be useful in the tech community, by advocating for more sustainable ways of designing, coding, hosting, and even considering the use of technology itself. And my main tools remain the Green IO podcast, as you mentioned it, as well as the Green IO conferences, which I started to organize in several cities, but I'm sure we'll get back to this point later. And besides, as you already mentioned, besides as volunteering activities, I do public speaking and consulting on systemic strategy for mostly for tech companies, for both paying the bills, but also to have an impact.

Chris Adams: Cool. Thank you for that. I will also share a link to the previous episode if you're for people who are listening, so they can get an idea of some of the other things that Gaël has been working on and been discussing previously. If you're new to this podcast, I should probably introduce myself as well. As I mentioned, my name is Chris Adams. I am the executive director of the Green Web Foundation, which is a Dutch nonprofit based around reaching a fossil free internet by 2030. And I also work as one of the policy chairs in the green Software Foundation, particularly in the policy working group. Okay, and before we dive into this podcast fully, just a quick reminder, everything we refer to, every story, we'll share in the show notes and there will be a transcript as well for you to kind of search through and look into later. Okay, so as per usual with this week in Green Software, We run through some of the latest stories or projects that have caught our eyes over the last few weeks. And Gaël, I should probably ask you, are you sitting comfortably, ready to go?

Gaël Duez: Perfect. Everything is fine.

Chris Adams: Brilliant. Okay. Let's start with the first story then. So this is the first one that kind of came up on my radar. There's a website called Green Coding AI. And we've spoken about the environmental impact of AI on this show multiple times.

We've also spoken about there is different models that you can actually use to maybe ask the same question and get something back. This one is actually a project from, the Berlin based group, Green Coding Solutions. And what they've actually done is put together a service, running on their own hardware, where you can basically try various, models to prompt them to, like, try out, say, Llama 3, one of the big versions, maybe a smaller version of some of these to see, to ask a question and see what kind of responses get back. And one of the things that's particularly interesting in my point, from my perspective is A, the transparency, but also it gives you a software carbon intensity score for every single inference when you ask a question. So you can start comparing the utility of a large model versus a small model. Gaël, I think you might've had a chance to play with some of this already.

When you looked at it, what, were there any things that sprung to mind for you when you first had a bit of a kind of kick the tires and mess around with it at all?

Gaël Duez: Well, what I was pleasantly surprised with is, as you mentioned, the use of the software carbon intensity. I, as far as I know, this is the first time that I've seen it being used for this kind of tools. I know that what is very interesting is that we can see now, not blossoming yet, but several initiatives have popped up with that kind of approach and I need to shout out and at least I need to give a big kudos to Arne and his team because once again it's very thoroughly done for what they've done in Berlin.

What I also like is the work that has been done with another initiative, and I hope that they will start discussing, with each other, which is, it used to be a data for good, project, but now it's a, an, an association on their own called Gen AI Impact. Yeah. So the association is called Gen AI Impact and they created this tool Ecologits, which is kind of just kind of the same approach.

With a very strong focus on transparency, how they calculate everything and all the code is accessible, et cetera. And there is also obviously Code Carbon by Jürgen Fais, which was kind of the trailblazer, if I recall. And I think it says something positive about the trend in the AI industry, that they need to tool up to Assess more and more precisely what is the carbon impact and hopefully soon the water impact.

And that's still, I know that it's not feasible at the moment because of the lack of transparency around water, but we will discuss water later in this episode. I'm pretty sure regarding the other news that, that we have, but it, is very important. I cannot count how many times I've got potential customers or heads of IT telling me what are the tools we can use to measure the footprint of AI.

And Also to use them to push back a little on the AI hype. everything should be AI'ed. I don't know about it. With a pinch of AI everywhere, but it comes with a cost. Now, going back to your question. So I really love the multiplicity of tools that we see at the moment. And now going back to more specifically in Green Coding AI, I think the way they, try to cover all the angles when it comes to energy, having both the CPU joules, the GPU joules, but also the temperature.

It's very interesting because the temperature, it connects a lot with a discussion I had with Professor Lee. One of the big experts on water cooling, especially in data center in tropical island or in tropical climate, sorry, not island. I'm the one on a tropical island, but there are a lot of tropical climate without tropical island, starting with Virginia in US that we tend to forget.

And he was telling me how important it is to understand that there are thresholds when you use a chip. And there's a threshold that impact massively the energy consumption. So it's not the same to have a chip being used around 25 degrees Celsius, for instance, and 30 degrees Celsius. It's just a linear progression in energy consumption versus, temperature.

And I was really delighted to see the temperature being put as a metrics, because that's where we are getting more and more in details and understanding that it's not that easy, it's not that linear, and we need to investigate things in a more systemic view. And the temperature for many people operating data centers, is absolutely pivotal when it comes to anticipating energy consumption.

So I think, yeah, I was very pleased with this CPU temperature metric that they added. I have no idea how they made to calculate it. So that's something I'm going to investigate, but I like the idea of putting this kind of metrics on the table as well.

Chris Adams: So, thank you for that, Gaël. So, what I can share with you, there's a chap, one of the chaps working at Green Coding Solutions, Didi Hoffman, was one of the people doing a significant amount of the work on this, and I've used some of their tooling, they've made open source before, and they basically pull some of the temperature figures from the, from basically the CPU itself, I mean from the system that they use, because for example, most machines can tell you how hot or cold they are, for example.

And that's for pulling some of those numbers actually from. The thing that I might share with you that I think is interesting here is, normally, Green Coding Solutions, they tend to open source as much of the work as possible. And they have one of the nicer tools out there for looking at the environmental impact of pretty much any kind of service, for example.

And the thing that So the, I did actually ask Diddy about this and said, "Diddy, this looks really cool. Are you planning on open sourcing?" He says, "yes. The only thing that has stopped me open sourcing it so far is tidying up because it is a bit of a mess." But the thing we can point you to in the meantime is a paper that he worked with, I think Professor Verena Majuntke from HTW Berlin. They submitted a paper specifically for, to Hot Carbon talking about, essentially being able to optimise different models for optimize the use of AI services specifically for particular tasks. And that paper is really interesting in my view, because they say, well, given these particular tasks, some tasks are amenable to running on small models.

And some tasks are better for large models, for example. And they were basically demonstrating how you can pretty much use a system, which looks at the prompt and then will kind of route it to the appropriately sized model to reduce the environmental impact of this. But one of the things they said was that they can look at the energy used when making an inference like, asking for a response back, but it's still a massive challenge to get any numbers from the embodied energy in making some software. And this is one of the key things that we might, maybe wasn't so obvious when we first spoke about this, that right now there are kind of three broad life cycle stages in an AI project. There's a training part where you create a model. There's an inference part where you have the use of the model. And then there's the kind of embodied carbon around this. And this is one of the first projects we've seen that really makes the figures for the inference part quite visible and quite easy to work at, and one of the key differences between this and the Ecologits project that you just mentioned before, was that this is actually running on the software itself, so they have access to the hardware, they're running the hardware themselves, and they, actually document how they do this. The, when I spoke to Samuel Rince at GenAI, I asked him, "how does your thing work?" Because that's really cool, I really would love to see some figures for the inference prices. Well, what we do is we have to make some educated guesses and estimates based on how big a model might be, how much memory might be allocated for it, and how long it's run and how big the response is.

So they're essentially annotating responses that come back from Open AI or from Mistral or from Claude or anything like that to give you some numbers for this. And this kind of does beg the question, if we see green AI instrumenting their own physical hardware, and if we know that the best we can see elsewhere is us having to instrument things ourselves. Based on guesses, why is it such a challenge to find services that provide these numbers as part of how they work? This is still a challenge in 2024. And I think these projects here demonstrate that yes, there is demand and there's also ways to do it. So if these aren't being exposed, they really should, because it becomes much easier to be a responsible technologist if you have access to the figures about the environmental impact of what you're using AI, for example. So yeah, those are the things I'd share, and I'll add some links to that to follow on from this.

Gaël Duez: I wasn't aware that they were using their own materials, and that, that's a massive change indeed, because no more educated guess, as you said, and the ability to measure yourself. Of course, it means that they create their own assumption when it comes to the hardware setup, but Yeah, I really love the idea.

And I have a question for you because I wasn't able to unpack yet all the publications from Hot Carbon. So how was this article aligned or not aligned with the last one from Sasha Luchoni when she was actually testing? So once again, on the inference side, because we know that this is where most of the impact comes from, when she was categorizing the use of a different LLM

in front of different use case and, she was, for instance raising the alarm that for speech recognition text or other, I would say, basic AI solution, using large LLM was a complete waste. And that's something that really struck me that how important it is to pick the right model for the right task and not using LLM, especially GPT 3 or GPT 4 for everything.

So I was wondering, do you know how well that was aligned or not? Their own findings.

Chris Adams: Okay, so it's been a couple of months since I read the paper, or about one month, because the actual paper was released, I think, on the 11th of July or

something. The key thing was this paper from Verena and Didi, the people at the University HTW Berlin,

and, Green Coding Solutions. Their one, while they do talk a little bit about classification of tasks, they don't go into the same detail that the paper from Sacha Luchoni, which was, that was the one that we saw in MIT Technology Review, which very much said, generating an image has the same kind of energy demand as charging a phone, for example.

So they don't provide that same kind of breakdown because their primary focus was working with text. However, there is something we should share a link to, which is some recent work in April where the, where Sasha Luchoni and other people, other luminaries at Hugging Face spent some time working on what you might refer to as like Energy Star for AI.

This idea that for different tasks, you do have models and so you can start making some more informed choices about your choice of model when you're doing some of this stuff. So we should share links to some of these things because

there is quite a lot of work taking place that I think not everyone is aware of right now and it's a really useful kind of jump off point for this.

So yeah, useful questions Gaël, thank you.

Gaël Duez: It connects well with what you've mentioned, that there's a rating should be provided by the one making these models or operating these models. And it's great to see all those initiatives, but eventually, and I think this is a nice connection to the next stories, but when you operate something, you need to be transparent about the environmental footprint of what you do.

Chris Adams: So this is actually, you're right, that's a very good point, Gaël, and maybe just the final thing I'll share with this before we move on to the next story is that for training, we've seen pretty much the industry start to settle on the use of CodeCarbon, which is an open source Python project that you can use to essentially work out the energy use for any piece of Python code, but specifically it's used for quantifying the energy usage for training.

So, this is what's used for some of the papers by, from Hugging Face. I believe this was also the tool that was used for the most recent information from Meta when they shared the carbon footprint of Llama 3, their big model as well. So you do have some existing tools that are out there. One thing I've actually tried looking at recently was I realized that you can use these tools and there are tools like GitHub and GitHub Spaces which allow you to run a virtual machine, leave and run, like some inference locally, for example, to try and like test something out, for example. And, what I found was that I was preparing a workshop for, to deliver at DjangoCon in June earlier on this year to help people, like, figure out, okay, what's the environment impact of maybe using an AI service or figure out what some of the kind of service side environmental footprints might be. I found that, there's a competitor to GitHub spaces called Gitpod. They use a slightly more up-to-date version of Ubuntu, which basically means that if you're running a virtual machine inside this, you can actually use Code Carbon and get numbers back. But when I try to use this with GitHub spaces, because they're using a slightly older version of the underlying operating Linux that's used for this, you can't get the same numbers back. And I think this is important or worth being aware of because there's a recent release from GitHub, I think. I'm not sure if it's totally available for everyone yet, but there are now some tools specifically for using inference in GitHub spaces specifically. So it'll be really lovely, and I know I'm kind of nerd sniping the GitHub team here, if they could expose some of these numbers, because the tooling totally exists now, and the bar is so low that even just having something like CodeCarbon, which is useful, but has some, when you look into the details, has a few kind of compromises and few issues. That would still be massively more useful than what we have available to right now. So yeah, there's there's definitely worth, there's definitely tools out there that organizations can use to make it easier for consumers like yourself or me to use these in a more kind of responsible fashion. All right. Thanks, Gaël. Should we look at the next story now?

Gaël Duez: Let's do it.

Chris Adams: All right. The next one. This is a story from a former VP of Sustainability Architecture, I believe, at Amazon, Adrian Cockcroft. he's written a piece for The New Stack and the title is, How did Amazon, Azure and Google perform in 2023 Sustainability? So this is a piece by Adrian Cockcroft where he's basically read through the three sustainability reports. And as someone who actually does have a significant amount of context working on the platform side as well as, since leaving Amazon looking at the tools out there and trying to collate a kind of like useful data set with the Green Software Foundation Real Time Cloud framework, he's been able to say well these things which are easy to use this is where some of the data is helpful this is where there are real challenges and he's it's really useful to get to have someone who, in my view is very much seen as like a kind of real kind of trusted message of saying look,

these bits are okay, this is where the bar is really low and we probably should be expecting quite a bit more given the amount of resources available to people. And yeah, I wanted to just check, is there anything that caught your eye or that really leapt out at you when you were looking at this? I really like this piece and I'm really glad it's actually out in the public domain.

Gaël Duez: Yeah, I love Adrian's work. I love his optimism. I'm not, I would be a bit more cautious is when he, well, especially regarding Amazon, but I guess we're all biased at some point. And what caught my eyes was we discuss a lot numbers that are not that easily assessed and separated. The very big first issue that I had is when we talk about Amazon, it's not the main, the same thing that when we talk about Google or Meta, because, or Microsoft, because there is this big on premise brick and mortar, as we used to say, chunk of Amazon's carbon footprint.

And what is strictly related to AWS should be extracted. And that's not the case with all the numbers.

Chris Adams: You're referring to AWS and

Amazon the retailer, like there's

been separate business, that's what you're talking about here. fact they're not breaking down makes it harder to understand, right?

Gaël Duez: It's, very hard because it's a bit like if we were discussing the, Fusabi report of Google plus Walmart or Sainsbury's or whatever, or Carrefour, and, I'm always very concerned about how globalized the data are.

And I've, got some cave hats with what Adrian said. I won't challenge the numbers or his analysis because I think it was well done and the trend are there. Like AWS, Amazon is doing a bit better and Google and Microsoft are slipping up as the, as he mentioned, but, focusing a bit more on Amazon, for instance, there are a few things that I'm a bit concerned about.

So the first one is that they gather everything in this big Amazon and what actually as techies, we would like to understand better is AWS on its own. I think it's big enough to have its own sustainability report. The second is that they continuously provide numbers on the market-based approach, especially for energy. And I think that there are now countless examples where it's not really how Sustainably is done. Sustainably is as much a global matter than a local matter.

And I'd like just to take the example of Ireland. So if you run as many European techies, your instance on AWS, there are a great deal of chance that by default, you will be using the Ireland region. And when you log in the dashboard or your Sustainably dashboard, exactly as Adrian mentioned, you will see that everything is fine.

You're zero percent, you're carbon neutral, everything has been offsetted, and ciao, bye bye, well done, you can, do business as usual. Now the reality of the Ireland electricity grid is that one year ago, in 2023, the amount of electricity consumed by data center equals residential urban residentials.

It means that every houses, every buildings in Ireland consumes now less, a bit less electricity than data centers. So it has put a tremendous pressure on the electricity grid and the Irish electricity grid is not the cleanest or sorry, the lowest carbon on Earth at all. So, technically speaking, when we add resources, when we add instances on AWS Irish region, we are adding pressure and pushing the Ireland electricity providers to emit, to produce more energy, which is kind of high intensive energy.

And now you've got this market based approach, which has is it's prone. I'm not like, it's not black and white here, but saying, okay, but we invested energy elsewhere. And we show it either by a, power purchasing agreement or AAC. And so that's all good on the market because everything shall be offset.

But the local realities matters and that's even more true for water. But let's put that aside for the moment. So as long as they don't at least try to localize a bit more the carbon emission and the related energy carbon emission. I think it will be always very hard to say, okay, the trend is okay, the trend is not okay.

So that's my first issue. My second issue is that, and I think it was our dear friend from SDIA, Mike Schultz, who once said, one of the most precious resource on earth today is renewable energy. Because of course it's growing, but we don't have that much. And we should always question how much we allocate to which use.

And by having this 100 percent focused on offset or net zero approach, that is the one from Amazon, Google, etc. We cannot leave the elephant in the room, which is, but what are the absolute numbers? And when the absolute number are getting higher and higher, almost from a logarithmic perspective, it's almost exponential, not fully, but almost, we should question ourselves, but where is the limit?

Because we do know that in systemic and in environmental ecology, there are, there is always a limit to how much a system can grow. So, that's my two big issues. It's not localized enough. And it doesn't talk about absolute values. It only talks about the potential of things being offset or being carbon neutral.

And we need to think more about when we slow down or even we reduce our energy consumption. That's not on the table at all. So yes, of course, there are a lot of progress being made. They buy a lot of renewable energy, but is it the best use what we can do about renewable energy? And what are the trends? I know I could speak for ages about it, but sorry, Chris, and I didn't even mention water.

Chris Adams: So just, so let's just check if I understand the key things you're referring to there. So one of them is, just to be clear, we're talking about the Republic of Ireland, as in the island of Ireland here inside Western Europe.

That's we're talking about here. And if I understand what you're referring to here, there is one of the big things, big parts of this story this year is that Amazon has made a big song and dance about saying "yes, we are now 100 percent renewable powered for all of our infrastructure." And what you're, what it sounds like you're saying is that The physical reality in Ireland doesn't necessarily match this claim because it may be that the kind of the way people are substantiating this green claim is that they're basing this on credits like renewable energy credits and while these may be kind of considered kosher or like considered like legitimate in like maybe a trade electricity trading market kind of sense, the fact that we don't see the actual location based figures for these data centers brings up all kinds of questions. And also, the, there is also questions about, are renewable energy credits the correct way to actually basically back up any claims around the use of green energy, particularly when we know that the underlying grid, there may be more power being used than it actually, than renewable energy is actually generated in Ireland itself for this, right?

Gaël Duez: Absolutely. And then to, give them credits, they use less and less, renewable energy credits, which are highly questionable tools, and they use more and more PPA, which are Purchasing Power Agreement, where actually they commit to add new renewable energy via partnership, long term partnership with electricity producer.

So it actually increases the amount of renewable energy available for everyone on the grid. So I'm not saying that everything is bad or everything is great. My question is if you, for instance, just staying within Europe, invest in northern Germany in a wind farm to produce that amount of gigawatt of renewable energy, that's great.

That's necessarily, that's something that is very useful for the German market and for German users, but it will not offset the fact that there are still gas and even, correct me if I'm wrong, coal based port plant in Ireland, and that the use, the rise of energy use in this part of the world will emit more greenhouse gases.

So, once again, it's the incredible ability of humankind to tell itself stories, which has made us what we are today, has also a dark side, which is it's not because we decide that we create a fancy story called the market or the energy market, et cetera, that it is completely disconnected of the physical reality of thing, as you mentioned.

And the physical reality of thing is, it's great to add more renewable energy to the grid every day. Anywhere on earth, because anywhere on earth, we need more renewable energy, but it cannot really offset or compensate the fact that if we put some stress on electricity grid somewhere, it will add the emission of greenhouse gases and, eventually everywhere around the world, because I think it's any like carbon molecule that take 15 days to do a round trip.

So, it's a global challenge.

Chris Adams: If I understand what you're saying, basically, the instruments being used do not fully capture the physical realities of what's taking place. And while there may be progress, we probably need more progress in order to actually face the challenges that are being kind of set out by the actual, the real science that we're seeing. I'll share a couple of little points around Ireland specifically before we move on to this. So Ireland is actually one of the few countries where Green and IT claims around green energy have actually been challenged by the Advertising Standards Agency, specifically saying if you're a green energy firm and you're saying you're using green energy. We've, there have basically been cases where the Advertising Standards Agency in Ireland has said, you can't make these claims in Ireland if you're using just renewable energy credits as the basis for making this claim. So that's one thing we've seen. And that has interesting implications for technology firms that are using these green energies if they're substantiating their kind of claims around green energy by using these certificates.

If you've already had a ruling saying, "nah you're not allowed to do that." The other thing that surprised me, when I was looking into this, because the Renewable Energy 100 is a ranking of the top of a large number of firms who are significant investors in renewable energy. They actually don't accept the use of these kinds of renewable energy credits if they're not physically deliverable.

And one of the challenges you see in Ireland is that there's a limited amount of capacity to move the kind of like green energy that might be generated elsewhere in the world to there for this. So that's one of the challenges that you see. And we'll share links to both of those two things because for people who are kind of wonkish and want to get down to some of the bottom of this, they're really, I think they provide some interesting background to this.

We'll also share a link to the real time, to the Green Software Foundation Real Time Cloud dataset where there's been a bunch of work into trying to find some location based figures for this stuff so you can come up with some more accurate numbers than what we're seeing here. And I think, okay, I'll leave the last word with you, then we'll move on to the next story.

Go for it.

Gaël Duez: There are two things that I'd like to give credits to Adrian in his article. It's like stressing how much there are two sides of the story. And there's reports that focus a lot on sustainably of the cloud. And that's definitely what Amazon, Google, et cetera, are trying to do. But there is also this question of sustainably in the cloud, which is how as a user I can do or not a better job mitigating, reducing my carbon emissions, my water consumption, et cetera.

And he's right to say that not significant, no significant progress has been made on Amazon side and on AWS side story. And they are still infant phase at Google and even at Microsoft when it comes to transparency. And as a CTO, as an software engineer. And when you look at these dashboards and you see that everything is fine, everything is offsetted, you've reached carbon neutrality, it doesn't empower you to do the right things, which is optimizing, reducing your carbon emissions, your water consumption, etc. So that part, empowering consumers is still lagging of what we should expect from these tech behemoths. And my last comment is that I was very pleased that he mentioned and he reviewed, thoroughly the water consumption because for water and that my message about global versus local, it doesn't really matter.

It doesn't really make any sense to analyze the water consumption in terms of global consumption. It's water is a local matter. And it's really region per region, even data center per data centers. How much water comes in? How much water comes out? And in which state? Is it reusable, not reusable? Is it a closed loop or not?

In most of the data centers, including the one from the hyperscalers, are far from a closed loop. I know that Google has experienced once and they told quite a lot about it and it makes total sense. But we need more. precise and localized information on water. And that's a massive challenge as well. We focus a lot on carbon, but water is the next big issue that we need to pay attention to.

Chris Adams: Alright, water, that's the next horizon. I'm going to park that because we'll come back to it a little bit later. The next story is actually from the Financial Times. This is talking about Big Tech's bid to rewrite the rules on net zero. Now, at the time of this going out, it may be that the really nice looking piece may be hard for people to see, but no, the link does seem to work actually still, thankfully. The Financial Times has a really interesting piece, basically talking about the large technology firms that we often see coming up again and again. And this is a bit of a deep dive into some of the things you just referred to about like location based carbon footprints for electricity, because that's one of the key drivers of emissions for our use of digital services, and the market based approaches. And this pretty much dives deeply into something of a bun fight that's taking place where you have two kind of schools of thought where there's one set of companies like, to an extent, Microsoft and Google are pushing for this notion of 24/7 renewable energy and are having a quite kind of tight accounting process. And then you have another approach being largely put forward by Meta and Amazon talking about their kind of emissions first approach saying, no, what we should be looking at is decarbonizing the entire grid, not so much looking at our carbon footprint. And there's a couple of things that are really interesting inside this.

There's a few nice interactive graphics for you to see how people make green claims around energy usage. But one thing that I think is actually really stark is this set of charts showing the difference when you try looking at these figures. So, if you were to look at, say, the carbon footprints from, say, Microsoft, you can see, like, from 2018 to, like, now, you've got a figure of maybe, you see one chart showing the market based footprint, which is, pretty close to zero for Microsoft and close to zero for Meta and likewise for Apple. And then you see the location based figure for Microsoft. It's something in the region of like 8 million tons or zero tons, for example. And likewise with Meta, you're seeing 4 million tons versus zero tons. And Amazon's got the same issue where you're looking at like 15 million tons of location based carbon footprint from using electricity versus 3 million tons from using this.

So you, this really gives an idea of how these two different perspectives end up changing how you might report on this and how you might think about the environmental impact of using some of these tools. And like, to an extent, there is, there are reasons why you have a market based approach because, these come out of the fact that people who are inside large firms are looking for ways to be recognized for the investments they're making so they can justify this internally.

So there is a role that some of these play, but it often, it obviously gets quite a bit more complicated than that, especially because this is the year that the Greenhouse Gas Protocol, the kind of gold standard for reporting, is currently being overhauled to rethink how you should report this stuff and how you should be allowed to talk about energy being green or not green in this context. So Gaël, is there anything that kind of leapt out at you when you had looked through this? Because I would love more people to see this. I think it's a really fascinating story.

Gaël Duez: I think I've already commented it in advance when I was referring to the struggle between market based and local based approach. And once again, I think we should stress how important it is to understand that the way we build things in our mind and in our society as humans is one thing, and the physical reality of the world is another thing.

And when you add energy on a grid, wherever, et cetera, you have no clue on how it will be used, even if it will be used, because when you create PPA, it's potential energy to be used. You create new capacities, whether those capacities will be used or not remains a challenge. Obviously, they will be used, but not necessarily 100%, etc.

And I think the right approach is clean up your own mess. Everyone should start with this. So, I'm fine with having part of the sustainability report explaining what has been done and what could be the approach of market based, but the truth is local based approach. And when you see these figures, they're actually very consistent.

Yes, they're increasing massively their investments in data centers to fuel the AI boom. Their entire business model is based on infinite growth. The numbers go up, that's pretty, pretty logical. And what I've just kind of, when I read this piece of news, it also connected a lot with the crisis at the SBTI, the Science Based Target Initiative, that happened this year, when there was very strong push to allow more offset techniques to be recognized as science based,

Chris Adams: Ah, you're referring to the Scope 3 thing. The push people being able to use offsets in their supply not just electricity, as a way to kind of decarbonize that without having to necessarily make some of the changes to like reform the supply chain. Is that what you're referring to here?

Gaël Duez: Absolutely. Thanks for making it much clearer than I was about to say. So, I think the struggle is everywhere because we see that the low hanging fruits, most of them has been already taken care of in this big corporation and they're entering the bumpy road where you've got harder choice to do. And when you face this kind of choices, well, either you do the right things and you go back to the physical reality of our world, or you try to change a bit the narrative or change a bit the rules, and I think this is exactly what we've been seeing at the science based target initiative where, some companies were obviously not able at all to meet the decarbonizing plan that they proposed just a few years ago, and they were trying to change a bit the rules.

And that should really question ourselves when it comes to transparency and acknowledge that even the most well intentioned CEO, the most well intentioned C suite, they cannot really do the right things without a bit of external help, whether it comes from pressure from activists or governments or UN, you name it, but we cannot fully rely on companies agreeing on how they should measure their own environmental impact.

Even if they are well meaning with tons of great people trying to do the right things, et cetera. It's not a black and white world out there. But there is a question at some point of financial pressure, shareholder pressures in many of these companies. They're just stronger than the entire stakeholders pressure.

Chris Adams: I think I know what you're referring to here, and what I'll do is we'll share a link to some of the stories about the SBTI, the Science Based Targets Initiative, the whole kind of bun fight that took place there, that ended up resulting in the then CEO stepping down from the organization, partly because there was so much pushback against this idea that you might be able to use offsets for looking at your supply chain emissions is in this same fashion. So we'll definitely add a link to that, thanks for that Gaël. The other thing that we maybe might maybe you must referring to inside this is there are some really nice quotes. Is that what you're talking about here does actually there's some relevance back to the Financial Times piece here, about when you look at the decarbonizing energy in this particular space. One of the approaches being used, which is described as Emissions First, for example, this is the one that's being largely put forward by Meta and Amazon. And one of the arguments for this is that we should be able to be optimized for absolute impact, not necessarily our own carbon footprint.

So given a hundred euros or a hundred dollars, we should be able to deploy that in the most effective place globally, wherever it might be. So one of the arguments being made is that we, if we want a data center in say, North Virginia, we should be allowed to basically purchase the right to claim the emission reductions in somewhere like India, which has a very coal heavy grid, and then kind of count that against our own emissions. And on one level, yes, you are reducing the environmental impact, you can make the argument, but it also means that some of the cheapest possible emissions in India are now being bought by one of the richest, some of the richest companies in the world. And this is very similar to the dynamic you were referring to with offsets, where, essentially, if you have a kind of, climbing scale in terms of how difficult it will be to reduce emissions over time. If you grab all the low hanging fruit, that raises all kinds of equity issues about what's left over for people who don't necessarily have the same resources available to them as Facebook or Meta or Amazon, for example, when looking at this. So, this is one of the challenges you do, find. But I'd really urge people to look over the FT piece and what we can do is we'll share a link to an archive link for this if the one, if it's behind a payroll, because it's a really fascinating piece and it's really worth looking into. Okay, Asim, sorry, Gaël, should we move on to the next story here?

Gaël Duez: Yes. I'm honored that you thought was... I

Chris Adams: Bit of a Monday morning moment here. Okay, so this is actually a story speaking about AWS again, by moving an AI workload to AWS, you can reduce the carbon footprint by up to 99%. That's the argument being laid out inside this. And this is a piece from The Stack, which is not the same as The New Stack, who covered our previous story.

It's another organization. This one basically takes apart some of these questions, or at least dissects some of these claims and say, well, what's the basis for this? Gaël, I wanted to kind of give you a bit of space to talk about this because I know that you've read this and I know that this is something that is... and you often advise firms who are actually trying to figure out how do I reduce the environmental impact of the services I'm using. So when you read this, what leaps out at you and what kind of things are the kind of most salient points would you say?

Gaël Duez: Well, I think my main reaction was, "oh no, yet another one."

Okay. So there is the SCI piece, which is interesting, but let's go back to basics about it. And I encourage listeners to have a look at two great resources. The first one is an article from the Boavizta Association investigating the claims made by several hyperscalers that when you migrate from your services, from a traditionally hosted perspective to a hyperscaler perspective, you save 60, 70, 80, 90 percent of carbon emissions.

And the second source is actually a man, and I was very pleased to see him being quoted and interviewed in the article, which is, he's Mark Butcher. He has been very vocal about hyperscaler claims, how they measure things, the scope three not being that well taken into consideration. Mark works at Positive Cloud and he works with a lot of clients across the UK on these topics.

And why I'm mentioning this too is that hyperscalers, when they say, basically "move your instances more, move your computing power to our facilities, because we are so much more efficient than the other data centers or the other hosting solutions," they might be right, but the others are a role, a world on their own.

And this is really the question of the baseline. Yes. If I run two or three servers in my office building, in a small room with a very old air conditioning, yes, I might reach a PUE of 2, 2.2, maybe 3, whatever, et cetera. But most of the clients, they don't start with these baselines. They start with servers and instances being managed in already quite professional and by seasoned providers, okay, and I won't provide any name here, who have already reached a decent PUE and the gain is much smaller.

So that's. On, from which baseline do you start?

Chris Adams: I see, like the example given here is like a really inefficient thing. If you compare really an inefficient setup compared

to this idealized, going to give you a disproportionately large saving. That's

Gaël Duez: And this is when you study the case studies, which have been provided by hyperscalers. it's literally my grandma managing a data center. Worst practices possible that are accounted as a baseline. And this is not the truth. Many data centers already reach pretty decent or, pretty good actually,

power efficiency. So where you start from matters. And that's my, point number one, my point number two, and we go all the way back to this local versus market based approach. But I'm sorry to say I'm, I, feel a bit like I'm, rambling here. But once again, if, even if, okay, I run the worst possible data center on Earth, I've got

maybe a PUE of two, maybe three, et cetera, but I run it in a very low carbon place, say France, if you account, for, the nuclear, a share of energy, say Scotland, for instance, and I recall Mark wrote something about it, Mark Butcher wrote something about comparing the energy intensity of North Scotland and Ireland.

And once again, just by migrating your instances there. You might, let's say, divide by two, your energy consumption, thanks to better energy efficiency by AWS, because they're very active in Ireland, but you can do the same math with Google or Azure, but then you start operating in a country which is, and that's the case between, for instance, Northern Scotland and Ireland, six times more energy carbon intensive when it comes to electricity production.

Chris Adams: So let me check if I understand, well, I think with the point you're getting at, you're basically saying, yes, you may, the infrastructure may be more efficient, but if the local energy is dirtier, it doesn't necessarily matter that it's six times more efficient. If, say, the infrastructure is twice as efficient, but the energy is six times worse, then it's still, you're not coming out ahead. That's what I think you seem to be making the argument there

Gaël Duez: I actually, I was actually, I was not the one making the argument. Mark did it and

Chris Adams: but that's, that's you're saying. Okay.

Gaël Duez: but that's exactly what I'm saying. And so this question of where do we start from the baseline is super important. Then there is a specificity of the AI itself. And we should always remember that most of the time by AI people, and I think this article is a bit misleading here as well, they think generative AI, which is part of AI on its own.

We start from so high. It's still very infant. The LLM are not that old. And of course you can decrease very significantly the energy consumption and the emissions of your AI model, because we are, we've just started to do so, but what will be the trend for, I would say, everyone rather than AWS is an open question.

Obviously, they're doing a better job reducing and optimizing everything, but it's fair to assume that other actors as well are doing the same. So I'm always very concerned when we take a very new algorithm, I would say, or a new part of the AI industry and say, "Oh, look, we're going to reduce by that, that much."

But of course, it's like with cryptocurrency. They've started at such an inefficient way that they made a great progress. Still, they consume a lot of energy. So you see that's my point. So I will be always super cautious with this kind of stuff. And then comes the good part of it, which is using the, as the software carbon intensity, but maybe Chris, you want to elaborate a bit more on

Chris Adams: So, it's nice to see an international standard being used in reports like this, but in order for a standard to be used, you need to follow the standard. And one of the key things that the standard actually does is it, basically, when I've looked through this, I look through the report, I can't see any numbers for any of this.

So, it's like you have a car saying, well, this car is twice as efficient as this other car. If you don't have any numbers about, like, the miles per gallon or the equivalent like that, it's very hard to be able to trust that number, for example, or trust any of these claims here. So you have an issue about, well, there's a lack of underlying numbers. Also, the thing we see is that a significant part of this is based around the environmental impact of the energy itself. Again, we don't have the numbers for this, but in particular, one of the key things, one of the key claims being used in the report was we're going to take into account these market based figures here.

Now, the Software Carbon Intensity specs explicitly says we don't use these inside this. So, you've essentially got people using this term, Software Carbon Intensity, without actually following any particular nuances of this. And this, It makes it very difficult for me to recommend this report for anyone else because it essentially is going against how this is intended to be used. And the firm that was working with Amazon, they are very involved inside the Green Software Foundation. This really needs to be a thing that we can't do if we want to see this to be adopted and respected because this essentially, in my view, undermines a huge amount of work that's gone into developing a standard here because this makes me trust the Software Carbon Intensity less after reading this report and seeing people cite it, because it's being used incorrectly. So that's one of the things I would actually raise and something that does need to be addressed. Like, there are mechanisms that the GSF has to say, please do not use it in this way. It's misleading and it undermines some of the work we have. And I think that's something that will need to happen because, yeah, I cannot recommend anyone looking at this report or even recommend using or referring to this standard like this because it's an incorrect use of the standard. So yeah, that's my take on it.

I'll move on from this because that was a bit of a downer, but it's really important if you want people to trust this in my view.

Gaël Duez: And I think also that there is a way to protect the SCI and the Green Software Foundation tools, which is using the Impact Framework manifest,

Chris Adams: Yeah, I'm glad you mentioned this, Yeah.

So there is a mechanism. The whole thing around the Impact Framework was specifically set up to say, "hi, you're going to make a claim? Make this transparent. Show you're working inside this." And that's like, there's a huge amount of work that's gone into providing this, and if you're not going to share any of the numbers or share the basis, and there are now lots and lots of really helpful case studies demonstrating how to do this, like we'll share links to this to show this is, these are the correct ways to use this. When you have it being used in a way which is so unhelpful, it's, really problematic, and you can see why people are going to struggle, and why, you can see why people end up essentially dismissing so much of these, efforts as greenwash, when people aren't sharing the underlying numbers for this stuff.

So yeah, that's like the framing I would take, and I would really like to see this addressed, because it's going to be, it's going to be a real challenge going forward, in my view.

Gaël Duez: Transparency, transparency, that's all that matters with this kind of claims.

Chris Adams: So, this probably takes us to the last story I think we have time for. This is one actually, this isn't so much a story, but more of a kind of discussion about some of this. So we've been talking about how it's real, really difficult to actually get some numbers from this, and how the way people report this is also a real challenge.

The thing I kind of want to share with you, and I realize I won't have time to talk about this while I, this is partly what I'll be talking about at some events in September, is the fact that we have, we Some regulation, which is forcing some of this. So in Europe, all across Europe, there is a law called the energy efficiency directive, and basically any data center that uses more than 500 kilowatts of power, which is, that's not a small data.

It's not a tiny data center, but pretty much every single hyperscale you imagine would have to do this. There is now a law, which basically says, every data center and any organization operating a data center has to make publicly available, Information like the name of the data center, who runs it, how large it is, how much power is used. It talks about the amount of energy used, the water used, all this stuff. So we do actually have laws which are kind of forcing some of this now. The, there is one caveat in that where companies consider this information to be a trade secret, they don't necessarily need to publish this information into the public domain, but where companies are not saying this is a trade secret, we now, as technology professionals, can ask and say, this information should be in the public domain if you're in, if you're in Europe. And the thing that I might share as a follow on from this is that for companies that are not sharing this information, they are now mandated to report to a centralized database with the idea that some of this information will be shared in an aggregated format.

So for the first time, we can actually get some meaningful numbers that come out of this. So, companies that are prepared to be transparent, you can ask for this stuff. Companies that are then saying, "we're not going to make this transparent because it's a trade secret." There is still a mechanism by which they will need to report so that we can finally have some data informed policy around this. Because one thing that's come up again and again in this discussion has been that we don't have access to this information. And there is so much pressure or there's so much, there's so many incentives to construct a message which makes you look good that it becomes very difficult to trust a lot of the statements around green software that come from lots and lots of large firms right now.

So yeah, this is, I'll share a link to the issue where this has been discussed inside the real time cloud working group inside the Green Software Foundation. But Gaël, I wanted to just check, as someone who's not in Europe, what's your take on this? Is there anything that kind of caught your eye when you were looking at this?

Gaël Duez: Well, the first thing is that it is required to disclose numbers in both relative, but also absolute numbers. And that's very important. It's not only PUE. It's also how much energy did you consume overall? And that really connects well with what we've discussed before. I also believe that this is something that is

pivotal for country with weakest electricity grid to consider. It's always claimed that for instance, Kenya, my neighboring country of La Reunion island, has a very strong policy of attracting data centers. They want to become a computing power.

Chris Adams: Iceland of Africa, because have more geothermal than

Gaël Duez: Absolutely. And I reckon that this kind of disclosure will also have them a lot anticipate what is needed for them to prepare the electricity grid to this kind of increase in electricity consumption caused by data centers.

And also making sure that what has been dealt and agreed when they do this big hyperscale deal are actually what is provided, that the energy is there, that the water is there, and I believe that local populations, which are often caught in between, like, "oh, it will create a lot of jobs, but when you do the mass, not that much," so it's not that an obvious investment to say, okay, we will welcome a lot of data centers in our country.

It might be, but it's not like a big investment. A clear win or the case, they will have the ability to scrutinize, how, what are the impacts and environmental impacts. And I must admit that, if you look a bit at the history, in 1982 in Europe, a European directive created the Seveso listing, and the Seveso made it compulsory for every state in Europe to list what are the facilities, industrial, agricultural, mostly industrial, which can create significant environmental risk.

And we were talking about chemical industry, et cetera, et cetera. And to some extent, data centers, they impact a lot of the environment. It's just that they will not blow away like a chemical industry, but on the long run, they've got a lot of impact on their environment and it makes a lot of sense to, at least for the bigger now, the biggest data centers, to be able to provide

environmental information, in a comprehensive way, a comparable way, and to make sure that we monitor the environmental impacts of these big facilities. we're talking about facilities that are built on hectares of land. There's not like the small, tiny warehouse that we might still think of.

They're like massive industrial facilities. So having open and transparent reporting seems to be quite straightforward.

Chris Adams: So maybe we might see some of these ideas adopted in other parts of the world, especially because on the underlying data for this. We'll share a link to this. We've been doing some research ourselves in the Green Web Foundation. A lot of these data points are based on the EU code of conduct, which is a public document for people. So it may be the case that you might see some of these data points being reported in other parts of the world as well to set a precedent. So they actually have the data to make data informed decisions about how, about the role digitalization plays in society and

how the impacts are actually shared around this.

Thank you for the link about Seveso as well. I didn't know about the Seveso directive. That's totally new to me. What I might share is a link to some of the work that the Green Web Foundation has been doing. We have a fellowship and one of our fellows, Samantha Nidwalana, she's based in South Africa and the Netherlands, and she spent a bunch of time looking through, trying to get some numbers in South Africa for, to basically explore okay, what's the environmental impact of data centers inside this?

And we'll share some links to her challenges in this because she's been trying to find these numbers and it's been a real, it's been a real challenge in many cases actually. And it does give you some idea about like where some of this might go, but also hopefully stories like this and seeing some of these laws being passed might help set a precedent so that we can have more transparency in other parts of the world as well. Okay, I realize we're coming up to time. Let's do a quick just run through. We've got some events coming up. I know that I'll be catching a train in a few weeks time to go to London, and I think I'll see you in person for the first time for quite some time actually. This is an event called Green IO Conference. Maybe you could just briefly touch on that before we move on to the next set of events and then close out for the day.

Gaël Duez: Yeah. Well, absolutely. I will be pleased to see you again for the second time. The first time was in Berlin, if I recall, and I think it was for being interviewed on the GreenEye And what. I launched with my partners RPI Days last year, was a series of global conferences called Green IO, not very original, sorry about this.

And the idea was to have also on site events. We can see a lot of hybrid and a lot of online events when it comes to digital sustainability and they're great. We can give kudos to the Green Web Foundation, the Green Software Foundation, CNCF as well. They're doing a lot of these events. Now, what I also realized is that if you look at other specialties in our IT industry, let's take cybersecurity, for instance, accessibility, design, or you name it.

When you work in a city with a significant enough workforce in cybersecurity, for instance, let's say Barcelona, Berlin, Paris, New York, Singapore, Beijing, you have at least 2, 3, 4, sometimes 5 different on site conferences when you know that you will meet your peers. Today, if you are an IT sustainability specialist or green IT folks or whatever, like the dude who anyone reached out to because, "oh, it's about green, et cetera.

He's a person, she's a person we'd like to talk to." You've got basically nowhere to go except for one conference in Paris and one in Belgium now in Brussels this year, where you know that the topic will be a hundred percent your specialty. So don't get me wrong. We can give kudos to a lot of tech conferences like QCon, like even Reinvent AWS, you've got to sustainably track, that's fine, but having a 100 percent green IT focus or IT sustainably focus conference, where you know that basically this is the place to be to meet all your peers, most of the cities around, they don't have it. So that was really the idea of creating this Green IO.

Even since Singapore, London, and Paris this year, we are like 90, 90 percent sure that we will add New York and Munich next year also. And the idea is really, it's just It's a place that has been created for the local communities to do what they want with it. And so you, this year in London, it's a bit like go back to the basics and let's talk step by step.

What about low carbon infrastructure? What about design and eco design? What about beyond, understanding the organizational challenges and HR challenges that when you want to rule out more sustainable approach, but that's a one day conference. It's two days in Paris this year, but in London, it will be one day.

And we expect to have like several hundreds of people joining. And I know also that we will have a great keynote speaker that some of the listeners might known named Chris Adams. So I'm really delighted that you agreed to join Chris. Thanks a lot for this.

Chris Adams: Cool, thanks, so that's the event. It's the 19th of September, taking

place in Bishopsgate in London. And thank you, I am indeed keynoting in the morning, so I hope my trains are on time. But there's a number of people from the Green Software Foundation, for example. I can see Sara Hsu, who is one of the people leading the Green Software patterns project in the Green Software Foundation. There's some people from the W3C on working on web sustainability guidelines. There's also, I'll, share a link to Therese Gale who is working at Salesforce. She'll be talking a little bit about some of the experience, some of her experiences as well. So there's a number of people who

Gaël Duez: Mark Butcher will

Chris Adams: and Mark, yes, the, yes, Mark Butcher of Positive Cloud.

He's been one of the people who's been really instrumental in some of the most recent work inside the UK government, put together a kind of Digital Sustainability Alliance. There's a bunch of things there. So I'm actually looking forward to this Gaël, and I want to just give people a heads up that it's taking place. Gaël, thanks so much for coming in for this and telling us the stories about Wales, And we're watching and all that sharing all your insights elsewhere. Gaël, if people do want to follow up and see what you're up to, what's the best way to find you for future work?

Gaël Duez: Well I would say on LinkedIn and I'm pretty easy to find Gaël Duez. There is not that many. And more generally, if you are interested in what we do in Green IO, it's greenio.tech. That's the website. You've got access to the podcast, the conferences you've got link to my own websites. But I think greenio.tech is the best place to start.

Chris Adams: Brilliant. Well, thank you for that. We'll share some links to all of those websites and to this event coming up. And, for people who aren't able to go to Green IO, the Cloud Native Computing Foundation, the TAG ENV essentially the green part of the Cloud Native Computing Foundation.

They have a series of remote events taking place in October. We'll show a link to that so that if you still want some, to get an events fixed, that's where to look. Alright, Gaël, thank you once again for making the time and I hope you have a lovely week. Okay. Take care of yourself, mate.

Gaël Duez: Thank you, that was great being there. Bye, have a nice week as well.

Chris Adams: Hey everyone, thanks for listening. Just a reminder to follow Environment Variables on Apple Podcasts, Spotify, Google Podcasts, or wherever you get your podcasts. And please, do leave a rating and review if you like what we're doing. It helps other people discover the show, and of course, we'd love to have more listeners. To find out more about the Green Software Foundation, please visit greensoftware.foundationon. That's greensoftware.foundation in any browser. Thanks again, and see you in the next episode!

Hosted on Acast. See acast.com/privacy for more information.

"There IS no Housing Crisis, Akshually ☝️" with economist Cameron Murray

Uncomfortable Conversations with Josh Szeps

COVID Stimulus and Rental Dynamics

The AI-powered Podcast Player