Environment Variables

In this episode Asim Hussain is joined by guests Scott Chamberlin formerly of Microsoft and Henry Richardson of Watttime as they discuss how time-shifting, location-shifting, curtailment and other terms are important to Carbon Aware Computing. How can we build sustainable software that reduces the impact on the environment and how these decisions may just lie in the hands of the developers instead of the CSR teams.  Learn more about our guests:Scott Chamberlin: LinkedInAsim Hussain: LinkedIn / Twitter Henry Richardson: LinkedIn / Website Episode resources:The Green Software Foundation Website Sign up to the Green Software Foundation Newsletter Green Software Foundation SummitMagazine: Branch MagazineIf you enjoyed this episode then please either:Follow, rate, and review on Apple PodcastsFollow and rate on SpotifyConnect with us on Twitter, Github and LinkedIn!TRANSCRIPT BELOW:Asim Hussain: And one way I think about carbon awareness is actually I'm building software, which responds to natural cycles of the earth. And it connects me with nature in an indirect way, but it's one of the few ways you can connect with nature. I think in software. Hello and welcome to environment variables brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding green software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software. I'm your host, Asim Hussain, welcome to the Environment Variables Podcast.We have an exciting episode today talking about carbon aware computing.Scott Chamberlin: Hi, I'm Scott Chamberlain. I was previously at Microsoft leading some of the sustainability efforts in the windows organization. And the day, this podcast is airing. I'm actually starting a new role at Intel leading their software sustainability effort.Henry Richardson: And I'm Henry Richardson. With Wattime we're a nonprofit really focused on making grid emissions, available to partners to achieve impact through load flexibility of sighting of renewables. And so we're really excited about the kind of expansion of capabilities in software to take advantage of flexibility in grid emissions.Asim Hussain: Henry cause this, this, this is your, this is your bread and butter. This is your space. Do you think you can give a, go trying to explain, you know, carbon intensity in these concepts to the audience here.Henry Richardson: Absolutely. One of the things that we spend a lot of time thinking about is how. Clean or dirty that the electric grid is. And what we mean by that is when you make a change on the electric grid by increasing or decreasing load, how does emissions change? So if you decide to schedule a compute load at a specific hour, a certain set of power plants will be responding to that change in load and they'll have an associated emissions.And so you can see how by scheduling. Load updates or sorry, windows updates to specific times you could actually affect which power plants are operating. Ideally, we would be scheduling those two when there's excess solar, excess wind, which can happen pretty often in the great Plains. There's a lot of excess wind, a lot of excess solar in California, but you can also pick between coal and natural gas, if you can have that flexibility.So. We measure the intensity, the electric grid, and then we make that information available the software and making it available to software as what we consider carbon awareness. So can the software take advantage of that? The time varying emissions, intensity of the electric grid and actually change when it trains machine learning loads, as Scott was saying updates, major pieces of software, can you run.At different times because they're, they're run chronically or regular. So things like that. So we see lots of opportunities in software to be kind of carbon aware and take advantage of this, this flexible.Asim Hussain: Yeah, I think there's lots of other examples. I mean, there's, there's other big, big examples of well as well from other organizations, but where they started to apply the ideas relate to, to a carbon Alanise I think broadly, it, it, if you have the kind of. Software workload, which can respond to a signal, not all software can, that's the chatter challenge.Not all software can respond like this, but if you do have the kind of workload that can respond like this, and that's why the windows update is such a perfect type of workload, because it's something that, you know, you need to do at some point. But you can have a reasonable amount of flexibility over when, when that happens.If you're, if you're, if I'm visiting a webpage, I don't have that. You know, I need that web. And in three seconds, the other kind of famous use case I've seen is a large, large scale implementation of this I've seen is, is the work that Google's done with their carbon way data centers, which I think is quite an interesting, like the work that Microsoft's done with the windows is, is, is, is on a device.And then Scott you've told me before I keep on saying, I'm sure I get the number wrong. Is it 10 billion devices around the world? Use windows? Is it? What is it?Scott Chamberlin: think the last public number is either 1.4, 1.5 billion devices use windmill point for 1.5 billion client devices use windows, and then there's a separate, separate staff for our servers and data. Asim Hussain: Well, so there's a, quite a few people in the world who now think is 10 billion, because I just threw that stat out so many times in the past, one of the things you're involved with at your time at Microsoft Scott was well windows, but specifically, and then an announcement that windows made recently.Do you want to give a quick summary as to what that was.Scott Chamberlin: sure. Totally. And again, this is in partnership. Wattime and Henry's organization as us in partnership with electricity map and the tomorrow organization. One of the first things that we did in windows was figure out how do we bring carbon awareness into the. Operating system, right? The operating system is responsible for scheduling tasks.One of the things that does, and the question we had was if we had a CO2 intensity signal, could we number one? Change the behavior of the operating system in a way that was beneficial for the environment and had minimal user impact. And number two, would that have a significant impact in the emissions associated with the energy used by PCs around the world?It was a PC focused saying not necessarily a server or a data center focus thing. I'm still, yeah. Recently in preview, Microsoft released the. First implementation of carbon aware, scheduling for windows update. And so windows update is essentially how windows applies new features for users. And there's this whole set of criteria which go into when is the optimal time to apply an update for a couple reasons.Number one, in a lot of cases. It causes a reboot to happen on your machine, not all cases. And in some other cases, it requires CPU cycles requires a bunch of things to happen, perhaps to close that kind of stuff. So we added to that list of criteria, carbon awareness. And so if we can, you know, with this within a certain timeframe, Find a period of the day where we think that the CO2 intensity and the grade is going to be lower.We're going to try to do the update during that time, rather than at a time of day, which might be optimal from the other criteria, point of view, but might have higher suit to intensity. So that's. Feature, that's just been released to windows preview. And in an upcoming version of windows 11 is available to windows insiders, not available to the general windows population yet.And they're doing evaluation and testing of, of that feature at this point, that's the ability to shift across the time of day, which we would call time shifting. And then there's the ability to just shift appropriate workloads. Yeah. To the place where it's being run and that's what we would call it.Location shifting right. In, in the, and there's two, I think, critical challenges with each, right? In, in time shifting, you have to have some ability to be able to move the load to a different period of time. And that's where you were kind of referring out. Some is like, Hey, when I'm using a webpage, I can't really move the processing of that visit to a different time of day.So. When you, and you have to be able to sometimes predict when that is, because you have to, a lot of times you're scheduling into, you're always scheduling into the future, but you don't always have. Long period of time to look forward and wait for a real time signal. So sometimes you have to prep your workload and predict when that is.There's some interesting ML and AI stuff that while time invest in to predict when that period would be, so you can get ready, your load can get ready and do it at that time. And, and that's one thing that is really important for time shifting now, location shifting. It also has to be appropriate workload, but in a different sense, many compute workload.Art require huge amounts of data to be able to read in and data shifting is really hard. It's actually probably a it's something that would block a really large scale location shifting implementation. If you had a huge large data dependency on that. So things like training machine learning algorithms are pretty hard to.Location shift. If you're not already geo distributing your data to multiple data centers around the world. And many people are, you know, I'm sure Google in a lot of senses, geo distributing their data to many data centers around the world. And then they shift to, they could, I don't know what they're doing.They could shift their. Processing to those, you know, regional locations, which had lower carbon intensity at the appropriate period of time and, you know, say follow the sun or follow the wind around the world. As long as the data was already there.Henry Richardson: And that doesn't speak to the political challenges of shifting data, which is like, you might have different jurisdictions, like you have specific rules and won't let the data outside of the boundaries. So they're not only technical challenges, but also geopolitical challenges. I would say.Scott Chamberlin: Totally agree Henry. Yeah. Asim Hussain: Every time you try and have this conversation with anybody about location. Shifting that's the, the word data sovereignty just comes up almost immediately in the conversation and it's challenging. But then again, like within large countries, like the United States, there is still a lot of variability between east coast and west coast.And the date, I believe the United States is one data sovereignty region and the same thing can, is it not? Um, she's shaking her head, Scott.Scott Chamberlin: A lot of the data sovereignty laws, the privacy laws are being written by the States, today as though like say Illinois has a really strong one in California is really strong. it depends on the nature of the data, whether it actually falls within that data, sovereignty law, not all data is going to, there's a lot of data that's just generic and is not tied to individual privacy stuff.And so that certainly wouldn't apply, but when you're doing, you know, Machine learning or algorithmics, or, you know, big data processing on things that are associated with users or have data privacy policies associated with their collection and use. yeah.there's going to be even in the United States, a lot of times per state laws, you would have to comply by.So again, it depends on the nature of the data about exactly. You have to consider when thinking about these kinds of things, there's a lot of things like in processing, you know, batch processing, a lot of these cloud concepts there start when you, when you think of a cloud native, you know, world, right.There's a tons of cloud concepts that are really appropriate for. Both time shift and location shifting, you know, in NLS to me, you did a lot of work on, on batch processing. You know, there's the, the work that has been done both in Google and Microsoft on, on cargo or Kubernetes, like how do you build it into the infrastructure so that if you do have appropriate data, you can start to have the data center operating in a carbon aware way and that's, that's analogous, you know, like We in windows, On the client and data center, you have similar concepts, but are more operating on those cloud data workloads, which are very different than what the client workloads are like.So.Henry Richardson: We focused on a lot on the challenges. We were just surprised by seeing, by seeing how many people are actually figuring out how to navigate a lot of those. Like maybe they identified instead of, because data can be so. Maybe they identify two data centers that are in different regions and just have local copies of both of those so that they can pick when they train.So they're not picking amongst the entire set of data centers, but a specific set or like the windows opportunity. I, would've never thought of updates as an opportunity for flexibility, but it's a huge, like you have up to a week of flexibility. Whereas a lot of the conversations we have are like, we need this job to be done by the morning.We only have. 12 hours of flexibility, but so the more flexibility you have, the more savings potential you can achieve. So I think we talked to a lot of creative engineers who have identified opportunities within their very specific software to figure out how to make it.Asim Hussain: And I think that's one of the exciting things about this space is that there's just a lack of knowledge. And this is kinda one of the things I believe for a while is if you pass on this knowledge to people, I mean, hopefully some people listen to this podcast now, or then have an idea regarding some aspects of their workload or something that they can maybe explore with, with carbon where computing one of the things I've always, there's always been a lot of interests.You know, one of the things we do in the green software like movement is we, as you look at kind of various, you know, as you know, various touchpoints to reduce the emissions of software and carbon awareness is just one of them. There's always been a lot of interests. The interest comes from the fact that relative to the investment, the return is quite high.It's not, this is not going to be the solution, the one solution and organization adopts to, to, to reduce all of their emissions. But relative to the investment you've got to put in, you know, the return is quite high. I think I've seen those even that there was a paper recently. I'm not too sure. Much adheres to what I've heard from people I know in this space, but it's talks about an upwards of up to about 30% emissions reductions from workloads.Although I've heard kind of at a top start more about 10%. I don't know how kind of, what are your, have you, have you guys heard anything about this, about the potential improvements from, from the.Scott Chamberlin: I think, well, I can't really reference the Microsoft saving specifically. I don't think they've released that information. I think what you can say is that it's highly dependent on the parameters of your problem. You're trying to solve like Kendra is referencing in terms of the amount of timeframe you have to be able to shift, or the amount of locations you have to choose from, for shifting and what the marginal emissions are in those locations.Right? Like shifting from you know, coal to natural gas might have a certain percentage opportunities shifting from coal to. You know, a hundred percent renewable, like wind or solar is going to have a much different, and if you can completely shift or partially shift, you're going to have a bunch of different stuff.I think almost every implementation is going to have a different upper bound at what the savings is and, you know, getting good at measuring that and identifying what that is, I think is if you were to like break down, if somebody's thinking. Building some time shifting or some locations shifting carbon awareness into their application.Certainly in the number, one thing that any user, any developer would need to do is model their potential parameters that they're going to be con they're going to constrain their problem. You know, come up with some estimates. Like if I can S if I can move 5% of my workload within 24 hours in DC location.I have this much potential savings. And then going back to ask them your previous point about the potential, like the cost versus benefit and the modeling, the work like there's, it costs this much development work to be able to do that versus this much savings and any developers are going to need to do.Like modeling and that estimation before they go forward with an implementation, because I can certainly think of many problems, which might not benefit greatly, especially if they already are very small and the amount of emissions they're generating and it might not be worth the implementation for that.You might focus on other things, but there's certainly ones that are, you know, generating a lot of missions. The, the attributes necessary, like the flexibility and time or location, the.data dependency stuff we've already discussed. There's certainly problems that could. Great benefits in terms of implementation time shifting.But again, the prereq for all of this is To model that out, understand what that potential is before implementing. And then if I can, I want to talk about one of the things you mentioned asked them in terms of the cost is a benefit. I think you're totally right. Like I, you know, in other places as some, I'm just, you haven't mentioned here, but you talk about software sustainability.The first, if you were to like, create a. A classification of software sustainable, and you've done this previously and I've seen it. And the first branch and that is, you know, making carbon efficient applications. And then the other branch is making carbon aware applications. Right. And so those of us who are new to software sustainability might think of.Efficient applications in the past, we might've talked about this performance engineering or improving the efficiency of your algorithms or stuff like that. Almost all of that is really hard. If it was easy, people have Buddha probably have done it already. Right. The nice thing about carbon awareness is that it's a different.Way of thinking about your algorithms that are already running and it doesn't necessarily require you to reengineer your algorithms or to change the underlying. Implementation of your software. You're instead changing the scheduling about how that, how those underlying things work. And yes, I totally agree with you.Like from a concert has been at Fort point of view in a lot of cases, the low-hanging fruit is in carbon awareness and software that I've seen.Henry Richardson: To kind of build on that. Once you've identified a piece of software that could have flexibility, both spatially or temporally there kind of tend to be two big factors that drive the potential one is how variable is the, is the location that you're in. So is there a lot of variability in the admissions rate?And can you take advantage of that with your flexibility and then comparing across regions? But then the second piece is how capable are you a forecasting that variability, because then, you know, can you take advantage of that variability by scheduling it? So do you have 24, 72 hour week long forecasts that you can begin to say, how well does that forecast match?What's actually going on and can I take, can I use that forecast actually think about when to schedule? So the first step is really saying, what software do I have that can take advantage of flexibility? And then the next section is. Once I had that flexibility, is there an opportunity to actually reduce emissions with that flexibility?So you need both of those pieces to really, to be successful. Scott Chamberlin: Yeah. And Henry, so, you know Wattime provides a forecast, correct me if I'm wrong, it's up to about 24 hours. Right. And Henry Richardson: We just extended it to 72, but yes. Scott Chamberlin: great. And do you have any stats that say how they speak to the accuracy of the forecast over certain periods of time and. Time parents start to be really unpredictable and it is, it like correlated with whether it is correlated with a bunch of other stuff.That's becomes more unpredictable. The further out you look, you look Henry Richardson: Yeah, that's a, that's a really interesting question. We we've shifted away from an accuracy metric towards an efficacy. So, if you were to shift based on this signal or this forecast, the signal, how effective are you at reducing emissions? Scott Chamberlin: okay. Henry Richardson: And so if we get the magnitude a little bit wrong, but we get the rank order or the, the, the, the time, right.That's much more important than the absolute magnitude, but that's just a training trick that we use on our backend, but it kind of can be represented as accuracy as well. But to answer your more, the deeper question of like, what characteristics do we see? We see that like solar dominated regions tend to.Slightly easier to predict because solar is much more reliable when the sun's up. And if you have cloud cover, you'll be reasonably okay. When can be much more unpredictable. So wind dominated regions tend to have variability. That's hard to detect far ahead, but we might be say, we think that this hour is likely to have curtailment.We might not know the exact five minute period when we're throwing away wind, but generally we can shift load to. The periods that are much more likely to see that high variability or low emissions period.Asim Hussain: You just used the word curtailment and I feel like we need to educate people as to that's what that magic.Henry Richardson: Yes. It's a very jargony, I apologize, but in the industry we often refer to when we throw away wind and solar, because there's an excess of it or where there's not enough capacity in the transmission system to, to move that wind or solar to other places as curtailment. And we're starting to see. Certain grids throw away wind and solar at kind of pretty prodigious rates.California throws away quite a bit of it in the spring because there's an oversupply of solar because the sun is shining, but it's high temperatures haven't arrived yet. So we're not running air conditioning also in the great Plains. There can be a lot of wind at night, but low load periods. So there's an excess of wind, I believe even in the Pacific Northwest occasion.In the spring that the same low load situation, when there's lots of wind and solar, they'll actually spill hydro over the dams and not generate with it because they have to release it. So you can see how, like, if we can take advantage of these opportunities through load flexibility with software, that's, that's an amazing opportunity.We also talk about devices often, too. So smart devices, EVs, that type of thing can anything that has load flexibility. We're very focused on software in this conversation, but you can see how it could be other things. Scott Chamberlin: Right. And again, I think I was thinking about that very concept Henry, in terms of, you know, you have to think about software in a very broad sense. When you talk about the total opportunity here for only talking about PC software, the total opportunity is, you know, going to be limited by the number of.PC's in the world as, you know, windows devices, Mac devices, you know, and throw in, obviously, you know, the mobile devices in the world, which kind of sip power, but, you know, we all need to think about the broader definition of software it's software running, you know, in our thermostats, even though it's driving, you know, Both, you know, energy, if you're, uh, you know, electric heat or electric heat pumps and stuff like that, or natural gas use, which doesn't have the same benefit of the time shifting, but it's software, that's running like, you know, you, my robot vacuum cleaner, that's sitting right here.It's software, that's running, you know, almost everything, you know, Future how homes and businesses are being controlled by software and have differing abilities to take advantage of the topics of carbon awareness that we're talking about. Right. And so, you know, the IOT space is huge relative to the PC space.When we typically think about when we think about software or the cloud space, when we think about the Microsoft software, but those are all software developers and they all. It a lot of sense have connected, you know, internet connections and can take advantage of some of these signals. And, you know, I think another area that, you know, we, we talked about, I don't have the ability to talk about too much, but we need to think about what, how does this take advantage of disconnected environments?Not every, not every phone is connected at all times to the internet. And can you still do carbon awareness when you're disconnected? I think there's a lot of work to be done there. A huge impact or not. I don't know, but some of the modeling and stuff like that, it has seasonal variability as well, which might be able to be built in as a baseline.If you don't have, you know, rich live internet connections on at all times. Henry Richardson: We've had some conversations with our partners. Like what about fall back schedules or. That's still to still do some scheduling that can maybe not perfectly identify that variability, but can still take advantage of some of the grid emissions, variability. And we've also had conversations around like how frequently should that device be connecting to the, to the kind of grid emission signal and making decisions based on that, because that has a, a carbon penalty to it as well.Because every time you reach out, run the computation to decide when to schedule load, there's a, there's a carbon cost to that. And are you, you have to make it. The flexibility is achieving savings greater than the cost. Scott Chamberlin: Yeah, we ran into and out of a feature, another feature, I won't name the feature, but we did investigate a machine learning, you know, approach to reducing, you know, matter of power, use a windows. It turned out in that case. The amount of processing and power used for the processing to run the algorithm was greater than the potential savings for it.So, yeah, you're touching on a really great point. Then that goes back to the point I was trying to make in terms of, you got to model all of this, but yeah, Henry, you totally got a model. The. New stuff you're writing as well to make sure you're not, you know, stripping out all the potential savings by the new code.You're going to start running here. And hopefully you're looking at loads that are large enough that, you know, the amount of algorithmics and. You know, connections and, And services you need to ride to do time awareness is probably going to be much, much smaller for an appropriate workload that you're looking at.But again, that's where the modeling and the measurement is super important to be in with.Henry Richardson: And we've seen people scale that level. Like if it's a small workload, they'll just pull the forecast once, make a decision and then not check it. Or they can even do it every like three hours instead of every 15 minutes or something like that. So there's lots of ways of like adjusting the workload to the, to the job.Asim Hussain: One thing I wanted to cover. I think, I think it's quite interesting to also cover the future because one of the things I think Henry you've mentioned to me and I think is quite important, is that everything that we're doing today, like if you talk about modeling something today, That's today's impact yet.The world is actually when we're moving towards a future where more and more of the energy is coming from renewables. And therefore the impacts. If you were to build something today where your carbon, where workload has an impact of 10% in five years’ time, it might have an impact of 20% because the world's becoming kind of a lot more variable.I mean, do you have any estimates of how that's, how that's gonna go in the future?Henry Richardson: We definitely are seeing an acceleration of renewable deployment, which is increasing the variability of grids. I mean, historically. The electric utility is balancing authorities. Grid operators have always matched generation to demand. And I think we're shifting into a paradigm where we're going to have to be matching more of the demand to the variable generation coming from wind and solar.And so as that variability increases, we're just seeing kind of dramatic increases in curtailment. Renewable deployment that enabled just much greater savings. You're kind of shifting from a world where you're occasionally trying to pick up that excess renewables to a world where you're trying to avoid the peaking fossil plants, which is just a much greater opportunity from an emission savings span, where you just move a load as far away from the peaks, instead of trying to find those trucks.Asim Hussain: Can we just dig into that for a second? Cause I think that's quite, that's quite interesting because that's almost the opposite of curtailment because you, are you talking about peaker plants? They're so.Henry Richardson: Exactly. So you could have a hundred percent renewable all the time, except for occasional periods where they have to turn on those really dirty peaker plants, whether they're fossil oil, fossil oil, fossil gas, or fossil coal, as you just want to avoid those periods at all costs. Instead of right now, we're saying that seeing occasional periods.Where we're throwing away renewables, and you want to move as much load into those. So it's kind of like this expansion of opportunity, which is really.Asim Hussain: Because because those, my understanding those peaker plants is, you know, the grids need the capability. The energy very, very, very fast. And they tend to be natural gas don't or some sort of gas. Cause you can just burn that quickly interrupted workloads, I think is what it is. It's interrupting. So not running something for five minutes could be as valuable as shifting your workload to another hour because you're avoiding the worst emissions.Henry Richardson: Absolutely. And of course you want longer periods, the more flexible. Great of that opportunity. So there could be a two hour period in the afternoon where they have to bring those peakers on. And if you can avoid that, that can be really good. Scott Chamberlin: But, but talking to this variability, one question that comes up pretty frequently is the nature of different grids and the makeup. And especially when it comes to grids that are. Are more towards the grids. We're going to need say 30 years from now, which a lot of times, you know, has the peak and Henry you're more of an expert on this.So, correct me if I'm wrong that the peak loads are going to be handled by in some of these. Nuclear. I think I take long-term ideally in some cases, and then the base load is going to be, you know, renewables for the most part. And so in those grids, you know, I, when I think of a couple regions today that we deal with Iceland, you know, typically we treat it mostly a hundred percent renewables.France is another one because it has high investment nuclear. We kind of treat this hundred percent. Carp R zero carbon region. And again, Henry, you have to correct me. I'm wrong, those two regions, but it's because they have this nuclear renewable, you know, not the nuclear in Iceland, but in France, high Metro renewable.And in that case, the carbon awareness is just kind of a flat clean signal. And so as the grid evolves to these things that are like zero carbon grids, like the techniques we're talking about, they don't have as much impact. Henry Richardson: So, this is a really interesting disconnect that we're seeing right now. And especially in the near future load, flexibility will have a lot of emissions savings potential because it'll be able to shift out of those dirty periods into the curtailment periods. But once we eventually attain those a hundred percent or near a hundred percent clean grids, the flexibility won't be saving emissions directly, but it will be enabling a hundred percent clean grid because will be following wind and.And if we didn't have that flexibility, we would have to be fossil resources. So like it's an essential piece of a clean future grid, but it's going to be harder to quantify the benefit of it in that view. Scott Chamberlin: That's a great way of putting it Henry it's it's we get to clean grids faster. The more we have carbon awareness because carbon Alinea, or this allows us to maximize the use of our renewables. Whereas today we're already curtailing them. Right. I think that's, that's an excellent way of putting that. Henry Richardson: Exactly. And so it's, it's like a critical piece of that future grid without it. We wouldn't be able to obtain it as quickly as efficiently as, as chief.Asim Hussain: It makes me realize I had like a, an epiphany moment a year or so ago when I, I realized the way this is interesting in computing, but just generally the way we consume electricity. Is based upon the way the energy grid was created. Like a lot of other things we do in our life, we flex based off of what's going outside in the world.Like, I, I don't try and grow plants in winter and my garden because it's cold. Right? So we, we normally have this thing where we flex and we change what we do based off of the natural cycles of the earth. But because we've just been, had this thing called coal, which you could just burn whenever we wanted, we've not had to have.That pressure in the rest of our world. And what renewables is bringing into the world right now is like, well look you, you can actually do I find it? I actually find it quite beautiful because oftentimes we're way disconnected from nature. And one way I think about carbon awareness is actually I'm building software, which responds to natural cycles of the earth.And it connects me with nature in a, in a, in a kind of abstract, an indirect way, but it's one of the few ways you can connect with nature. I think in software,Henry Richardson: It's a really good, interesting point about the electric grid in that it's very unique in the sense that it needs to be balanced instantaneously at all times. There's no flexibility in terms of timing. So if there's a demand on the electric grid that has to be met immediately, you can grow rice and store it in a silent.Or it's an, a grain elevator for a while and then release it. The electric grid has to be instantaneous and until we have a lot more storage or pumped hydro, we're not going to have that flexibility. And so demand has to kind of follow supply much more closely.Scott Chamberlin: I think it was in California, right. Where they're starting to look at in terms of increasing storage by starting to enable all this, the growing amounts of EVs in the world. Connecting those batteries and utilizing them as a bunch of local storage. So we have this like future that's way more complex in the sense that, you know, we're, we're drastically increasing the amount of storage.Sometimes it's centralized storage. Sometimes it's this municipal storage. That's like, you know, you're, you're , and it's just can be a local buffer, not only for your house, but also for your driving. And you could charge that in a carbon aware way. And then. You know, we're adding municipal, solar, ratting, you know, utility, solar, adding all this.And I think, you know, the grid gets way more complex, but like you're saying, ask them, I think it all slowly, you know, you get these new patterns, these new natural patterns that start to arise out of it. Yeah. The technology is going to play a really key role in all of this, how to, how to implement. Henry Richardson: I'll toss another bit of jargon in here, V2G vehicle degrades. So there's both smart charging of your vehicle, but there's also can that vehicle actually push power back to the grid at important periods? Whew. It's all software driven, even though it's living on hardware. One other question for you.Awesome. Cause I, I think we're approaching the end here, which is. One of the things that we're excited about, this is that it puts the capabilities in the hands of developers to actually affect this. This is something that a developer can make a decision about actually make, may affect the software that they're working on and have a real emissions effect.Is that something that you've explored with?Asim Hussain: Oh, you talking about some of the projects that are working in the foundation, because we have one particular project, which is a carbon aware of soft SDK, the software development kit, which is a lot of what we're describing here. I mean, the logic is the same. Every single company wants to implement carbon awareness is pretty much just creating the same logic.And so one of our projects is create the carbon where software development care, which is gonna enable people to enable developers to much more easily implement some of this logic and functionality. And I remember when I was having, we were having conversations with our team and kind of the same thing came up, which was I w w we started this with this idea that we're going to reduce the carbon emissions of software, but actually software drives everything.So this SDK could be used. For, I have an E vehicle in my driveway. So it could, I could build something to leverage it and charge my Evie based off of, you know, some sort of signal or heating my house or something like that, because I think that's, that's one of the things we're seeing is if we can just make it easier for people to do this, then the more likely to, to, to implement a lot of these things.And I'm seeing it, I'm seeing it implemented another place as well as lots of websites that, and I love the impact might not be so good. In terms of carbon savings, but impact's quite high in terms of making people aware of what the potential is. There's a really great magazine called Branch Magazine from one of our colleagues here, Chris, Chris Adams, and you know, the it's an online web magazine, which changes its behavior based off of electricity grids.And so the images will disappear if it's, if it's high carbon intensity and replace it with text and words. And that's really good because everybody reading that is suddenly then aware of this entire concept, because people aren't aware of it in the.Henry Richardson: One of the pieces that I think I really like about it too, is that a lot of this is being driven by the developers themselves. Not necessarily the sustainability to. Like the corporate sustainability team at these organizations, they're like the developers see an opportunity. They understand how the code works and can actually make a decision about how to drive emissions or that.Scott Chamberlin: I a hundred percent agree with you. I mean, some of that is natural in the sense that the. Inability to step in. Most corporations are driven a lot of times to the supply chain organizations. And a lot of times that's because the measured CO2 impacts a lot of the majority of is sometimes coming from.Supply chain, but I think the opportunity in terms of cost versus benefit on the software side, I think is it's, it's an area where we can change faster and have some initial impacts greater than some of the supply chain teams, which supply chain changes, which are our longer term kind of things. And to be clear, all of those, both sides of the thing are totally interleaved.There's not a fine line between them. Asim Hussain: So I think that's all we've got time for today. So it was really wonderful conversation that we want. And just a final thought for me. I just want to give a shout out to an event that's happening in the middle of June. So the foundation has a summit, a global summit, which has been run over 20 locations around the world.If you want to meet other like-minded people, people like us kind of thinking and talking about these topics. You know, come find us a confined your local event at summit.greensoftware.foundation.Scott Chamberlin: I think the final thought is that, you know, having gone and tried to build carbon aware software, it was as long as you're making sure that you have the ability to measure and that you are. Actually doing the engineering that is going to have an impact it's actually super motivating to look at. And it's actually the technologies.It's a rich, rich area of technology. And it's may seem intimidating when we add these new kind of terms about to, about how the grid operates. You have to think about a, yet another thing is software, but it's, it's really, once you get in there, the concepts are pretty straightforward and adjusting yourself.Do this kind of stuff is actually not too hard. So I'd encourage folks to, to try it out at least. Henry Richardson: I think one of the things. About, as people are coming up with use cases that we never thought about. So scheduling windows updates, like we hadn't even considered that as a possibility. So people come up with very creative ideas, but with region shifting location, shifting that, that we would never have thought of.And so we're always excited to see that kind of expanding possibilities for loaded flexing.Asim Hussain: Thanks for listening to Environment Variables, all the resources for this podcast, including links to our guests and more about carbon where computing as well as the Green Software Foundation. The show description below. We hope you enjoyed the show and see you on the next one. Hey everyone. Thanks for listening.Just a reminder to follow environment variables on apple podcasts, Spotify, Google podcasts, or wherever you get your podcasts. And please do leave a rating and review. If you like what we're doing, it helps other people discover the show. And of course we want more listeners to find out more about the Green Software Foundation, please visit green software.foundation.Thanks again and see you in the next episode.[END] Hosted on Acast. See acast.com/privacy for more information.

In this episode Asim is joined by guests Chris Adams, Sara Bergman and Danielle Erickson and they discuss the impact that Amazon’s Customer Carbon Footprint Tool is having on the green software landscape. How do services like AWS affect climate change and what are the effects on the environment of these huge data centres? We also learn about how you can use heat from greenhouses to grow tomatoes!Learn more about our guests:Chris Adams: LinkedIn / GitHub / Website Sara Bergman: LinkedIn / Twitter Danielle Erickson: LinkedInEpisode resources:The Green Software Foundation Website Sign up to the Green Software Foundation Newsletter Person: Adrian Cockcroft Blog: David MyttonBlog: SRE Methods & Climate Change - Benoit Petit Presentation: Green Cloud Triangle Magazine: Branch MagazineIf you enjoyed this episode then please either:Follow, rate, and review on Apple PodcastsFollow and rate on SpotifyConnect with us on Twitter, Github and LinkedIn!Transcript below:[background music]Danielle Erickson: We're looking at the AWS tool, the Google tool, and the Microsoft tool and understanding the broader strategy, so when you combine those two things, I think that's what we have to do right now to strategize in the best way to reduce our emissions.Asim Hussain: Hello, and welcome to Environment Variables brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding Green Software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software. I'm your host, Asim Hussain. Welcome to Environment Variables, our new podcast. My name is Asim Hussain. I'm the executive director of the Green Software Foundation.Chris Adams: Hi, there. My name is Chris Adams. I am one of the directors of the Green Web Foundation.Asim: The Green Web Foundation, the Green Software Foundation, there's a story behind that.Chris: There is indeed.Danielle: Hi. I'm Danielle Erickson. I am the product manager of the Cleantech service line at Thoughtworks and the product manager of an open-source tool we created called Cloud Carbon Footprint.Sara Bergman: Hi. My name is Sara Bergman. I am a software engineer at Microsoft and I'm also the chair of the writer's project in the Green Software Foundation.Asim: All right. I think we're here today to at least start off talking about Amazon's Customer Carbon footprint tool that they announced recently. This is something I saw they announced in re:Invent which if I remember right, was it November or December in last year, in 2021? They announced it in November or December and it's a method of measuring the carbon emissions of customers' workloads on Amazon.I don't know how all you all feel about it but I was really impressed with just the speed with which they made their second announcement, which was just, when was it? Weeks ago now or maybe February, they then announced it in preview. It was incredible breakneck speed. I was expecting from their re:Invent to wait a year for them to publish something. For me, that was a really, I felt like I had a whiplash. I was like, "Whoa." Has anybody else had a chance to look at their announcement?Chris: I have. I think it's really cool. I'm really, really glad that something like this exists because if you consider yourself a responsible technologist, I figure the electricity has to come from somewhere, and being able to have this information allows you to optimize for carbon and given the information that is shared with us increasingly in the news, it's worthwhile actually referring to this. For example, the WPC, so the IPCC report explicitly mentions digital and the role we have to play now in that. Having the tools to instrument that is really, really handy. We actually used to work on something like this. A couple of years ago, we built a tool called Amazon Green Cost Explorer, which basically used some of the billing API to work out which regions were green and which ones were not green, so you can act on this. To actually have something much more fine and green is really, really cool, but it's not the only project going in this field, I suppose. It might be worth talking about that, yes, there are other ways that you can understand the environmental impact if you compute than just using this, for example, even though it's very, very useful and very, very welcome.Sara: Yes. I think that's a really good point. I think, for example, if you think about performance, some people are very interested in getting into the nitty-gritty. I want to read the logs, I want to really get down to it but others are just I want to see my latency. I think the same goes for carbon, right? For some, this will be revolutionary and this will be a great way to get any kind of insight but there are others who are ahead of the pack where maybe the granularity isn't enough or isn't timely enough or whatever but this isn't a really good first step in my opinion.Danielle: This is something that customers have been asking for for a while, so to see Amazon respond in this way and give some visibility that I believe is generally available, this tool. Anyone who uses AWS can see their carbon emissions over time is really incredible. It's a great step in the right direction and as Chris already said, there is a lot more regulatory pressure to be reporting on this. Everyone's going to need this and I think if Amazon can make this first step, we can hope that in the future, they'll continue to respond to this demand and this need the customers are having, so really excited about this.Asim: This is my mistake as well. It's generally available, is it now, Danielle? I just assumed it was a preview.Danielle: That was my understanding, but Sara, you may know better?Sara: No, I heard the same. It sounded like it was for everyone who uses their services.Asim: Yes, it's done.Danielle: Yes.Asim: Wow. That is very impressive to go. I mean, it means they were working on this for a long time, I think. You don't come up with a tool like this in two, three months. This has been something they've been working on for a while under the covers, I imagine.Chris: I think you're right, Asim. I mean, if you've been following this, Amazon have been hiring for sustainability specialists for the last two or three years. Also, if you look at the VP of cloud at the moment, I think, or one of the VP of Sustainability, Adrian Cockcroft, he's been speaking about this for a very long time. If you followed him on Twitter, he's actually had a lot to say about this even in the pre-Amazon days, actually. It's really, really cool to actually see some leadership here on this. I'm quite impressed with this as well because between 2019, there was actually a talk by AWS specifically at a conference called, Map Camp where they were explicitly calling this stuff out.They said, "Look, if you want to do this, you're going to need to tell us as customers because we're not seeing the customer demand for this." There's a slide of Mr. Cockcroft standing with a big thing behind saying "The thing you can do is move to the green region." Now the thing they've created now is something which provides a bit more visibility to that, so rather than just having that as your option, you've got ways to optimize the actual compute in place rather than having to take on what in many ways could be a risky or scary migration that you might have to weigh up against other things like feature development or the other things that product manager or a CEO might be asking for.Asim: For me, it felt like the cherry on the top because Microsoft announced their, I think it's now called the Emissions Impact Dashboard a year ago. Then I think it was six months ago that Google announced their dashboard as well and so with Amazon coming out on it as well now, that's all three of the major clouds, all [unintelligible 00:06:39] use that major clouds now have a capability of customers, basically being able to answer questions along the lines of, 'Well, how much carbon emissions is all of my emissions?" This is where I think Thoughtworks would be leading the pack as well because I've forgotten what you call it. I keep on getting confused, Cloud Carbon Footprint. Is it?Danielle: Yes, that's it. Yes. [laughs]Asin: Okay. Yes. Cloud Carbon Footprint tool because you've actually been developing essentially an open-source version of this for a while. You must have some really deep insights into how do you actually go away and calculate some of these numbers?Danielle: It's been really cool seeing the three different tools come out and right now, our team is going through an internal process of reviewing each of the different tools and understanding the variety of features that each of them has is what they have to offer. As much as we can understand how they're getting the data and the methodologies they use to calculate carbon emissions, we're trying to do so because each of them are going to give you really the best numbers you can get for each of those individual cloud providers, but one thing that they're unable to do at the moment is compare between each other.For many organizations, the majority are multi-cloud users and if you're trying to look holistically at your sustainability strategy and your cloud emissions, you likely want to see them in one place and also using a similar methodology. If you're looking to compare, if you're looking to really optimize, take action, you'd want to compare them apples to apples, not oranges to apples. For our perspective, using all of these tools together is really the best strategy. Have a lot of tools in your pocket to understand what's going on and then begin to understand the areas you can start to make changes.[music]Asim: I think we've talked enough about all the glowing praise for all these three tools. We now dig into the issues with them. I think one of them, like you just touched on there is exactly how is Amazon calculating its numbers? How is Microsoft calculating its numbers? How is Google calculating its numbers? There's a lot of opaqueness because they're not revealing that. They're just revealing, 'Here's your total number."Chris: This is one thing that I could share some light on, I suppose. There is some good news in that increasingly organizations are now talking about essentially, how they share which parts they do measure, which parts they do not measure inside this. We have established ways to track some of this stuff. There are things from the GHG protocol, which is an organization that pretty much sets some standards for this and they talk about things in terms of Scope 1, which is burning for carbon emissions on-site, Scope 2, which is electricity, and then Scope 3, which is stuff in your entire supply chain.A lot of the time when you might look at some of this, you might have people talking about just Scope 1 and Scope 2, for example, without necessarily talking about the Scope 3 part. If you look at say, I know this is one thing that both Google and Amazon don't include in their numbers is basically the environmental impact from creating the servers in the first place. This is one thing that's probably worth talking about because well, they have to come from somewhere, and it's obviously an energy-intensive process to turn sand into silicon chips.This is one thing that I've been quite impressed with because there actually are a few open issues on cloud carbon footprint to start piecing together some of these numbers because this is actually very much considered the next step, now that stuff is being done on the energy front. There's a really good blog post in the show notes by David Mitten, who's been writing about this. I'd really recommend his blog because he provides a really, really useful set of incisive analyses in this field.Sara: Yes, I think that's an excellent point. Depending on the type of application that you have, the hardware emissions just from creating a server or whatever, the network devices, whatever you use can actually outweigh the pure energy cost of it. It depends, of course, on multiple factors, but it definitely can be the case. When we talk about engineering and engineering enablement, there are some pretty easy things that you can do to decrease the amount of hardware that you use, but if you're not getting measured on it, how will you be incentivized to do those actions?If it's pure cost, well, we are very much relying on cloud providers being kind enough to give us a cost which is mapping to carbon, but that isn't necessarily true always. Right?Asim: Yes. I think just essentially from my understanding, Microsoft's emissions dashboard gives you Scope 1, 2, and 3, so it tells you the carbon emissions of your workload, your energy consumption, just to break it down a simpler format, your energy consumption and your hardware. Google currently just gives you your energy consumption of your workload. I have actually assumed the Amazon one was all three but is it just energy again? It's just energy again, so Amazon is just energy?Danielle: Yes, one and two.Chris: I've read the 451 report. In the announcement, there is a report by 451, and they explain what's in the model and outside the model. They basically said, "We're not looking at embedded energy, and the actual machines themselves, and we're not looking at Scope 3 at present. We're not necessarily looking at Scope 1 because it's not quite so tight." This is primarily about the energy part and this is also why the numbers, as far as I'm aware, there is a lag, because they're looking to get the most accurate numbers, just like how Google do where they basically say, "We will move as fast as we can, but we are working with very, very large providers who might not bill on the same monthly basis. We wait until we have the information from energy providers, so we can give you an up-to-date number." This is what is actually shared to my understanding.I've got to stress, I do not work at Amazon, so there may be much more detail that may be there that I'm not so aware of.Danielle: I'm not exactly sure the full reason for that lag but my understanding is it's about three months, which if you're getting very accurate information, can be helpful to look back and understand over the course of the year. I do see a challenge to the consumer actually trying to make changes and use this data. How can I act on data today that's three months old? It becomes a little bit difficult to build into your workflow, to make decisions day to day based on three-month-old data. That's something to consider, I think, with this tool, and maybe something they can improve in the future.Sara: Yes. What they're stating in their announcement is that it is the underlying billing cycle for the electrics utilities and I believe that Google is doing the same, but they are also quite late in showing. It really limits what you can use it for. It's still great for some type of comparisons. If I have two applications that are similar, which one do I continue with? Those sort of things, it's very good for, or comparing over time, but doesn't really tell you what I should do tomorrow. I think as more and more software companies move away from the waterfall and move into more and more aggressive agile- three months, no, is anyone going to be really happy with that? I don't know, maybe.Chris: Maybe there's one thing that you can talk about here in terms of, there may be different uses for this data, for example. I know that when I've spoken to people who are looking to use things like cloud carbon footprint, they've told me that there's two main use cases that you tend to have. There might be engineers like yourself or me when we want something like an SLO for carbon, I want to be a green SRE and there's a really nice post by a guy called Benoit Petit, who is one of the lead contributors to a French project called [unintelligible 00:14:34] which basically provides per-process level energy usage information that could provide these numbers.He talks about this stuff as an SRE saying, "Well, these are the numbers I need to basically optimize for, and I should have dashboards like this." There's some really nice work by the folks at Mapbox, who've been speaking about this for a while. They were some of the early contributors to the early green cross-explorer stuff for this where they were talking about, "Well, if we review our bills on a weekly basis, and we use that to shape our usage, it'd be awesome if we could do this for carbon because we're already good at optimizing for some kind of metrics, so it would be really nice to have something that."Increasingly, we do actually see numbers like that now. There are schemes which do make this stuff possible. Just last week, for example, there was a new standard which has been proposed called granular certificates by a number of organizations. This gives you hourly settlement for this stuff, which is really, really, really impressive. This isn't that well known yet, but this is the kind of stuff that what the future looks like in my view. I look forward to the time where this is actually a thing that you can optimize for as an engineer, and you can see on a dashboard, for example, for your team.Asim: I think really, the issue here is that we want to celebrate this work on the work that we're doing, but it's not quite there as a dashboard, that from an engineering perspective, teams can use to actually give them information to make decisions. That's basically the challenge that we've all got. As we're sitting there and we've got options between one, two, and three different architectural types or different choices. This doesn't quite give you that level of granular-- Regardless of even the methodological differences between the different platforms, even the granularity won't give you that.I can't speak for how Amazon does it. I do have some experience for how the Microsoft dashboard works, and it is very averaged out data. Multiple servers will always report the same energy consumption regardless of what you do because that's just how it's been calculated. That works great the thing you're talking about what is it used for? They're designed for reporting purposes. They're designed so organizations can calculate and report their carbon emissions to CDP or perhaps have an understanding regarding what are the offsets or neutralization strategies we need to employ. That's just what they're designed for. It's not built for engineers with the caveat of-- I think Google is on an interesting track.Sara: Yes. If you think like a person in an individual team with a small portfolio, then I completely agree, but maybe if your step up, so if you're someone who is responsible for a larger portfolio of services, then suddenly this means you're able to compare them. Sure the data is older, but I can then start to evaluate, "Okay, how much value is this service provided me compared to service B, and how much is their carbon footprint."If one is vastly higher, but providing me less business value, then that's a decision on a leadership or a planning level that you can take that these dashboards enable that you would not have been able to reach without this. It really depends on what kind of decisions you're trying to make based on this dashboard.Danielle: Yes, this is something that we thought about a lot when building the open-source Cloud Carbon Footprint tool. Our perspective has been trying to reach that engineer level, that day-to-day decision-making level with as much granularity as we can build in and as much real-time as we can try to make the tool, taking billing data immediately and turning it into carbon emission estimations. Not to repeat myself, but I think the benefit of having multiple options here is that you can combine them for these different uses that you have.Your engineers can look at both tools, combine the data that they're seeing from the Cloud Carbon Footprint on a day-to-day basis, and then talk to their infrastructure leads who are looking at the AWS tool, the Google tool, and the Microsoft tool, and understanding the broader strategy. When you combine those two things, I think that's what we have to do right now to strategize in the best way to reduce our emissions.Asim: Especially the fact that because Cloud Carbon Footprint is open source, not only is your methodology public, but your data and the underlying data assumptions are very low, granular level are public. I can see what is the energy-- If I'm using this particular server, this particular load that data is public. We're actually using that in the foundation in the software carbon intensity standard, where you're leveraging that data because it helps engineers calculate the carbon emissions of processes or estimate the carbon emissions of processes, so they can then make those kinds of decisions.It's the openness of the data is, I think, also missing with these tools. I've also heard it's extremely difficult for Amazon and Google and Microsoft to make this data public. It's not only they're revealing competitive information, there might also be legal constraints. If you reveal some of this information, the SEC might come after you because you're revealing proprietary information. There's actually lots of complications around that, from what I've heard. I wonder if others have thoughts on that, on the openness of data.Chris: I can actually weigh in a bit on this, which might be of use because you have a similar thing happening in the energy sector just the one layer below right now. One thing that we've seen pushback from people who run the energy grids in various places, they've typically said, "We are not able to share information about how congested various parts of the big transmission wires that move power around because we see that as a security risk." but this is actually a thing that we've heard in lots and lots of places. In many cases, a lot of the time you could see there's a trend towards opened for a bunch of this stuff.I feel like a lot of the time, if you're not designed or if you aren't used to sharing things open by default then you can come up with a lot of- it's understandable that you might not want to share a bunch of this stuff. There will be cases where you might not want to share this for very valid reasons. For example, there are probably valid reasons for not listing where geographically every single data center might actually be. Even if this may be information that as a customer you might want know if you want to understand what climate risk is associated with all the machines running in a particular place.Especially when we refer to examples like say- a most recent risk example might be the big Facebook data center, the big data center from Meta and Zeewolde in the Netherlands. That's eight meters below sea level, that gigantic data center. That's the thing you might want to know about in a world of rising sea levels. That's some of the stuff which is useful to know about, but going back to the original point. not everyone's ready to share information on a very, very open basis just yet, but I suspect that over time this will have to come up because well, environmental factors will increasingly push this and necessitate this kind of disclosure.This is actually one of the things that's been driving a lot of this stuff right now. It's because investors are basically saying, "I need to understand the disclosure in my supply chain." or "I'm invested in you as a company. I need you to share this information so I can end up with a net-zero portfolio. If you don't have these numbers, it's going to be very, very hard for you to share that." In many cases, organizations will basically say, 'Well, I'm not going to invest in you. I'm going to invest in someone else. At least I know whether risk is there." We're not open yet, but the more often we do get, the easier it is to make data-informed decisions as we move forward into this changing climate world.Sara: Now, I see the same security issues for hardware as well. Do you want to state exactly what type kind of servers are on your server [unintelligible 00:22:25] floors? Maybe not because there has been hardware security incidents in the past, I'm sure we'll see them in the future. Then you might not want to say exactly what you have, but there can also be an argument for finding what is a valid enough proxy that you don't state explicitly that this is this type of server, but exactly this hardware. I built it like this. I specify carbon cost or some other tangible number that gives you the information that you need without being a security risk.That is, of course, a lot of work especially if we think across all cloud providers even if your company is your own cloud provider while being on Preem, you would want to be able to compare across the stack. The lining on that without being open is difficult. That we're going to guess what our competitors use. I don't think that's a good approach. It's quite exciting from an engineering perspective, just the complexity of some of those things.Asim: That's a really good point that you mentioned about what data can you reveal? Because this is what we're talking about with the software carbon intensity specification in the foundation is what we want. We're talking at one level about give us all the data, but really, why do we want this data? We're actually trying to calculate our carbon emissions. Well, what really would be quite useful is just the carbon intensity. It's like this server, I don't necessarily care what the components are. I want to know how much carbon per CPU, per minute of this [unintelligible 00:23:55].I want to have that data, and if I have that kind of data, that's actually probably all that I need from an engineering perspective. That's probably all that I need in order to make decisions. It'll be a wonderful world in the future where everybody is essentially giving you this data is what is the carbon intensity of my service? What is the carbon intensity of this streaming service we're using right now? What's the carbon per minute? That's all I really need.Chris: It might be worth looking at some work that's happening in the web world that I've seen. There are tools like website carbon and increasingly there are tools that plug into analytics like Google Analytics to give you an idea of what the environmental footprint of some digital services over time might actually be. One thing that I've seen in the web world right now is this real push for having carbon budgets for websites. One company, Wholegrain Digital, they basically say, 'No website that we build will cost more than two grams of CO2 emissions per page load. That sounds really silly on a per-page load basis. Some websites get quite a lot of page loads, so over time this stuff adds up.If we just zoom out for a second and think in the outside world, there is a huge amount of science saying, "Yes, we have a budget we need to stay inside." If you look at the energy sector, they themselves have a carbon budget that they have agreed to stay inside which is why you have massive compliance markets. It makes sense. Probably we would also need to have something like that as well if we want to stay inside, like I guess the dictates of science. We don't get to change the physics of climate change but we can at least change some of the economics around climate change.We can at least do something around this so we can optimize for carbon as developers, so when we're building services, they tread more lightly or as lightly as possible given the various other requirements would be nice to meet as professionals, I suppose.Asim: It's interesting that you mentioned the carbon budget as two grams per page load, because that's an intensity, not a total. I think that's the thing that I talk about a lot that total budgets are really challenging in our world because there's just- how can you set a one-ton budget for our website? You have no idea how many users are going to land on it, but an intensity is, "Oh, [unintelligible 00:26:10]. '"Chris: I'm not so sure about that.Asim: Oh, interesting. Let's go.Chris: Here's the thing, Let's say you're going to go with this. You have a $2,000 budget or something like that because you know that you're probably going to get this many page views over a given time. This is the thing that you're seeing in procurement and contracting these days. They're basically saying, "Well, we have been given legally binding targets that we need to reduce our emissions by 5% a year, a year between now and 2030. That's it. We don't get to do not do this now it's in the law. If they have that, then they're going to have to say, 'Well, we're going to spend €100 million, £100 million on this contract for the next two or three years. We basically have an implicit budget that we need to stay inside."You do have something like that now. It may be the case that okay, having just one number over three years isn't very useful. You might want to have some smaller timeframes or something like that. This is why it might be useful to have a rate for this because you can say, "Well, given that I have this, I now have something I can act upon. I can either change the size of a page for example or I can change the intensity of the electricity so that's going to allow me to stay inside it. It gives me more options." I think it's useful to have the total number because this is essentially what's driving things from a science and regulation point of view. As a developer you might not be able to use it on a daily basis.If you have CI for example, you're going to want to have a unit because that's what you're used to using for your score from say, Google's web vitals. A web vital score is going to be a rate that you can refer to or something you can look at. It's not going to say over six months. It's a kind of volume basically. I think you need both, Asim, not just one, but it's very, very useful to have the ratio, absolutely.Sara: I think tools like the tool we've been talking about today, Amazon's new tool, that can give you that from an OKR perspective because you can see, "Okay, what's my cost? What's my page views? I do the simple division and I do get these numbers, but once again, it's for reporting purposes. If you've never reported on it, this is better than not reporting on it for sure. Totals are also interesting because you can go to a rate assuming you have the other end of the fraction, but sometimes you want to go the other way. That can be a lot trickier without the really granular data.Chris: Sarah said something really interesting here about going both ways and about if you've got a total number, you can go down from there. I've mentioned Wholegrain Digital before because I'm a really big fan of what they do and EcoPing is another group that do this stuff as well as Mightybytes who built a tool called Ecograder almost 10 years ago where they were tracking this kind of stuff. The model that is used, they call it the sustainable web design model that is basically based on a global figure for all the initial used by all the entire internet tech sector divided by all the data transfer that is facilitated by this.This is a bit of a course figure, but at least gives you something that you can act upon and work with. This is actually one thing that I think it's going to be live next week, as being able to use some of the tools that you do use if you build websites and have things like that. It's useful to have those kind of stuff. In many cases, you need to understand what the model is actually representing to see what's going into that, for example. In the example of CO2-GS, for example, this is using network transfer as a proxy to talk about things like, say, usage at a device level or usage at a data center level.Once again, without having access to the open models, it's very difficult to know where your interventions are going to make a meaningful difference. This is why I'm actually quite happy that things like Cloud Carbon Footprint are open enough and are accepting [unintelligible 00:30:07] requests. You can basically say, "Well, this is what I think is going on. This is what I'm trying to do in good faith to reduce the emissions of whatever service I'm building."Sara: Yes. I think we should also maybe mention that the granularity only matters if you have an application or service that changes rapidly. Not every software does that right, that we have stable software that's in a maintenance mode or for whatever reason, isn't that interesting to change frequently. Then this is honestly good enough. You don't need hourly data for a service that you're going to update twice a year. That's not needed. I think it's interesting to compare to, for example, the transportation sector.I was taking a train recently in Norway, and on the app, they showed me by taking this train, you've saved this much carbon. I got annoyed. What is this magic number? Then I clicked on it and they actually showed the entire methodology. I'm like, compared to flight, if ou are one person, da, da, da, this is how much. If you were to take one four-person car, you were one of those four people, this is how much you would save. I was pleasantly surprised. The reason they can do that is that is because the cost of fuel and the cost of building a car doesn't change that frequently. They can calculate this once and then use it for a really long time. For software that's tricky. You have rapidly changing software, so this is something to keep in mind.Danielle: The thought that you brought up, Sarah, about stable applications made me think of a trend work seeing with usually more mature organizations to try to start understanding, if I have stable applications and I also have typical architecture practices that I continue to use, can I start to understand particular architecture choices that impact different carbon and energy use? How can I understand and learn from those architecture choices over time, and then maybe even automate that process? I learned that this particular architecture choice is less carbon-intensive. Can I just make a dashboard or facilitate provisioning certain services in that way?Asim: Danielle, I think you're right. This is one thing that we haven't really got around to developing a language for yet about how you optimize for carbon at various places. in the show notes, I've shared a link to a thing called the Green Cloud Triangle, but we've spoken about this. There's a kind of iron triangle of compute cycles, response time, and cost that you might want to be doing trade-offs of. For example, there may be cases where you want to optimize for response time and cost. This is stuff where you might say, if you want something to be cheap spun quickly, you might go for say, static pre-build stuff.For example, you're not doing too much dynamic stuff. This is stuff we know already a lot of the time. It may be that there are some cases where you don't need to have things happen immediately, right? You might be more interested in keeping the costs low, but making sure you've got lots of compute cycles. We might use this in terms of having queues or tools or things like that.Then finally, this is like the default that a lot of us end up using when we're not thinking about this, which is basically optimizing for compute cycles and response time, not really thinking too much about the cost part or not really knowing that the cost can change in this way. This is like speaking to the fact in many cases and what I think we're going to see more of every time is that the cost of electricity changes depending on the time of day. This is not really exposed to us right now, but it's something that is definitely visible, that does definitely happen especially if you look at the markets such that sometimes the cost of electricity can go negative, so you can be paid to use Compute, for example.I feel like there needs to be a set of tools or a way to describe this stuff so you can take advantage of these changes that have been happening one layer down in the stack, so that you can basically architect for better, more responsive things, but also in a way that's actually very. very planet-friendly as well as wallet-friendly. I think there's a couple of good posts on the Green Software Foundation blog specifically about this. I might have written one of them, but the other ones have also been written by other contributors.[laughter]Sara: Well, I can speak about electricity because I think that's interesting. Many of these cloud providers, they say that it takes a long time for electricity utilities to get to them. During this winter in Norway, we only have hydro here, so if it doesn't rain, electricity becomes expensive or if it's really, really cold, all our water is frozen, it becomes expensive. There were these newsletters reporting almost hourly on like, 'This is how much it costs to charge a smartphone right now. This is how much it costs to use the oven right now." Obviously, we know what the electricity costs right now. I wonder how hard it would be to propagate that.It doesn't always correlate to carbon though. My carbon cost was the same throughout the day because it was hydro, but their cost changed.Asim: Yes. If cost- [unintelligible 00:35:05] cost is a proxy for carbon, really at the end of the day, I think here.Chris: All right, you're talking about cost as a proxy for carbon, and that's because a lot of the time when you have more electricity than you need on the grid, it's lunchtime and it's sunny as hell, or it's windy as hell, you've got more than you need. The problem with the grid is that the grid has to basically be balanced the entire time. Otherwise, basically, very, very bad things happen and very expensive hardware gets damaged that falls off-grid. You can end up with incentives to basically-- If you operate a grid, it's cheaper. It's easier for you to basically just set the cost of electricity to be lower than it is to ask someone who runs a big nuclear power station to please turn down the output to make sure stuff is balanced, for example.This is the thing that you are often doing. This is why the cost will change over time, depending on how much demand there is compared to how much supply there is. Most of the time we're shielded from this, but it's actually quite fascinating. It's stuff that you can absolutely take advantage of because companies do this. Google make a really big thing about shifting loads to when energy is green, but the reason that they do that is it really saves a bunch of as well as just carbon, basically.Asim: Yes. That's the secret to a lot of our spaces that there is cost savings, a lot of this stuff as well.Sara: I don't necessarily think that's a bad thing. Capitalism runs large parts of this world. If we can get those forces to work with us, promote us to a greener future, we shouldn't necessarily be against it. I'm saying capitalism is always awesome, but you take the wins where you can find them.Chris: This speaks to incentive design, basically and who's making good use of this. There's a really nice example of organizations. There's one organization in America called Lancium, and there's another one in Germany called XMesh. They take advantage of this. They basically take data centers which is basically a shipping container full of machines, which will otherwise be thrown out from hyper-scalers like Facebook. They put them on renewable energy parks. What they end up doing is they end up providing stuff for either-- okay, I'm not a huge fan of the cryptocurrency stuff, but you can use the same thing for machine learning models as well.Anything, which is a plausible load that is quite compute-intensive is a really good fit for this use of an oversupply of renewable energy in many cases. This is what Lancium and XMesh both do now. By being able to be plugged straight into places where they have energy, that they otherwise would not be able to use, they basically end up being able to provide Compute for a much, much lower cost. You can get your machine learning models done at a fraction of the price from some of other larger providers by going with this because they're taking advantage of the economics and how they've changed over the last 10 years that in many cases say some other there haven't taken advantage of yet.Asim: Why is that more economical? At the end of the day, those servers are servers that they got essentially maybe for free or very, very, very reduced cost. They must be four or five years old.Chris: There are two reasons. Moore's law has slowed down over the last, say five years, for example, before you could just rely on [unintelligible 00:38:20] to do this work. As a result, servers which are maybe two or three years old, aren't actually that much slower than they were previously. If you've got something like a plausible load, because you are not trying to run it 24/7 all the time, if you have different requirements for keeping this stuff cool, for example. Unlike, if we're going to talk about keeping cool, for example, there are some really nice examples in the Netherlands where they basically have shipping containers full of servers.Once again, these are servers which are end of life. They plug them into greenhouses with the idea being that the waste heat, rather than basically vented into the sky, or you spend loads and loads of money trying to get rid of it because you see it as a waste product, they use it to pump into greenhouses, so they end up with really nice juicy tomatoes. This is a really, really cool use of heat because the greenhouse folks, they were like, 'Well, we can either burn fossil fuels for heat or we could just use that heat from over there." This is an example of taking advantage of-- If you understand the underlying energy systems, then there are all these fascinating, new, pretty cool use cases. I don't know about you, but the idea of, I don't know, a greenhouse connected to a data center and juicy tomatoes, that sounds cool. I like that idea.Asim: I do know, actually that heating greenhouses is one of the biggest costs for greenhouses. That's wonderful. I love that.Chris: There are loads of examples here. When you look at the next challenges we are facing between now and the next say, five to 10 years, one of the big ones is heating things up. What we have right now is we have a massive data street full of data centers thinking, "I've got all this heat, how do I get rid of it?" It feels like maybe the people saying, "Well, if only we could find a way to get heat." and then we'll tell the people saying, "Oh if only I could go find it, we'll get a way of getting rid of this heat." If they could talk to each other, then maybe you could end up with a slightly more efficient system.Now, this isn't going to happen all the time because if you put a gigantic hyperscale data center miles away from everyone else, it's going to be harder to integrate that into in an urban environment. Then maybe that speaks to the fact that our idea of what a data center needs to look like could change over time to end up with a different topology for the internet because the internet did use to be quite distributed. What we've seen right now over the last 10 years is that the energy sector has ended up looking a lot more like the internet and the internet is now looking a lot more like the energy sector was 10 years ago.I feel like maybe there's scope for us to find some happy medium rather than just zipping past each other in mad decentralization or centralization mania that we have at present.Danielle: I think, Chris, with all that you're saying there's so much opportunity. My question is, where does the responsibility lie to provide that information to consumers, and who is responsible to make these choices of shifting workloads, taking advantage of the energy at different times of day, that type of thing? How much can the cloud providers do and how much can the consumers do and what is that balance? How do we get there? I think it's going to be a really interesting problem that, hopefully, we get to solve in the next few years.Sara: I'd love to see a carbon throttling thing that you can add to your services, whatever cloud provider you have. It's like, Yes, you can carbon social this application. That's fine for me."Chris: There is loads of cool stuff happening in this field right now. Brunch Magazine does examples of this. If you go to brunch.climateaction.tech, it throttles based on the carbon intensity of the grid right now because this is exposed to it. There are also tools [unintelligible 00:41:49], for example, that let you do this stuff. This stuff exists and there are examples of it being built. I think it's a really exciting fun place to be looking at this but there's a whole policy piece that would map to what we're doing here.[background music]Asim: This has been a wonderful conversation. I love all the places we've been to. Maybe let's just end with just one quick thought or idea from each of us and think about the future and something from our conversation. I might start because if I don't, I'll forget. There's something you touched on, Chris, earlier on, I thought was fascinating. You talked about what of Meta's datacentres was going to 8 meters below sea level.One of the things for one of our future podcast episodes I would love to explore is the SEC has just had a proposed I think ruling, I don't know if that's where you got the data from. The SEC has proposed ruling now that organizations have to disclose their climate risks. I'd love to have a conversation about what are the climate risks related to software and green software and sustainability and technology? That's a great example that you gave, and I just thought that's something I'd love to explore in the future.Sara: Final thoughts, placing my tomato plants next to my laptop, number one. Number two, it will be interesting to talk in the future about how the pure economical aspects of where to place a data center will impact the grid. If you're only placing data centers where the grid is green, will that power a green shift in the energy markets?Danielle: I'm having trouble wrapping up all these thoughts. There were so many different avenues. I think something that stuck with me that I'll continue to think about is the idea of carbon intensity and viewing that in conjunction with totals, using these variety of numbers to come up with a strategy. I thought that was really interesting.Chris: I guess that's me left now, actually. Asim, I'll keep it short. I think this points to us having a carbon-aware internet. I think that's a really cool vision, personally. I'll leave it with you, philosophwith you, Mr. Hussein.Asim: Thanks for listening to Environment Variables. All the resources for this podcast, including links to our guests and more about Amazon's customer carbon footprint tool, as well as the Green Software Foundation, and everything else we read discussed today is going to be available in the show description below. We hope you enjoyed the show. See you on the next one.[background music]Asim: Hey, everyone, thanks for listening. Just a reminder to follow on Apple podcasts, Spotify, Google podcasts, or wherever you get your podcasts. Please do leave a rating and review if you like what we're doing. It helps other people discover the show and, of course, we want more listeners. To find out more about the Green Software Foundation please visit greensoftware.foundation. Thanks again. See you in the next episode.[music][00:44:52] [END OF AUDIO] Hosted on Acast. See acast.com/privacy for more information.

Join your host Asim Hussain on Environment Variables, a podcast from The Green Software Foundation, and a bunch of experts from varying software and tech fields. In each episode we’ll talk about the latest news regarding how to reduce the emissions of software and how the industry is dealing with its own environmental impact. Find out more:The Green Software Foundation Website Connect with us on Twitter, GitHub and LinkedIn!Asim Hussain: Twitter / GitHub / LinkedIn Hosted on Acast. See acast.com/privacy for more information.