Why is the model of a Nobel prize winner not necessarily good science? What is “good” modelling? Is access to information enough to change a system behavior?

This episode is the second part of a long interview with Laetitia Bornes, a Doctor in Human-Computer Interaction, Systems Engineering and Systemic Design who is one of the co-authors of a research paper investigating the claims made by the second hand digital platform Vinted about the avoided carbon emissions thanks to its operations. As presented in the first part, their findings were surprising, enlightening for the IT sector and nuanced!

In this second part, Gaël Duez and Laetitia Bornes moved away from the Vinted use case and discussed modelling, the scientific method and Systems Thinking in general. You can enjoy this discussion without having listened to the first episode however we would suggest you do so to enjoy all the references, especially to the Vinted study. Among the topics covered in this second part are:

An impressive transparency exercise about the limit of the model used for the Vinted use case,
Why models are “wrong” and how to still use them purposefully,
Why a Nobel prize modeling in his lab without publishing isn’t doing science (yet),
Access to information and its connection to the four main categories of leverage points,
The concept of protopia,

And much more!

❤️ Subscribe, follow, like, ... stay connected the way you want to never miss an episode, twice a month, on Tuesday!

📧 Once a month, you get carefully curated news on digital sustainability packed with exclusive Green IO contents, subscribe to the Green IO newsletter here.

📣 Green IO next Conference is in Munich on July 2nd and 3rd. Green IO listeners can get a free ticket using the voucher GREENIOVIP. A small gift for your huge support. 🎁

Learn more about our guest and connect

📧 You can also send us an email at contact@greenio.tech to share your feedback and suggest future guests or topics.

Laetitia's sources and other references mentioned in this episode

Transcript (auto-generated)

Laetitia Bornes (00:01)

The problem is when you put a model that has been made for a given purpose in the hands of someone who doesn't know who doesn't want to recognize its scope and who therefore uses it outside of its field of validity.

Gaël Duez (00:18)

Hello everyone, welcome to Green IO I'm Gaël Duez and in this podcast we empower responsible technologists to build a greener digital world, one bite at a time. Twice a month, on a Tuesday, our guests from across the globe share insights, tools and alternative approaches enabling people within the tech sector and beyond to boost digital sustainability. And because accessible and transparent information is in the DNA of Green IO. All the references mentioned in this episode, as well as the full transcript, are in the show notes. You can find these notes on your favourite podcast platform and, of course, on our website greenio.tech.

This is the second part of my long episode with Letitia Born, a doctor in human-computer interaction, systems engineering and systemic design, who is one of the co-authors of a research paper investigating the claims made by the second-hand digital platform Vinted about the avoided carbon emissions thanks to its operations. As we saw in the first part, their findings were surprising, enlightening for designers and nuanced, with potential applications for the entire digital industry regarding how we assess its potential footprint and handprint, that is the positive impact it brings on our planet and our societies. In this second part, we moved away from the Vinted use case and discussed modelling, the scientific method and system thinking in general. You can enjoy this discussion without having listened to the first episode. However, I would suggest you do so to enjoy all the references, especially to the Vinted study. So Laetitia, when we were discussing how you model the vintage use case and all the results that came along, I had like a ton of question about what are the limit to modeling? What are these assumptions that you built your research paper on, et cetera, et cetera. And I think it's a great opportunity to have a trained researcher, a doctor now on the show to discuss a bit about not the vintage use case itself, but how we actually can do systems thinking in a proper scientific manner. And maybe it's a bit provocative here. My first question about it is, because we know that there is limit to model everywhere, where are you unhappy with the way you model the vintage use case? Whether it's a boundary that you would have changed, some proxy data, which you would have preferred having primary data instead, or it's assumption that you couldn't really double check or shortcuts that you had to take. It's just a sort of transparency exercise that, it's not because you've got a published research paper that everything is flawless and that you didn't have actually to put limitations to the scope of what you wanted to investigate.

Laetitia Bornes (03:43)

Yeah, So I would say that ⁓ I regret that my model focuses mainly on carbon emissions and not on the other impacts, apart from the loss of income from charities. But this is linked to the fact that I drew on the value study, which focused on avoided emissions in carbon equivalent. I've also made a big shortcut by displaying the number of sales per month to assess the economic viability of a scenario. Because if you change the business model in an intervention scenario, instance, by to a subscription then the economic viability would no longer depend on the number of sales with delivery, but on the number of users. It's not really complicated to change in the model in reality. It's just that I've never taken the time to do it. also, I must admit that the system that calculates the sales per month is quite basic and not really realistic. But all of this is not really a problem for the purpose of my modeling, which was only to demonstrate the sensitivity of assumptions and the fact that very different results can be obtained on the basis of the same study if we include other indirect effects. I wanted also to demonstrate the value of transparent and dynamic modeling and of comparing interventions to strategies. So I did it with the help of the that participated to my workshops. ⁓ And yeah, so I wanted to carry out a formative evaluation of magnitude, my modeling tool with professional designers that I ⁓ could do so as well. it depends on the purpose of your modeling. In my case, the way I use modeling is really not, let's say, rigorous, ⁓ even if it's shocking, because the aim is to predict precisely something. So yeah, in my case, it was to demonstrate some things. If I would have done it for Vinted in a real we would have made it more rigorous, for sure. Yes.

Gael Duez (06:09)

That was the exact question that I wanted to ask to bounce back on what you said, because thanks a lot for this transparency exercise, but also to remind people that we model things in a certain way to achieve certain goals. And your goal was not to provide Vinted a better model to assess their claims on avoided emissions. And that being said, as you just mentioned, what would what have done differently if the model had a different objective which is providing to Vinted a more accurate or a more nuanced way to assess their avoided emission? Could you just share maybe two or three examples? Obviously, you're not going to write a new research paper live on the show. I mean, you can do so if you want.

Laetitia Bornes (06:54)

Okay. It's difficult to anticipate the things that we want to include in the model rely on the methodology. So before doing the methodology the actual people from Vinted, it's difficult to anticipate. But I think if I had all the time and resources that I want, I would have more investigated. The way people use the money they earn from sales and the impact of Vinted on the fact that people renew their word more frequently. So can we quantify how many users buy more items because they can sell it on Vinted? That's something that I would definitely investigate and yeah, have a better idea of what people do with the money they are from but it's anyway you can do a new survey you will still have a lot of uncertainties and something that is not in the model but in my software and that is definitely lacking is a way of visualizing the quantitative uncertainties so you have uncertainties that you can't quantify but you have some uncertainty that you can quantify. And for the moment, my software doesn't help with So for the moment, get a range of uncertainty, you'll have to do for each context two scenarios, one optimistic with every parameter at an optimistic value and one pessimistic. And then you say, OK, I'm between those two values or graphs. to conclude, I said that it would depend on the whole methodology that I would conduct but those were the main things that I would like to investigate personally.

Gael Duez (08:49)

Okay, so the level of uncertainty with as well as the direct connection between the sales and the amount of opportunistic buy. Am I right to say or am I missing some other items here?

Laetitia Bornes (09:09)

Yeah, the rate of sale and the way people renew their wardrobe and also the way they spend their money on from sales.

Gael Duez (09:19)

And actually, the third point was actually the way they spend it. And you beautifully explained it previously with a significant share, which is not quantified in this research paper, but that you could assess is that it is spent on fast fashion items. So it's actually not helping the planet that

Laetitia Bornes (09:43)

Yeah.

Gael Duez (09:45)

So, you know, while you were explaining in a very transparent way, all the limits and all the choices you made for your research paper I had this sentence bouncing back in my head. I think it's from a British statistician, George Epibox. All models are wrong some are useful. And I had this sentence a lot in my mind also when we were preparing this episode. And I wanted to ask you how wrong was your Vinton model and was it still useful? But I think you already perfectly answered this question. But I had another question related to it because this sentence, all models are wrong, some are useful, is sometimes used in the wrong way to, for instance, push back against IPCC models defend other models which would question anthropic climate change. So to broaden a bit our perspective, I'd like to ask you this question. Why should we consider with more scientific robustness the modeling exercises done by the IPCC than the one done by a climate denier who holds a PhD in physics, let's say, or even a Nobel Prize, as in the case of the Dr. John Closer, And another way to put my question is, can you give us a scientific methodology, one-on-one courses in two minutes?

Laetitia Bornes (11:20)

So first about the quotation. Actually, I included it in my thesis So that's funny that you mention it. And in its long version, it says in fact, all models are wrong, but some models are useful. So the question you need to ask is not, is the model true?

Gael Duez (11:31)

Excellent.

Laetitia Bornes (11:43)

Because it never is, but is the model good enough for this particular application? And in fact, this quote is absolutely not a criticism of models in general, but rather a way of saying that models should not be taken out of context. So the problem is when you put a model that has been made for a given purpose in the hands of someone who doesn't know who doesn't want to recognize its scope and who therefore uses it outside of its field of validity. So it's also important to understand that there are many, many different kinds of use for models. for instance, in my case, and it can be also the case if we have a look at models that are used by IPCC, ⁓ you have some models that are predictive models. So they seek to represent reality as closely as possible and to predict the future, which is possible if we are in a linear situation or if we know perfectly the mechanisms of evolution. So that's the case for weather forecasts, for instance, you have always a kind of uncertainty. And then you have exploratory and prospective models which seek to facilitate a reflection in the what-if logic. So we wonder what might happen on the basis of different sets of hypotheses which we consider to be likely or to be extreme and so on. So that's the way I use models for instance. So to come back to the question about the IPCC, there are different things. So the IPCC produces or rather evaluates and shares that are based on different sets of assumptions as a basis for reflection and political decision making. So these are what-if scenarios and they are exploratory. is kind of predictive of a situation assuming a set of hypotheses and they are perfectly legitimate within this scope. if we accept the So for instance, regarding the anthropogenic nature of climate change, there is exploratory about it. statement is actually unquestionable because despite the huge number of researchers and increasingly accurate climate models, no model has ever been able to explain the observed climate change without including human activities in the hypothesis. So… a proof of the anthropogenic nature of climate change. And I'm not necessarily able to go into more detail about it, I recently read a book by Pablo Jensen is called Your Life in Numbers, Modeling Society for Data. So this is the version in English, but in French, it's totally different. I think it's literally why we can't put our society into equations, something like And yeah, he is really good about explaining the different kinds of models and how the weather forecast model has become so accurate and why the anthropogenic nature of climate change has been proved.

Gael Duez (15:19)

That's interesting what you mentioned because you say no model has ever been able to prove that the climate change has not a massive part of a human made, causality. However, we can find models of people and sometimes people with some degree in physics. We've got even, as I mentioned, the case of one Nobel Prize, not experts in climate, but that's a different question, but still a Nobel Prize, who would push a model saying that anthropogenic nature of climate change is not assessed. So I guess modeling is just the first part of the story. What makes a model the model or the source of truth at a certain point of time in history? Because science is always evolving. And what makes you so sure now, thanks to the IPCC work, for instance, that you say that it is an absolute truth that climate change is human made? And I know that I'm playing the devil advocates here, all the listeners in you, you know about it, but it's me playing a bit dumb saying, but I see other model and they're written by people who seem to be pretty clever. So which model shall I trust? Because you know, all models are wrong and blah, blah, blah, and blah, blah, blah. So what would you say to these people and how would you debunk this partial approach to what is modeling and what is science?

Laetitia Bornes (16:53)

Yes, so the problem is that unfortunately there are, as you said, some people who sound very serious but who use their influence to say things that are completely outside their field of competence. So it's a bit like getting a model to say something that is outside its field of validity. It's quite the same thing. So for example, although I'm a doctor, I don't have any legitimacy to talk about quantum physics. And if I did, I shouldn't be given any more credits than a bricklayer or a baker or whatever. what ensures the validity of science is the research system, although it has many flaws that we clearly don't have the time to go into here, I think. It is organized to verify the knowledge produced. So when an article is published in a serious journal or conference, it has been reviewed by several researchers anonymously who have judged it to be solid and well-funded. And even if a questionable article has managed to be published, then the scientific community can intervene to have it withdrawn. A scientist should only speak publicly about his field of research and particularly about his published research work, because the science is by the publishing system. So you can do ⁓ whatever model if you don't publish it. And if you made the assumption and the hypothesis behind this model in your lab alone, with nobody to check it, it's not science. And in the case of IPCC reports, there is a new assessment of the published literature. So this literature has already been assessed and published. And then the people from the research of IPCC, there is a consensus building in the relevance research community. So it's really difficult to get something much more solid than the reports produced by IPCC.

Gael Duez (19:05)

I think that was our one-on-one crash course on scientific methods. Thanks a lot for reminding this to us. Going back now to the question of a model and systems thinking, I promise I will stop grilling you and you can go back to your normal life because otherwise this episode will last for four hours and we don't want to do this. But to prepare our episode, and to be honest, also to use it in my new Green IT course, I reread Thinking in Systems, the Seminole, Donnellas, Meadows book. And it connects a lot what you explained previously, starting with the importance of system boundaries and using the model for answering the right question and not trying to use the model to answer all questions and really fit for purpose, I would say. However, there's something that really puzzles me in her book, which is an emphasis on free information and short feedback loop. And she sees it really as the number one tool to influence the system, mostly for positive outcomes. And today we have access to all the information about climate change, resource exhaustion, etc. And it seems that our willingness to act for the greater good of the entire planet Earth system, or would say our capacity to embrace complexity is actually reducing. And I was wondering, and that's super incredibly pretentious to just state it, but did she miss something? Or much more likely, is it me who's missing several things? Like, is it a case of bounded rationality, especially our biases, a bit like you explained with the researchers out of their scope of knowledge, I would say, or the information flow being crippled by the frequency and quantity of inflows in our digital world, or information being too manipulated. But I really wanted to get your feedback. And I know it's really like pure systems thinking theory about the importance of information to enhance a system and why she puts so much emphasis on it and why it doesn't seem to work despite having such a great access to information today. You've got a lot of time to answer this question.

Laetitia Bornes (21:36)

Perfect. So first, thank you for mentioning Danila Meadow's work and the levels for intervention that she worked on. So I obviously rely on Danila Meadow's work and on her identification of levels for intervention. I just don't use the 12 leverage points which seems to me to be too detailed and too complicated to explain to people. But I rather work with the four main categories ⁓ of leverage points. So the fourth one would be acting on the constants and the parameters of the system. The third one would be acting on the feedback loops and delays. The second one would be modifying information flows, as you mentioned, and changing system rules. And the first one would be changing the goals of the system and maybe altering the underlying So as I understand it, these levels should be ideally combined because they are moving from the fourth category to the first one, levers are from the fourth category, are quicker or easier to mobilize, but more effective just in the short term. And the first category of levers longer and longer to mobilize, but they are increasingly powerful in truly transforming a system over the long term. So you might want to combine them and not to act only on the long term ones. unfortunately, as we mentioned, providing information is not always enough, especially when it remains abstract.

Gael Duez (23:32)

Environment, you mean by abstract?

Laetitia Bornes (23:34)

Yeah, because you know about the climate change, you know about the impacts of what you buy, not always in fact actually. So have the information in our world society, we have it partially, but anyway we have a lot of information about climate change and environmental impacts and so on. But it remains abstract because it's figures or because it's events that are not always we are actually living. So you have some people that are quite shocked because they knew a landscape and this landscape changed affected by climate change.

Gael Duez (24:13)

Like the glaciers or things. Like this.

Laetitia Bornes (24:16)

Yes, so I think that if the social environmental impacts of a certain type of conception or a certain type of actions were to be felt immediately and in a very tangible concrete way by the people at the very point of the this would mean that we would somehow act on the delay of the propagation of information. And I would say on the intensity of the information or the way it is felt. I think that this would change things radically. And above all, from what I have understood, beyond information, you have this profound paradigm shift. And this paradigm I think that the way that the information remains abstract is kind of a barrier between this information and the paradigm shift that we need. So when I say paradigm shift, it's what's acceptable in our society, we value, when we make decisions and so on. So yeah, I think we have information, but it remains partial to just some people and this information in an abstract form. I'd like to say that it's not a problem to question the work of Donela Medos because actually she didn't hide the fact that she identified those categories of leverages intuitively and that further research was needed on this subject. There were not a lot of research afterwards. And in fact, a researcher called Ryan Murphy, so he's a systemic designer. He has published a year ago or maybe two years ago, an article on this subject, which explains this very well and which invites the scientific community to continue research into the categories of leverage. So yeah, that's supposed to be a work in progress, but people use it as a recipient. I think it's really useful for thinking about possible intervention. And for instance, I used it in my case study, for instance, in Vinted. As I said, there are different levels of intervention. And I gave these four categories to my designers, my participants, so that they can think of different kind of interventions. for instance, about information, they would informing the users about the environmental cost of a clothes, of an item. Or system rules would be banning certain ⁓ brands of fast fashion from the platform. But then if you move to the paradigm shift, there would have other ideas like enabling closing exchange or promoting clothing repair and customization. So you know you can have a strategy that would combine these different kinds of approaches and of levels.

Gael Duez (27:24)

So that is so much interesting. And I could go on and on and on and asking you tons of other questions. thanks for the honest answer that it's still working progress. It really had this feeling and that was just a feeling thought pretty much worthless that she comes from a generation where internet was not invented and she lived long enough to say it's wide adoption, but sharing information was key and they lived on this assumption. Like you just share information and that's it. And now with all the elaborate answers that you provided like here, but there is information and information there is abstract information and comprehensible information. There is information in how people are ready to receive it because of all these values and all the, as you mentioned, the potential paradigm shift that will be required to people to actually accept this information or accept to do something with this information and so on and so on. So I'm parroting what you say. So I'm going to stop here, but really I think this work in progress approach that you mentioned is super important because it's not information for the sake of information and we will share obviously the link to Murphy's article. Now, if you indulge me one last question before the usual closing questions. It will be also related obviously to systems thinking. And I was wondering also rereading this book, is systems thinking compatible with the way our societies work now? And by this, mean, I'm especially concerned with this crazy focus on short term that we have. can, for instance, see it with listed companies having to display quarterly revenues and so on. And even more like physiologically speaking, the chase for this immediate shot of dopamine that is provided a lot by our digital platforms. I was really like my fist in thinking is somehow the quest for some sort of long-term understanding of things and is our species, at minimum is our society and especially our Western world still compatible with this sort of long-term thinking? And once again it's a question I guess you can take several days to answer but I just love to get your immediate reaction on it and I promise after that we're done because otherwise you will stay with us for days.

Laetitia Bornes (30:06)

So I think that systems thinking is not only compatible with the way our societies work, but it's essential more than ever because we are in an increasingly complex inter-twin set of social-technical systems. So yeah, I think it's more necessary than ever because otherwise we can't understand the systems we are evolving in. Regarding the short term and ⁓ shoot-up dopamine things, think certain thinking should be maybe, yeah, it requires some time. ⁓ It's not as immediate as other stuff. So it has to be explained. Maybe we have to prove its efficiency on the long term to make people accept it. But I think there is a lot to do in the education system. And without exactly mentioning it, there is a really nice book about that that is wrote by Sean, and that is called The Reflective Practitioner. And he explains how we have specialized education in silos and why it's not relevant to the complexity of real world. So that's, I think, a good reference to mention here. So I think maybe it should come from education. I don't know exactly how, but I think that it's necessary more than ever and that in all sectors. So I think it's essential to use system thinking to debunk or at least reframe false good ideas of innovation. If you look at AI with a systemic perspective, the claim benefits the adults on the floor long. So yeah.

Gael Duez (32:11)

You see things in a very different way when you embrace systems thinking. That's for sure. And actually, and that's going to be a smooth transition. For people interested in system thinking and being sort of beginners or just like amateur, I would say, besides Donnela Amido's book, would you have one or two resources that you would like to share with them so that they could sort of ramp up their knowledge in this field?

Laetitia Bornes (32:41)

Yes, sure. So ⁓ if we hold the first generation of systems thinking, which is more about modeling, I would recommend definitely Limits to Growth, which is also written by Danila Meadows and her husband and other folks. I also would Business Dynamics written by Sterman. And the more recent book by Smaldino, which is called Modeling Social Behavior, Mathematical and Agent-Based Model of Social Dynamics and Cultural Evolutions. So that's for the of systems thinking, but there are many branches of systems thinking. And I would also definitely recommend Checkland, Senge, and Michael Jackson, which is a system thinker and not a music artist.

Gael Duez (33:40)

And as usual, we will share all these references in the show notes. So it will be super easy for people to double check these books, all these articles. Maybe just for people already quite literate in systems thinking. mean, what is the latest cool stuff in the research field? Beside your work.

Laetitia Bornes (34:01)

I don't know if it's the latest cool stuff, I think that if you're already familiar with systems thinking, should follow conference called Relating Systems Thinking and Design. So I'm making an advertisement for systemic design, which I think is really useful and interesting and I think it's definitely worth following the work of Ryan Murphy, whom I mentioned earlier, because he a lot of really interesting stuff.

Gael Duez (34:36)

Okay, excellent. Thanks a lot. That's going to be a lot to digest for the readers. Maybe, and that will be my final and usual question, we shared a lot. was both very hands on with this Vinted case and very theoretical. And that's good sometimes to tool up with theories and models to be able to embrace the complexity of the world. But just to finish on a more optimistic note, would you share a piece of good news regarding sustainability, with the listeners.

Laetitia Bornes (35:07)

So it can be maybe related to systemic design, but what interests me a lot at the moment is the concept of protopia. I think that we have a lack of future perspective and that's a real problem. And usually in future scenarios, we used utopias and dystopias, which are extreme future scenarios. So they are provocative, they are thought provoking, but they don't bring us together or they don't prepare us for the future. They don't open up any perspective, because they are so extreme and protopia is somewhere in between. And it's a way of imagining more plausible and desirable futures and I think that it's something that we need or or personally that I need so recently I've read a book called the Ministry for the Future written by Robinson and I think that we need more of these kind of books because it gives actually a perspective on what could really happen in the future without necessarily. Let's say, caricaturing things to the extreme. even if it's not a perfect perspective, that gives us some perspective.

Gael Duez (36:30)

I love the book that I didn't find the time to finish, to be honest, but I love the concept and I love this concept of protopia. I've never heard of it, but I was very aware of the pushback reactions that often come with either utopia or dystopia, with people saying, well, it's not plausible, as you've mentioned. Thanks a lot. Love the word. And thanks a lot for joining for this super long double episodes. I really do hope that listeners will at least pick one of the many resources that you shared with us to embrace a bit more system thinking systemic design and so on. Thanks a lot Laetitia and once again it was great to have you on the show, was great to have you on stage not technically speaking but represented by David at Green IO of Paris and I hope that we will meet again either in a conference another episode.

Laetitia Bornes (37:32)

Thank you very much for inviting me. Really, I really appreciated this discussion. So yeah, thank you.

Gael Duez (37:39)

You're welcome. Thank you for listening to this Green IO episode. If you enjoyed it, please take 30 seconds to give us 5 stars on Apple Podcast or Spotify. Sharing this episode on social media or directly with other responsible technologists seems also a good idea to provide them with tools and resources about modeling and systems thinking. You've got the point, being an independent media. We rely mostly on you to spread the word to more environmentally aware peers. In our next episode, we will meet Erika Pisani, a seasoned software engineer, to debrief the last QCon edition from a Green IT perspective. Erika was indeed the MC of the sustainability track of this prestigious conference. Stay tuned. One last thing. Visit greenio.tech to check the next conferences we organise. New York was a blast two weeks ago and Munich is in five weeks with almost all the speakers already disclosed. As usual, you can get a free ticket to any Green IO conferences using the voucher GREENIOVIP Just make sure to have one before the 30 free tickets per conference are all gone. I'm looking forward to meeting you there to help you fellow responsible technologists build a greener digital world.

❤️ Never miss an episode! Hit the subscribe button on the player above and follow us the way you like.

📧 Our Green IO monthly newsletter is also a good way to be notified, as well as getting carefully curated news on digital sustainability packed with exclusive Green IO contents.

#100. Hans & Christ: de 100ste DenkTank!