Understanding Data Revisions and the Importance of Natural Capital Measures

In this episode Miles is joined by the National Statistician, Sir Ian Diamond, to reflect on what has been a busy and transformative year at the Office for National Statistics. 

Transcript  

 MILES FLETCHER  This is “Statistically Speaking”, the official podcast of the UK Office for National Statistics, I’m Miles Fletcher. This is our 20th episode, in fact, a milestone of sorts, though not a statistically significant one. What is significant is that we're joined, once again, to look back at the highlights from another 12 months here at the ONS by none other than the National Statistician himself, Professor Sir Ian Diamond. Ian, thanks for joining us again. The year started for you with being reappointed as the national statistician. As 2023 developed, how glad did you feel to be back?   SIR IAN DIAMOND 

Of course, you know, I was hugely privileged to be invited to continue. It's one of the most exciting things you could ever do and I will continue to do everything in my power to bring great statistics to the service of our nation.   MF  To business then, and this time last year, we sat in this very room talking about the results of Census 2021, which were coming in quite fresh then. And we've seen the fastest growth of the population, you told us, since the baby boom of the early 1960s. Over the course of the year much more data has become available from that census and this time, we've been able to make it available for people in much richer ways, including interactive maps, create your own data set tools. What does that say about the population data generally and the way that people can access and use it now? How significant is that there's that sort of development?   SID  Well I think we need to recognise that the sorts of things that we can do now, with the use of brilliant technology, brilliant data science and brilliant computing is enabling us to understand our population more, to be able to make our data more accessible. 50, 60, 70 years ago, 150 years ago, we would have just produced in about six or seven years after the census, a report with many, many tables and people would have just been able to look at those tables. Now, we're able to produce data which enables people to build their own tables, to ask questions of data. It’s too easy to say, tell me something interesting, you know, the population of Dorset is this. Okay, that's fine, but actually he wants to know much more about whether that's high or low. You want to know much more about the structure of the population, what its needs for services are, I could go on and on. And each individual will have different questions to ask of the data, and enabling each individual to ask those questions which are important to them, and therefore for the census to be more used, is I think, an incredibly beautiful thing.   MF  And you can go onto the website there and create a picture...   SID  Anyone can go onto the website, anyone can start to ask whatever questions they want of the data. And to get very clearly, properly statistically disclosed answers which enable them to use those data in whatever way they wish to.   MF  And it's a demonstration of obviously the richness of data that's available now from all kinds of sources, and behind that has been a discussion of, that's gone on here in the ONS and beyond this year, about what the future holds for population statistics and how we can develop those and bring those on. There's been a big consultation going on at the moment. What's the engagement with that consultation been like?   SID 

Well the engagement's been great, we’ve had around 700 responses, and it addresses some fundamental questions. So the census is a really beautiful thing. But at the same time, the census, the last one done the 21st of March 2021, was out of date by the 22nd of March 2021, and more and more out of date as you go on and many of our users say to us, that they want more timely data. Also by its very nature a census is a pretty constrained data set. We in our country have never been prepared to ask for example, income on the census yet this is one of the most demanded questions. We don't ask it because it is believed that it is too sensitive. And so there are many, many, many questions that we simply can't ask because of space. There are many more questions that we simply cannot ask in the granularity that we want to. We've been doing some work recently to reconcile the differences between estimates in the number of Welsh speakers from surveys with estimates on the number of people in the census who report they speak Welsh. Frankly, it would be better if we were able to ask them to get information in a more granular way. And so while the census is an incredibly beautiful thing, we also need to recognise that as time goes on, the technology and the availability of data allowing us to link data becomes much more of a great opportunity that we have been undertaking a lot of research, a lot of research which was asked for by the government in 2013, following the report by Chris Skinner, the late Chris Skinner, Joe Hollis, and Mike Murphy, which is a brilliant report. We said at the moment we need to do another census in 2021. That's what we have done and I believe it to be one of the best coverages there has ever been. And yet we need to assess whether administrative data could be used in future to provide more timely, more flexible and more accessible data and that's what the consultation is about. I will be making a recommendation to the UKSA (UK Statistics Authority) board in the future. In the near future we have to say, and I think it is worth saying that what the consultation says to us is that people are very, very, very much in favour of the direction of travel but at the same time as yet accepting our prototype, unconvinced about the data flows and the sustainability of those data flows to enable us to do it and so, we are looking at how to respond to other very important analyses and we will do so in the near future.   MF  When can the people who contributed to that consultation, roughly when should they expect to hear from us?

SID  I think the expectation is we'll publish something by the end of quarter one in 2024.  

MF  Surveys have continued to be a very important part of what the ONS does, these very large national surveys, and yet one of the biggest challenges of the has been maintaining coverage and particularly response rates and obviously, particularly with the Labour Force Survey recently that has been a particular issue for the ONS hasn’t it. Where do things stand now as we move into modernising the traditional Labour Force Survey and moving to a new model because it's an issue statistics bodies around the world have been dealing with, it's harder to get people to complete surveys like they used to.    

SID  I think it's a fair point that response rates globally are a challenge and response rates globally, not only in national statistics issues, but in the private sector organisations that also collect data, are a challenge. So we need to recognise that. A part of that is that historically, one could find people at home, knock on doors, have that conversation with people, and perhaps post pandemic people are less willing to have a conversation at the house. Also, people are very busy. They work in multiple occupations. They are not always in, they live in housing accommodation which is more and more difficult to access. This there is no kind of single magic bullet here that we could press all we would have. The first thing to say Miles is that we recognise that and that's why we worked with our colleagues at His Majesty's Treasury to provide a project to go to what we call a transformed Labour Force Survey. And I think that that's a hugely exciting project for a number of reasons. One, the labour force survey which has been around for a long time, the questionnaire had become a little bit unwieldy. And also we wanted to enable people to have much more flexibility at the time of which they answered the question. We are in the field with the pilots for that service. We've been pretty good. There are good response rates. There are also some challenges around getting the questions right. These aren’t challenges that stress me, that's why you do a pilot, but at the end of the day we're hoping to be able to transform into that new Labour Force Survey early in 2024, in the first half of 2024. We're working very closely in doing that with our major stakeholders and the Bank of England, His Majesty's Treasury and the Office for Budgetry Responsibility (OBR), is you take a joint decision on when people feel comfortable that we have had enough dual running to enable us to move forward. The other question that I'd have to raise around surveys more generally, is on inflation, which we have all been subjected to in many, many areas in the last couple of years, inflation in survey collection has increased massively and so in the last year we've had to make real judgments about how we maintain quality. And in the next few years, we will really be needing to think through exactly how we conduct our surveys and the cost of doing so.   MF  Yes. Of all the people who should be aware of inflation are the people who report it, and certainly the impacts of those relatively high rates of inflation have impacted us as much as anybody else. The challenges not withstanding of running surveys, the interest of government bodies in getting that information directly from people does continue to underline the unique value of surveys. Some people say Oh, well, they surely they can get this information from other sources I've even seen it suggested that social media could provide the answers, but there is a unique value isn't there and actually getting a statistically representative sample of people and speaking to them directly.   SID  It depends Miles, I think it absolutely depends on what the question is you're trying to answer. If you're trying to get some answers to a question where the answer can be obtained through administrative data sources, then you don't need a survey. Surveys are difficult to conduct and difficult to pilot and plan and extremely expensive to undertake. So you should only do a survey if you can't get the information from somewhere else. Therefore, you know, I do think that we need to be very, very careful in thinking through when we need to do surveys. Does that mean to say we don't need to do surveys? Absolutely not. There are reasons why you need to do surveys. It may be that you need to really spend some time identifying whether someone really is eligible for hte questions you're going to ask or you may want attitudes. I don't know how to get someone's attitude without asking them. And so there are reasons why you would want to do a survey, but I would argue that you should only do a survey when you cannot get the data from elsewhere. And you also mentioned social media. Social media is an incredibly interesting and important source of data. Now, I wouldn't necessarily say it was statistically representative, but we absolutely have to be flexible in what we call data. We have to be sure of the quality of those data and we have to be sure that we are really aware of what the population is that are represented by those data. So we are using many, many, many types of data now that we would not have used 50 years ago, we simply couldn't have used things like telephony data, things like card data, things like data from satellites to address questions which those data are the best way of providing answers.    MF  And there are some fantastic examples of that around the ONS. If you look at how we've changed prices over the last couple of years, again, the measurement of inflation, bringing in new data sources most recently from the US car industry, from the rail industry as well and it all means that the estimates of inflation are now based on many hundreds of thousands of price points, where it used to be just a few things.   SID  It doesn't matter what the numbers are, frankly, it matters that you've got a good coverage it matters that you have the most appropriate method and that your data are as accurate as possible. And I do think it is incredibly important. We use a wider range of data sources. I think it's incredibly exciting what those data sources are, but we should only do so being unbelievably careful about what the metadata are that go with them, what the coverage is, why we are using them and whether or not they represent an improvement over what we could do before.   MF  Okay, so we've seen in the area of prices, the measurement of inflation, there's new innovative data sources coming from outside, coming from industry. What sort of an improvement does that represent in how we measure inflation, when it's such an important time for cost of living?    SID  Well, I think it helps because we have more accurate data. We have more timely data, we have data that are real. So on rail prices, we know what people pay as opposed to what the price as advertised necessarily is and I think that is important. And so being able to properly understand what the consumer is doing, therefore, what inflation is, is to me, incredibly important. I would say that all this effort that we're putting in would not necessarily just be about prices. Here it is about do we understand more about what is going on in the economy, and there are many more questions that we can ask from those data when you've got them, and simply from some of the fixed price point data that we have previously.   MF  Now one massive change we've seen lately, and this is another area we've managed to improve coverage, is of course the private rental sector. It’s become much more important as we've seen house prices coming under pressure and mortgages under pressure by high interest rates and so forth. It's revealed a very interesting picture of long-term change, and also in more contemporary terms, what's actually going on with the economy right now.   

SID 

Oh 100% 

 MF  Talking about areas where we've been able to form a new view of what's really been going on. An area that attracts a particular commentary during the course of the year is expenditure on research and development. Regarded as a very important area of activity if you're talking about productivity, future economic growth...we substantially upgraded our estimates of R&D. What was the story behind that? Why was that necessary?    SID  Well it was incredibly important because we looked very carefully at our data, we look very carefully at our samples, we looked at our coverage and we decided that we needed properly to to bring in a much wider range of business. And we were reflecting very much those businesses from a very wide range of areas who were able and available to claim R&D tax credits, and therefore to be able to get a decent sample, and the critical thing here is not only were we making good estimates, but we were able to understand much more about what, particularly for smaller tech and creative industry companies, was R&D. And I think that is something that we need to recognise particularly in those smaller companies where there's a much greater flexibility about what people would call R&D.   MF  It’s a reflection perhaps that startups are the sort of firms that do R&D these days, and less so the sort of industrial behemoths with huge R&D departments. But there was an interesting change nonetheless, and obviously considerable improvement in measuring that very important area. This all I guess comes under the umbrella of future proofing practices and systems and this all came under a refreshed data strategy that we launched during the course of the year. One of the fundamental principles underlying that, where is it taking us?   SID  I think, I mean, just where I've been coming from, are to do a much more holistic view of what data are and how we really use data which are most appropriate to answer the questions that we have, and we recognise that the economy and indeed society are changing very quickly, and therefore we need appropriate data to be able to answer those questions. For example, if you look at employment, there are many, many people in our society who have three, four, even five jobs, we need data which enable us to find out what the distribution of the number of jobs people have is, what they're spending their time doing, and how that impacts on our understanding of the labour force.   MF  Worth perhaps recognising some of the particular areas where new data has also been able to shed new light and particularly think of the payments industry which obviously digital payments happen very quickly. They provide almost a daily update on the state of consumer spending. With it obviously the state of the of the of the wider economy. We've managed to strike up partnerships with a huge cross section of the payments sector. What is the particular value of that? And what do we say to perhaps other data providers who might wish to enter into similar arrangements?   SID  Well I think we’d say we do everything ethically, and with complete privacy, but at the same time in the public good. And that is, to me, incredibly important. And so understanding what the consumer is spending money on understanding what the consumer is not spending money on, and the transitions, is incredibly important to enable policy which impacts very positively on all of our fellow citizens. So we are very proud of those partnerships. We value them greatly. We don't take them for granted. And those data, entirely ethically provided, with great security but at the same time enabling us to understand what is going on at an early stage in the economy is incredibly important.   MF  And of course it’s worth restating, as mentioned already, that of course all of these data are anonymized and aggregated, and no individual would ever be able to identify themselves or be identified from that fast payments data which of course is helping to inform economics policy.   MF  Providing data to the people who do make policy and around government and to make sure that policies are really informed by evidence of course that is the major purpose behind the new Integrated Data Service, which was accredited this year under the Digital Economy Act. And that's enabling data to be shared around government in a way that simply wasn't possible before.   SID  80 datasets available now and indeed, that number going up more or less by the day. And one of the most important things here is that there are very few challenges which government face which simply can be addressed by data from one department. Therefore, what we need to be able to do is to link data from different sources to enable us in a very granular way to be able to answer questions about topics for which the answer requires data from many sources. And the Integrated Data Service allows us to do that. It allows us to do at a pace and allows us to do it in a way which brings a wide variety of analysts to the party. And I think that, you know, this year major milestone in getting Digital Economy Act accreditation. And we will be looking to streamline the process of using it over the next year, as well as seeing more and more and more projects on it having successful results.   MF  And sharing between departments at the national level is important, but also it's been a long-term aim of the ONS to improve its coverage at local levels. And again, there's another important initiative kicked off this year, and that's the launch of ONS Local.   SID  Yes and I’d say that the two are linked. It doesn't matter whether you are at a national level or whether you are at a regional level, linked data are important, but we are very pleased working with funding from our colleagues at the Department of Housing, Levelling Up and Communities to have been able to place ONS staff in regions. So we're not talking about teams of people in Manchester or teams of people in Exeter, but we are talking about interlocuters in the southwest, northwest for example, who can really work with the leaders there to ensure that we've got local data for local leaders to make local decisions and that's incredibly important because the questions that people wish to ask are different in different parts of the country and therefore we need to recognise that so it is a good initiative, which I hope will bear fruit in 2024.   MF  And the importance of data in government has been underlined by a big initiative, which takes in everybody, not just statisticians and analysts, but everybody in the civil service, has been engaged in what's called the One Big Thing campaign to spend time learning about data that's important to the use of data. How has that initiative been going? The ONS has been a central part of that. How's it been going? How important is it?   SID  It is critical. We do not need every public servant to be able to be a brilliant statistician, but we need every public servant to be data literate. We need every public servant to be able to understand data and the best policy comes about when analysts and policymakers and potential beneficiaries work together. And that requires that you can have that data literate conversation. And so I think One Big Thing is a great thing.   MF  In fact that the need for people to better understand data became evident early this year, of course, when our GDP revisions were quite dramatically revised in the early part of the autumn as the estimates for the big peak pandemic years, 2021 and 22. There was quite a reaction from some parts of the media and beyond, who reported that our original figures were, because they had revised so dramatically, were simply wrong. I mean, that's not the case. revisions of course have always been integral part of the process. Indeed the OSR, the statistics regulator, found as part of its review our approach to be, and their words were appropriate and well managed, however, it also found the ONS could communicate better the uncertainty in those early estimates of GDP and that's a learning point for the future.   MF  We saw particular attention recently for the natural capital outputs, measures of the natural environment, and they attracted a degree of media interest we haven't seen so far, helped by the fact we're able to bring it to life with an analysis of time spent in nature and so forth, and you spoke to BBC Countryfile about that particular work. What's your overriding thoughts on that release?  Are we moving to the point where these kinds of measures are getting more exposure? Are they being recognised for their value?   SID  I thought the national Natural Capital stuff was brilliant. I've always thought, as I said last year, that we should put alongside GDP measures of the environment and measures of well-being, but you need a concise picture and that's where we're moving in the future.    MF  As we speak, we're heading into the bleak midwinter of 2023. The nation is doing all it can to avoid a seasonal bout of flu and the other viruses that traditionally do the rounds at this time of year. And that's seen a revival of our surveillance effort. The Winter Coronavirus Infection Study (WCIS). Tell us about that. What's the purpose of it and what's happening?   SID  Yeah, working very much for our colleagues at the UK Health Security Agency who asked us whether we would be prepared to stand back up some of the work we do on surveillance of winter flus, COVID and other issues. and we're of course pleased and proud to be asked. We’re using a different strategy to the one we were using in the past, this is very much simply a mail out of tests enabling people to take a test and then to make estimates, and at the moment the good news is that the estimates of positivity are relatively low, but the bottom line is we need to recognise that without some good hard data on those levels it's pretty impossible for government to plan, and so I think it’s a really exciting initiative. It's a smaller survey than one in the past. It's a survey which will make national estimates rather than many regional estimates, but it's one that we think is extremely exciting, and builds on some of the work we've done in the past.   MF  And now of course everyone knows how to self-administer a COVID test and that ability makes it much easier to run these big.   SID  Oh 100%. I do think we need to recognise the way in which the world moves on. And certainly, when we first set up the COVID infection survey in 2020. We were not aware of the extent to which people could self-administer, we learned pretty quickly that's why we were able to transition to self-testing, but I think we are in a world where we can do this at pace and provide estimates very, very quickly.   MF  Well, thank you very much for joining us. Great to have you with us again at the end of the year. You could choose just three words to sum up your 2023   SID  Exciting, full of change and high-quality statistics.   MF  And looking ahead to 2024, which pieces of work are you looking forward to most?    SID  The economy is changing quickly, society is changing quickly. We will continue to change and to be ever more effective. We've talked about some of the things we're bringing on board and looking forward to a brand-new website to improve our communication. And I think it's going to be a very exciting time.  

MF 

Professor Sir Ian Diamond, thanks very much for joining us.   That's it for another episode of Statistically Speaking, you can subscribe to future episodes of this podcast on Spotify, Apple podcasts and all the other major podcast platforms and also follow us on X, previously known as Twitter, via the @ONSFocus feed. I'm Miles Fletcher, and from myself, our producer Steve Milne, and everyone here at the ONS, we wish you seasonally adjusted greetings, goodbye.   ENDS