Statistically Speaking A survey like no other: Tracking the spread of COVID-19 in the general population
Since April 2020, the Coronavirus (COVID-19) Infection Survey has provided vital weekly snapshots of the level of SARS-CoV-2 virus circulating within the community. We’re joined by three central figures in the project, Ruth Studley, Tina Thomas and Professor Sarah Walker, for the inside story on one of ONS’s most ambitious surveys, set up in a matter of weeks as the pandemic took hold. (This episode was recorded in September 2021, before the emergence of the Omicron variant) Transcript:
Miles Fletcher, Head of Media and Public Relations at the ONS
Numbers, numbers everywhere, but here we’ll take some time to think about where they come from what they mean, and where they're going. Welcome then to the first episode of statistically speaking the new official podcast from the UK’s Office for National Statistics. In this series the nation's number crunchers, as some people still insist on calling them, chew the fat and spill the beans on the stories behind the stats. Lately, they've been making headlines, some would say ruling our lives, like never before.
I'm Miles Fletcher and in this first episode we'll be looking at how millions of swab tests and finger prick blood tests allowed the Office for National Statistics and its partners to track the progress of COVID-19 across the UK. During the pandemic, the COVID infection survey has proved a vital source of regular data on Coronavirus infections, antibodies and symptoms. We'll hear why this huge study was needed in the first place, how it was set up in double quick time and what it's told us about the virus and its human impacts, and why it remains important now.
Joining us are three central figures in the project: Tina Thomas, who runs the survey operation itself, leading a force of thousands of study workers out gathering data in the field; Ruth Studley Head of Analysis for the ONS, whose job it is to turn those test results into fast statistical estimates that we hear about in the news every week; and from the University of Oxford, the chief investigator and academic lead of the infection survey, Professor Sarah Walker. Sarah, to start with you first, how did this study get underway? And well, why was it needed in the first place?
Professor Sarah Walker, Chief Investigator and Academic Lead for the COVID-19 Infection Survey
So it was back in April 2020, when a lot of people had, you know, been sick with COVID in the first wave. But we really didn't know how many because at that point, we didn't have the PCR tests that are done in the laboratories, we didn't have the tests on a stick, the lateral flow test that lots of people do before going to school or work. And we really have no idea how many people had actually already had COVID. And at the time, there was actually a hope that we might even be close to herd immunity then. And so initially, in the middle of April, the infection survey was first of all a study looking for antibodies in the blood. And the initial plan was to sample just around a thousand households in the first month, then a further thousand households a month for a year to just find out how many people had had COVID already. But over literally the course of two or three days from the 17th to the 19th of April, we realized that actually, we didn't know so much more, in particular about how many people were infected without having any symptoms, how many people were passing it on to other people in their household, how many children were infected. And very rapidly, the survey increased to sampling over 11,000 households in the first month with an initial plan to then resample another 11,000 households a month for a year.
Miles Fletcher
Quite simply, you needed to have that representative sample of the whole population, it wasn't enough just to rely on people coming forward who suspected they had COVID.
Sarah Walker
Well, exactly, because what we cared about was what was really going on in the community. And it's well known that people who come forward for testing tend not to represent their communities completely. And so this was why initially, just because we had to get going so fast, we did actually approach at random people who had been in previous ONS surveys and said they would be interested in taking part in future research. But very quickly, we moved to just sampling from addresses. So, to really get a completely random sample of people living in private households across the UK.
Miles Fletcher
And how did that connection with the ONS come about? Because it's a new departure for the ONS, we normally measure the economy and migration and so forth, but not medical testing. So how did that partnership get started?
Sarah Walker
So, I really think it was a case of everybody just working as hard as they could together to make this happen fast. And what ONS do have huge experience with is these very large population representative sampling frames, and they also had access to this databank of people who had been randomly selected for previous surveys and who had said they would be interested in taking part. It's a huge field operation and obviously ONS has got huge expertise in that. So, I think they were in many ways, you know, the obvious partner to really take it on. And it was a huge collaborative effort between the Department for Health and Social Care as well as the University of Oxford and ONS.
Miles Fletcher
And this all had to happen in a fraction of the time that's normally available to plan a big survey, for example a census, and it was almost on that sort of scale. You need every community, every age group or socio demographic group represented in that massive sample. All this had to happen in a matter of a few days to start with to get the first estimates.
Sarah Walker
We wrote the first draft of the protocol on Friday the 17th of April, we submitted it for ethical approval on Monday the 20th of April. So that's just four days later, during which time we had gone through three major changes in scope and size. It was approved on Tuesday the 21st of April, we recruited our first participant on Sunday, the 26th. So literally ten days after the first draft, and we published our first estimates two weeks later, on Sunday the 10th of May. And interestingly, the positivity rate was 0.24%, around 136,000 individuals in England which we thought was enormous.
Miles Fletcher
Oh, well we’ll have a lot to say about what we've actually found. But just thinking about those early days and having to achieve in a matter of just a few days, what would normally take months. Tina Thomas, what was your reaction? What was your reaction when you first heard about this project and what it was trying to achieve?
Tina Thomas, Deputy Director for the COVID-19 Infection Survey
My reaction! So, before COVID, I was running the ONS social survey field community, and that's about 1,200 people, 1,200 interviewers, so they were keeping me busy. I had a phone call from one of our deputy directors on a Sunday night. They said, they want us to run an infection survey and they need me for the operations. And to be honest with you Miles, last year was a little bit of a blur. Usually, when we do surveys like that we take our time in working out the actual survey model, how are we going to do it, what we need, what our end goal is. And like Sarah has just articulated, everything was needed within days and weeks. To submitting a protocol on a Friday to recruiting our participants and the field staff out in the field, collecting the swabs and asking the questions was just short of 10 days, I believe, which was just something that we had never, ever done before. Did we think we could do it? There was a lot of nervousness but there was also so much passion to get this out there because we knew how important this data was. And everybody who worked on this, as Sarah said before, a huge collaborative effort to get it started. But it was just something that none of us had ever tried to tackle before. And it just proves what you can do. At pace, under pressure – we did it.
Miles Fletcher
And it's a huge collaborative effort involving not just Sarah and her academic colleagues at Oxford, but also the University of Manchester, a whole fleet of specialist contractors helping us to run the field study and specialist providers of all sorts right across the country. And, of course, government partners, chiefly the Department of Health and Social Care in England and the devolved administrations in Scotland, Wales and Northern Ireland. This is a huge UK-wide effort. Now in those early days, of course, it's the start of the survey, it started in a relatively small scale. It was, as I recall, about 12,000 households to start with, but then it grew rapidly didn't it?
Tina Thomas
Yes, that's right. I think it was, it was changing hourly some days. But yes, it started off relatively small and then it was within a matter of days “we want 150,000 unique participants providing swab samples across the UK”. We started off in England, we didn't actually bring any of the devolved nations onboard until around about the summertime, so a couple of months after the study had been running. That involved Scotland, Northern Ireland and Wales, which really brought its own operational challenges. Scotland and Wales are obviously a bit more rural than England. They wanted us in Scotland to go to the highlands and the islands and of course, going into Wales everything had to be translated into Welsh. And we had to make sure that we had Welsh speakers who could answer the phones for queries and also study workers that could speak Welsh as well. So really what was going on in the background from an operational perspective was very, very much like the swan analogy. We seemed quite calm on the surface, but with so much going on underneath - it was just incredible. And it's not just about recruiting those participants and getting the study workers out there. There's all the logistics around it, like how do we get the swabs to the labs? How do we get the test results back? Who's going to do all our printing? Who's going to send all the letters out? I'd wake up in the morning and think okay, so what challenge are we going to have to deal with today and I was never disappointed.
Miles Fletcher
And meanwhile always this huge expectation, from government, from the media, from ordinary citizens wanting to know exactly where the infections were, where the path of the virus was going. And during those very hectic early days, just one operational upset could throw the whole schedule into disarray. Fifteen months on, the survey is settled into a regular, pretty steady pattern now of hundreds of thousands of enrolled participants. And from their number, a large group taking a regular finger prick blood or swab test. How is that? Has it become easy to run now or are there still huge challenges out there? How are people, how about the participants themselves? How have they responded? And what's been their role in keeping this study running and keeping its findings meaningful?
Tina Thomas
So yeah, I mean, at this moment in time, we've got just over 457,000 live participants in CIS, 5.5 million swab tests have been taken to date. And just under a million blood tests. A daily rate for swab test is around 14,000, there's about four and about 5,000 blood tests being taken as well. We couldn't do this without our participants. Last year, when we had lockdowns, we had a bit of a captive audience. So, the study workers could get through their appointments. As we've seen lockdown restrictions lifted, it started to make operations a little bit more difficult. So, we've had to kind of pivot and change the way our study workers operate. So that's maybe working more in the evenings, offering weekend appointments. But our participants are just absolutely critical to this. And we can't thank them enough. It takes a lot of their time. We're asking a lot of them, but they majority of our participants are just really willing to do this. And you know, actually quite enjoying being in surveys while really feeling like they're making a difference.
Miles Fletcher
Tina, talk us through - we've got this small army of people out there in the field, covering households up and down the UK, tell us how the process works.
Tina Thomas
It's not such a small army, there's just over 3,000 study workers that are contracted to work on CIS, as you said, spread all across the UK. They are given what we call an allocation each day. So that's about visiting households to keep within protocol. So, some are still on weekly visits, some households are on monthly visits, they have to think ahead and have all their kit together: barcodes, test tubes, the swabs, the blood kits, and their mobile devices. So, they ring a household, usually the night before and agree a date and a time for them to visit. When they arrive at the household it’s a non-contact visit and they hand over the kits to the household members and scan the barcodes to make sure that we get the right test against the right person when they go to the labs. That's for swabs only. If the household is blood too then it's on to the fingerprick blood tests, which the participants usually take inside to do. And then they come back to the doorstep and there is a questionnaire that they'll need to answer, which Ruth touched on, which is about if they've had COVID, if they've had symptoms, what their social interaction has been like, and obviously that's more and more important now that lockdown restrictions are being lifted. Then the study worker completes the visit. And at some point during that evening they will drop the swabs off at a courier point where they'll go overnight to the labs. And the test results are usually back within two to three days. We've recently just introduced a new process, which saves quite a bit of money and also a lot of trees in that the test results now go out to our participants by email rather than by letter. If it's unfortunate to be positive, it gets notified to test and trace who then get in contact with the household.
Miles Fletcher
And of course, running something on this scale, it doesn't mean things always run entirely smoothly. We've had some quite colourful encounters on the doorsteps…
Tina Thomas
We have! So we have had a couple of study workers who've arrived at a pre-agreed date and time for the door to be opened by somebody who wasn't wearing any clothes. That's happened a couple of times and I think actually one study worker did ask the gentleman to go and put some clothes on and he came back with a T-shirt on, which just about covered his top half. And with the finger prick blood test as well, so it's quite a neat kit that comes in a cardboard box and it's got a plaster and a lancet and the test tube that we need them to collect the blood. The lancets are not needles, they're tiny, tiny blades because the actual volume of blood that we need to test is quite high. But we have had a couple of participants who I think were laborers on building sites, whose hands were very calloused from the job that they do, where the lancet just would not pierce the skin. And because they were so willing to provide that blood sample and wanted to provide that blood sample, I think we've had a couple of instances where they've come back with Stanley knives and actually managed to get blood out the finger with that. We wouldn't approve that or suggest that's the best way to do it. But what it does show, certainly the last one, is that people are just so keen and passionate to be included in this survey that they'll do what they can to help us.
Miles Fletcher
Made of sterner stuff than me, willing to take a stanley knife to yourself in the cause of science. And please, everybody don't try that yourselves. Some people haven't been quite so robust on the doorstep though.
Tina Thomas
No. Since we introduced the fingerprint blood test for antibodies, we have had a number of participants who have fainted whilst trying to get a blood sample for us. And unfortunately, a very high percentage of those are men. But we do have procedures in place with a study worker to help people when that happens. And obviously it remains a non-contact visit. But they have got a list of instructions if somebody does, unfortunately, pass out. And I think at this point as well, Miles, I'd really like to just say a massive, massive thank you to our study workers working on this who have been out doing this, during those early dark days of the pandemic, through every type of weather you can think of, to get us these samples and the data that we need.
Miles Fletcher
Thank you, Tina. So Ruth, this is where you come in – the field force have gone out, they've done their job, they've gathered in these thousands and thousands of samples, what do you do first?
Ruth Studley, Head of Analysis for the COVID-19 Infection Survey
So, what happens first is all of those swabs are sent to the different Lighthouse Laboratories to be tested using different PCR arrangements. That is part of the national testing programme and we use exactly the same process as the rest of the UK, that then gives us a set of data which we could use. And that is sent to us securely in ONS, where we process that data to understand exactly what is going on in the raw data before we use our modelling arrangements to produce our headline estimate. So, as part of looking at that data, we want to know things like what the different cycle threshold value is, for example. Now that is a bit of a technical term, and if I try and describe it very basically, it's the number of times, the number of cycles that each PCR test has to go through before a positive result is detectable, for example. And if there's a high quantity of the virus, you would expect that to be identified after a low number of cycles. So we would say that that was a low CT test, and it will be regarded as a high viral load. And so we look at things like that. And there's lots and lots of different things that we would look at in the raw data before then moving on to doing our modelling. The modelling that we use is a Bayesian multilevel regression post stratification model. And that's used to calculate breakdowns of positivity by region, and age across England. That all happens at breakneck speed. So the data arrives on day one, and we are virtually ready to produce information by day three, we publish by day five. It's very, very rapid. I'm not sure if any other official statistics are produced that rapidly.
Miles Fletcher
But such as the need for that data. Have the findings ever surprised you?
Ruth Studley
Yes and no. So, you wouldn't expect it to change very rapidly in the course of a few days. And usually, if there is something that you were surprised by, it would usually be an indicator that you want to do some further analysis. And there have been occasions where we've seen things and thought, does that make sense? And you dig a bit deeper, and you find that there's something going on in the data. But whilst, like Sarah was saying, you would never propose to predict what is going to happen, you would expect the changes to be relatively smooth.
Miles Fletcher
So, out on the doorstep every day, 12,000 swab and blood tests being taken, on average, at the moment nearly 6 million in all gathered under this survey. But what happens to those test results when they come in? Ruth, it's your job to make sense of them, and to turn them into statistics that can be relied upon. What's the secret to keeping those estimates reliable and trusted?
Ruth Studley
So that's a really great question Miles. I was thinking when I was listening to Sarah and to Tina then, what is it about this team that has allowed us to produce such fantastic results that have been so vitally important to the UK? And I think it’s the three Ps you know, we've got people who are passionate, people who work at pace, and we work in partnership. And it's all about wanting to do the right thing for the country, actually. So, what do we do with all of that data? We have a plethora of information as you have all described. And we do collect information from every participant every time we visit them on their socio demographic characteristics: whether they are experiencing symptoms, whether they are self- isolating, what their occupations are, whether they're working at home, questions about long COVID and whether they've been vaccinated, social distancing, physical distancing, etc. We ask all sorts of different questions, because it's really important that we're able to provide as much information as we can, not only about the direction of the pandemic, but also what people's experiences are in the community. And it's probably worth just reflecting there about who it is that we are sampling and who we're representing. This is a survey of the community of the population at large. So we will be testing people who are both symptomatic, but also are not experiencing symptoms. And that's really important because we know that our data has shown us that over the last 12 plus months, that people very often have tested positive but not had symptoms. And so this is a real reflection of what people will experience at large. And every week, we use a number of modelling techniques to estimate the number of people who are testing positive for the virus. And we produce that every week. You will see it in the media and on our website every Friday lunchtime, where we provide estimates of the number of people that are testing positive for infection. But as I mentioned, we asked lots of questions. So we break that down by lots of different characteristics, so by age by region, we do it for the four countries in the UK, and so on. And we do that very much in partnership with our academic partners of which Sarah is a really, really key partner for us.
Miles Fletcher
So that every week when you produce the estimates, we can say not just what's going on in Scotland or England at that level, but how local can you make the data to make people aware of what's happening in, you know around the corner?
Ruth Studley
We're really mindful that people are interested in what's happening locally to them. So we've also been able, because we have quite a large sample, to do quite sub-regional estimates. And that has provided a granularity of information for both decision makers, but also for the general public who are interested in what's happening in their locality. The geographies that we're able to get into the detail of are, within England, about 100 sub regions to give you a feel for the size of it.
Miles Fletcher
Now, tell us then about the ongoing story the data have been telling us after the first wave. After that, not surprisingly, we found quite a low level of prevalence in those early readings. But what was the story from then on, what happened after that? And what have been the key moments from an analyst’s point of view do you think in the path of this pandemic?
Ruth Studley
That's an interesting question. Over the course of the autumn last year, we did start to see a general steady rise, or just before Christmas with the identification of the Alpha variants as we now know it. And an awful lot of work went into that because as part of the swabs that we take and the analysis we do, we are able to do additional analyses to try to identify different variants and that is absolutely critical. There were some key points there obviously, because our information was part of the suite of information that government uses to help make their decisions around all the different interventions that they wanted to put in place, such as the different lockdowns across the UK. So the infection rate was steadily rising, and it peaked in January 2021. All of the days over the last 18 months have very much gone into one, but it peaked in January 2021. And then we started to see a steady decrease over the next couple of weeks and months with various different interventions occurring and then obviously we saw the arrival of Delta variant, and that has had an impact. And within certainly months, if not weeks of identification of that it became the dominant variant across the UK. And we are now seeing that the data has increased.
Miles Fletcher
And as you say, it's the Delta virus that really changed things suddenly midway, if you like, through the pandemic. Sarah, would you say that that's been the most important finding of the study so far?
Professor Sarah Walker
Well, I mean, I think it's easy to focus on Delta. But you know, frankly, Alpha was pretty terrible in December. I think it's actually been incredible to think that, frankly, in the space of only six months, the virus went through two such massive changes, which basically doubled transmissibility every time. So Alpha was twice as transmissible and Delta was twice as transmissible again. Of course, what is different is that we now have vaccinations and I think that is somewhere where the survey is increasingly really making a major contribution, because linked data is really brilliant and we can do a lot of stuff with it. But we're not really able to adjust very well with the kind of large scale NHS linked data, for characteristics that do affect people's chance of testing positive, whereas in the survey, because as researchers we collect this detailed information every month, we're able to adjust for things like whether people are healthcare workers, when they work in care homes, smoking status. Things that actually do make a difference to your risk of testing positive: whether they have been to a hospital. And so we're able to get much better estimates of the impact of vaccines on infection rates, really in the community, than many of these other big studies. Of course, we can't do it all, we can't look at hospitalizations, because they're quite rare in our study, but we can actually make a really big contribution.
Miles Fletcher
And that's all because the study worker as well as taking the taking the test sample sits down and actually goes through the questions with gets a lot more information from them.
Sarah Walker
Absolutely. And then also at the antibody testing, which we're now doing on around a half of our participants who are giving us finger prick blood every month, and that's enormously valuable in tracking levels of antibody protection in the community as a whole and then trying to understand how that relates to infection rates in different age groups and different parts of society.
Miles Fletcher
And that's the particular value of testing the same group of people again, and again.
Sarah Walker
And actually also because it turns out that one of the really fascinating findings is that people who've had COVID before, if they get vaccinated they do even better. But in order to find that out, you need to know who's been infected before. And because we've been testing our participants every month, some people now for nearly 18 months, we have a really good history. We won't catch every single infection but for most people we have a really good idea about who's had COVID or not. And again, that allows us to make much better estimates of the impact of vaccination, the impact of natural infection, and then how the two work together.
Miles Fletcher
And that's because the survey has covered the period right before the introduction of the of the vaccines right through their mass rollout. And according to the data you produce, towards the end of the summer, we're starting to actually see some of that vaccine induced positivity actually reducing once again, and new questions arise about booster jobs and so forth. What remains to be learned about the effect of those vaccines and about the longer-term impacts of COVID more generally?
Sarah Walker
There's a really huge emphasis on boosters at the moment, Miles, and I understand from the point of view of individual people, particularly if they are having their antibodies tested, and they see it go positive to negative, they may feel some concern, but it's really important to understand that the immune system has got memory. And actually you can have low levels of antibodies, but actually, the immune system remembers, and if you get exposed to COVID again, you get a burst of activity, and you are actually protected, particularly from hospitalization and death. And ultimately we've got to find a way to live with this virus, we aren't going to eliminate it and we can't keep vaccinating 50 million people a year. And so what we're trying to do as we move forward is to stop people ending up in hospital and stop them dying, whilst understanding that the virus is going to be with us. And I think the survey has really got a crucial role in answering some of the questions around what kind of levels of background infection can we live with. Are there thresholds of background infection, which then do trigger increases in hospitalizations, which obviously we can't live with? You know, who actually needs boosting in order to stop them ending up in hospital, as well as who maybe needs boosting in order to stop them getting infection, but infection that particularly leads to bad consequences. Obviously, antibodies are only part of the picture, we also have T cells and other things that help protect us from infection. I think over the next six months, the survey will really help us unpick some of those answers and really, it's about helping us learn how to live with this, because we aren't going to get rid of it. It's still a brand new virus. I mean, 18 months ago, we didn't know it existed. The beginning of March 2020, there was stuff in the papers about this virus in this Chinese city, but no one had any idea. And no, we still have a huge amount to learn. And the survey can really, really help.
Miles Fletcher
Vital then, that it keeps going. It’s one thing I’ve discovered in working on the media side of ONS, and like everything at ONS we try and show with numbers, it’s interesting how media attention for our estimates goes up and down according to the level of infection. Is there a danger more broadly do you think then that people might think at this stage that we’re seeing deaths and hospitalisations thankfully at fractional levels of what they were before, but people might still think that it’s all over really, that we don’t need to take this as seriously as we did, and Ruth how do we get the message out about the importance, the continuing importance of trusting the data?
Ruth Studley
So Miles I think one of the key things, you talked about the benefits of the longitudinal aspects of the survey, going back to the same people week in, week out. One of the critical benefits of this survey has been our ability to respond and be relevant to what decision-makers need. Sarah’s done some fantastic work and I’ll let her talk about the work that she’s done around vaccine effectiveness, but actually being able to understand and pre-empt and work with our users to understand what is critically needed is one of the real benefits of this survey. Because not only do you have that breadth of data and that wealth of data, but being able then to link it across ONS to some of our other data sources, whether that is the immunisation data or other data sources that we might hold within ONS, all adds a huge amount of value.
Miles Fletcher
Sarah, just looking at that international dimension again, are other countries running studies like this?
Sarah Walker
So, to my knowledge there isn’t another study like this in the world, both in terms of its length and the fact that it has been going since April 2020, but also in particular its size and representativeness of the general population. Generally what other studies are doing is relying on testing data, so relying on linking information about people who come forward to be tested in national testing programmes, either because they have symptoms or other reasons like workplaces. And of course, whilst that data is very powerful and is very large, not everyone who has symptoms takes a test and certainly there are plenty of people who have Covid without symptoms who never know they need a test. So, you know there are some real limitations of using that data, so from that point of view the survey really has got huge benefit and that benefit is recognised by policy bodies including the WHO in terms of particularly informing questions around vaccine effectiveness where being able to do the kind of adjustments that we do is really so important.
Miles Fletcher
So the survey has established itself as well, what the Times no less calls the most reliable measure of infection rates provided by the UK government. That speaks loudly because the media in the UK don’t hand out compliments for nothing: that’s a powerful testimony to how this study has established itself and it remains central to the UK’s research effort, and we’re told is to some degree unique in the world. Thank you to all three of our guests Tina Thomas, Head of Survey Operations, Ruth Studley Head of Analysis and from Oxford Professor Sarah Walker. Next time we hear the inside story of the 2021 Census and hear about the challenge of getting responses from every household in England and Wales during a period of national lockdown. You can subscribe to new episodes of this podcast on Spotify, Apple Podcasts and all the other major podcast platforms. You can also get more information by following the @ONSfocus Twitter feed. The producers of Statistically Speaking are Elliot Cassley and Julia Short, I’m Miles Fletcher, goodbye.