Martin has been kind enough to put together a Guest Post on Data Modeling in the Epidemic.  Part 1 was posted at approximate 2:30 pm on Wednesday.  This is Part 2.

Once again, Martin is standing by in case we have questions.

Take it away, Martin!

Questions on Data Modeling in the Epidemic: Part 2

So, how do we know if containment is working and for how long do we need to do this?

We can answer this! Well, we can get close, with a few caveats, because we can look at what happened in China, and we can do a little bit to confirm that model with what’s happening in Italy a bit ahead of us. So how do we build it?

Well, what do we have to work with, and what do we need to know? We have a few data elements – confirmed cases, fatalities, recoveries. And we have time. We know this data for each day. We know this for the whole world, for different countries, and for different cities and states. Now, the experts have a whole bunch of other data, hospitaliation, ICU cases, intubated cases, tests administered but waiting on results, etc. and all in infinitely more detail than we have.

Confirmed cases is kind of garbage. I’ve been largely ignoring it because I don’t know if it’s telling me reproduction rate R₀ or testing rate. It may get reliable, but I’m not counting on it.

The most accurate bit of data is likely fatalities. Unlike determining if someone is infected or not, we’re really good at determining if someone is dead or not. And if they are dead, we can test if they’re infected, so we can probably rely at this point on that being a pretty reliable number. Time can be a bit more uncertain than you might think because when data is collected and reported in a human administrative dependent process (as opposed to an automated weather station that does things on precise and unwavering schedules) you have problems of people going to the dentist and not getting their data in until the next day. So, we’ll expect this to be a bit noisy from day to day.

Now, because we’re at the start of an epidemic, where there’s almost nothing holding spread back like herd immunity, we can probably expect to see something like a perfect exponential curve. A model for infected is more complicated because people recover. Nobody recovers in our model. And when we plot that out, that’s exactly what we get. People aren’t very good at intuiting variation from an exponential curve, or even extrapolating on an exponential curve, but if you take the logarithm of your data, you wind up with a straight line, and we’re pretty good at intuiting a linear function. Below is a plot of the log of our fatality data for the US, and that’s a pretty darn straight line. I suspect that recent uptick in the slope is due to NYC dominating the national data and having a higher R₀.

If you want to play along at home, the fatality rate for the US is approximated by e^0.273t where t is days since the first fatality (Feb 29).

So we have a well behaved exponential function, and that doesn’t tell us when things will change, but it does give us a sense of urgency. You can look forward and see projected fatalities that make you pucker and decide let’s make sure we don’t get there and then work backward.

Understand, we’re building a very simplified model here. Our goal isn’t to give us any real long-term predictive value of how many people may contract this, or how many people will die. Our goal is to give a good approximation of the worst case sceniario for early in this epidemic and then look for when the model breaks on the assumption that our actions will break the model before other normal factors like herd immunity does. The model gives us a sense that if we want to keep fatalities below a certain number (and we’re assuming that number is relatively small) then we need to act before a certain date. In terms of actual fatalities, the model is probably accurate to about an order of magnitude, and that’s all we’re looking for. Are we looking at thousands or tens of thousands or hundreds of thousands of fatalities? What should I emotionally try to prepare myself for, and how loudly should I scream at my governor to shut my state down now, even if things may not seem too bad locally.

What does China tell us?

China gives us some good data to work from. They did a bunch of minor things just as the US did, but they locked down Wuhan on Jan 23, and all other urban areas the next day. Jan 23 is our day 0. And China saw a nice exponential curve as well – it was a little different in magnitude (the slope of the log is different) so it might grow a bit faster or a bit slower but either way it grows incredibly fast.

The first sign their lockdown was working was on Feb 5 (day 13). That was the first day that new cases fell, and they generally continued to fall after that. That doesn’t mean that people stopped getting sick on Feb 5, it means they stopped getting sick on Jan 24, but we couldn’t measure it until 13 days later (give or take a few days, plus a few days to confirm that it’s a trend and not just an outlier). So, if we are modeling infections and we want to know if a given action had an effect, measure any change that occurs around the 13 day mark. That also tells us that any action needs to remain in place for probably around 3 weeks before we get any real sign it is working or not. But this is our inflection point for R₀ going from greater than 1 to less than 1.

The next sign came on Feb 13 (day 21), the first indication that the rate of fatalities was halting. Now, the fatalities didn’t immediately fall, but it stopped growing and that’s key. Fatalities per day stayed relatively flat until Feb 24 (day 32) when it started to consistently fall. Then on March 9 (day 46) the number of daily fatalities fell to about the level of day 0.

So, what does this tell us? Well, look at that date where the fatality projection makes you pucker, go back 21 days and make sure your most aggressive mitigation action is in place by then, because if not, you will hit that number, and you may maintain that daily rate of fatalities for days.

Now, a few caveats here. The 13 day and 21 day numbers are largely a function of the virus, and not the population. Those should be roughly equally true in China as New York City as Montana. If you blow through day 13 (give or take) and have no reduction in infections, then you didn’t dream big enough, need to throw down some much more restrictive actions, and wait another 13 days (give or take).

So, does Italy validate that? Possibly. Italy quarantined their first area on Feb 23 and then did national quarantines on March 8/9. We should see some slowing of new cases on March 7 (day 13 for the smaller area) but without an infection model we can’t see that, and a larger reduction around March 21/22 (day 13), and we did see that on March 22. We should also see some sign of reduction in the fatality numbers around March 15 (day 21 for the smaller area), and we do. Their numbers are still climbing, because that wasn’t the national lockdown, but it definitely slowed right around that date. The next and larger data point should come around March 29/30.

The dates after that are largely a function of the population, the effectiveness of the actions taken on day 0 and the compliance of the population. The 11 day long plateau in the fatality rate that China saw might be shorter or longer here. The 14 days to reduce from the plateau back to day 0 might be shorter or longer here.

My assumption is everything will be longer in the US than China. Despite Wuhans high population density, China has an unprecedented ability to control their population and an unusually high level of compliance by the public. The US is struggling with compliance, and has very little control. That doesn’t mean it won’t work, it just means we probably won’t see that nice sharp inflection that China had. It’ll probably be messier and slower, possibly much slower. Italy should give us a little more insight in how much things can vary. Their lockdown was national, but Italians are notoriously defiant of government guidelines, so they should look closer to US efforts.

Days 0 in the US:

Bay Area: March 16
California: March 19
New Jersey: March 21
NYC: March 22

Bay area is already showing some evidence of improvement, presumably from their work from home, public gathering orders back in early March. We’d expect to see real new case declines on or just after March 29. CA as a whole, April 2, New Jersey April 4, NYC April 5. We’d expect to see fatality growth halt in the Bay Area on or after April 7, CA April 10, NJ April 12, NYC April 13.

If nothing else, we’re trying to establish the importance of acting quickly because once fatalities starts to go, it goes fast. And while we’re in this state just waiting for something to happen, roughly when we can expect to see results and where to look.

Additional information added at 3pm, based on new information and an additional model:

So, I wrote this yesterday, and some new information has come out, and some new models have been made.

Specifically, I built a model of NY. To start, it’s a bad model since there are only 13 days of fatality data. A model built on 13 datapoint is going to suck. It can change wildly with just one more data point. To give an example, a model built of Washington State built after 13 days would look apocalyptic because all of the data was dominated by a single nursing home, that you would expect to have a vastly higher mortality rate than the general population. The mode suggests a trend that simply can’t hold, mainly because everyone in the nursing home was pretty much accounted for. You can’t grow fatalities in a 150 person facility above 150, but the model is too primitive to reflect that. So that’s just a limitation we need to keep in mind. And sure enough, after some more data was collected, a more reasonable and less apocalyptic trend developed. We just had to be aware that there were a lot of unknowns and wait to see how things shook out. That how this kind of field data modeling works. It’s messy and limited but can help to tell us where to look or can change the urgency of a decision.

As to New York, their numbers look pretty apocalyptic right now. But New York isn’t getting the kind of detailed reporting that Washington did. For all we know, most of these fatalities came out of a single project in the city and once it tears through the few thousand people that live there, the underlying general population trend will emerge. I’m hoping that’s the case, but we just have to wait. I want my model to break, because that provides us with a new bit of information, and we can then go and try and figure out why it broke. We have a theory for when lockdowns will break the model, but there can be other causes as well – as happened in Washington state. That’s not a bad thing. In fact, that’s sort of the point.

Another key bit of information is out of Italy that they may be significantly underreporting their fatalities. That was a concern of mine. Any human collected dataset has to deal with this stuff, so you just asterisk everything. The testing data is kind of garbage because everyone has different access to tests, and different criteria for testing, but it also changes for a given location. When the 10th person this hour walks in presenting the same set of Covid symptoms, do you really take the time to give the test, or do you just get them in a bed ASAP. Of course testing is good information, but you may no longer feel you have time to do it. So testing data just gets that much more unreliable. Same goes for fatalities. When 13 people a day are dying in your hospital, the effort you put into determining cause of death is going to change. Do you run the Covid test so you can list that as cause of death, or do you just put down what you know, respiratory failure, and move on.

Italy’s fatality data seemed to fall off faster than I expected. I attributed that to the smaller quarantine in northern Italy having an oversized effect on the data (they quarantined for a reason, so it’s not unreasonable to assume they’d dominate the data) but now it’s looking like it may (also?) be a lack of attribution of fatalities to the disease. We just have to deal with that. China’s data saw a similar pattern, possibly for a similar reason. Maybe fatalities don’t drop off at day 21, hospitals simply get too overwhelmed to count them accurately, and the real dropoff is day 32. More likely it would be somewhere in between. So, we’ll look more closely at Italy’s data as it continues to come in and see if we can figure that out. California may provide a better data point. We locked down earlier than Italy in terms of number of fatalities, so I don’t expect CA will get quite as overwhelmed as Italy. That should give us somewhat more consistent data.

Martin Guest Post: Questions on Data Modeling in the Epidemic: Part 2Post + Comments (68)

Martin has been kind enough to put together a Guest Post on Data Modeling in the Epidemic.  Part 2 will be posted on Thursday afternoon.

Martin is standing by in case we have questions.

Take it away, Martin!

Questions on Data Modeling in the Epidemic: Part 1

So, like Cheryl (such good company I find myself in!) I thought I’d provide some information on data modeling this grand experiment we find ourselves all in. I do have a statistics background and do a fair bit of data modeling of population behavior. I’m not an epidemiologist, but good data modeling always requires some degree of understanding the context the model operates in so that means spending some time becoming a ratchet epidemiologist – knowing just enough to understand how to make the model work.

Epidemiologists arm us with a bit of information regarding the behavior of this particular virus. We have the R₀ (r naught) or the reproduction number that says that each person who is infected will infect R₀ other people. That is determined by the infectious period (how long you can give it to someone else), the mode of transmission (air, water, touch, etc.), and the contact rate (how many people you are likely to interact with). We also know how long before symptoms appear, how long before fatality is likely, how long to recovery. These are statistical values and vary from person to person. Early in an epidemic you have fairly small sample sizes so it often just looks like chaos, but later when sample sizes are much larger those statistical values start to really show up.

Why is this such a big thing?

Well, there’s a few things going for it. An incomplete list:
1) Nobody has natural immunity, so for the contact rate above, every person an infected person comes in contact with is a potential new infection. One reason why the flu doesn’t do this is that depending on the strain of flu, there’s people walking around immune, so it doesn’t spread through them. That’s what herd immunity does – if enough people are immune, an infected person can’t find enough others to pass it along to for it to explode through the population. (This is a good place to ask Cheryl about how nuclear reactions and nuclear moderators work – basically the same process)
2) Nobody has artificial immunity. No vaccine. Again for the flu, that flu shot not just protects you personally, but helps add to the herd immunity.
3) Good modes of transmission – inhalation, contact, and fecal transmission. Hong Kong had some apartment blocks spread it through the sewer system, even when people were locked down. This is a combo that is notoriously hard to protect against.
4) Durability – it can hang in the air for hours and on surfaces for days, boosting that contact rate. It can live long enough that if the stocker at the grocery store spread it to the jug of milk overnight, it can still be there when you put it in your refrigerator.
5) It looks like other diseases – early symptoms are flu-like, so rather than rush off to the doctor, we tend to keep acting like it’s the flu, treating it like the flu, and protecting ourselves like it’s the flu. But it’s not the flu, so those things don’t work sufficiently. Ebola struggles to spread because it’s in the ‘holy fuck this guy is dying’ category, not the ‘take some Advil and go to work’ category.
6) It doesn’t affect everyone the same way. This is more important than it might seem. The 1918 flu killed young people more than old people. That allowed old people to unknowingly spread it much further than if they had fallen ill at the same rates. Covid is the opposite. Where that really matters is with testing. If you only test people who are really sick, you leave a whole bunch of people only slightly sick to run around and spread it. And it’s hard to get people to stay home sick if they don’t feel sick.

R₀ is important because it tells us roughtly just how fucked we are. An R₀ value of 1 means that everyone who gets it will give it to one other person. That leaves you with a fairly constant population of sick people which is very easy to deal with. An R₀ value that is less than one will eventually die out on its own, or at least shrink to the size that it’s no more than a nuisance. An R₀ greater than 1 means that the population of people who get infected will grow expoentially until you see things like herd immunity show up and start to knock that rate down by shrinking R₀ by slowing the contact rate. An R₀ value well above 1 means we’re going to go through some shit. The estimates out of China was that Covid had an R₀ of around 2.7.

But R₀ is not a property just of the virus but also of the population and their behavior. We calculate it for the population at large but every context will be different. A highly dense city like NYC will almost certainly have a higher R₀ than a less dense city like Des Moines. When I lived in NYC we lived in a 5 story walk-up. There was no elevator, so you had to climb the stairs to our 4th story apt. There were about 100 people living in that building, all using the same front door handle, all touching the same mailboxes, all running their hands along the same handrails. That’s a lot of contact. I now live in a detached single family house. Only 4 people ever touch the front door or the railings. Much lower contract rate. When my wife comes home with a week to two weeks worth of groceries everyone heads out to unload the car and bring them in. But when you live in a 5 story walk up, you don’t do multiple trips to haul in 2 weeks of groceries, instead you maximize each trip. You know you’re going to climb those stairs at the end of every work day so you bring a single load of groceries with you every time – about a days worth. That means you’re going to the store roughly every day, interacting with others, etc. Again, that’s a lot of contact. In a university, a 500 seat lecture hall has roughly 10x more contact than a 30 seat K-12 classroom. That’s why universities moved to online instruction so early. Our R₀ value was going to be massively higher than your kids 2nd grade classroom. Nurses have more contact than accountants, etc.

Why are some places blowing up worse than others?

Well, for NYC, the population density and community behavior of more regular store trips, more frequent eating out, and so on means that their R₀ value could be much higher than 2.7, while a lot of California which saw the virus earlier than NYC has a value that might be somewhat lower than 2.7. Italy is denser than the US on average, so they’re having a bit harder time with it. Canada is lower than the US, so they’re having a bit easier time.

Containment

A containment strategy basically means to identify and isolate the infected so that their R₀ reaches 0. For ebola that might mean a perimeter around the village so nobody goes in or out until nobody is infectious. You’re going to knock down the contact rate to 0 until the infection period is over, and it can’t be spread. For measles, it means isolating the person’s home the same way. But it relies on being able to identify who is infected, and then tracing where they’ve been, who they’ve been in contact with and prophylactically isolating those people before they become infectious and only releasing them after testing.

This strategy failed in the US because we couldn’t test and isolate as fast as the infection spread. It succeeded in South Korea because they could. It failed in Wuhan for a few reasons – a little bit of denial at first, but that was only a few days, but then a lack of a test.

The goal of containment is R₀ = 0.

Mitigation

When containment fails, you mitigate. You’re giving up on R₀ = 0. You can’t do it. But you can do R₀ < 1, so you’re trying to get the number of cases to shrink to a small enough value that you can either maintain that state until something like a vaccine shows up, or you get the problem to be small enough that you can try to make containment work again.

That’s what China did. They mitigated, shrank the problem, and outside of Wuhan appears to have achieved containment. Jury is still out on whether they can get there for Wuhan as well.

That’s what the US is doing – reduce the contact rate. What is needed? Well, probably more for NYC than for Des Moines. Social distancing and hand washing might work in Montana but not Des Moines. A lockdown sufficient for Des Moines might not be sufficient for NYC. And if people don’t follow the rules, you might need to do more.

The goal of mitigation is R₀ < 1

So, how do we know if containment is working and for how long do we need to do this?

That’s where the modeling helps us in Part 2.

~Martin

Martin Guest Post: Questions on Data Modeling in the Epidemic: Part 1Post + Comments (76)