When our middle and youngest daughters were married, we hosted their weddings out back on one of the sheep pastures at New Fadum Farm. These were wonderful affairs. We rented a tent, put down a dance floor, and had a really good dance bands come and play. Family and friends provided some of the meals served in the tent, but we got a caterer for the main events. These were tons of fun. Deborah and I hosted a somewhat scaled down version of these events the year when we both stepped down from full time teaching at the University. We are planning ahead for a really big event on our fiftieth wedding anniversary. That will be in 2023.
In contrast to those happier times, this COVID-19 lockdown is becoming a bit dreary. Deborah and I are dreaming about not waiting for our fiftieth wedding anniversary and just throwing a big party when this whole thing is over. However, in the back of my mind, I am a bit concerned about this “second wave” that keeps popping up in Ali’s simulation model. These concerns have led me to take a deep dive into the details of Ali’s model.
Here is what I discovered. First, Ali’s model makes some pretty standard assumptions about how pandemics grow and spread. His model predicts a cumulative death rate for Iran that is great enough to provoke a strong public reaction. Ali’s model makes the optimistic assumption that the government and people of Iran would react to a rising death rate by increasing social distancing, reducing contacts, and ordering lock-downs. Ali modeled these reactions as a feedback effect within his model. These feedback effects internal to his model serve to “turn off” a soon-to-explode pandemic. I think that Ali is correct. It is precisely the feedback loop that he placed in his model that averted a much worse crisis as the government and population of Tehran reacted to “turn off” the pandemic.
However, Ali’s feedback effect is a double-edged sword. When the death rate subsides in his simulation, the population backs away from social distancing, fewer contacts and lock-downs. What happens then? Well, in Ali’s model the pandemic bounces back. Ali’s model provides a fully specified causal theory of what is probably driving the bounce backs that we have already observed in some places around the world.
Here is where I hope that Ali got it wrong. I hope that if and when a second wave hits Tehran, if and when a second wave hits New York, the authorities will be better prepared. The second time around they should be prepared to implement a strict Test-Trace-Quarantine policy. In the first wave, our global public health system was caught unaware and short of time. We lacked adequate testing capabilities and did not fully see the danger, making us slow to react. I hope that Ali is wrong with two of his assumptions (1) the population’s cautious behaviors will fade away as the danger of the first wave subsides, and (2) the response to any possible second wave of the pandemic will be the same as our belated response to the first wave (because we were not fully prepared). I hope we can learn from experience.
There will be no celebratory party on New Fadum Farm this summer. No tent. No band. No caterer. I do not know for sure whether there will be a second wave. Nobody knows for sure. But Ali’s model has laid out a fully specified causal theory of why a second wave is a critical contingency that we and all human actors should be aware of. If Deborah and I were to proceed with the celebration, we might inadvertently contribute to a second wave. Worse, this second wave would include our closest friends and family members. We won’t take such a risk, no matter how much we might want to celebrate a break from the first wave.
I hope and pray that Governor Cuomo’s and President Trump’s mental models, indeed the mental models of all those deciding when and how to reopen schools, churches, restaurants, and indeed the whole economy have an important space in them reserved for thinking through the implications of this second wave.
This Story Has Two Lessons
(1) Ali’s model lays out a powerful and simple-to-grasp explanation for why a second wave of COVID-19 is likely to be in our future—human actions, including reduced contacts and social distancing, served to turn around the pandemic in the first instance. If (or rather, when) these are relaxed, the pandemic will resume unless a vaccine has been developed or strong test-based Test-Trace-Quarantine policies are followed
(2) Ali’s explanation for a second wave, like all model-based explanations, is a hypothesis that needs careful evaluation. Decisions are made based on the mental models of human actors, not by mathematical models. That said, it would be foolish to make plans for the future until we can reject Ali’s hypothesis with confidence. What we all decide to do next is of great importance to the health of our population and to the prosperity of our economy.
Supplemental
Babak's Model
Read More to Dig Deeper
In this short story, the important question for me, and for everyone, I suppose is how much should I trust Ali’s model? How much confidence in his model is enough for me to cancel our party? I suppose that I could just trust Ali’s model and cancel any plans for a big party this summer. After all, I have a lot of confidence in Ali. But should I transfer my confidence for Ali as a person into confidence in results from a model that he created? Is that good science? Fortunately, I don’t have to have this kind of blind confidence because Ali has provided a detailed record of his reasoning in three PowerPoint shows and has even provided the whole world with access to his source code. Anyone can check out Ali’s model and his technical conclusions in the “Technical Modeler’s Notes” at the end of this story. No need to know Ali personally to have confidence in the results of the model that he produces.
When I am trying to evaluate a model—when I am trying to increase my confidence in a model—I always look at the model as a huge and complicated “what if” hypothesis. My job becomes to attack this hypothesis to see if I can find ways in which it seems to be wrong. If I cannot break apart the model no matter how hard I shake it or poke at it while challenging every piece of theory and shred of data, then I become more confident that the model is somehow true. On the other hand, if I can easily poke holes in the theory or data embedded in the model, then I lose confidence in its conclusions and would not want to base any serious decisions on such a model. Of course, professional modelers have lots of tests that can be used to challenge a model, so they are in a better position as a professional community to complete these evaluations of a model and its conclusions. This, I believe, is the essence of the scientific method. If you have a hypothesis, then you have to try as hard as you can to refute it. Scientific theories (and good system dynamics models) are just hypotheses that no one has yet been able to strongly refute. Scientific communities consist of experts who have the knowledge and skills needed to try to refute hypotheses. Through a process of testing, revision and refinement they create theories more worthy of our confidence.
In my search to refute Ali’s model, I might turn to another model of the COVID-19 pandemic as an alternate hypothesis. Personally, I like the ISEE Systems COVID-19 Simulator as an alternative hypothesis. This simulator seems to show only one peak, though that peak may present as a double-bump with early and late sub-peaks. So if I accept the ISEE model as valid, do I have to reject Ali’s model? At his point it would be wiser to look “under the hood” of both models and examine them in more detail. Each of them is a complicated bundle of data and scientific fact put together for a distinct purpose. So I need to keep that purpose in mind as I evaluate how the model incorporates data and other scientific facts (such as fact about how humans contract and recover from the virus).
The ISEE model makes a number of key assumptions. First, it sets the spread of the virus in a hypothetical city of one million persons that they call “Anytown”. Even though “Anytown” does not exist as a point on any map of New York State or the United States or indeed the world, I am OK with this assumption because it clearly lays out what this hypothesis is all about. Next, ISEE assumes that the population of Anytown is uniformly mixed. That is, in this hypothetical place called Anytown, people are like so many identical particles bumping into each other at random. Clearly, this is not true out here on New Fadum Farm (I am not making too many random contacts these days), but in a big city such a Wuhan, China or Tehran, Iran or New York City, United States I think that this is not such a bad assumption. Third, the ISEE model shares the same causal mathematical model of infection and recovery that is used in Babak’s and Ali’s models and is consistent with epidemiologists have learned about epidemics in the real world, and it explains why epidemics peak and fall. I know that this is a good assumption for Wuhan, Tehran, and New York City because that is exactly what happened in those three places. Overall I like the basic assumptions of the ISEE model. It sets up an interesting and plausible story for understanding the pandemic.
As I see it, the ISEE model has two major uses or purposes. First, the model serves to identify and explore most important epidemic scenarios that might arise in Anytown. The model does not purport to “prove” that any of its basic assumptions are true. On the contrary, the model starts with well-considered assumptions about basic features of the pandemic and does one huge “what if” analysis to explore possible scenarios that could emerge from such a system, were it to be found in the real world. The ISEE simulator presents four basic modes of societal response that might appear during a COVID-19 pandemic: (1) Do Nothing, (2) Effective Quarantine, (3) Additional Testing, and (4) Containment. The model’s four base scenarios showed a wide range of possible behaviors. In the worst case (do nothing), the epidemic came to a peak within 100 to 120 days, with over 500,000 persons engaged with the epidemic at that peak and cumulative deaths at round 22,000 persons. In the best case (containment) by contrast, the epidemic came to a much shallower peak, engaging only around 80,000 persons at all levels of severity with only around 5,000 deaths.
The second major purpose of the ISEE simulator as I see it is to explore the sensitivity of these four basic scenarios to uncertainties in our knowledge of key aspects of the infection, such as how infectious people are before they experience symptoms. As I write these words, the jury is still out on what is the best estimate for the overall death rate for persons infected with the Corona virus. The ISEE simulator deals with that particular uncertainty by allowing model users to easily and quickly test how much overall mortality changes as this uncertain fact varies over its full plausible range. This is as easy as pushing a slider in the simulator to watch the whole pandemic swell to greater proportions or shrink dramatically. This is a sobering and dramatic learning experience.
I like just about everything that I know about the ISEE simulator. In my opinion, it meets its purposes well. I can easily imagine the basic scenarios and I can learn about how these scenarios will change in the face of differing not yet known factors. But this simulator does not speak directly to the question of second or subsequent peaks. In that sense, this model neither agrees with nor refutes Ali’s model of a second (and third and possibly fourth) peak. So now I need to go under the hood of Ali’s model to understand his basic set of assumptions and mechanisms in more detail. I can study Ali’s model, its assumptions and its results by clicking on the links in the Technical Modeler’s Issues below.
Ali’s model and Babak’s model and the ISEE simulator are the same in terms of the basic mathematics of the infection/recovery process. Initially the number of active cases grows exponentially since each active cases continues to produce additional infections and few active cases have had time to either recover or die. In a later stage, the number of new infections declines, since there are fewer susceptible people to become infected and persons with active infections recover or die in larger numbers. I am OK with these infections/recovery assumptions in all three of the models. Another point in common is that in all three models the population is represented as being completely homogenous. That is, there is one pool of susceptible people and they’re all alike, and there is one pool of infected people and they’re all alike, and so on. Now I have to stop and think hard. Would it actually be true that the same explosive forces of contact and contagion operating in downtown Tehran would be at work in rural communities in Iran? That would be like assuming that contagion out here on New Fadum Farm matched that in downtown Manhattan. I am not confident that assumption holds water. This leads me to suspect that because of this assumption of uniform mixing and uniform contagion equations, Ali’s model could mis-estimate infections and deaths in Iran if, as is likely, its parameters are derived from the urban experience. We could test for that by disaggregating the model into a rural and an urban sector, with different parameters for contact frequency and probability of transmission per contact, to see what difference that would make. As interesting as this line of inquiry may be, it does not help me to understand Ali’s phenomenon of a second and (and maybe more) waves of infections.
I can explain this second wave to myself by comparing Ali’s model to the ISEE model more closely. In the ISEE model, the model’s user (that would be me when I am running the model) decides how cautious the simulated population will be with respect to issues of social distancing, limiting contacts, and lockdowns. Once I (the user) set these levels of caution, the simulator unfolds the consequences of those set assumptions. By contrast, in Ali’s model a feedback loop internal to the model acts to “turn on” and “turn off” social distancing and lessened contact, leading to repeated waves being simulated in this model. That is, whereas in the ISEE model, the user had to decide to turn up the assumed level of caution by the pubic, in Ali’s model an internal (or “endogenous”) feedback loop did the same job.
Notice that in the telling of this story there is a third model. It is the “mental model” of David Andersen. As David unfolds this story he expresses confidence in some things and doubts about others. Such mental models of the interplay between human behavior and natural processes are the key models that we need to pay attention to. Computer simulations do not make decisions, they do not make real changes in the world. Deborah and I will sit down with our daughter and her husband to decide what we will do on New Fadum Farm this spring. Four mental models, hopefully informed by results from Ali’s model and the ISEE model (and other models), will meet and reflect on what to do next.
All of us who are making decisions about holding parties, reopening schools or churches, or restarting meetings for a 4-H club or Boy Scout troop should carefully evaluate the possibility of the important bounce back effects that are found in Ali’s model. In the end, what Deborah and I do about a party could be very important for our family and friends, though our decisions will have little impact on whether or not there is a second or third wave. The important mental models driving the second wave are probably those of political leaders such as Governor Cuomo and President Trump who need to most carefully assess a complex future fraught with the possibility of unintended consequences of the gravest nature.
