The Bayesian Workflow: Building a COVID-19 Model by Thomas Wiecki

pymc_devs_bot · October 17, 2020, 6:04pm

Talk Abstract

In this tutorial we will build a COVID-19 model from scratch.


Thomas Wiecki	Twitter `@twiecki`	GitHub `twiecki`

Talk

Part 1

Part 2

Thomas Wiecki

Thomas is the founder of PyMC Labs, a Bayesian consulting firm.

This is a PyMCon 2020 talk

Learn more about PyMCon!

PyMCon is an asynchronous-first virtual conference for the Bayesian community.

We have posted all the talks here in Discourse on October 24th, one week before the live PyMCon session for everyone to see and discuss at their own pace.

If you are available on October 31st you can register for the live session here!, but if you are not don’t worry, all the talks are already available here on Discourse (keynotes will be posted after the conference) and you can network here on Discourse and on our Zulip.

We value the participation of each member of the PyMC community and want all attendees to have an enjoyable and fulfilling experience. Accordingly, all attendees are expected to show respect and courtesy to other attendees throughout the conference and at all conference events. Everyone taking part in PyMCon activities must abide by the PyMCon Code of Conduct. You can report any incident through this from.

If you want to support PyMCon and the PyMC community but you can’t attend the live session, consider donating to PyMC

Do you have suggestions to improve PyMCon? We have an anonymous suggestion box waiting for you

Have you enjoyed PyMCon? Please fill our PyMCon attendee survey. It is open to both async PyMCon attendees and people taking part in the live session.

Baskoes · October 27, 2020, 9:48pm

Hey Thomas, thanks for this explanation of this model. Certainly makes something accessible that wouldn’t otherwise be to me! so feel free to ignore these questions if they are too onerous: So the Gaussian random walk is flexibly capturing all the structure related to policy / individual interventions that decrease the virus’s reproduction rate. Can you explain any more intuition behind ‘the sampler will try and fit the rope to best explain the data’? In a longer term predictive setting would it just go back to random or would it actually learn some structure? Could you somehow include an observed variable for the social distancing policies that were implemented and then pass into the prediction a range of forward policy options and compare them?

AllenDowney · October 30, 2020, 4:16pm

Really nice talk, Thomas. Thanks very much! One thing we see in the data is a clear weekly cycle, which I assume is due to some combination of the workweek (fewer people tested on weekends?) and the data reporting/collection process. Knowing that these effects are artifacts, I wonder if it would improve the results if you smooth the data before running the model.

junpenglao · October 30, 2020, 4:43pm

Or add a weekly component to the Gaussian Random Walk, maybe something from Hierarchical Time Series With Prophet and PyMC3 by Matthijs Brouns

twiecki · October 30, 2020, 5:26pm

Thanks, Allan.

This weekday effect is mostly driven by the different numbers of tests administered. This is actually already included in the final model when we take testing frequency into account (I only mention that part briefly but the NB has more info). The effect of that is the smoothing you’re talking about.

twiecki · October 30, 2020, 5:30pm

Yes, exactly, the random walk is modeling the changes in people’s behavior, policy etc which all affect the effective reproduction rate.

It does so where it has data. The further you forecast, the more uncertainty you will have in the reproduction factor.

What you mention about including things like social distancing policies is a great idea. You could add this (and other factors) as covariates to the model, like in a regression model and then use that for prediction.

docmrichards · November 3, 2020, 10:12am

Nice complete talk thanks Thomas! Knowing your challenges(!) I looked at your notebook and it seems like you fixed the delay model with an exposure clipping prior, it all worked really well for me

Slightly off topic, my question is when trying to apply a random walk model into a new dataset, with Data[], it recalls the shape and mu of the random walk steps. But I want to relearn these with the new random walk path, on the fly as it were, as otherwise the old data biases the new data. I guess I would like the random walk to be a distribution I can sample from as a transient to learn other model parameters and posterior, but that the walk steps are not embedded into the model (mu_0, mus_1 … etc), but I can’s see a way to do that, and grw does not have a dist, random sample method so can’t use it like that. Seems I have to set the new data as ‘missing data’ in the model and run the model again, so it imputes the out of sample data. Hope this makes sense! Anyway, great talk, very useful, and great conference

twiecki · November 5, 2020, 11:08am

Thanks Marcus.
I don’t quite understand, do you talk about prediction?

I think the GRW has a .random() method, no?

In any case, you can always do pm.Normal().cumsum() to create a random-walk manually.
Not sure that helps but I didn’t quite understand the question.

docmrichards · November 5, 2020, 5:33pm

thanks Thomas,
yes - I am trying to do prediction with a pre-trained model which uses GRW but I do not want to persist the random walk steps into the model state as they are just transitory in calculating my principal model parameters. You could say I am just using the GRW for smoothing.

Yes, the GRW has a random() method - I was looking at the multivariate GRW so got that wrong Thanks for the tip about cumsum() - that sounds great, I shall investigate further.

HG1 · December 1, 2021, 1:06pm

Hi,

I cannot find the notebooks on your github repo.
Is it still available somewhere?

Regards Hans-Peter

twiecki · December 2, 2021, 4:10pm

Here you go:

gist.github.com

https://gist.github.com/twiecki/43def0aa16ee6a23f822a124eb429958

Section4_1-Bayesian_Workflow.ipynb

{
  "cells": [
    {
      "metadata": {
        "papermill": {
          "duration": 0.016911,
          "end_time": "2020-03-27T06:09:14.400757",
          "exception": false,
          "start_time": "2020-03-27T06:09:14.383846",
          "status": "completed"

This file has been truncated. show original

gist.github.com

https://gist.github.com/twiecki/fc63488e7c81d162af3f58ae68a32cd4

Section4_2-Generative_Modeling.ipynb

{
  "cells": [
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "# Generative Model Tutorial\n\nIn this lecture you'll learn:\n\n* How to build generative models that take into account knowledge we have about the data generation process\n* How can apply Bayes formula to reason backwards from observed data to unobservable hidden causes\n* How to model time-varying processes\n* How to iteratively improve a model\n\nThis notebooks explains the model powering https://rt.live with its nowcast of the effective reproduction number. It is based on [version 1.0.0](https://github.com/rtcovidlive/covid-model/commit/50bf8e4ace5b973b3d323f058ad613d4b02f4c2f) of the model.\n\n## What is Rt?\n$R_0$ (\"R-naught\") describes the reproduction factor of a disease -- i.e. how many other people does one infected person pass the disease to. If this quantity is larger than 1 we have an epidemic on our hands, as is the case for COVID-19. \n\n$R_0$ assumes, however, that there are no counter-measures being implemented to curb spread of the virus. Thus, the more critical measure to track is $R_e(t)$ -- the time-changing effective reproduction factor, i.e. on a given day $t$, how many people does one person infect. \n\nAs lockdowns and social distancing measures are put in place we expect this quantity to drop, ideally below the critical quantity of 1 because then, over time, the disease would just wimper out.\n\nUsually we'd extract $R_e(t)$ from something like an [SIR (susceptible-infected-recovered) model](https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology) or an SEIR (which adds an exposed category), which are classic epidemiological compartment models. \n\nAn SIR model is in fact what `rt.live` used in the beginning. However, SIR/SEIR models are also just approximations of the real thing and come with quite a few assumptions baked in. The current model is simpler and makes fewer assumptions. In addition, the SIR model is described as an ODE which causes various technical problems. Solving the ODE is quite time-intensive and while closed-form approximations exist and are faster, we found that they are quite unreliable.\n\nInstead, the current model uses a simple generative logic to explain how a initial pool of infected people spreads the disease at each time-point, according to the current reproduction factor.\n\n## The generative model\n\nWe need the package from the rtlive repo to download data and get our distributions."
    },
    {
      "metadata": {
        "trusted": false

This file has been truncated. show original

generative-rtlive.ipynb

HG1 · December 3, 2021, 11:40am

Very much appreciated!
Thank you.

HG1 · December 3, 2021, 11:41am

And the 1 million dollar question.
Did you find out what caused the model to go awry at the last step?

twiecki · December 5, 2021, 8:07am

Ha, no I haven’t. Internet fame points if you do and post it here :).

Topic		Replies	Views
New PyMCon Talk Released: Bayesian Causal Modeling by Thomas Wiecki & Ben Vincent PyMCon Web Series	9	1153	October 2, 2023
Exponential, Logistic and Generative Bayesian Models with Gamma Prior	3	171	March 1, 2024
Need Help creating models in PyMC3 as a beginner Questions	4	344	November 2, 2021
PyMC3 for COVID-19 Research: Now Published in Science Questions	1	600	December 19, 2020
Scallable Bayesian Models PyMCon Event this week! Announcements	0	369	March 19, 2023

The Bayesian Workflow: Building a COVID-19 Model by Thomas Wiecki

Talk Abstract

Talk

Thomas Wiecki

Related topics