Time series modeling question

Questions
1

I am not sure if I should ask this kind of question here, but let me know if Cross-Validated would be a better place. This is more a question about modeling than PyMC3. But since PyMC3 is the only tool I know to do Bayesian Modeling, I am wondering how I might do that in PyMC3.

We have a tool that allows companies to audit their stores. They can make custom questionnaires and auditors go in the field and fill the questionnaire and the store gets back an evaluation. Stores getting audited with different questionnaires, by different auditors. Stores are in different companies. So hierarchical model seems like a good fit here.

I am modeling the score of the next audit in a store. This is useful to help managers better plan their resources and assign coaches/auditors to stores that might need the most help.

Right now I am modeling this as a binomial hierarchical model where for each questionnaire there’s a maximum # of points (# of trials in the binomial) and the audit had a final score Y (successful trials in the binomial). I use the score of the last two audits in the store as predictors. I am getting very good performance out of this model but I still feel like I don’t exploit the time dimension enough. I feel like each new audit in the store is not independent from the audit before because the store will actively try to become better or maybe if there’s a lot of time between two audits become worst…

I would like to be able to add this time dimension as part of my hierarchical model where each store could use something like a Gaussian random walk, but their weights would be partially pooled by store, by auditors etc to allow shrinkage. Not too sure where to start learning for this. Maybe if there’s no examples in PyMC3 could anyone suggest other resources I could read for other similar tools?

Thank you!

2

You only have two timepoints and you are trying to predict the third time point right? In that case I think the simplest thing to do is adding a new predictor using the score at t-1 to the linear function. I dont know how you parameterize the model, but if you are doing matrix multiplication essentially you are adding a new column into the design matrix.

Another way to think about it is that, at each time point the variance of the observation is divided to 3 part: noise, variance explained by the predictors, and the variance explained by the previous timepoint. You can then model the total variance (eps_total ~ HalfCauchy(10.)), and generate a simplex (e.g., [.01, .2, .79]) to divide the variance. You can have a look at this notebook which demonstrates the idea.

3

I already modified my data so that I have two columns Shift1Score and Shift2Score (those are in percentage) in my predictors which are the score of the last at t-1 and t-2. I use those predictors to predict the score of the next audit.

Thank you for the link I will look into that!

4

Also I found this paper yesterday that pretty much sums up what I would like to try. Effective Bayesian Modeling of Groups of Related Count Time Series

I am already having really good performance with just using Shift1Score and Shift2Score. Not sure if I want to add the complexity of a model like that since it would make it much more less interpretable for me. But this sure sounds interesting for other types of count time series.