@Michael_Johns Thank you so much for this - You have got me into Bayesian statistics!
Some questions
90 minutes runtime was on what sort of spec machine?
Does that runtime increase dramatically as you add additional channels?
Did you ever find Variational Inference to provide satisfactory approximations?
I’m not sure I fully understand the 2 stage approach. How is the CAC calculated for those bottom of funnel channels? Or do you put the adjusted spend for these like search etc. into a 3rd all inclusive model? So a top of funnel channel like Radio or TV would include spend + estimated spend on search over the number of attributes sales.
Runtime is on a vanilla MacBook Pro running 2.8 GHz i7. We’ve been subsequently doing some work with @twiecki and PyMC Labs (shameless plug!) to understand the causes and improve model fitting speed. I’ve also found independently that using stronger priors helps shrink the sampling tree and can reduce the fitting time by up to 2/3.
Adding additional channels does increase runtime but I wouldn’t call it dramatic. Maybe adds a couple of minutes; ultimately depends on how collinear the variables are.
We did some experiments with VI and didn’t have great success. I couldn’t get the model to converge and the results were quite unstable as a result.
At the moment, we don’t use any models to estimate CACs for those bottom funnel channels. Our focus is primarily on measuring channels above that level because those are the channels that drive most of our spend. We are working on a third model that will tie the main two together and allow us to get at direct, indirect and total effects for all channels, however.
I’m currently trying to build up a Marketing Mix Modeling for my research and I’m really glad that I’ve found your presentation. It’s very amazing, well explained and insightful. Thanks a lot for sharing the knowledge.
I would be very grateful if you could share with me your sample code, so that I can try it out on my data.
My email address is: qnhu90@gmail.com
Wow this was excellent. I’ve loathed doing MMM because I always fear my models would be (very) wrong due to multicolinearity and frequent low sample sizes in certain channels. When I first heard of pymc3, I thought MMM could work better in a Bayesian framework. Your presentation and model have given me hope. Two questions:
How did you learn the Theano part of the code? The website is the only resource I am familiar with
If you don’t mind, I like to review your code in more detail. If you are OK in sharing it, my email is mrwjblake@gmail.com.
Thanks again for a your great thinking and clear presentation,
Bill
Thank you very much for the presentation, it was really interesting.
I have a question and maybe you can help me:
I want to add the ADSTOCK function to the MMM model but I also want to add a function that allows me to identify the saturation point of advertising expenses, I found the ADBUDG function relevant for saturation, however I don’t know when to add it to the MMM model, yes before or after the ADSTOCK function, and I don’t know if it is possible to add the two functions (ADSTOCK and the ADBUDG saturation function)
Additionally, I thank you if you have any other ideas for a saturation function and how to implement it.
Finally, I would appreciate if you can share the code of your presentation, it would be very useful for me.
email : oscargc0110@gmail.com
Thanks for the informative presentation, it was a joy to watch. As I’m pretty new in Bayesian modelling especially in the field of marketing I would love to have a peek at your sample code. Would you kindly provide it for me too?
Thanks for such an interesting explanation of MMM and sharing your presentation. I am new to PyMC and doing my best to understand the model and made up the solution for my data. I don’t want to bother you but it would be very useful if you could share your sample code with me, of course if it is possible.
Hi Michael, hopefully you haven’t left the building in this thread yet - I was curious about MMM in general, but also in your implementation, about the time series components. It would seem to me that perhaps there is an auto-correlative pattern in sales data (and possibly in the predictors, too). I know you include some seasonality predictors (such as month), as well as the lag/adstock function - but other than that, it seems we have a pretty standard linear model? Have you thought about auto-correlation at all?
@Michael_Johns I’m also curious how you approach lead-in effects for holidays. For example, sales may be low compared to ad spend in the week before Black Friday.
And if it’s not too much trouble, I’d love to see your code, particularly for the charts you’ve created if you don’t mind! My email is jared [at] stufft [dot] us