Getting started with pymc-marketing - questions and doubts

1

Dear community,

I am new to Bayesian modeling and am currently working my way through the “Bayesian Analysis with Python - Third Edition” book. As being a Data Scientist in a travel company busy with marketing analytics, my main motivation behind going into this issue is Marketing Mix Modeling. So i came upon pymc-marketing and its mmm. This somehow promises to make life easier even for people who do not have years of experience in Bayesian modeling and PyMC.

A few days ago, I watched the “Qonto’s Path to MMM with PyMC” session in the PyMC Labs YouTube channel, which made me doubt a little bit. They are talking about steep learning curves and about months of work (at least) to get a reasonable model done. Also, they switched from using pymc-marketing to building up their model with PyMC on their own.

My imagination was that after understanding the basics of Bayesian modeling and PyMC, there should be a satifying solution possible with pymc-marketing, just as I am used to build up quite predictive models using Gradient Boosted Machines for different kinds of problems in the non-Bayesian data science world. After hearing the Qonto talk, this imagination now sounds a little naive to myself.

Also, I have a feeling that it is much harder and more expensive to find good learning resources to enter the Bayesian world. While there are tons of online courses and books giving you even deeper insights into different kinds of models in the Machine Learning world, you don’t find any high quality online course about PyMC and advanced Bayesian modeling based on it on the usual platforms. Instead, comparable online courses cost > 10k €, which is quite difficult to explain to the management who has to pay it.

After having this thougts, I am now in doubt if I am on the right track. I would of course like to add Bayesian modeling to the tool box of our small team, but the team is indeed small and our resources are limited in both time and money.

I would like to have some honest feedback if in this situation, it might be more reasonable to rely on easier or more established / documented kinds of models, which do not have such steep learning curves and need larger investments in learning and consulting. After all, PyMC labs is a consulting company and has their own interests, and the pymc-markeing stuff might from some perspective look like a sort of bait which seems to make life easy, but after starting with it I find myself with an uncomplete and unsatisfying model and will not be able to succeed with it in a reasonable amount of time and without buying lots of consulting.

I hope its ok to share this kind of thoughts and doubts and appreciate any honest feedback.

Best regards
Matthias

1 Like
2

Hi @bayesian_padawan! This is a valid and interesting question; we can discuss it for ages :wink: But here is my short take:

Media Mix Models are for key decision-making processes, so they are not just prediction models and, therefore, have to be causal models. Sometimes, people have a hard time understanding this, so I wrote a case study: Using Data Science for Bad Decision-Making: A Case Study - Dr. Juan Camilo Orduz (see also Be Careful When Interpreting Predictive Models in Search of Causal Insights | by Scott Lundberg | Towards Data Science)

As you can read, whether the model is Bayesian or not does not matter. The important thing is understanding the problem and having a solid formalism on what you want to estimate and how you will use it. There is no free lunch!. Throwing a generic ML algorithm to a random problem and hoping for the best is easy. Making good decisions from a good model is a different story. Again, I want to emphasize this has nothing to do with Bayesian modeling itself but with the fact that there is always an investment in any serious molding task for decision-making.

Back to your initial question: From my (personal!) experience, the Bayesian way of thinking requires the modeler to be explicit about the assumptions and, therefore, think about the problem thoroughly. This has proven to be very positive when working in real applications. Of course, there is a learning curve. I decided to invest this way, which has paid off 100%.

Why Bayesian MMMs? For me, it is the ability to calibrate using lift tests very transparently and customize it with a lot of freedom (e.g., time-varying parameters). Last but not least, uncertainty quantification.

You can, of course, choose not to use Bayesian models, which is totally fine! However, the data analysis, scoping of the project, having a causal model with good evaluation schemes, and having a clear way of making it actionable is something you will have to do either way :wink:

I hope my personal view helps you :smiling_face: !

Learning Bayesian modeling is a long path, and here we are learning together :muscle: .

4 Likes
3

About learning resources: We are working hard to keep this example gallery (PyMC Example Gallery — PyMC example gallery) active, adding examples for different levels. I also like to share knowledge myself: https://juanitorduz.github.io/. I hope these free resources can help you and your team get started. We are also working hard on having top documentation for the PyMC Marketing project. If you feel there is something missing please do not hesitate to open an issue :pray:

3 Likes