Help on how mmm budget optimizer works

Questions v5
1

Hello! I am looking for some understanding of what budget optimization does in pymc-marketing MMM.

Lets say specifically this:

bo = BudgetOptimizer(num_periods = n_period,model = mmm,response_variable = “total_contribution”)

optimal_budgets, scipyresult = bo.allocate_budget(total_budget = ttl_budget,budget_bounds=bounds)

Given Adstock, saturation and even optional interactions within the model, the budget optimization method would seem to require these steps for a given set of candidate values (channel spend amounts) to be repeated:

  • create a new X with all required dates, control variables and spend candidates, either distributing the channel spend evenly or weighted across the time periods of the future period. Then also creating channel interactions manually if they are needed. This would be exactly like an out-of-sample prediction.
  • Run

forecast_predictions = mmm.sample_posterior_predictive(
X=new_X,
extend_idata=False
)

  • Summarize the outcome variable as a mean of the sum over the time periods and return to the optimizer.
  • Repeat

This is likely computationally expensive.

If the budget optimizer is instead using the saturation curves from

curve = mmm.saturation.sample_curve(
mmm.idata.posterior[[“saturation_beta”, “saturation_lam”]], max_value=2
)

isn’t this assuming that 1) there are no channel interactions in the model and 2) the values for X which are put forward by the optimizer hit the saturation curve directly without adstock and thus would not be accurate? Or are the spend values X being adstocked and then hitting the saturation curve (which begs the question if the total outcome contribution is being summed even outside of the range of time in num_periods?)

@cetagostini seems like the expert if possible to shed some light on this for me?

Thanks!

2

The optimizer doesn’t predict the observed quantity but the expectation. controls and non optimized parameters are frozen during optimization. Only budgets change.

Furthermore the optimizer can and uses gradient information to search the space much more efficiently than computing a dense grid for different values of x like in your description

1 Like
3

Hi @ricardoV94

No I wasnt suggesting a grid, I was just saying that the ideal way to optimize the budget seems to be forecasting future values from the model taking into account everything, where the loop I mentioned is the obj function for the minimizer (pass in candidates, evaluate and return to the minimizer - repeat) ,

But, Is the process the optimizer uses just the saturation curves so that adstock is not taken into account or was my description of it above incorrect?

4

By default the optimizer is looking at total_contribution, which should include both adstock and saturation. I’ll let others confirm that, it has been sometime since I worked on it.

1 Like
5

@ricardoV94 confirm what you mentioned.

Optimizer takes into account saturation and adstock based on your model, replicating the full generative process until contribution (here `total_contribution`), if you adjust the transformation to consider interactions or anything like it, will be consider (If you are doing vanilla MMM API does are not by default there).

the ideal way to optimize the budget seems to be forecasting future values from the model taking into account everything

That’s precisely what optimizer is doing @jmallard but using a more “*clever*” pytensor methods under the hood than looping.

6

Thanks for the info @cetagostini !

Can we intuitively then understand that during optimization, the candidate spends for a channel in a time period (e.g. week) have adstock applied and then the adstocked spends are input into the saturation function?

If we have say 6 weeks for the optimization period, will the process take into account lags that fall into week 7+ due to adcost?

Also - Control variables are all set to zero correct?

7

@cetagostini checking if you have a chance to comment?

8

On the adstock topic: yes, the optimizer counts adstocked contributions that fall outside of the official optimization window.

The optimizer doesn’t consider the value of the control variables, as far as I can tell. Because the model is strictly additive, the control contribution and the media contribution are independent.