Inconsistent ROAS when migrating to multi-dim MMM class

Questions v5
1

Hello,
I trained an MMM using pymc-marketing and realized that it’s probably better to migrate to the multi-dim MMM class for budget allocation analysis, even though I have a single geo to model. Despite keeping everything the same (input data, model configs, random seed for model fit), I see a significant difference in the ROAS between the two models. In my understanding, the results matched better with the real-world scenario for the single geo MMM. I don’t understand what could have caused this — looking for advice. Please let me know if I should provide more context. Thank you.

prior_sigma = total_spend_per_channel / total_spend_per_channel.sum()

my_model_config = {
“intercept”: Prior(“HalfNormal”, sigma=5, dims=(“geo”,)),
“saturation_beta”: Prior(“HalfNormal”, sigma=prior_sigma, dims=(“channel”,)),
“gamma_control”: Prior(“Normal”, mu=0, sigma=0.05, dims=(“control”,)),
“gamma_fourier”: Prior(“Laplace”, mu=0, b=0.2, dims=(“geo”, “fourier_mode”)),
“likelihood”: Prior(“TruncatedNormal”, sigma=Prior(“HalfNormal”, sigma=6), dims=(“date”, “geo”)),
}

my_sampler_config = {“progressbar”: True}

mmm = MMM(
date_column="date",
target_column="y",
channel_columns=channels,
control_columns=controls,
scaling={
“channel”: {“method”: “max”, “dims”: ()},
“target”: {“method”: “max”, “dims”: ()},
},
adstock=GeometricAdstock(l_max=10),
saturation=LogisticSaturation(),
yearly_seasonality=2,
model_config=my_model_config,
sampler_config=my_sampler_config,
dims=("geo",),
)
2

An important difference is that the multidimensional MMM defaults to pooled modeling where no explicit dimensions are specified, whereas the single dimensional MMM defaults to separate model.

Looking at your model_config, you might want to add in priors with dims=”channel” for adstock_alpha, and saturation_lam. This dhould unify the model to what you had with the single dimensional class.

3

Hello,

Thanks for writing, I tried adding dims argument in Prior for adstock_alpha and saturation_lam in my_model_config and it looks like the following

my_model_config = {

    "intercept": Prior("HalfNormal", sigma=5),

"saturation_beta": Prior("HalfNormal", sigma=prior_sigma, dims=("channel")),

"saturation_lam": Prior("Gamma", alpha=3, beta=1, dims=("channel")),

"adstock_alpha": Prior("Beta", alpha=1, beta=3, dims=("channel")),

"gamma_control": Prior("Normal", mu=0, sigma=0.05, dims=("control")),

"gamma_fourier": Prior("Laplace", mu=0, b=0.2, dims=("geo","fourier_mode")),

"likelihood": Prior("TruncatedNormal", sigma=Prior("HalfNormal", sigma=6), dims=("date")),

}

I could see improvement in low spend channels, but for big channels ROAS are high and not matching with the regular mmm. Multi-d MMM gives the following

{'Channel-a': 9.21,
'Channel-b': 0.22,
'Channel-c': 0.76,
'Channel-d': 1.11,
'Channel-e': 0.86,
'Channel-f': 1.74,
'Channel-g': 1.07,
'Channel-h': 2.33,
'Channel-i': 0.185}

and ROAS from regular MMM are as follows

{'Channel-a': 2.13
'Channel-b': 0.82,
'Channel-c': 0.56,
'Channel-d': 1.71,
'Channel-e': 0.89,
'Channel-f': 1.23,
'Channel-g': 0.15,
'Channel-h': 1.89,
'Channel-i': 0.38}

In my case, ‘Channel-a’ is the one with highest spend share of 50% and multi-d MMM is really off in predicting its ROAS (with a huge credible interval). I also tried adding the ‘geo’ argument in dims but it didn’t help either. To add more context, prior_sigma which in my case is spend share is the following

array([0.49728545, 0.08402681, 0.10286331, 0.0533288 , 0.02376035,
       0.06204508, 0.15416075, 0.01652881, 0.00600064])