Using PyMC for Thompson Sampling

Questions version agnostic
1

I’m exploring using PyMC for Thompson sampling to solve a multi-armed bandit problem.

At a high level, we want to personalize which day of week, either Friday or Saturday, we send a user a message. So each week, we’d like to create a distribution of a users likelihood to to engage on each day. We then draw a random value from each of these distributions and select the day with the highest sampled value to send the user the message.

For this I would use a formula like clicked ~ user + day_of_week + user * day_of_week, then generate a posterior of clicked for each user / day of week combination and sample from that.

However, the content of the message changes each week (but is the same for all users this week). I’d like to incorporate the content in the model as well to account for the fact that they may not have clicked because the content wasn’t appealing, essentially adding something like a content term to the formula.

I’m not sure how I would account for this when trying to get a posterior of clicked for each user / day of week combo. Any direction on how to accomplish this?

2

You would need to model the effects of the content somehow and infer them overtime. You could add predictors for content length / topics / word bag or some sort of PCA. Without knowing the type of “content” it’s hard to say more.

3

In this case, I’m assuming that day of week and content don’t have any interaction effect in order to keep things simple. So I’m not looking to get an accurate prediction of a user’s likelihood to click on the upcoming content, but rather, how likely they are to click on each day, regardless of the content. Does that make sense?

4

Does that mean you want to add something ~ week as a predictor?

5

I was thinking of it as an identifier of the content, say content_id, which I should have clarified, but you’re right, week as a predictor is a better way to think of it.