It was recently debated on Twitter, that updating prior (beside in most cases un-interesting conjugate models) under Bayesian framework is actually not trivial. Personally, approximating the posterior with some heavy tail distribution (say a t distribution) is what I would do. Otherwise, Bayesian filter (Kalman filter, particle filter) sounds like a promising framework that I would love to explore more.