"Regularising" Measurements

Questions
1

Hi Folks,

Problem

I’ve been trying to solve a problem where I have measurements that I want to regularise.

I have some prior knowledge that

  1. The mean of the measurements is should be around \mu_0
  2. The measurements should be “close” to each other i.e. have low variance.

Potential Solution

The solution I have is to model the data as being iid with a Gaussian likelihood and parameterised by a mean \mu and precision, \tau = 1/\sigma^2

\begin{align} x_i | \mu, \tau {} & \overset{iid}{\sim} \mathcal{N}(\mu, \tau) \\ \mu {} & \sim \mathcal{N}(\mu_0, n_0\tau) \\ \tau {} & \sim \textsf{Ga}(\alpha, \beta) \end{align}

Starting with parameters for \mu_0, n_0, \alpha, \beta, the posterior mean and precision can be calculated as

\begin{align} \mu | \tau, x \sim {} & \mathcal{N}\left( \frac{n}{n+n_0}\bar{x} + \frac{n_0}{n+n_0}\mu_0, (n +n_0)\tau \right) \\ \tau | x \sim {} & \textsf{Ga}\left( \alpha + \frac{n}{2}, \beta + \frac{1}{2}\sum_i (x_i - \bar{x})^2 + \frac{nn_0}{2(n + n_0)} (\bar{x} - \mu_0)^2 \right) \end{align}

where \bar{x} is the sample mean of the data. [1]

By setting \mu_0 to be equal to the prior knowledge of the mean and n_0, \alpha, \beta arbirtrarily so that the variance of posterior is low, I, can compute the the posterior mean and precision.

My solution is then to adjust the data such that it matches the posterior mean and precision.

  1. Step 1: Find the z scores with the mean and precision calculated from the data:

    • z^{\text{old}}_i = \sqrt{\tau_x}(x - \bar{x})

  2. Step 2: Convert the z scores back but with the posterior mean and precision

    • x^{\text{new}}_i = \frac{1}{\sqrt{\tau}}z^{\text{old}}_i + \mu

Is this valid?

Is there a better way to do this?

References

[1] The Conjugate Prior for the Normal Distribution, Stat260 UC Berkeley