Simple imputation difficulties

Questions
1

I’ve been trying to learn the basics of modeling distributions, inferring paramaterization from observed data, and imputation but I seem to be at a standstill and cannot resolve the error I am getting. These are the steps that I’ve taken so far:

  1. Created a representative distribution for something I am interested in, an inverse gamma with alpha=1.75, beta=.45, and mu=0 (no issues)

  2. Draw random samples from representative distribution (no issues)

  3. With fully observed data (nothing missing) allow alpha, beta, and mu to vary as follows. Yes I realize the priors and test values are overspecialized but this was just a conceptual test to make sure I had the code correct (no issues here either)
    alpha_m=pm.Normal(‘alpha_m’,1.75,.25,testval=1.75)
    beta_m=pm.Normal(‘beta_m’,.45,.25,testval=.45)
    mu_m=pm.Normal(‘mu_m’,0,.25,testval=0)
    x = pm.InverseGamma(‘x’,alpha=alpha_m,beta=beta_m,mu=mu_m,observed=data_full)

    image
    image872×574 81.4 KB

  4. At this point compared the data generated using bayesian inference (blue), with data drawn from the specified distribution from step 1 (orange), and the true data (green). Results are pretty solid for my purposes so I was happy to move forward. (no issues)
    image

  5. Now I tried removing a few values from the observed data and forcing imputation and I can’t get it to work for the life of me, even referencing all the material I could find. (issues start here)

Here I create the mask, where example.csv is the same as data_full above but with random values removed and only a single column.
data = pd.read_csv(“example.csv”)
data=data[‘example’]
data = data.fillna(-999)
data=pd.Series(data)
masked_values = np.ma.masked_array(data, mask=data==-999)

Then the only change I made from the above was
x = pm.InverseGamma(‘x’,alpha=alpha_m,beta=beta_m,mu=mu_m,observed=masked_values)

and I get the following error:

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{‘alpha_m’: array(1.95737899), ‘beta_m’: array(0.27115254), ‘mu_m’: array(0.60721553), ‘x_missing’: array([-2.56459914e-01, 1.03271340e+00, 5.37146175e-01, 6.33337610e-02,
4.46804232e-04, 7.07904288e-01, 6.51092972e-01, -2.85362739e-01,
-1.34877999e-01, -2.40307384e-02, 2.20646390e-03, 7.50916427e-01,
1.07499807e-02, 1.03219510e+00, -5.43253045e-01, 1.25069329e-01,
1.13071463e+00, -3.68609644e-01, -5.00859329e-01, 6.44641718e-01,
3.02196631e-01, 7.10830251e-01, -6.50745530e-01, -2.75524564e-01,
8.97536559e-01, 3.10161264e-01, -6.28564407e-01, 8.13093317e-01,
3.40975576e-01, -4.06067127e-01, 8.94295936e-01, 1.14401623e+00,
4.63153067e-01, 2.83155347e-01, 7.67496900e-01, -1.77194550e-01,
-4.92041228e-01, -1.92180471e-01, 1.08655159e+00, 1.13854780e+00,
6.50842041e-01, -7.76757997e-01, -8.28662086e-01, -4.40533628e-02,
5.00030806e-01, -5.11285545e-01, 5.22047637e-01, 3.07830888e-01,
4.36935613e-01, 2.49150643e-01, 5.91741801e-01, 5.18811499e-01,
4.48478431e-01, -6.37065703e-01, -6.59090284e-02, 2.51518293e-02,
-4.79413363e-01, 9.24105286e-01, -2.77504141e-01, 5.10672431e-01,
-2.43128670e-01, -8.28511637e-01, 3.46825354e-01, -6.61335616e-01,
1.12870138e+00, 1.04598754e+00, 9.57362172e-02, -7.03424979e-01,
4.21512006e-01, 1.15254613e-01, 1.04043248e+00, 1.02420792e+00,
6.15783208e-01, -8.15735917e-02, -5.36452271e-01, -3.44379983e-01,
1.10631342e+00, 4.89865570e-01, -1.11638787e-01, -3.98604578e-01,
-7.33014532e-01, 1.05443790e+00, -8.94335887e-02, -1.16933764e-01,
7.12966183e-01, 1.06018731e-01, 2.99466567e-01, -3.96176767e-01,
3.14421904e-01, -3.39604280e-01, 2.02820039e-02, -1.82236380e-02,
-5.16778523e-01, 1.23122683e-01, 7.57843743e-01, 5.65357486e-01,
-4.70341508e-01, -3.68338967e-02, 4.80370397e-01, 4.66817047e-02,
-1.45233374e-01, 1.06635217e+00, -7.32898172e-01, 9.34686376e-01,
-1.81057892e-01, 7.68732761e-01, 1.06358696e+00, 7.48028224e-01,
-6.04678999e-01, -6.94510056e-01, -6.85895238e-01, 7.78919793e-01,
5.18228775e-01, -2.52505997e-01, 8.74225444e-01, 2.01164461e-01,
-1.77455776e-01, 8.03973335e-01, 5.09187160e-01, -3.97334265e-01,
3.33646107e-01, -7.22424482e-01, -7.90567392e-01, 9.46931608e-01,
4.42225158e-01, -3.39355668e-02, -6.42603310e-01, -1.36781130e-01,
9.03648090e-01, -4.48264407e-01, 2.31177891e-02, 3.12000522e-01,
8.09640712e-01, 9.23472504e-01, -5.76922686e-01, 1.44668022e-01,
-3.33292233e-01, -5.62655982e-01, 2.00679035e-01, -6.84508844e-01,
2.72634609e-01, -1.60958413e-01, -7.47801868e-01, 7.21855177e-01,
6.59993731e-01, 1.79267220e-01, 7.60192257e-01, -1.26609691e-01,
-5.75451698e-01, -5.74451747e-01, 1.21968783e-01, 9.29336393e-01,
5.03099105e-01, -3.29075561e-01, 7.97619337e-02, 5.36553556e-01,
8.51725841e-02, 9.60629071e-01, -5.70581360e-01, 3.08534673e-01,
-4.62936397e-01, 2.68766416e-01, 1.06902441e+00, 9.56821482e-01,
-6.03160081e-01, -4.39749345e-01, -2.62619842e-01, -2.02623735e-01,
-6.30836979e-01, -3.10702161e-01, 4.92058788e-02, 2.18492934e-01,
-3.91710090e-01, -8.35793585e-01, -8.14986854e-01, -6.50303893e-01,
-6.20502588e-01, 8.29080002e-01, 7.98622156e-01, -7.39731899e-01,
2.04722846e-01, 3.37888112e-01, 1.05744134e+00, -4.68054718e-01,
5.68051216e-01, 4.54996500e-01, 9.57258777e-01, -4.38313418e-01,
6.17406583e-01, 5.68564437e-01, 2.64769545e-01, -4.06590773e-01,
-5.15943320e-01, 1.10693556e+00, 6.27224075e-01, 4.47963652e-01,
-5.28520017e-01, -7.85044718e-01, 6.04798724e-01, 1.10570548e+00,
6.14846291e-01, 6.62096367e-02, -4.98149813e-02, 3.55018806e-02,
7.93869515e-01, 7.19357083e-01, -2.28674395e-01, 1.76282367e-01,
5.14703033e-01, -9.95450711e-02, 4.56675623e-01, 3.06323933e-01,
-3.65915616e-02, 5.95568017e-01, -7.81243362e-01, -3.89103160e-01,
2.58007089e-01, 3.48775831e-01, 3.97266998e-01, 1.65673192e-01,
1.02494894e+00, -6.17231334e-01, -2.44706365e-01, -3.46887241e-01,
7.33965565e-01, 1.67138792e-01, -5.03121804e-01, 2.69277465e-01,
-7.87492330e-01, 1.71290377e-01, 9.88027591e-01, -5.81938694e-01,
-9.39554299e-02, 9.32132207e-01, 1.09399593e-01, 1.14537264e+00,
1.00376178e+00, -2.20333325e-01, -5.87701700e-01, -1.81933910e-01,
-6.14112184e-01, -2.47261447e-01, 9.06232447e-01, 1.82538397e-01,
-2.60925249e-01, 7.07406162e-01, 4.93799703e-01, 6.67394024e-01,
-5.66590283e-01, 4.76422669e-01, 8.04897680e-01, -6.36313360e-01,
1.23005148e-01, -2.22585079e-01, 6.01666352e-01, -5.36924972e-01,
3.90864390e-01, -1.37414951e-01, -2.12561188e-01, -6.84518709e-01,
-4.20595565e-01, -3.21448387e-01, 2.88550399e-01, 1.29410848e-02,
3.29857690e-01, 1.18617903e-01, -1.36511924e-01, -5.21363922e-01,
-5.40567637e-01, -5.93197521e-01, -8.06814167e-01, -1.05633848e-02,
-3.34406171e-01, -7.23689193e-01, -2.27606679e-01, -7.79320965e-01,
-1.61230978e-01, 4.39346368e-01, 9.12884036e-01, 9.33810058e-01,
-3.43104986e-01, -7.00654124e-01, -8.43643034e-01, 6.95241843e-01,
9.80458972e-02, 1.19264558e-01, -8.19958625e-01, 6.46437371e-01,
3.82216022e-01, -6.44374357e-01, -4.57326281e-02, -2.29383889e-01,
2.35227038e-01, 6.61437206e-01, -7.71843095e-01, 5.48744140e-01,
5.67675707e-01, 5.05607610e-01, 3.44427337e-01, 9.37343000e-01,
-3.17446819e-01, 3.75114840e-01, -4.30117218e-01, 6.73423259e-01,
-2.46037740e-02, 3.84708395e-01, 8.14323098e-01, -4.94280202e-01,
4.70559557e-01, 2.67137973e-01, 2.37196691e-01, -8.49772610e-02,
-1.22207183e-01, 1.58913862e-01, 6.56943440e-01, 5.38479902e-01,
-2.75215537e-01, 9.87847408e-01, -2.49354850e-01, 1.08353641e+00,
-4.71361023e-01, 6.20380648e-01, -5.41340586e-01, 1.13275671e+00,
1.82028064e-01, 2.52682690e-01, -3.53110368e-01, 8.39909678e-01,
-7.84978737e-01, 5.59785346e-01, -7.40121611e-01, 3.08248993e-01,
1.87618184e-02, 1.11850411e+00, -8.73014920e-02, -5.82830843e-01,
9.11510602e-01, 9.73281391e-02, 3.72651078e-01, 7.81425783e-01,
-4.88189527e-02, -3.22987762e-01, -4.62371172e-01, -3.62229191e-01,
1.03583483e+00, -2.71650850e-01, -4.72472089e-01, 7.97923271e-01,
5.50564639e-03, -4.31219761e-01, -6.74398512e-01, 2.60820662e-01,
6.20602454e-01, -8.27120695e-01, -6.00090523e-01, -3.03066200e-01,
5.43842528e-01, 9.59047435e-01, 7.79362827e-01, -4.01242697e-01,
-4.06707478e-02, -4.08998035e-02, 9.43912674e-01, 7.50789024e-01,
6.82774224e-01, -1.03838442e-01, -7.57533232e-01, 4.74712047e-01,
9.38673882e-02, 5.34081834e-01, 8.37113177e-01, 3.52958510e-01,
6.80246936e-01, 4.48863848e-01, 7.50892742e-01, 6.74895853e-01,
-7.65295370e-01, 1.00340138e+00, 7.67204609e-01, -1.14722971e-01,
-1.72789770e-01, 1.11663739e+00, -5.79648896e-01, -1.37486789e-01,
2.74624430e-01, -6.63923934e-01, -2.01149537e-01, -2.14804408e-01,
-3.20468840e-01, 5.26730410e-01, 2.40151985e-01, 5.88595485e-01,
8.11224950e-01, -6.11320605e-01, 4.08912494e-01, -7.12412141e-01,
-6.24680746e-01, -3.28105081e-01, 5.32133850e-01, 1.13200833e+00,
-6.50560180e-01, -2.14474109e-01, 1.51234663e-01, -4.28403109e-01,
8.79545472e-01, -5.42864357e-01, 2.70665550e-01, 7.00253476e-02,
-5.93296633e-01, 3.67868374e-01, 4.36895109e-01, 3.94929488e-01,
1.04475518e+00, 9.98391793e-01, 1.65014620e-01, -2.98689733e-02,
-2.30775183e-03, 2.48217341e-01, 9.37975197e-01, 1.06603388e+00,
2.93165306e-01, 8.18188549e-01, 9.19764897e-01, -5.14422125e-01,
8.98353554e-01, 5.11028592e-01, -1.69008863e-01, 8.97246872e-01,
8.08287348e-01, -1.87466630e-01, -5.28613765e-01, 1.12218661e+00,
-5.93021772e-01, 7.30250084e-01, 6.42074017e-01, -8.18192782e-01,
1.00239839e-01, 9.88923614e-01, -3.10001428e-01, 2.46454078e-01,
-4.13664637e-01, 8.21737326e-01, 1.06732958e+00, -6.19809008e-01,
3.53594331e-03, -1.64752842e-01, -5.38201016e-02, 1.10427621e+00,
-5.08927501e-01, 9.06846751e-02, 1.10560940e+00, -6.27900826e-01,
-3.17044883e-01, 9.85148605e-01, -2.30829778e-01, 4.21157757e-01,
-6.90133476e-02, -2.70168780e-01, -1.85635989e-01, 2.17639278e-01,
7.84081354e-02, -5.87984231e-03, 9.33497519e-01, 4.62538833e-01,
-5.57575529e-01, -4.51798820e-01, 1.01453264e+00, -4.17870963e-02,
9.85197477e-01, 3.08248866e-01, 5.72786230e-01, -4.43709635e-02,
-3.70075639e-01, 3.75132811e-01, -5.02186752e-01, 9.19913382e-02,
-1.32161157e-01, -4.73387619e-01, -7.87756766e-01, -7.74884320e-01,
-5.41263607e-02, 6.69663919e-02, 2.45469236e-01, -7.70099204e-01,
-4.71313464e-01, 4.55770493e-01, 1.14733435e+00, 7.36029562e-01,
-2.55022817e-02, -5.75726453e-01, -3.74514860e-02, 5.21692261e-01,
1.10592715e-01, 2.42727755e-01, -5.45106181e-01, 6.06658854e-01,
6.58806665e-01, 5.58714397e-01, 5.96672250e-01, 8.50808845e-01,
7.97021088e-01, -7.50697127e-01, 1.10269369e+00, -4.25210155e-02,
-6.67030463e-01, 9.57110275e-01, -2.01779713e-01, -2.62756404e-01,
5.27046270e-01, -6.19545849e-01, 4.89554181e-02, 8.76140811e-01,
-4.03444120e-02, -1.70470182e-02, -3.54574817e-01, -4.70762552e-01,
-1.49992544e-01, -2.14569050e-02, 6.71152226e-01, 3.81883432e-01,
3.08547858e-01, -3.12834488e-01, 8.69520247e-01, -4.50501333e-01,
-6.20451557e-01, 8.30503073e-02, -1.00635101e-01, -5.92101385e-01,
3.00197241e-01, 7.15049902e-01, -7.87283265e-01, 8.19160203e-01,
-6.58372182e-01, -6.63878919e-01, -7.63949206e-01, 1.02158354e+00,
2.35490974e-01, 7.53303449e-01, 5.73751893e-02, -4.10189077e-01,
1.00169817e+00, -7.72634182e-01, -4.51386468e-01, -9.62293507e-02,
-5.51378941e-01, -3.13544891e-01, 1.94586358e-01, -1.83047701e-01,
2.58916833e-01, 8.91864692e-02, -8.24070557e-01, 4.54045752e-01,
7.18356621e-01, -4.78902086e-02, 1.13658476e+00, -1.77413793e-02,
2.10789613e-01, 4.78393325e-01, -3.18737505e-01, -2.07510422e-01,
-4.53585643e-01, -6.38697507e-01, 4.51824438e-01, -5.98307774e-01,
4.36445138e-01, -2.45530876e-01, -4.58214026e-02, 1.69865539e-01,
1.49425398e-01, -9.31978947e-02, -6.98981619e-01, 2.48866918e-01,
-3.12783311e-01, 1.77934579e-01, 4.42273158e-03, 9.66036821e-01,
1.09900959e+00, -8.04267025e-01, -7.76647389e-01, 4.92112303e-01,
6.81821242e-01, -9.65296174e-03, 1.00013977e+00, 5.35181654e-01,
-3.48940770e-01, -1.88848063e-01, 4.96932810e-01, 6.02560222e-01,
6.61135665e-01, 6.41645712e-01, -8.03792741e-01, -6.19240925e-01,
-8.04987929e-01, 9.87950205e-01, 1.03375027e-01, -7.72768898e-01,
7.32514156e-01, 5.72388994e-01, -2.77453224e-01, 1.83476831e-01,
-7.78523101e-01, -1.31645041e-01, 3.13369843e-01, 9.38654277e-01,
-6.88153959e-01, 6.52375333e-01, 3.90078896e-01, 8.22146111e-01,
-7.75139117e-02, 3.38795647e-01, 6.41132059e-01, 6.07171657e-01,
9.98381148e-01, 4.75946690e-01, 4.14240264e-01, -5.25588929e-01,
-4.21840210e-01, -6.12347194e-01, -5.32664103e-01, 3.91295493e-01,
-6.94383108e-01, 1.82223591e-01, 2.01448456e-02, 9.68092064e-01,
1.10139241e+00, 1.01755575e+00, 6.38607329e-01, -6.35417164e-01,
7.69340781e-01, -7.43816296e-01, 2.12096725e-01, -7.02895152e-01,
4.85680095e-01, -1.45453538e-01, -5.66534232e-01, 3.48062015e-01,
1.12640775e+00, 1.37068984e-01, -7.47301120e-01, -1.07739537e-01,
3.35242418e-01, 1.09768724e+00, -8.40599753e-01, -2.05698376e-01,
-6.77909926e-01, -6.83483875e-01, -5.47916754e-01, 8.73645431e-01,
-1.83509084e-01, 8.92757998e-01, -4.41769484e-01, 1.08047224e+00,
-3.78252672e-01, -5.78227714e-01, 9.30599804e-01, -3.80077488e-01,
-8.56229593e-03, -2.75377295e-01, 3.51853098e-01, 1.07728202e+00,
1.07754331e+00, 8.30488570e-01, 6.72477257e-01, -5.82664854e-01,
5.03939486e-01, -5.07036398e-01, -1.67606208e-01, -2.13355461e-02,
-6.09921357e-01, 7.55487175e-02, 1.06811976e+00, -6.61867398e-01,
-6.08405289e-01, -3.30709299e-01, 6.36844562e-01, 8.98779528e-01,
-1.33711938e-01, -8.85466119e-02, -2.62464991e-02, 4.04080231e-01,
8.09564914e-01, -2.32567126e-01, 6.77700762e-01, -1.52613848e-01,
-4.63054062e-01, 2.56083231e-01, -7.82265914e-01, -2.10998911e-01,
-6.56034429e-01, 2.83645111e-01, -1.72606892e-01, 7.34287403e-01,
9.17772914e-01, -1.70149060e-01, 6.84285666e-01, 8.47158198e-01,
1.01290900e-01, 2.78184807e-01, 6.35388286e-01, 9.15795397e-01,
4.89190447e-01, 5.37625283e-01, -3.10941311e-01, 9.91059098e-01,
8.71523115e-01, -5.62491304e-01, 6.49748645e-01, 6.37519146e-01,
-6.53754981e-01, 5.38771024e-01, -7.26868130e-01, -5.71947151e-01,
7.37708371e-01, -3.23205469e-01, 9.06148238e-01, 4.06172599e-01,
-2.62262339e-01, -4.30141460e-01, 3.69134046e-01, 1.01818540e+00,
9.05391118e-01, -1.11615178e-01, 9.28056554e-01, -8.46007092e-01,
-3.37947942e-01, -1.71351448e-01, 7.41940500e-01, -3.04153110e-01,
-1.22483252e-01, 2.02714325e-01, 7.79633487e-01, 9.76160015e-01,
3.27097684e-01, 2.63360039e-01, 5.04703993e-01, -4.81579544e-01,
1.00136585e+00, 7.65422574e-02, 9.72817061e-01, 4.33211420e-01,
1.29102903e-01, 9.93491163e-01, -3.08160945e-01, -7.61277976e-01,
5.56883847e-01, 8.54171594e-01, -8.49569254e-01, -2.60009899e-01,
2.18376581e-01, 5.34141761e-01, 3.33024143e-01, 1.07866875e-01,
-3.36490343e-01, -7.78025534e-01, 8.17143111e-01, -7.16239001e-01,
1.10727750e+00, -6.36596951e-01, 4.57176171e-01, 5.51260069e-01,
9.23478680e-01, -5.99099063e-01, 7.85822426e-01, -6.01352589e-01,
-4.34501430e-01, -6.80988438e-01, 5.31697405e-01, 1.13775995e+00,
1.09182554e+00, 7.47558366e-01, -5.22622386e-01, -8.24418730e-01,
7.93111084e-01, -4.27691406e-01, -1.67025146e-01, 1.08155407e+00,
5.08177198e-01, 6.97228464e-01, -6.81310135e-01, 3.55355528e-02,
8.28703974e-01, 6.65960982e-01, 1.04133172e+00, 3.37361944e-01,
3.07931240e-02, 1.25039315e-01, -3.91048293e-01, 6.01323634e-02,
8.20092529e-01, -5.87103234e-01, 6.53732287e-01, 2.87718323e-01,
-1.15875614e-01, -7.03673091e-01, -5.17124986e-02, 3.34023071e-01,
9.91155540e-01, 4.91869498e-01, -2.28771751e-01, 9.52154112e-01,
5.30609167e-01, 9.09957899e-01, 1.00627609e+00, 4.95259105e-01,
5.09559006e-01, 4.52183653e-01, -6.00446641e-01, 3.91769995e-01,
6.14983140e-01, -4.51842636e-01, -7.09525126e-01, 9.33811749e-01,
1.13277140e+00, -8.19891916e-01, -6.55163980e-02, 1.08409684e+00,
1.74897030e-01, 1.85505586e-02, -7.97932191e-01, -3.60606347e-02,
-6.56632307e-01, 7.32553965e-01, 5.44024126e-01, 8.31116693e-01,
3.81948905e-01, 1.48960446e-01, -5.27252527e-01, 5.58746133e-01,
5.59189967e-01, 1.07528296e+00, 7.06351985e-01, 2.76152869e-01,
-2.03706167e-01, 6.41606526e-01, 4.26715879e-01, 8.81461171e-01,
-7.61175857e-02, 3.90744554e-01, -8.52655544e-02, -7.57701621e-01,
9.65326087e-01, 4.97840028e-01, -6.80530370e-01, 7.44948092e-01,
1.96565216e-01, -4.89055553e-01, 1.07175890e+00, -3.82622356e-01,
-2.02043620e-01, -3.75539461e-01, -5.38168360e-01, 3.91490779e-01,
-2.21179566e-03, 5.48862332e-01, -7.81219172e-01, -2.82922852e-01,
3.05117686e-01, 4.09763506e-02, 3.99409014e-02, -5.85405833e-01,
1.11035452e-01, -3.53369297e-02, -5.86839660e-01, -1.91243051e-02,
-7.44125060e-02, 8.17138432e-01, -1.64659903e-01, 4.38983320e-01,
-5.50249794e-01, -8.25428749e-01, -5.52450217e-02, 2.01280030e-01,
-3.60247957e-01, 9.72125318e-01, 7.34631139e-02, 6.31586719e-01,
-1.07213971e-01, -1.47839741e-01, 1.08105336e+00, -3.04130151e-01,
7.02198116e-01, -7.69998198e-01, 1.03635335e+00, 6.91679327e-01,
2.73107110e-01, 3.75496802e-01, 2.87140469e-01, -3.28785584e-01,
-2.53450623e-01, 1.10732281e-01, -7.97984215e-01, 1.13228421e+00,
3.42160606e-01, -4.03459508e-01, -7.17337852e-01, 5.83681873e-01,
-1.04698137e-01, 1.12520041e+00, -2.49704592e-01, -2.45091485e-01,
-4.65941228e-01, -1.16521503e-01, -8.02513033e-01, 1.17121608e-01,
-6.33920932e-01, 1.07216390e+00, -7.91070027e-02, -3.45338244e-01,
8.03606252e-01, 8.53971327e-01, 1.04461080e+00, -6.31679477e-01,
-4.84830415e-01, 1.06729652e+00, 4.31727588e-01, 8.98712623e-01,
-6.45629877e-01, -3.14269743e-01, 1.02464028e+00, -2.73710348e-01,
-5.95485519e-01, 6.00847501e-01, -6.12442024e-02, -4.72349080e-01,
-7.80746430e-01, -5.58552566e-01, 1.27313960e-01, 4.12808791e-01,
5.55861612e-01, -5.13657990e-01, -3.96937845e-01, 7.17477667e-01,
5.01302806e-01, 4.61853645e-01, 6.20444328e-01, -7.82934792e-01,
1.12935868e+00, 7.17585645e-02, 8.19209436e-01, -7.81397064e-02,
5.54123162e-01, -3.76452144e-01, 3.58072595e-01, -7.56953125e-01,
6.89424182e-01, -7.22657968e-01, 6.63910581e-01, 6.12898038e-01,
-2.15668775e-01, 1.96890845e-01, 7.21777725e-01, 9.42057524e-01,
-5.43890435e-01, -6.14606610e-01, 8.58383168e-01, -1.75387939e-01,
5.11258504e-01, -2.57554944e-01, 2.41800843e-01, -7.95447185e-01,
-5.52088484e-01, 9.64690200e-01, -1.01828110e-01, -5.81692506e-02,
-1.84184640e-01, 5.95523049e-01, 1.90804280e-01, 1.78102151e-01,
1.01482546e+00, 1.77709694e-02, 7.60093204e-01, 3.30146033e-01,
2.68024445e-01, 2.20051120e-01, 6.94798387e-01, 6.80583409e-01,
-4.19974063e-01, -5.87924827e-01, 2.90762268e-01, 1.50444170e-01,
9.88100829e-01, 8.89763136e-01, 1.03792686e+00, 1.06137138e+00,
5.99424794e-01, 5.43644498e-01, 6.05431880e-01, -2.15749487e-01,
-7.80601612e-01, -8.16608617e-01, -3.00618623e-01, -5.80638036e-01,
8.18752259e-01, 5.03614742e-01, -1.23036115e-01, -8.25765694e-01,
2.26381114e-01, -7.23113749e-01, -3.71539725e-03, -4.74133583e-01])}

Initial evaluation results:
alpha_m 0.22
beta_m 0.21
mu_m -2.48
x_missing 0.00
x -inf
Name: Log-probability of test_point, dtype: float64

  1. I tried evaluating some test points because of the strange log probabilities listed in the error. For example the error states the the test point for mu_m is evaluated to -2.48, but when I simply test:

z = pm.Normal.dist(mu=0,sd=.25)
z.logp(0).eval()

I get that it should evaluate to array(0.46735583) so something seems completely wrong.

Any help would be greatly appreciated!

2

The problem with the automatic imputed variables is that they are not transformed to the unconstrained domain as conventional unobserved variables are (in your case your variable is lower bounded at zero). This is something that we fixed in the codebase but will only show up in the next major release.

You wouldn’t have issues with variables that are unconstrained by default such as the Normal.

In the meantime you can split the observed and non observed in two separate distributions:

obs_x = pm.InverseGamma(‘obs_x’,alpha=alpha_m,beta=beta_m,mu=mu_m,observed=observed)
unobs_x = pm.InverseGamma(‘unobs_x’,alpha=alpha_m,beta=beta_m,mu=mu_m, shape=n_unobs)

Which should give you the right results.

Edit: You also seem to have serious sampling issues in your fully observed model, gauging from your plots, so you will need to fix those as well.

2 Likes
3

Ricardo, thank you for the input. To test this out on a smaller data set I started using the data I generated in step 1 above. Generating 20 observations with 5 missing and the following code I get these results:

with pm.Model():
    
    init = np.random.randn(ma.count_masked(test_masked)).astype('float32')
    #init = np.abs(np.random.randn(14990,1))
    
    alpha_m=pm.HalfNormal('alpha_m',1,testval=.25)
    beta_m=pm.HalfNormal('beta_m',1,testval=.25)
    mu_m=pm.Normal('mu_m',0,1,testval=0)
    x = pm.InverseGamma('x',alpha=alpha_m,beta=beta_m,mu=mu_m,observed=test)
    #x_missing = pm.InverseGamma('x_missing',alpha=alpha_m,beta=beta_m,mu=mu_m,shape=init)
    
    approx=pm.sample(2000,tune=2000,progressbar=True,cores=1,chains=4,target_accept=.9999)

image
image874×579 145 KB

As you can see I left out piece trying to specify x_missing explicitly. However, when I attempt the method you suggested I get an error telling me that I can’t specify it since it’s being generated automatically.

with pm.Model():
    
    init = np.random.randn(ma.count_masked(test_masked)).astype('float32')
    #init = np.abs(np.random.randn(14990,1))
    
    alpha_m=pm.HalfNormal('alpha_m',1,testval=.25)
    beta_m=pm.HalfNormal('beta_m',1,testval=.25)
    mu_m=pm.Normal('mu_m',0,1,testval=0)
    x = pm.InverseGamma('x',alpha=alpha_m,beta=beta_m,mu=mu_m,observed=test)
    x_missing = pm.InverseGamma('x_missing',alpha=alpha_m,beta=beta_m,mu=mu_m,shape=init)
    
    approx=pm.sample(2000,tune=2000,progressbar=True,cores=1,chains=4,target_accept=.9999)

x_missing---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-150-f4bc58ab6bfd> in <module>
      8     mu_m=pm.Normal('mu_m',0,1,testval=0)
      9     x = pm.InverseGamma('x',alpha=alpha_m,beta=beta_m,mu=mu_m,observed=test)
---> 10      = pm.InverseGamma('x_missing',alpha=alpha_m,beta=beta_m,mu=mu_m,shape=init)
     11 
     12     approx=pm.sample(2000,tune=2000,progressbar=True,cores=1,chains=4,target_accept=.9999)

~\anaconda3\envs\pymc3gpu2\lib\site-packages\pymc3\distributions\distribution.py in __new__(cls, name, *args, **kwargs)
    120         else:
    121             dist = cls.dist(*args, **kwargs)
--> 122         return model.Var(name, dist, data, total_size, dims=dims)
    123 
    124     def __getnewargs__(self):

~\anaconda3\envs\pymc3gpu2\lib\site-packages\pymc3\model.py in Var(self, name, dist, data, total_size, dims)
   1156                 )
   1157                 self.deterministics.append(var)
-> 1158                 self.add_random_variable(var, dims)
   1159                 return var
   1160         elif isinstance(data, dict):

~\anaconda3\envs\pymc3gpu2\lib\site-packages\pymc3\model.py in add_random_variable(self, var, dims)
   1194         """Add a random variable to the named variables of the model."""
   1195         if self.named_vars.tree_contains(var.name):
-> 1196             raise ValueError(f"Variable name {var.name} already exists.")
   1197 
   1198         if dims is not None:

ValueError: Variable name x_missing already exists.

So for the moment that issue seems resolved. However, there is something new. As I tried to increase the size of the test data set, both the number observed and the number missing, I got a new error. Right now it seems that if there is a small number of missing covariates then the method works as it should, shown above, but I get an error in my starting values when the numbers are large. The error below is truncated because there are 1000 x_missing values.

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{'alpha_m_log__': array(-1.14815613), 'beta_m_log__': array(-1.60186875), 'mu_m': array(-0.03255755), 'x_missing': array([-6.82604916e-01, -2.79261278e-01, -7.98570009e-01, -2.89974966e-01,
        2.47602847e-01, -3.44261805e-01, -1.44073574e-01, -6.88639315e-01,
        1.51019834e-01, -3.44758260e-01,  5.23698558e-01,  1.16905209e-01,
       -2.37173179e-01,  1.11750718e+00, -5.73059114e-01,  3.71448114e-01,
        1.16254121e+00, -2.11479592e-01,  3.10421730e-01,  1.16562478e+00,
Initial evaluation results:
alpha_m_log__   -1.42
beta_m_log__    -1.85
mu_m            -0.92
x_missing        0.00
x                -inf
Name: Log-probability of test_point, dtype: float64

Here I would expect an error to be thrown for x_missing because I incorrectly formulated it’s component’s and allowed a negative value to be passed somewhere where it shouldn’t be, resulting in a inf/-inf logp but it’s showing up in x itself. Explicitly passing the correct array shapes has no effect either. I saw mention in other threads that there could be some issue with incorrectly passing int/float32/float64 values to different pieces of the code, so I’m curious whether that could be a driver or if it’s something far more simple that I am just missing. Thank you again for the feedback!

4

Thought I’d share the solution. It turns it that the issue was actually just the use of jitter during initialization. After I removed it and forced only the use of adapt_diag I was able to get decent results for the full size of the data set, ~15000 records and 1000 missing. It seems jitter was pushing my initial values outside the feasible range even when specifying a test value.