# Advice for speeding up sampling for models with multiple evaluations of Cholesky factors

**URL:** <https://discourse.pymc.io/t/advice-for-speeding-up-sampling-for-models-with-multiple-evaluations-of-cholesky-factors/4041>\
**Category:** Questions\
**Created:** [October 29, 2019, 7:33pm UTC](https://discourse.pymc.io/t/advice-for-speeding-up-sampling-for-models-with-multiple-evaluations-of-cholesky-factors/4041 "2019-10-29T19:33:56Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mind-The-Data](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/mind-the-data/32/2241_2.png) [@Mind-The-Data](https://discourse.pymc.io/u/Mind-The-Data)\
**Post date:** [October 29, 2019, 7:33pm UTC](https://discourse.pymc.io/t/advice-for-speeding-up-sampling-for-models-with-multiple-evaluations-of-cholesky-factors/4041/1 "2019-10-29T19:33:56Z")

</div>

I am trying to estimate the posterior for hyperparameters of a covariance function. Here is a Kruschke diagram of my model.

 ![GP_param](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/b/bcb578e69fd017d1997f9d684aa0d505f95d7501.png)

I am looking for speedups preferably using NUTS as it seems to sample the hyperparameters better. Although, Metropolis is ~100x faster for a given sample (although, not for a given effective sample!). I am using a non centered parameterization. But am using the generic inference button. Any hints, tips or tricks to speed this up? Here is my model code:

> Blockquote  
> with pm.Model() as GP\_uncentered:  
> # uncentered version:  
> #to sample from MvN(0,K) multiply L z where LL^T = K  
> # and z = N(0,1) the scale parameter of in this case the normal dist

```
# ============== kernel ===============
sigma_sq = 1 # pm.Gamma('sigma_sq', 5,10) # sets maximum covariance ij
lsq = 1e4 #pm.Gamma('lsq', alpha = 5, beta=7e-5)
sigma_n = .01
Kij = simga_sq*(np.exp(-Dmatsq/lsq)+np.diag([sigma_n]*N))

# ========== kernel informing prior slope covariance ========
z = pm.Normal('z', 0., 1., shape=Num_farms) # scale parameter
b = pm.Deterministic('w', tt.dot(tt.slinalg.cholesky(Kij), z)) # prior slope

a = pm.Normal('a', 0, .3, shape=Num_farms) # prior intercep

# ========== Linear Model =============== 
u = pm.Deterministic('mu',a[zero_farm_idx] + b[idx]*X 
                    

#============ Likelihood ==============
sigma = pm.Gamma('sigma',5,10)

y = pm.Normal('y', mu=u, sd=sigma, observed=Y) 

# ============ sample =================
#trace = pm.sample(500,tune=500,chains=2)

```

Matrix inversions scale O(n3). So this is likely a significant portion of the time spent sampling. However, the MH algorithm is ~100x faster. Is this evidence the gradient calculations necessary for HMC are slowing down NUTS by about a 100x factor? Any advice?

-I am not getting divergent transition warnings.  
-N\_eff is fairly high for hyperparms with NUTS (about half-a third of samples).

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 31, 2019, 2:38pm UTC](https://discourse.pymc.io/t/advice-for-speeding-up-sampling-for-models-with-multiple-evaluations-of-cholesky-factors/4041/2 "2019-10-31T14:38:36Z")

</div>

Maybe try the pm.GP with approximation? From a quick look these is nothing obvious to speed up 😅