# RandomWalk(Diffusion) Covariance Kernel for GPR

**URL:** <https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554>\
**Category:** Development\
**Created:** [July 12, 2019, 4:31pm UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554 "2019-07-12T16:31:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![wjgregory92](https://avatars.discourse-cdn.com/v4/letter/w/df788c/32.png) [@wjgregory92](https://discourse.pymc.io/u/wjgregory92)\
**Post date:** [July 12, 2019, 4:31pm UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554/1 "2019-07-12T16:31:17Z")

</div>

Would it be possible to implement a Random Walk Covariance Kernel (a.k.a Diffusion Kernel) in the pymc3.gp.cov module? I have inferred a network among my predictors (X) and would like to implement something similar to, e.g. Equation 1 in Urry and Sollich, 2013 for my covariance function ([Random Walk Kernels and Learning Curves for Gaussian Process Regression on Random Graphs](https://arxiv.org/pdf/1211.1328.pdf)).

This takes the form: Cov\_func = a x exp(M x l), where M is a NxN covariance matrix of X (txN), with diagonal elements equal to the negative sum of each respective row: M\_{ii} = -1\* \sum\_{j\neq i} M\_{ij}. a and l are then hyperparameters of the covariance function.

---

<div class="post-metadata">

**Author:** ![wjgregory92](https://avatars.discourse-cdn.com/v4/letter/w/df788c/32.png) [@wjgregory92](https://discourse.pymc.io/u/wjgregory92)\
**Post date:** [July 13, 2019, 11:37pm UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554/2 "2019-07-13T23:37:40Z")

</div>

I ended up adding my own function to my pymc3/gp/cov.py file for this, in case anybody is interested. You will have to supply your own network matrix M (adjacency matrix of weights). Something like:

```
M = np.cov(X, rowvar=False, bias=True)
M = M - np.eye(len(M))*M.diagonal()
M = M - np.nansum(M,axis=0)*np.eye(len(M))

```

and then assign it as a variable when you call the function, along with hyperparameters sigmaf (signal variance) and ls (characteristic length-scale).

`Σprior = pm.gp.cov.Diffusion(input_dim=X.shape[1],active_dims=np.arange(X.shape[1]),M=M,sigf=σf,ls=ℓ)`

For the pymc3/gp/cov.py file:

```
class Diffusion(Covariance):
    R"""
    Random Walk (Diffusion Kernel) based on linear covariance.
    W. Gregory
    """
    def __init__ (self, input_dim, M, sigf=None, ls=None, ls_inv=None, active_dims=None):
        super(). __init__ (input_dim, active_dims)
        self.ls = tt.as_tensor_variable(ls)
        self.sigf = tt.as_tensor_variable(sigf)
        self.M = tt.as_tensor_variable(M)
        
        
    def full(self, X, Xs=None):
        X, Xs = self._slice(X, Xs)
        if Xs is None:
            K = tt.dot(tt.dot(X,(self.sigf*tt.exp(self.M*self.ls))),tt.transpose(X)) #K(X,X) / K(X_*,X_*) - nxn(n*xn*)
        else:
            K = tt.dot(tt.dot(X,(self.sigf*tt.exp(self.M*self.ls))),tt.transpose(Xs)) #K(X,X_*) - nxn*
        
        K2 = tt.patternbroadcast(K, (False, False))
        return K2
```

---

<div class="post-metadata">

**Author:** ![wjgregory92](https://avatars.discourse-cdn.com/v4/letter/w/df788c/32.png) [@wjgregory92](https://discourse.pymc.io/u/wjgregory92)\
**Post date:** [September 11, 2019, 10:28am UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554/3 "2019-09-11T10:28:13Z")

</div>

There is a mistake above. The exponential should be a matrix exponential:

i.e. `tt.slinalg.expm(self.M*self.ls)`

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [September 11, 2019, 12:18pm UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554/4 "2019-09-11T12:18:05Z")

</div>

Thanks for follow up on your own post! Would you be interested to check this into PyMC3?

---

<div class="post-metadata">

**Author:** ![wjgregory92](https://avatars.discourse-cdn.com/v4/letter/w/df788c/32.png) [@wjgregory92](https://discourse.pymc.io/u/wjgregory92)\
**Post date:** [September 11, 2019, 2:35pm UTC](https://discourse.pymc.io/t/randomwalk-diffusion-covariance-kernel-for-gpr/3554/5 "2019-09-11T14:35:35Z")

</div>

Sure