# Theano shared variable error

**URL:** <https://discourse.pymc.io/t/theano-shared-variable-error/4918>\
**Category:** Questions\
**Tags:** theano\
**Created:** [April 22, 2020, 6:04pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918 "2020-04-22T18:04:47Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 22, 2020, 6:04pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/1 "2020-04-22T18:04:48Z")

</div>

Hi  
I have a problem that has been totally eating me up all day. I believe there is a simple solution to the question…

If I specify a theano shared variable for use in a pymc3 model so that i can loop without re-specifying the model, it seems the initial value of the shared variable effects the result. If the initial value is too far from the new value, I get a “The derivative of RV `test1`.ravel()[0] is zero.” error. Why is this??

See my code below. changing the nf value to 10 or change the new value to 100000 would fix the error…

def test():

```
nf = 10000000000000000000
sharednf = shared(nf)

with pm.Model() as model:
    test = pm.Normal("test"+str(1), mu=sharednf, sd=1**100)
    sum = test + 1
    pm.Deterministic("sum"+str(1), sum)
 
for i in range(10):
    sharednf.set_value(float(1))
    with model:
        trace = pm.sample(5000)
        trace1 = trace
        print(pm.summary(trace))
        pm.traceplot(trace)
        plt.show()
```

---

<div class="post-metadata">

**Author:** ![OriolAbril](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/oriolabril/32/2497_2.png) [@OriolAbril](https://discourse.pymc.io/u/OriolAbril)\
**Post date:** [April 22, 2020, 7:28pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/2 "2020-04-22T19:28:42Z")

</div>

It looks like [this GitHub issue](https://github.com/pymc-devs/pymc3/issues/3884). I don’t know the reason behind it, but it looks like modifying the priors or using `init="adapt_diag"` have solved the issue in some similar situations.

Another option would be to stop using GitHub `master` and install 3.8 version instead if you do not need any recently added feature.

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 22, 2020, 9:14pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/3 "2020-04-22T21:14:43Z")

</div>

Thank you for getting back to me! These modifications did sadly not make a difference… I am already using pymc3 3.8. ☹ other suggestions?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 23, 2020, 8:38am UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/4 "2020-04-23T08:38:12Z")

</div>

PyMC3 assigns a default value to each Random Variable, for Normal distribution it is the mean. So in this case the underlying default value is sharednf, when it is super huge and the new `set_value` is too far away from it, the gradient is essentially 0.  
There are a few way to fix it:  
1, providing starting value to `pm.sample`:

```python
pm.sample(..., start={"test"+str(1): float(1)})

```

2, non-center parameterization

```python
with pm.Model() as model:
    test = pm.Normal("test"+str(1), mu=0., sd=1**100)
    sum = test + 1 + sharednf
    pm.Deterministic("sum"+str(1), sum)

```

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 23, 2020, 12:00pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/5 "2020-04-23T12:00:55Z")

</div>

Hi! thank you so much for getting back to me! I see the root cause of the problem now. However, setting start value = new shared variable value is not doing the trick. Am i misunderstanding something?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 23, 2020, 12:39pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/6 "2020-04-23T12:39:16Z")

</div>

Maybe there is error cause by caching? Try clearing out your theano cache…

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 23, 2020, 1:42pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/7 "2020-04-23T13:42:53Z")

</div>

i tried that without success. this is what i did:  
trace = pm.sample(5000, start={“start”+str(1):float(sharednf.get\_value())})

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 23, 2020, 2:07pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/8 "2020-04-23T14:07:02Z")

</div>

if the new and old values of the variable are far apart, the gradient is zero. ok. But if the variables are little enough apart for the sampler to converge, the resulting mean is still dependent on how far the variables were apart. Why is there a correlation here? I am really not able to use the theano shared variables the way i am hoping to ☹

Using the second suggested solution by non-center parametrize the model, I have no way of introducing new shared varibles for use in the sd? I was hoping to loop through a dataset with means and sds by the help of theano shared.

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 23, 2020, 4:50pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/9 "2020-04-23T16:50:28Z")

</div>

> [@BjornHartmann](#):
>
> trace = pm.sample(5000, start={“start”+str(1):float(sharednf.get\_value())})

`start` needs to be a dictionary-like object that each str name correspondent to the random variable name, ie `pm.sample(..., start={"test1": float(1)})`

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 23, 2020, 4:54pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/10 "2020-04-23T16:54:46Z")

</div>

> [@BjornHartmann](#):
>
> Why is there a correlation here?

Likely not enough burning/tuning, increase the tuning should help

> [@BjornHartmann](#):
>
> Using the second suggested solution by non-center parametrize the model, I have no way of introducing new shared varibles for use in the sd? I was hoping to loop through a dataset with means and sds by the help of theano shared.

location-scale distribution are all be able to reparameterized like `rv_you_want = unit_rv*sd + mu`. A bit more information could be find in: [https://docs.scipy.org/doc/scipy/reference/tutorial/stats.html](https://docs.scipy.org/doc/scipy/reference/tutorial/stats.html)

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 23, 2020, 5:52pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/11 "2020-04-23T17:52:23Z")

</div>

awesome! thank you so much for getting back to me!!

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 24, 2020, 7:01pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/12 "2020-04-24T19:01:26Z")

</div>

I have a quick followup question. Can one use theano shared variables to change a distribution?

From what I have understood, theano shared can only be used with np arrays.  
Is there a way to do something like:  
distList = [pm.Normal, pm.Lognormal,…]  
sharedVar = theano.shared(0)  
with pm.Model as model:  
distList[sharedVar](“name”, mu=0, sd=1)

```
for i in range(10):
    shareVar = i
    with model:
        pm.sample()

```

This would allow the change of what distribution is used inside the “loop”

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 26, 2020, 5:33am UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/13 "2020-04-26T05:33:51Z")

</div>

I dont think so - we cannot index a list with theano variables

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [April 26, 2020, 3:50pm UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/14 "2020-04-26T15:50:05Z")

</div>

ok. thank you. I have now implemented the method of non center parametrization as suggested. I am just curious, Why is it so that doing  
…  
pm.Normal(“test”, mu=1e-10, sd=1e-20)  
…  
gives a “derivative is zero” error for the sampling, while shifting a the distribution by using the below works? As far as I can tell, the resulting distributions are equal.

```
....
test = pm.Normal("test", mu=0, sd=1)
shift = test*1e-20 + 1e-10
shifted = pm.Deterministic("shifted", shift)
....
```

---

<div class="post-metadata">

**Author:** ![AlexAndorra](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/alexandorra/32/9142_2.png) [@AlexAndorra](https://discourse.pymc.io/u/AlexAndorra)\
**Post date:** [April 27, 2020, 8:12am UTC](https://discourse.pymc.io/t/theano-shared-variable-error/4918/15 "2020-04-27T08:12:00Z")

</div>

Yeah, the reparametrization trick feels like 🌟 magic 🌟  
Basically, it’s because it presents HMC with a much better geometry to sample from. Here is [a blog post](https://twiecki.io/blog/2017/02/08/bayesian-hierchical-non-centered/) where it’s explained in details.