Problem with hyperbolic discount model

Questions
1

Dear Pymc experts,

As a beginner of pymc3, I am learning how to use pymc3 to fit simple quantitative psychology/cognitive science models . Firstly, I want to try a simple hyperbolic discount model for intertemporal choice tas, I followed the procedure in this great post:
Fitting a simple Reinforcement Learning model to behavioral data with PyMC3 (Jupyter NB).
However, there always is an error, as a layman, I don’t understand what is going wrong, can somebody help me fix this stupid and naive problem? Thanks.
Here is my code:

import numpy as np
import pandas as pd
import pymc3 as pm
import seaborn as sns
import theano
from theano import tensor as t
# generate some fake data
def hyperbolic_rng(k,beta,ll_a,ll_t,ss_a,ss_t):
    dsv = ll_a / (1+k*ll_t) - ss_a / (1+k*ss_t)
    p = 1/(1+np.exp(-beta*dsv))
    size = p.size
    choice = np.random.binomial(1,p,size)
    return choice
ss_amount = np.random.normal(20,4,60)
ll_amount = ss_amount + 10
ss_time = np.random.binomial(1,0.5,60)*4
ll_time = np.concatenate([ss_time[0:30] +4,ss_time[30:] + 8]) 
fake_data = hyperbolic_rng(0.1,0.9,ll_amount,ll_time,ss_amount,ss_time)
def sv_diff(k,ll_amount,ll_delay,ss_amount,ss_delay):
    k = pm.math.exp(k)
    return ll_amount/(1+k*ll_delay)-ss_amount/(1+k*ss_delay)
def hyperbolic_llf(k,beta,ll_a,ll_t,ss_a,ss_t):
    #theano log likelihood function
    ll_amount_ = theano.shared(ll_amount)
    ss_amount_ = theano.shared(ss_amount)
    ll_delay_ = theano.shared(ll_time)
    ss_delay_ = theano.shared(ss_time)
    k = t.scalar('k')
    beta = t.scalar('beta')
    sv = sv_diff(k,ll_amount_,ll_delay_,ss_amount_,ss_delay_)
    sv_ = -sv * beta
    llf = pm.math.log(1/(1+pm.math.exp(sv_))) 
    neg_llf = -t.sum(llf)
    return neg_llf
with pm.Model() as m:
    k = pm.Normal('k', 0, 4)
    beta = pm.HalfNormal('beta', 10)
    like = pm.Potential('like', hyperbolic_llf(k, beta, ll_amount,ll_time,ss_amount,ss_time))
    tr = pm.sample()

And the error is:

TestValueError                            Traceback (most recent call last)
~\anaconda3\lib\site-packages\theano\graph\op.py in compute_test_value(node)
     80         try:
---> 81             storage_map[ins] = [ins.get_test_value()]
     82             compute_map[ins] = [True]

~\anaconda3\lib\site-packages\theano\graph\basic.py in get_test_value(self)
    421             detailed_err_msg = get_variable_trace_string(self)
--> 422             raise TestValueError(f"{self} has no test value {detailed_err_msg}")
    423 

TestValueError: k has no test value  
Backtrace when that variable is created:

  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2877, in run_cell
    result = self._run_cell(
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2923, in _run_cell
    return runner(coro)
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3146, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3338, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-6-0bf33b6fde4a>", line 5, in <module>
    like = pm.Potential('like', hyperbolic_llf(k, beta, ll_amount,ll_time,ss_amount,ss_time))
  File "<ipython-input-5-c3ad5e2e4ef5>", line 7, in hyperbolic_llf
    k = t.scalar('k')


During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-6-0bf33b6fde4a> in <module>
      3     beta = pm.HalfNormal('beta', 10)
      4 
----> 5     like = pm.Potential('like', hyperbolic_llf(k, beta, ll_amount,ll_time,ss_amount,ss_time))
      6     tr = pm.sample()

<ipython-input-5-c3ad5e2e4ef5> in hyperbolic_llf(k, beta, ll_a, ll_t, ss_a, ss_t)
      7     k = t.scalar('k')
      8     beta = t.scalar('beta')
----> 9     sv = sv_diff(k,ll_amount_,ll_delay_,ss_amount_,ss_delay_)
     10     sv_ = -sv * beta
     11     llf = pm.math.log(1/(1+pm.math.exp(sv_)))

<ipython-input-4-55b4b528b9e8> in sv_diff(k, ll_amount, ll_delay, ss_amount, ss_delay)
      1 def sv_diff(k,ll_amount,ll_delay,ss_amount,ss_delay):
----> 2     k = pm.math.exp(k)
      3     return ll_amount/(1+k*ll_delay)-ss_amount/(1+k*ss_delay)

~\anaconda3\lib\site-packages\theano\graph\op.py in __call__(self, *inputs, **kwargs)
    251 
    252         if config.compute_test_value != "off":
--> 253             compute_test_value(node)
    254 
    255         if self.default_output is not None:

~\anaconda3\lib\site-packages\theano\graph\op.py in compute_test_value(node)
     92                 detailed_err_msg = get_variable_trace_string(ins)
     93 
---> 94                 raise ValueError(
     95                     f"Cannot compute test value: input {i} ({ins}) of Op {node} missing default value. {detailed_err_msg}"
     96                 )

ValueError: Cannot compute test value: input 0 (k) of Op Elemwise{exp,no_inplace}(k) missing default value.  
Backtrace when that variable is created:

  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2877, in run_cell
    result = self._run_cell(
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2923, in _run_cell
    return runner(coro)
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3146, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3338, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "C:\Users\KLIPS\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-6-0bf33b6fde4a>", line 5, in <module>
    like = pm.Potential('like', hyperbolic_llf(k, beta, ll_amount,ll_time,ss_amount,ss_time))
  File "<ipython-input-5-c3ad5e2e4ef5>", line 7, in hyperbolic_llf
    k = t.scalar('k')

Thanks a lot.

2

You are overwriting k and beta inside the likelihood function.

1 Like
3

Sorry, what is overwriting k and beta? How should I correct it
? Thanks.

4

I believe these lines:

k = t.scalar('k')
beta = t.scalar('beta')

(re-)initialize the arguments k and beta, overwriting what is being passed into hyperbolic_llf().

2 Likes
5

This approach is perhaps more complex than it needs to be. See this thread for another example which has implemented discounting models Replacing nested for loops from JAGS code for cognitive model of temporal discounting - #6 by drbenvincent

1 Like
6

Yeah, thank you. Also, I am reading your last year published paper’s code(I forgot the name, that JEP:General paper) to learn pymc3. I think they are well organized, and thank you for making this paper’s code online!

1 Like
7

If I get some time over the break, I should really put it all together as a pymc-example.

2 Likes