pm.Potential without Theano?

Questions
1

When using potentials, Is it possible to define them in a way that does not “rely” on theano?

More specifically - I have a custom logp function which is already implemented and optimized for my usage. Following this thread, I’ve tried to do something like this:

def likelihood(params, data):
    return my_custom_logp(params,data)

with pm.Model() as m:    
    params = pm.Normal('params', mu=1.5, sd=0.3,shape=(3,))
    like = pm.Potential('like', likelihood(params, data))
    trace = pm.sample()

This raises the following exception:

TypeError: Cannot convert FreeRV to numpy.ndarray
Exception ignored in: 'custom_func.createFunction'
TypeError: Cannot convert FreeRV to numpy.ndarray
Segmentation fault (core dumped)

Which kills the python kernel.

Is this doable?

2

Maybe the easiest thing to try is wrapping the likelihood into a theanoOps: http://deeplearning.net/software/theano/extending/extending_theano.html#as-op-example

3

Thanks for the quick response!

I’ve tried this, and I think I’m making some progress, but I’m not familiar enough with theano to makes this work. I dod:

import theano
import numpy as np
from theano import function
from theano.compile.ops import as_op
import my_func as me
import pymc3 as pm

@as_op(itypes=[theano.tensor.dvector, theano.tensor.dmatrix],
       otypes=[theano.tensor.dscalar])
def likelihood(params, data):
    return me.model.getLogProbability(data.astype('uint32')).sum()

x = theano.tensor.dvector()
y = theano.tensor.dmatrix()
f = function([x, y], likelihood(x, y))
theano.config.compute_test_value = 'ignore'

with pm.Model() as m:	
	params = pm.Normal('params', mu=1.5, sd=0.3,shape=(5,),dtype='float32')
	like = pm.Potential('like', f(params, data))    
	trace = pm.sample()

The error I’m getting is:

Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?

As far as I understand, the call f(params,data) is expecting params to be a vector (“numeric array”) but instead gets a theano Variable. Is there some way around this? I’ve tried casting but it didn’t work…

4

try casting data into theano tensor: theano.shared(data)

5

I tried - it still raises the same error, and specifcally mentions the problem is with params:

Traceback (most recent call last):
  File "mcmc.py", line 57, in <module>
    like = pm.Potential('like', f(params,theano.shared(data)))    
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 813, in __call__
    allow_downcast=s.allow_downcast)
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/tensor/type.py", line 87, in filter
    'Expected an array-like object, but found a Variable: '
TypeError: Bad input argument with name "params" to theano function with name "mcmc.py:51" at index 0 (0-based).  
Backtrace when that variable is created:

  File "mcmc.py", line 49, in <module>
    x = theano.tensor.dvector()
Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?
6

How about:

like = pm.Potential('like', likelihood(params, theano.shared(data)))
7

Thanks! Some more progress (I think), still not there yet. :slight_smile:

Now the sampling process initializes, but then I get a different, looks-very-low-levelish exception:

(TL;DR - JoblibValueError and TransportableException)

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Initializing NUTS failed. Falling back to elementwise auto-assignment.
Multiprocess sampling (4 chains in 4 jobs)
Slice: [params]
  0%|          | 0/1000 [00:00<?, ?it/s]

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
ValueError: expected an ndarray

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/miniconda3/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 350, in __call__
    return self.func(*args, **kwargs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/joblib/parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/user/miniconda3/lib/python3.6/site-packages/joblib/parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 526, in _sample
    for it, strace in enumerate(sampling):
  File "/home/user/miniconda3/lib/python3.6/site-packages/tqdm/_tqdm.py", line 962, in __iter__
    for obj in iterable:
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 630, in _iter_sample
    point = step.step(point)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py", line 129, in step
    apoint = self.astep(bij.map(point), *inputs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/slicer.py", line 56, in astep
    y = logp(q) - nr.standard_exponential()
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/blocking.py", line 257, in __call__
    return self.fa(self.fb(x))
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/model.py", line 1029, in __call__
    return self.f(**state)
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/user/miniconda3/lib/python3.6/site-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
ValueError: expected an ndarray
Apply node that caused the error: MakeVector{dtype='float64'}(like)
Toposort index: 1
Inputs types: [TensorType(float64, scalar)]
Inputs shapes: [()]
Inputs strides: [()]
Inputs values: [-373.5]
Inputs type_num: [12]
Outputs clients: [[Sum{acc_dtype=float64}(MakeVector{dtype='float64'}.0)]]

....

___________________________________________________________________________
"""

The above exception was the direct cause of the following exception:

TransportableException                    Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/joblib/parallel.py in retrieve(self)
    698                 if getattr(self._backend, 'supports_timeout', False):
--> 699                     self._output.extend(job.get(timeout=self.timeout))
    700                 else:

~/miniconda3/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    643         else:
--> 644             raise self._value
    645 

TransportableException: TransportableException
___________________________________________________________________________
ValueError                                         Sun Mar 18 16:09:41 2018
PID: 8972                    Python 3.6.3: /home/user/miniconda3/bin/python
...........................................................................
/home/user/miniconda3/lib/python3.6/site-packages/joblib/parallel.py in __call__(self=<joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function _sample>, (0, True, 507879474, {'params': array([ 1.5,  1.5,  1.5])}), {'draws': 1000, 'live_plot': False, 'live_plot_kwargs': None, 'model': <pymc3.model.Model object>, 'step': <pymc3.step_methods.slicer.Slice object>, 'trace': None, 'tune': 500})]
    132 
    133     def __len__(self):
    134         return self._size

...


During handling of the above exception, another exception occurred:

JoblibValueError                          Traceback (most recent call last)
<ipython-input-2-3d46709a462a> in <module>()
     31     params = pm.Normal('params', mu=1.5, sd=0.3,shape=(n,))
     32     like = pm.Potential('like', likelihood(params,theano.shared(data)))
---> 33     trace = pm.sample()

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, njobs, tune, nuts_kwargs, step_kwargs, progressbar, model, random_seed, live_plot, discard_tuned_samples, live_plot_kwargs, compute_convergence_checks, **kwargs)
    417         _print_step_hierarchy(step)
    418         try:
--> 419             trace = _mp_sample(**sample_args)
    420         except pickle.PickleError:
    421             _log.warn("Could not pickle model, sampling singlethreaded.")

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in _mp_sample(**kwargs)
    947     jobs = (delayed(_sample)(*args, **kwargs)
    948             for args in zip(chain_nums, pbars, rseed, start))
--> 949     traces = Parallel(n_jobs=njobs)(jobs)
    950     return MultiTrace(traces)
    951 

~/miniconda3/lib/python3.6/site-packages/joblib/parallel.py in __call__(self, iterable)
    787                 # consumption.
    788                 self._iterating = False
--> 789             self.retrieve()
    790             # Make sure that we get a last message telling us we are done
    791             elapsed_time = time.time() - self._start_time

~/miniconda3/lib/python3.6/site-packages/joblib/parallel.py in retrieve(self)
    738                     exception = exception_type(report)
    739 
--> 740                     raise exception
    741 
    742     def __call__(self, iterable):

JoblibValueError: JoblibValueError
___________________________________________________________________________
Multiprocessing exception:
...........................................................................
/home/user/miniconda3/lib/python3.6/runpy.py in _run_module_as_main(mod_name='ipykernel_launcher', alter_argv=1)
    188         sys.exit(msg)
    189     main_globals = sys.modules["__main__"].__dict__
    190     if alter_argv:
    191         sys.argv[0] = mod_spec.origin
    192     return _run_code(code, main_globals, None,
--> 193                      "__main__", mod_spec)
        mod_spec = ModuleSpec(name='ipykernel_launcher', loader=<_f...b/python3.6/site-packages/ipykernel_launcher.py')
    194 
    195 def run_module(mod_name, init_globals=None,
    196                run_name=None, alter_sys=False):
    197     """Execute a module's code without importing it

...........................................................................
/home/user/miniconda3/lib/python3.6/runpy.py in _run_code(code=<code object <module> at 0x7f81ebd054b0, file "/...3.6/site-packages/ipykernel_launcher.py", line 5>, run_globals={'__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__cached__': '/home/user/miniconda3/lib/python3.6/site-packages/__pycache__/ipykernel_launcher.cpython-36.pyc', '__doc__': 'Entry point for launching an IPython kernel.\n\nTh...orts until\nafter removing the cwd from sys.path.\n', '__file__': '/home/user/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py', '__loader__': <_frozen_importlib_external.SourceFileLoader object>, '__name__': '__main__', '__package__': '', '__spec__': ModuleSpec(name='ipykernel_launcher', loader=<_f...b/python3.6/site-packages/ipykernel_launcher.py'), 'app': <module 'ipykernel.kernelapp' from '/home/user/m.../python3.6/site-packages/ipykernel/kernelapp.py'>, ...}, init_globals=None, mod_name='__main__', mod_spec=ModuleSpec(name='ipykernel_launcher', loader=<_f...b/python3.6/site-packages/ipykernel_launcher.py'), pkg_name='', script_name=None)
     80                        __cached__ = cached,
     81                        __doc__ = None,
     82                        __loader__ = loader,
     83                        __package__ = pkg_name,
     84                        __spec__ = mod_spec)
---> 85     exec(code, run_globals)
        code = <code object <module> at 0x7f81ebd054b0, file "/...3.6/site-packages/ipykernel_launcher.py", line 5>
        run_globals = {'__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__cached__': '/home/user/miniconda3/lib/python3.6/site-packages/__pycache__/ipykernel_launcher.cpython-36.pyc', '__doc__': 'Entry point for launching an IPython kernel.\n\nTh...orts until\nafter removing the cwd from sys.path.\n', '__file__': '/home/user/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py', '__loader__': <_frozen_importlib_external.SourceFileLoader object>, '__name__': '__main__', '__package__': '', '__spec__': ModuleSpec(name='ipykernel_launcher', loader=<_f...b/python3.6/site-packages/ipykernel_launcher.py'), 'app': <module 'ipykernel.kernelapp' from '/home/user/m.../python3.6/site-packages/ipykernel/kernelapp.py'>, ...}
     86     return run_globals
     87 
     88 def _run_module_code(code, init_globals=None,
     89                     mod_name=None, mod_spec=None,

Setting njobs=1 gives a slightly different exception:

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Initializing NUTS failed. Falling back to elementwise auto-assignment.
Sequential sampling (2 chains in 1 job)
Slice: [params]
  0%|          | 0/1000 [00:00<?, ?it/s]

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py in __call__(self, *args, **kwargs)
    902             outputs =\
--> 903                 self.fn() if output_subset is None else\
    904                 self.fn(output_subset=output_subset)

ValueError: expected an ndarray

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-1-0be912d9bdd4> in <module>()
     31     params = pm.Normal('params', mu=1.5, sd=0.3,shape=(n,))
     32     like = pm.Potential('like', likelihood(params,theano.shared(data)))
---> 33     trace = pm.sample(njobs=1)

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, njobs, tune, nuts_kwargs, step_kwargs, progressbar, model, random_seed, live_plot, discard_tuned_samples, live_plot_kwargs, compute_convergence_checks, **kwargs)
    437             _log.info('Sequential sampling ({} chains in 1 job)'.format(chains))
    438             _print_step_hierarchy(step)
--> 439             trace = _sample_many(**sample_args)
    440 
    441     discard = tune if discard_tuned_samples else 0

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in _sample_many(draws, chain, chains, start, random_seed, step, **kwargs)
    480     for i in range(chains):
    481         trace = _sample(draws=draws, chain=chain + i, start=start[i],
--> 482                         step=step, random_seed=random_seed[i], **kwargs)
    483         if trace is None:
    484             if len(traces) == 0:

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in _sample(chain, progressbar, random_seed, start, draws, step, trace, tune, model, live_plot, live_plot_kwargs, **kwargs)
    524     try:
    525         strace = None
--> 526         for it, strace in enumerate(sampling):
    527             if live_plot:
    528                 if live_plot_kwargs is None:

~/miniconda3/lib/python3.6/site-packages/tqdm/_tqdm.py in __iter__(self)
    960 """, fp_write=getattr(self.fp, 'write', sys.stderr.write))
    961 
--> 962             for obj in iterable:
    963                 yield obj
    964                 # Update and possibly print the progressbar.

~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in _iter_sample(draws, step, start, trace, chain, tune, model, random_seed)
    628                     strace.record(point)
    629             else:
--> 630                 point = step.step(point)
    631                 strace.record(point)
    632             yield strace

~/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py in step(self, point)
    127             return bij.rmap(apoint), stats
    128         else:
--> 129             apoint = self.astep(bij.map(point), *inputs)
    130             return bij.rmap(apoint)
    131 

~/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/slicer.py in astep(self, q0, logp)
     54         for i in range(len(q0)):
     55             # uniformly sample from 0 to p(q), but in log space
---> 56             y = logp(q) - nr.standard_exponential()
     57             ql[i] = q[i] - nr.uniform(0, self.w[i])
     58             qr[i] = q[i] + self.w[i]

~/miniconda3/lib/python3.6/site-packages/pymc3/blocking.py in __call__(self, x)
    255 
    256     def __call__(self, x):
--> 257         return self.fa(self.fb(x))

~/miniconda3/lib/python3.6/site-packages/pymc3/model.py in __call__(self, state)
   1027 
   1028     def __call__(self, state):
-> 1029         return self.f(**state)
   1030 
   1031 

~/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py in __call__(self, *args, **kwargs)
    915                     node=self.fn.nodes[self.fn.position_of_error],
    916                     thunk=thunk,
--> 917                     storage_map=getattr(self.fn, 'storage_map', None))
    918             else:
    919                 # old-style linkers raise their own exceptions

~/miniconda3/lib/python3.6/site-packages/theano/gof/link.py in raise_with_op(node, thunk, exc_info, storage_map)
    323         # extra long error message in that case.
    324         pass
--> 325     reraise(exc_type, exc_value, exc_trace)
    326 
    327 

~/miniconda3/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    690                 value = tp()
    691             if value.__traceback__ is not tb:
--> 692                 raise value.with_traceback(tb)
    693             raise value
    694         finally:

~/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py in __call__(self, *args, **kwargs)
    901         try:
    902             outputs =\
--> 903                 self.fn() if output_subset is None else\
    904                 self.fn(output_subset=output_subset)
    905         except Exception:

ValueError: expected an ndarray
Apply node that caused the error: MakeVector{dtype='float64'}(like)
Toposort index: 1
Inputs types: [TensorType(float64, scalar)]
Inputs shapes: [()]
Inputs strides: [()]
Inputs values: [-348.0]
Inputs type_num: [12]
Outputs clients: [[Sum{acc_dtype=float64}(MakeVector{dtype='float64'}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-0be912d9bdd4>", line 33, in <module>
    trace = pm.sample(njobs=1)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 381, in sample
    step = assign_step_methods(model, step, step_kwargs=step_kwargs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 149, in assign_step_methods
    return instantiate_steppers(model, steps, selected_steps, step_kwargs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 70, in instantiate_steppers
    step = step_class(vars=vars, **args)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/slicer.py", line 47, in __init__
    super(Slice, self).__init__(vars, [self.model.fastlogp], **kwargs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/model.py", line 190, in fastlogp
    return self.model.fastfn(self.logpt)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/model.py", line 666, in logpt
    logp_potentials = tt.sum([tt.sum(pot) for pot in self.potentials])

Debugprint of the apply node: 
MakeVector{dtype='float64'} [id A] <TensorType(float64, vector)> ''   
 |FromFunctionOp{likelihood} [id B] <TensorType(float64, scalar)> 'like'   

Storage map footprint:
 - <TensorType(float64, matrix)>, Shared Input, Shape: (500, 3), ElemSize: 8 Byte(s), TotalSize: 12000 Byte(s)
 - params, Input, Shape: (3,), ElemSize: 8 Byte(s), TotalSize: 24 Byte(s)
 - like, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - TensorConstant{(1,) of 0...0068542243}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{(1,) of -1..1111111111}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{(1,) of -1.5}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{0.5}, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - Sum{acc_dtype=float64}.0, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 TotalSize: 12072.0 Byte(s) 0.000 GB
 TotalSize inputs: 12056.0 Byte(s) 0.000 GB

Seems like the problem is with the type of the function return value?

8

Could you please try

    trace = pm.sample(step = pm.Metropolis())

Also check the output of

m.logp(m.test_point)
9

Still looks like a type mismatch somewhere (I think):

Sequential sampling (2 chains in 1 job)
Metropolis: [params]
  0%|                                                                                                                                                                              | 0/1000 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
ValueError: expected an ndarray

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "mcmc.py", line 58, in <module>
    trace = pm.sample(step=pm.Metropolis(),njobs=1)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 439, in sample
    trace = _sample_many(**sample_args)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 482, in _sample_many
    step=step, random_seed=random_seed[i], **kwargs)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 526, in _sample
    for it, strace in enumerate(sampling):
  File "/home/user/miniconda3/lib/python3.6/site-packages/tqdm/_tqdm.py", line 962, in __iter__
    for obj in iterable:
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py", line 624, in _iter_sample
    point, states = step.step(point)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py", line 162, in step
    apoint, stats = self.astep(self.bij.map(point))
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/metropolis.py", line 162, in astep
    accept = self.delta_logp(q, q0)
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/user/miniconda3/lib/python3.6/site-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
  File "/home/user/miniconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
ValueError: expected an ndarray
Apply node that caused the error: MakeVector{dtype='float64'}(like)
Toposort index: 11
Inputs types: [TensorType(float64, scalar)]
Inputs shapes: [()]
Inputs strides: [()]
Inputs values: [-309.92989822093455]
Inputs type_num: [12]
Outputs clients: [[Sum{acc_dtype=float64}(MakeVector{dtype='float64'}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "mcmc.py", line 58, in <module>
    trace = pm.sample(step=pm.Metropolis(),njobs=1)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/step_methods/metropolis.py", line 136, in __init__
    self.delta_logp = delta_logp(model.logpt, vars, shared)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pymc3/model.py", line 666, in logpt
    logp_potentials = tt.sum([tt.sum(pot) for pot in self.potentials])

Debugprint of the apply node: 
MakeVector{dtype='float64'} [id A] <TensorType(float64, vector)> ''   
 |FromFunctionOp{likelihood} [id B] <TensorType(float64, scalar)> 'like'   

Storage map footprint:
 - <TensorType(float64, matrix)>, Shared Input, Shape: (500, 3), ElemSize: 8 Byte(s), TotalSize: 12000 Byte(s)
 - inarray1, Input, Shape: (3,), ElemSize: 8 Byte(s), TotalSize: 24 Byte(s)
 - inarray, Input, Shape: (3,), ElemSize: 8 Byte(s), TotalSize: 24 Byte(s)
 - Constant{0}, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - Constant{3}, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - TensorConstant{(1,) of 0...0068542243}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{(1,) of -1..1111111111}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{(1,) of -1.5}, Shape: (1,), ElemSize: 8 Byte(s), TotalSize: 8 Byte(s)
 - TensorConstant{0.5}, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - like, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 - Sum{acc_dtype=float64}.0, Shape: (), ElemSize: 8 Byte(s), TotalSize: 8.0 Byte(s)
 TotalSize: 12112.0 Byte(s) 0.000 GB
 TotalSize inputs: 12096.0 Byte(s) 0.000 GB

Regarding m.logp(m.test_point) - if I comment out pm.sample (which throws the exception) and add m.logp(m.test_point) instead, I get the same error. Is this the usage you intended?

10

Yes that’s what I meant - it is a way to check if the model is setup correctly

in this case, it means the evaluation of logp is still not correct. How about setting the input to theano tensor ivector?

@as_op(itypes=[theano.tensor.dvector, theano.tensor.ivector],
       otypes=[theano.tensor.dscalar])
def likelihood(params, data):
    return me.model.getLogProbability(data).sum()
11

But the observed data in this case is a matrix, not a vector… why should this work? Or am I missing something?

EDIT:
When it raises ValueError: expected an ndarray - do you have an idea which object was expected to be of this type?

12

In your liklihood function, seems that it’s not evaluated on params? Could you please try rewriting the function to actually evaluated on params and data?

13

This is just because I wrote a shorter version of the likelihood to make everything shorter and more readable, sorry about that. The full function is:

@as_op(itypes=[theano.tensor.dvector, theano.tensor.ivector],
       otypes=[theano.tensor.dscalar])
def likelihood(params, data):
    model = me.Model(params)
    return model.getLogProbability(data).sum()

So it’s probably not there… I tried the ivector version but it didn’t work.

The me.Model function uses an external C code - is there any chance the as_op decorator is having trouble because of this?

14

I am not completely sure - but the problem is either itypes or otypes. How about changing the otypes to theano.tensor.dvector? Otherwise, since data is already set maybe remove that from the input so that the input to likelihood is likelihood(params)

15

Perfect! Now it’s working (also with njobs > 1).`

Solution:

  • Changed otypes to theano.tensor.dvector.
  • Changed return model.getLogProbability(data).sum() to return np.array(model.getLogProbability(data).sum()).reshape((1,))

This returns an array instead of a number, which (I assume) theano handles better for some reason. Somehow related to this post?

Thanks for all your help! Now to make sure the whole thing actually converges. :slight_smile: `

1 Like