Shape Error, unsure where the problem variable is. Please advise

Questions v5
,
1

Running the model below, I receive the error message below concerning incomparable element shapes. Any idea where I am going wrong? I have included the graphviz output as well

Incompatible Elemwise input shapes [(9000, 9000), (33, 33)]

engine_idx, engine = model_data['ENGINE_ID'].factorize(sort=True)
travel_time = model_data.travel_time.values
alarms_per_hour = model_data.alarms_per_hour.values
dist_from_engine = model_data.dist_from_engine.values
service_time = model_data.service_time.values
area_of_alarm_district_in_miles = model_data.area_of_alarm_district_in_miles.values


coords = {
    'cov_params': ['mu','sigma','xi'],
    'engine': engine,
    'obs_id': np.arange(model_data.shape[0]),
    'X': ['alarms_per_hour','service_time','area_of_alarm_district_in_miles']
}

X_gev = model_data[['alarms_per_hour','service_time','area_of_alarm_district_in_miles']].to_numpy()

with pm.Model(coords=coords) as gp_model:
    engines = pm.Data('engines', engine_idx, dims='obs_id') #W:factorized stations
    X = pm.Data('X_', X_gev, dims=('engine','X')) #W: lats and lons

    # gaussian process hyper parameters
    ℓ = pm.InverseGamma("ℓ", mu = 50.0, sigma = 50.0) #W: for all covariance matrices
    η = pm.Gamma("η", mu=0.15, sigma=0.10, dims = "cov_params") #W: for all covariance matrices 

    # gaussian process prior for mu
    gp_μ = pm.gp.Latent(cov_func=η[0]*pm.gp.cov.ExpQuad(3,ℓ))
    μ_group = pm.Normal("μ_group", mu=3.0, sigma=1.0) 
    μ = pm.Deterministic("μ", μ_group + gp_μ.prior("μ_gp", X=X), dims='engine') 

    # gaussian process prior for sigma
    gp_σ_log = pm.gp.Latent(cov_func=η[1]*pm.gp.cov.ExpQuad(3,ℓ))
    σ_log = gp_σ_log.prior("σ_log", X=X, dims='engine')
    σ_log_group = pm.Normal("σ_log_group", mu=-1.0, sigma=2.0)
    σ = pm.Deterministic("σ", pm.math.exp(σ_log_group + σ_log),dims='engine')
    
    # gaussian process prior for xi
    gp_ξ = pm.gp.Latent(cov_func=η[2]*pm.gp.cov.ExpQuad(3,ℓ))
    ξ_group =  pm.TruncatedNormal('ξ_group', mu=0.0, sigma=0.25, lower=-0.99, upper=0.99)
    ξ = pm.Deterministic("ξ", pm.math.tanh(ξ_group + gp_ξ.prior("ξ_gp", X=X)), dims='engine') #W: keep this func
    
    # likelihood for all observations
    gev = pmx.GenExtreme("gev", mu=μ[engines], sigma=σ[engines], xi=ξ[engines], observed=travel_time, dims='obs_id')
   
    # mcmc sampling
    #gp_trace = pm.sample(1000, target_accept = 0.98, random_seed=314)

pm.model_to_graphviz(gp_model)

graphviz
graphviz2407×1440 225 KB

2

What is the shape of X_gev?

3

@jessegrabowski

9000, 33

4

But you gave \mu and \xi shape “engine”? And you index them by engine later?

5

@jessegrabowski

Well the goal was to recover the parameters for each of the gev parameters separated by engine group. As the structure of the data is a column for various parameters, each row being a seperate incident, and each incident associated to a particular engine. I hope that clarifies.

Edit: see below

incident number engine alarms per hour
inc1 e-1 1
inc2 e-15 3
6

What do you expect the shape of “engine” to be? And what is mu.shape.eval()?

7

I get 9000 for μ.shape.eval() and I would expect engine to yield a shape of 33 as in the end, there are 33 engines

8

@jessegrabowski

Could it be that I used the wrong dims for X? engine_idx shape is 9000 and engine shape is 33 for clarification.

9

Your problem isn’t dims, it’s the actual shapes of the objects. I’m drawing your attention to the dims to highlight some conceptual problems in your model. You need to look at .shape.eval() on different quantities in your model line-by-line, and figure out where your assumptions are breaking.