Problems about multi-objective regression

Questions v5
1

I am writing a multi-objective exponential fitting problem, the code for generating observations is as follows：

def log_transformed_functions(x,coef1,coef2,coef3):
    num_samples, num_features = x.shape
    assert num_features == 3, "x should have exactly 3 features"
    log_x = np.log(x)
    W=np.stack([coef1,coef2,coef3]).T
    log_y = np.dot(log_x, W)   
    return log_y
a1,a2,a3=0.3, -0.8, 0.9
b1,b2,b3=0.75, 0.45, 0.95
c1,c2,c3=0.99, 0.4, 0.8
a=np.array([a1,a2,a3])
b=np.array([b1,b2,b3])
c=np.array([c1,c2,c3])

num = 400  # number of samples
x = np.stack((np.random.uniform(0, 1, size=num),
              np.random.uniform(0, 1, size=num),
              np.random.uniform(0, 1, size=num))).T

log_y = log_transformed_functions(x,a,b,c)
y=np.exp(log_y)

in the Bayesian part I used multivariate normal distribution as the likelihood probability, but the model performs poorly, I don’t know the reason for that, my code is as follows:

def polynomial_regression_functions(x,coef1,coef2,coef3):
    log_x = pm.math.log(x)
    W=pm.math.stack([coef1,coef2,coef3]).T
    log_y = pm.math.dot(log_x, W)   
    return pm.math.exp(log_y)
with pm.Model() as multi_objective_model:
    para1=pm.Uniform('para1',lower=[-1,-1,-1],upper=[1,1,1])
    para2=pm.Uniform('para2',lower=[-1,-1,-1],upper=[1,1,1])
    para3=pm.Uniform('para3',lower=[-1,-1,-1],upper=[1,1,1])
    sigmas=pm.HalfCauchy('sigmas',[1,1,1])
    y_observed=pm.Normal(
        "y_observed",
        mu=polynomial_regression_functions(x,para1,para2,para3),
        sigma=sigmas,
        observed=y,
    )
    sample=pm.sample(draw=4000,tune=1000,target_accept=0.99,chains=4,cores=40)
    az.plot_trace(sample,var_names=['para1','para2','para3'])
    summary=az.summary(sample,var_names=['para1','para2','para3'])

The form of the distribution from the posterior is bad, the chains do not converge, the results are:

image
image2036×1350 399 KB

I can’t find the cause of the problem，all suggestions are helpful, Thanks.

2

Try adding noise to your generated data. Deterministic processes are degenerate: you’re asking the model to sample from a distribution with sigma=0, which is not a distribution. For model testing, I recommend you generate data from your model itself (using pm.sample_prior_predictive), then use pm.observe to see if you can recover the parameters in a given prior draw. See here for an example. This procedure will help you avoid problems like these.

Also setting cores=40 does nothing, the number of cores is used only up to the number of chains.

1 Like