Consider the following image:
which is an fitted GP. Note how 0 <= x <= 2 yield a much higher uncertainty than e.g 5 <= x <= 8.
Thus gps are good when dealing with the exploration vs exploitation dilemma which is useful in e.g bayesian optimization.
Now consider e.g y = c* x which we fit with bayesian linear regression. No matter where the datapoints are lying, the uncertainty measurements will be the same. This makes sence, we have an parametric model which assumes an underlying datageneration process and should therefore also hold outside of our range of inputs.
In bayes opt they say that we can have any sort of surrogate function that yields uncertainty estimates but this one i just stated doesnt seem like a good one since it does not capture the uncertainty outside of our inputrange.
Now my first question is: If i would want to include some domain knowledge in my estimates such that the response should exhibit y = c*x^{c_2} where 0 <= c_2 <= 1 and c > 0 is there any way i could do that by tweaking the kernels, does anyone have any examples of that, i.e building a model that exhibits high uncertainty in unexplored regions but still exhibits domain characteristics as the just stated saturation(c_2)?
My second question is this, are there other methods out there that also captures the high uncertainty when extrapolating(extra credit for the one stating such an algorithm that can also incorporate domain knowledge such as the one stated above)?