How to troubleshoot sampling error with custom, not differentiable likelihood?

@jessegrabowski , thanks for the response!
I’ve tried setting initial values that I know should produce positive, finite values, and I’ve tried catching IoUs of exactly 0 or 1 and changing them to 0.001 or 0.999. Neither of these approaches change the likelihood from -inf. I don’t know what else about the process could be causing this errors.

More importantly, I really appreciate your insights on my fundamental approach to the problem, but I don’t really understand them. I’ll lay out in more detail what I’m trying to accomplish and then hopefully you could help me better understand what you mean, if you don’t mind.

I have a PDE that maps changes in temperature over a space that would correspond to a burning wildfire. The data I have is a mask of the fire perimeters in the area at specific times. Just binary whether fire is present or not. My metric for comparing how well the predicted fire matches my data is described in get_IoUs(); I consider temperature of above 600 degrees to be on fire in my predicted temperatures, find the intersection between the predicted and actual burning areas, and divide it by the union of the two geographical areas. I would like to recover the parameters for the PDE that produce predicted temperatures at specific times that best match the true wildfire according to IoU score in the hopes of predicting how the fire will develop in the future.

So the output of theano_fem_solver is a vector describing the temperatures at each coordinate of our finite element mesh in 2D space at each time for which we have a true fire perimeter to compare. I don’t understand how Ts_observed can be used directly as data. The IoU metric (or adaptations of it) is the only reliable performance metric I know of and I’d like my likelihood function to be based on it. However, if I’m missing something fundamental here I really want to understand it. What would it look like if I implemented what you’re describing? Using the PDE as the mean of a spatial process intrigues me, however the actual temperature at individual coordinates of the mesh isn’t important so much as how much of the fire is correctly predicted, and how badly the fire is overestimated, which is what IoU captures, so I don’t understand how this would be done.

Thanks so much!

1 Like