Gpu and cpu showing different log prob numbers

I have a code which I tried running both in cpu and gpu. A typical number for log probability from cpu is
-66.78053371
while the gpu numbers are
-98.88531779

I know there is the difference in double precision for cpu, I can understand if the gpu crashes at certain point due to out of range problems, but how can the same code show such different numbers while its running?

Wow that is a huge difference. How did you check the logp? is the input point identical?

I have a theory how this happened. I think somewhere along the way, the gpu ran out of precision and started dropping bits. As a result, the end numbers show a ridiculous value. I played with a hyper parameters. Now the numbers match between gpu and cpu.