Thanks for the reply. Yea so in the third message I sent above, I did realize that and converted it to numpy in which now the sampler runs but estimates are not great. In summary what it does differently from the DiscreteMarkovChain is:
Because the state space is very large, using transfer matrices and pushing initial distribution of states is not feasible. So I generate a single trajectory from a single starting state (my initial starting state is fixed i.e initial distribution of states is a dirac ) and use that trajectory in the context of Simulator with an error.
The estimates I get are not great and I am wondering if that is because I am generating a single trajectory. If I generate multiple trajectories and take the average error over multiple trajectories maybe it will work better. Or maybe because the state space is large, this is not a feasible approach I don’t know. In any case I am trying to understand what is the best way to model markov chains with large state spaces without going to sth like diffusion approximation and solving SDEs instead.