@junpenglao Thanks! it would be nice to replace X with theta, but they are not exactly the same model, still want to stick to the original model specification. I have also checked the link post - it is not however on supervised LDA. Do you happen to know what is the reason that VI in PyMC3 could not handle discrete variables? There is really few resource available on this issue. I barely could find one. It does not seem an issue for VI in theory to handle discrete variable, not sure why PyMC3 could not. Thanks so much!