# A newbie question: How should I formulate a classification problem using PyMC3?

**URL:** <https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573>\
**Category:** Questions\
**Created:** [July 24, 2018, 8:35pm UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573 "2018-07-24T20:35:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://discourse.pymc.io/u/sam)\
**Post date:** [July 24, 2018, 8:35pm UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/1 "2018-07-24T20:35:07Z")

</div>

I have a classification problem  
I have a dataset of 1150 docs with variable number of words (from a vocabulary of 100,000) in each. The docs are labeled favorable / unfavorable  
Additionally I have 800 docs that are not labeled. and I have to predict the label.  
I read through the example below:  
[http://docs.pymc.io/notebooks/lda-advi-aevb.html](http://docs.pymc.io/notebooks/lda-advi-aevb.html)

I thought I should specify number of topics as two…but in the example topic is automatically predicted based on the words in the document. In my case at least 1150 documents already have a topic (favorable / unfavorable)

Can someone help me in formulating the problem so that pymc3 / lda / aevb or parts thereof to solving this classification problem.

My apologies for a rudimentary question.

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [July 25, 2018, 4:33am UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/2 "2018-07-25T04:33:27Z")

</div>

Depending on what is your assumption. If your assumption is that each document contains some latent topics and it is the topics that drive the favourable/ unfavourable labelling, you can first fit an LDA model as in the doc, and use the outputted label as input to a logistic regression.  
You can also specify the number of topics as two, that way you are doing a large sparse logistic regression. You can still use a neural net like in the LDA doc as the approximation for inference.

---

<div class="post-metadata">

**Author:** ![colcarroll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/colcarroll/32/9_2.png) [@colcarroll](https://discourse.pymc.io/u/colcarroll)\
**Post date:** [July 25, 2018, 7:56am UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/3 "2018-07-25T07:56:04Z")

</div>

I have not seen an example of naive bayes with PyMC3, but that might also be a good approach here, right? The advantage (over, say, `sklearn`) would be having posterior estimates for each word. If you did not care about such estimates, then `sklearn` might be the right way to go.

---

<div class="post-metadata">

**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://discourse.pymc.io/u/sam)\
**Post date:** [July 25, 2018, 12:18pm UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/4 "2018-07-25T12:18:11Z")

</div>

@junpenglao & @colcarroll

Thank you both !

---

<div class="post-metadata">

**Author:** ![mkesin](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/mkesin/32/992_2.png) [@mkesin](https://discourse.pymc.io/u/mkesin)\
**Post date:** [July 25, 2018, 12:41pm UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/5 "2018-07-25T12:41:35Z")

</div>

Here’s an older example, requires some translation to PyMC3

> **[PyMC Tutorial #2: Estimating the Parameters of A Naïve Bayes (NB) Model](http://alfan-farizki.blogspot.com/2015/07/pymc-tutorial-2-estimating-parameters.html)**
>
> Before you read this post, we suggest you to read our previous post regarding PyMC Introduction, several parameter estimation techniques, a...

---

<div class="post-metadata">

**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://discourse.pymc.io/u/sam)\
**Post date:** [July 31, 2018, 6:31pm UTC](https://discourse.pymc.io/t/a-newbie-question-how-should-i-formulate-a-classification-problem-using-pymc3/1573/6 "2018-07-31T18:31:30Z")

</div>

@mkesin,  
Thank you 🙏