Discriminative model vs. Generative model

Discriminative model: P(y|x)

• A conditional probability of y given x
• learning h(x) = {0, 1}
• As a rule of thumb, it's faster than generative models.
• Examples: Logistic regression, SVM, Neural networks

Generative model: P(x, y)

• A joint distribution of x and y
• P(x|y) * P(y)
• Possible to induce discriminative model by using Bayes' Theorem
• Possible to generate new data by sampling from P(x, y)
• Examples: Naive Bayes

References:

- Stack Overflow: http://stackoverflow.com/questions/879432/what-is-the-difference-between-a-generative-and-discriminative-algorithm

- Lecture note by Andrew Ng: http://cs229.stanford.edu/notes/cs229-notes2.pdf

- Machine learning lecture by Andrew Ng: http://openclassroom.stanford.edu/MainFolder/VideoPage.php?course=MachineLearning&video=06.1-NaiveBayes-GenerativeLearningAlgorithms&speed=100

- Quora: http://www.quora.com/What-are-the-differences-between-generative-and-discriminative-machine-learning