Talk:Generative model Source: en.wikipedia.org/wiki/Talk:Generative_model

This article is rated Start-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects:

Robotics Mid‑importance This article is within the scope of WikiProject Robotics, a collaborative effort to improve the coverage of Robotics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.RoboticsWikipedia:WikiProject RoboticsTemplate:WikiProject RoboticsRobotics Mid This article has been rated as Mid-importance on the project's importance scale. Statistics Low‑importance This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics Low This article has been rated as Low-importance on the importance scale.

In the context of Markov Decision Processes, the term "generative model" has a much different meaning. A generative model provides a way to sample a new state and resulting reward from the step given a current state and action and is usually denoted ${\displaystyle s',r\gets G(s,a)}$. The term is used widely in the literature, but I believe the original usage is in Kearns' 2002 sparse sampling paper^[1].

I haven't decided yet whether there should be two articles, "Generative Model (Statistical Classification)" and "Generative Model (Markov Decision Process)", or if this should be a Wikipedia:Broad-concept article. My leaning is towards two articles because I don't think the terms are actually related etymologically and an experts in either field (machine learning and MDP planning) may not be familiar with the other sense. Any advice or help on this is welcome!

Sunbergzach (talk) 19:01, 3 June 2020 (UTC)[reply]

Removed Generative grammar from the list of examples. Generative Grammar being a generative model in this sense seems to me too far fetched. Cagri (talk) 22:07, 2 October 2008 (UTC)[reply]

Hello.

I changed

If the observed data are truly generated by the generative model, then fitting the parameters of the generative model to maximize the data likelihood is optimal.

to

is a common method.

If you still prefer optimal, please explain under wich criteria the optimality is reached. Indeed it has been shown that ML estimation has drawbacks such as overfitting. Other methods are MAP (maximun a posteriori) and averaging over posterior distribution. Dangauthier 15:58, 2 February 2007 (UTC)[reply]

I think they misinterpreted the statement that if you actually select the correct generative model, then it will give optimal results in terms of minimizing misclassification rates or, in general, some expected loss. By "correct generative model", we mean not just selecting the correct parametric family, but selecting the correct parameters also if the true distribution is parametric. — Preceding unsigned comment added by 149.97.32.36 (talk) 01:20, 6 April 2012 (UTC)[reply]

I agree. However I think something stronger can be said than it simply being common, as well as some mention of ML estimates that can be directly derived from sufficient statistics vs. EM for more complicated models. I'm going to make some additions when I have some time to get my thoughts together. DaveWF 08:19, 6 February 2007 (UTC)[reply]

Generative models contrast with discriminative models, in that all the variables of a descriptive model are directly measurable.

Could this be clarified? I assume "descriptive model" refers to generative models? Perhaps words like "Generative models are descriptive models" would be helpful. Thanks, BenWilliamson 01:57, 18 October 2007 (UTC)[reply]

A generative model is a model for randomly generating observed data

For the newbie, can you clarify this too? If the data has already been *observed*, why do you need to generate it? I presume the idea is to take a hypothetical distribution of UNobservABLE data, and from that to generate a resulting predicted distribution of the observABLE data, then compare that predicted distribution of observABLE data with the actual distribution of the observedED data; but it would be nice to make this (or a correction of this) explicit. Mcswell 15:24, 12 November 2007 (UTC)[reply]

SVM does not output the posterior probabilities. In such a case why is it an example of Generative model? 121.244.161.2 (talk) 05:38, 25 April 2008 (UTC) Sunil Jagadish[reply]

I would like to know this as well. Did you get any answer to this? 2003:E3:472E:C400:68E4:79FF:297D:C378 (talk) 01:34, 19 June 2022 (UTC)[reply]

The section "Generative models in the context of Machine Learning" was apparently taken from two stack overflow comments without attribution ( http://stackoverflow.com/questions/879432/what-is-the-difference-between-a-generative-and-discriminative-algorithm ). — Preceding unsigned comment added by 188.174.28.141 (talk) 08:17, 28 March 2017 (UTC)[reply]

Hi !

In the intro it says:

Standard examples of each, all of which are linear classifiers, are:

   generative classifiers:
       naive Bayes classifier and
       linear discriminant analysis
   discriminative model:
       logistic regression
       non-model classifier:
           perceptron and
           support vector machine.

Maybe instead of linear discriminant analysis, it was Latent Dirichlet Allocation that was meant ? The confusion might come from the fact that they have the same acronym. It feels like linear discriminant analysis is more of a discriminative model. — Preceding unsigned comment added by Matthieu.heitz (talk • contribs) 10:09, 13 December 2019 (UTC)[reply]

Most probably, yes! See list here: https://www.quora.com/What-is-a-generative-model/answer/Jack-Rae --138.37.180.174 (talk) 14:43, 2 January 2020 (UTC)[reply]

If this is the case, then it is also wrong in https://en.wikipedia.orghttps://demo.azizisearch.com/lite/wikipedia/page/Linear_classifier . Interestingly, there is a special note devoted to the explanation of why LDA (as in linear discriminant analysis) is in fact a generative model. To be fair, I found that note rather confusing and reading the discussion here, I doubt it is correct. I cannot make any improvements either as I am just learning about this topic currently. Maybe someone else can clarify. 2003:E3:472E:C400:68E4:79FF:297D:C378 (talk) 01:39, 19 June 2022 (UTC)[reply]

"Generative model" most often refers to the following:

A model of a probability distribution from which *new samples* may be drawn; hence *generating* observations.

A generative classifier is a classifier that makes use of a generative model. However it is not typically used to generate data itself. For this reason, I propose changing the title to "generative classifier", while reserving "generative model" to the former notion. I think this is particularly relevant given recent progress in deep generative modeling - it's important to have a precise notion of what "generative model" refers to. 80.2.247.44 (talk) 22:02, 23 March 2024 (UTC)[reply]

agreed Cononsense (talk) 04:24, 3 January 2025 (UTC)[reply]

The article covers both generative and discriminative. It should however focus on the former, since there is already a dedicated page for discriminative model. I don't have sufficient knowledge on the topic to fix the issue myself

7804j (talk) 15:32, 11 May 2025 (UTC)[reply]