Improving Mixture Model Specification

4 replies [Last post]
Ryne's picture
Offline
Joined: 07/31/2009

Tim and I were discussing my recent botched growth mixture code, and have a proposal for a future version of mixture modeling which should be a little bit easier on the user. It consists of two parts:

-A new matrix type, "Prop" or proportional. A (row-wise) proportional matrix consists of k rows which must sum to a constant (1, unless someone really wants to be able to change it). A 1 x k prop matrix would be used for model-wide class probabilities, while an n x k prop matrix would be used for individual class probabilities. Prop matrices also open us up to compositional data and associated models.

-A mixture optimizer, named something like "mxMixtureObjective." This objective function would take the names of submodels (or their objectives) as arguments, as well as a matrix indicating class probabilities. In addition to making mixture/LCA specification simpler, it also opens us up to optimization of the mixture routines above and beyond improvements to the FIML objective.

Assuming that R objects model1 and model2 are models representing two classes with omx names "M1" and "M2", this model would look something like this:

mxModel("Mixture Example",
    mxData(myDataFrame, type="raw"),
    model1, model2,
    mxMatrix("Prop", 1, 2, free=TRUE, start=.5, 
        labels=c("classprob1", "classprob2"), name="class"),
    mxMixtureObjective(c("M1", "M2"), "class")
)
 
This could be shortened more with a default class probability matrix, but that's probably too far. This proposal frees users from specifying their own constraints on class probabilities (i.e., to keep them between zero and one with a unit sum) and simplifies the objective function with possible speedups in the future. It may be too big for a 1.0 release, but that's part of why we have a wishlist.

neale's picture
Offline
Joined: 07/31/2009
Agreed, a nice approach. It

Agreed, a nice approach. It would be even better if "class" could refer to an algebra or a matrix.

Personally, I'd like to see this as feature #1 to implement post 1.0. It should not, I think, be too difficult. If it is, then perhaps adding an argument weight="myweightAlgebraorMatrix" to the mxFIMLObjective() function would be simpler. This would be applied to every row in the event that vector=TRUE is set.

A major piece of flexibility is added if it is possible to make the weight matrix/algebra a function of definition variables, which would require re-evaluation of the weight algebra for each data vector, in the event that it depends on definition variables.

Ryne's picture
Offline
Joined: 07/31/2009
As the results of algebras

As the results of algebras are treated as matrices, I don't see a problem using them as arguments. The covariance and means matrix arguments in all of our other optimizers already accept algebras.

What is the difference mathematically between a weight and a class probability parameter? The class probability parameters are really just a set of freely estimated weights with some type of constraint on their sum that reduce their dimensionality.

neale's picture
Offline
Joined: 07/31/2009
I don't see a mathematical

I don't see a mathematical difference. In principle, both could be functions of definition variables and/or parameters of the model.

Sample weights are sometimes applied even when the number of classes is 1, which is a very special case of a latent class or factor mixture model. For programing purposes, it would not be necessary to use a mxAlgebraObjective() if weights were available as an argument within the mxFIMLObjective() call. Conveniently, definition variables would likely already be part of the dataframe in the mxData() call, so their use in weight formulae would be facilitated if they were part of mxFIMLObjective()

tbates's picture
Offline
Joined: 07/31/2009
that looks like a really nice

that looks like a really nice approach Ryne... I find it very intuitive!