3 Factor CFA in Mplus and OpenMX, what's missing?

9 replies [Last post]
smcquillin's picture
Offline
Joined: 08/28/2009

Greetings,

I'm trying to transfer a 3 factor CFA from Mplus to OpenMX, and I believe I'm missing something. I've attached the syntax and output from both; however, the primary difference is the notable absence of CFI, TLI, and Chi-Squared in OpemMx. I remember seeing another post re: missing CFI/TLI, but couldn't seem to find it at second glance. I'm pretty certain this is an operator error, rather than program differences, because I have transferred a similar model in the past. Thanks for any help,

Sam

AttachmentSize
open_mx.txt11.29 KB
mplus_syntax.txt8.93 KB
mhunter's picture
Offline
Joined: 07/31/2009
The biggest difference I can

The biggest difference I can see between the two models is that in Mplus you freely estimate all the factor loadings and fix the factor variances, whereas in OpenMx you fix the first factor loading and freely estimate the factor variances. This shouldn't be much of a problem if you compare the standardized output for both models. And for the estimates that I checked, they are quite close.

The missing Chi-Squared in OpenMx is the result of using raw data full-information maximum likelihood (FIML) for handling missing data. The missing CFI, TLI, and RMSEA are fallout from the missing Chi-Squared. OpenMx does not automatically report Chi-squared when using raw data because the Chi-Squared value is the difference between the -2 log likelihood of the fitted model (that is, the model you specified) and the -2 log likelihood of the saturated model.

When you have covariance data, the saturated model -2 log likelihood is very simple to calculate. When you have raw data, it is potentially as time-consuming as fitting the model specified by the user.

I know there was a feature request for a helper function to construct saturated models: http://openmx.psyc.virginia.edu/thread/979

But I do not know if that was filled.

Finally, if you have a saturated model that you fit named MySaturatedModel, then

summary(MyModel, SaturatedLikelihood=MySaturatedModel)

should give the Chi-Squared, TLI, CFI, and RMSEA.

Best,
Mike

smcquillin's picture
Offline
Joined: 08/28/2009
This is helpful. I freely

This is helpful. I freely estimated the loadings, set the factor variances to 1, and freely estimated the correlations between factors and the model now has identical estimates between programs. I attached the updated syntax for those with similar questions.

Sam

AttachmentSize
Open_Mx_revised.txt 11.14 KB
JVaske's picture
Offline
Joined: 06/08/2013
I have a very similar

I have a very similar question and decided to post it here rather than start a new thread. I ran a simple CFA model in MPLUS and Open MX, and was comparing the results. I attempted to set the first factor loading (for the "task" item) in Open MX, but I'm not sure if I correctly did it--this is my first time using Open MX and the factor loading definitely is not set to 1.00, so that may be a problem. The fit statistics look similar and similar intercepts, but the factor loadings drastically differ. Can you help me identify the difference/problem?

Also, does Open MX have an MLR estimator like MPLUS?

Thanks in advance for your help!

AttachmentSize
MPLUS input and output.txt 2.76 KB
Open MX input and output.txt 3.02 KB
tbates's picture
Offline
Joined: 07/31/2009
column order?

at a glance, this looks odd: As if the variables in the dataframe are not in the order in which you picked them out.
So the fixed factor one has happened, but is applied to the wrong variable.

But I can't replicate: Here's version that runs properly, even with jumbled data columns, can you try?

library(MASS)
manifests = c("task", "swear", "impuls", "lsc")
myCov = c(
  1, .3, .3, .3,
 .3,  1, .3, .3,
 .3, .3,  1, .3,
 .3, .3, .3, 1)
 
myCov = matrix(nrow = 4, byrow = T, myCov);
 
mydata = MASS::mvrnorm (n = 100, mu = rep(0,4), Sigma = myCov);
# jumble column order
mydata = data.frame(mydata);  names(mydata) <- manifests[c(2:3,1,4)];
 
oneFactorModel <- mxModel("Common Factor Model Path Specification", type = "RAM",
	manifestVars = manifests,
	latentVars = "F1",
	# latent and residual variances
	mxPath(from = "F1", arrows = 2, free = T, values = 1, labels = "varF1"),
	mxPath(from = manifests, arrows = 2, free = T, values = 1, labels = paste0("e", 1:4)),
	# factor loadings (first loading fixed @ 1)
	mxPath(from="F1", to = manifests, arrows = 1, free = c(F,T,T,T), values = 1, labels = paste0("l", 1:4)),
	# means
	mxPath(from = "one", to = "F1", arrows = 1, free = F, values = 0, labels = "meanF1"),
	mxPath(from = "one", to = manifests, arrows = 1, free = T, values = 1, labels = paste0("meanx", 1:4)),
	mxData(observed = mydata, type = "raw")
)
 
oneFactorFit <- mxRun(oneFactorModel)
summary(oneFactorFit)
oneFactorFit@output

tbrick's picture
Offline
Joined: 07/31/2009
Clear Workspace?

I think tbates' assessment is right; it looks like the wrong path has been constrained to 1--notice also that the path names are matched to the wrong paths. The estimates come out different because of the difference in factor scales, which is why the residual variances, means, and standardized estimates all agree with MPlus.

I can't seem to replicate the error on my machine either.

It's possible there's something hanging around in your workspace from a previous run that isn't being replaced properly because of the typos in the posted script. Try clearing your workspace (the broom icon in the corner of the Workspace section in RStudio) and running the script again. Note that clearing your workspace will remove any variables/history/previous fits that you have saved.

If that doesn't work, maybe it's a specific problem with the way the data is structured? I can't imagine what that problem might be, but it's worth checking.

Would you be willing to either post your data or run the fakeData command (available here ) on your data and post the simulated data? If you simulate data, please try the model once on the simulated data to make sure the path names still don't match.

JVaske's picture
Offline
Joined: 06/08/2013
Hi,Thanks so much for your

Hi,
Thanks so much for your quick responses! Yes this was a total user error. I wasn't aware that the data needed to be organized/structured in the same manner it was to be analyzed (unless you modified the code at the beginning of the syntax). Once I moved the "task" variable to be the first variable, it was constrained to 1.00 and all of the other parameter estimates matched those from MPLUS.

Also, does Open MX have a maximum likelihood estimator with robust standard errors (in case a continuous variable is not normally distributed)? I looked through the user manual but did not see anything on the topic.

Thanks again for your help!

tbates's picture
Offline
Joined: 07/31/2009
OpenMx doesn't/shouldn't care about order of data are frame

This doesn't seem right: OpenMx doesn't (shouldn't) care what order the data are in the dataframe.

Can you please let us know what version of OpenMx (and perhaps R) you are using?

mxVersion()
[1] "1.3.2-2301"
version$version.string
[1] "R version 2.15.2 (2012-10-26)"

We have really FIML standard errors via mxCI()

No WLS as yet...

JVaske's picture
Offline
Joined: 06/08/2013
Hi Tim, I moved the variable

Hi Tim,
I moved the variable back to it's original location, re-ran the analyses, and received the same results as MPLUS--perhaps the other Tim was correct that something was hanging around in my workspace and that caused my estimates to be different.

As for my versions:
> mxVersion()
[1] "1.3.2-2301"
>
> version$version.string
[1] "R version 2.15.0 (2012-03-30)"

tbates's picture
Offline
Joined: 07/31/2009
cleanliness (of the environment) is next to godliness :-)

cool: yes, R can muck you up by using variables that you have not set explicitly in this model but it finds lying around in the environment.

rm(list=ls()) can be a good help.