Fit Indices in OpenMx path modeling

8 replies [Last post]
HAMED's picture
Offline
Joined: 06/15/2012

Hello,

I am using OpenMx for latent variable path modeling. I have faced two problems in the results of the summary() function, about which any help would be much appreciated:

1- When I use the "raw" data type in my mxData() function, some of the resulted fit indices (i.e., RMSEA, CFI, TLI, and Chi-square) returns "NA" as the output! However, when I change my data type to "cov" in the mxData() function, the problem is resolved and it gives me some non-"NA" values for those fit indices. I have tested this on the examples in the "OpenMxUserGuide" and I have seen the same problem there too. Is there any reason for this? or is it a bug in the OpenMx?

2- The summary() function reports very limited number of fit indices (i.e., chi-square, AIC, BIC, CFI, TLI, and RMSEA). However, many other common fit indices that reviewers usually ask for them are missing from the output of summary() (e.g., GFI, AGFI, SRMR, NFI, NNFI, 95% CI for RMSEA). I was wondering is there is any function/solution for calculating these missing fit indices in OpenMx?

Thank you,

tbates's picture
Offline
Joined: 07/31/2009
until then use umx.lib.R ...

Until then, you can use the umx helper file.

It lives here https://github.com/tbates/umx

But you can pull a version into your script with this:

source("http://timbates.wdfiles.com/local--files/start/umx.lib.R")

You can either use umxRun() to run the model (it will detect that you have a raw-data RAM model and run the sat and ind models and put their LL into the model for your you:

m1 = umxRun(m1); 
umxReportFit(m1) # compact fit suitable for paper
summary(m1)

or this works too

m1_sat = umxSaturated(m1)
summary(m1, 
   SaturatedLikelihood = m1_sat$Sat, 
   IndependenceLikelihood = m1_sat$Ind
)

HAMED's picture
Offline
Joined: 06/15/2012
Thank you for your

Thank you for your information. "umx" sounds like very interesting and helpful for reporting the results of a SEM model in a paper. However, the problem is that when I insert the following line in R:
source("http://timbates.wdfiles.com/local--files/start/umx.lib.R")
It gives me this error:
Error in mxOption(model, "Calculate Hessian", "No") : object 'm1' not found

Am I missing anything?

tbates's picture
Offline
Joined: 07/31/2009
fixed now

That was a bit of debugging code running a model ... commented out now. Sorry.

mhunter's picture
Offline
Joined: 07/31/2009
We know about these issues

We know about these issues and I'm happy to say there is a solution for both.

For the first issue: when raw data is used, absolute fit indices (like RMSEA, CFI, TLI, and Chi-Square) require one or more null comparison models to be fit. The short answer is that OpenMx does not fit these null models automatically on raw data in the interest of speed. When covariance data is used, the null models are trivial and always fast to estimate, so we return the absolute fit indices in that case. The binary release of OpenMx does not yet have a single line of code solution to this issue. However, the svn version does have a a helper function to fit the null models in a single line, and then you can obtain absolute fit indices. In upcoming binary releases of OpenMx you will be able to type something like the following:

myModelFit <- mxRun(myModel)
myModelSat <- omxSaturatedModel(myModel, run=TRUE)
summary(myModelFit, SaturatedLikelihood=myModelSat)

For the second issue: OpenMx does return a limited number of fit indices. However, the semTools package has a function called fitMeasuresMx() which returns a vector of fit indices when given an mxModel. It has 37 elements the last time I checked. These include Chi-Square, CFI, TLI, NFI, NNFI, RFI, RNI, IFI, RMSEA (including confidence intervals thereon), SRMR, RMR, GFI, AGFI, PGFI, MFI, AIC, BIC, and so on. You should be able to simply obtain all the fit indices you want with code like the following:

# install.packages("semTools") # if needed
require(semTools)
myModelFit <- mxRun(myModel)
fitMeasuresMx(myModelFit)

Hope this helps!

ya's picture
ya
Offline
Joined: 11/28/2013
Hi, I am trying to get the

Hi,

I am trying to get the model fit indices and standardized estimates using semtools. The model terminated normally, but the error occurred when I ask for the fit indices and standardized estimates. Could you give me some suggestions please? Thank you very much.

Here is my code:

> model2=mxModel('model2',
+ mxData(HolzingerSwineford1939[,7:15],type='raw'),
+ mxMatrix(name='loadingregress',type='Full',nrow=12,ncol=12,
+ values=c(
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,0,
+ 0,0,0,0,0,0,0,0,0,0,0,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0
+ ),
+ free=c(
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,F,F,
+ F,F,F,F,F,F,F,F,F,T,F,F,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,F,T,F,
+ F,F,F,F,F,F,F,F,F,F,T,F,
+
+ F,F,F,F,F,F,F,F,F,F,F,T,
+ F,F,F,F,F,F,F,F,F,F,F,T,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,T,F
+ ),
+ byrow=T),
+ mxMatrix(name='vcov',type='Symm',nrow=12,ncol=12,
+ values=c(
+ 1,0,0,0,0,0,0,0,0,0,0,0,
+ 0,1,0,0,0,0,0,0,0,0,0,0,
+ 0,0,1,0,0,0,0,0,0,0,0,0,
+ 0,0,0,1,0,0,0,0,0,0,0,0,
+ 0,0,0,0,1,0,0,0,0,0,0,0,
+ 0,0,0,0,0,1,0,0,0,0,0,0,
+ 0,0,0,0,0,0,1,0,0,0,0,0,
+ 0,0,0,0,0,0,0,1,0,0,0,0,
+ 0,0,0,0,0,0,0,0,1,0,0,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0,
+ 0,0,0,0,0,0,0,0,0,0,0,1
+ ),
+ free=c(
+ T,F,F,F,F,F,F,F,F,F,F,F,
+ F,T,F,F,F,F,F,F,F,F,F,F,
+ F,F,T,F,F,F,F,F,F,F,F,F,
+ F,F,F,T,F,F,F,F,F,F,F,F,
+ F,F,F,F,T,F,F,F,F,F,F,F,
+ F,F,F,F,F,T,F,F,F,F,F,F,
+ F,F,F,F,F,F,T,F,F,F,F,F,
+ F,F,F,F,F,F,F,T,F,F,F,F,
+ F,F,F,F,F,F,F,F,T,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,T,F,
+ F,F,F,F,F,F,F,F,F,T,T,F,
+ F,F,F,F,F,F,F,F,F,F,F,T
+ ),
+ byrow=T),
+ mxMatrix(name='filter',type='Full',nrow=9,ncol=12,
+ dimnames=list(
+ c(names(HolzingerSwineford1939[,7:15])),
+ c(names(HolzingerSwineford1939[,7:15]),'f1','f2','f3')
+ ),
+ values=c(
+ 1,0,0,0,0,0,0,0,0,0,0,0,
+ 0,1,0,0,0,0,0,0,0,0,0,0,
+ 0,0,1,0,0,0,0,0,0,0,0,0,
+ 0,0,0,1,0,0,0,0,0,0,0,0,
+ 0,0,0,0,1,0,0,0,0,0,0,0,
+ 0,0,0,0,0,1,0,0,0,0,0,0,
+ 0,0,0,0,0,0,1,0,0,0,0,0,
+ 0,0,0,0,0,0,0,1,0,0,0,0,
+ 0,0,0,0,0,0,0,0,1,0,0,0
+ ),
+ free=F,
+ byrow=T),
+ mxMatrix(name='means',type='Full',nrow=1,ncol=12,
+ dimnames=list(
+ c('means'),
+ c(names(HolzingerSwineford1939[,7:15]),'f1','f2','f3')
+ ),
+ values=c(0,0,0,0,0,0,0,0,0,0,0,0),
+ free=c(T,T,T,T,T,T,T,T,T,F,F,F),
+ byrow=T),
+ mxRAMObjective('loadingregress','vcov','filter','means')
+ )
> fit2=mxRun(model2)
Running model2
> fitMeasuresMx(fit2)
Error in mxFIMLObjective(covariance = "S", means = "M", dimnames = colnames(data@observed)) :
'dimnames' argument cannot be an empty vector
> standardizeMx(fit2)
Error in standardizeMxSingleGroup(object) :
trying to get slot "values" from an object of a basic class ("NULL") with no slots
> fitMeasuresMx(fit2)
Error in mxFIMLObjective(covariance = "S", means = "M", dimnames = colnames(data@observed)) :
'dimnames' argument cannot be an empty vector
>

neale's picture
Offline
Joined: 07/31/2009
Null vs. saturated models

I agree with mhunter that the fit index issue with raw data is speed. However, it is fitting the saturated comparison model (every variance, covariance and mean or threshold is a free parameter) that is the problem for speed. The null model (diagonal covariance matrix, separate means or thresholds for every variable) can be fitted without optimization so it would be fast.

Unfortunately, most fit indices (chi-squared and those based on it) do need the saturated model fit.

tbates's picture
Offline
Joined: 07/31/2009
saturated model in the submodel case

hi all,
What is the correct saturated fit to return for a supermodel with submodels?
Just that supermodel, with it's contained models turned into saturated (or independence) models?

Currently the helper doesn't support these models.

I see that SEMtools handles multiple groups in its nullMx() and saturateMx() functions.
It does seem like core functionality to me, and best maintained inside OpenMx package.

Best, tim

HAMED's picture
Offline
Joined: 06/15/2012
Thank you so much for such a

Thank you so much for such a prompt and complete answer. It certainly helped.

Best