Comprehensive fit indices
- Is chi-square non-significant?
- Is RMSEA < .05 ?
- RMSEA is returned by summary(model) for RAM models not using Raw data
- Is the lower bound of the 90% CI of the RMSEA < .01 ?
- Is the upper bound of the 90% CI of the RMSEA < .10 ?
- Is p close non-significant ? [tests the null hypothesis that RMSEA in the population is < .05]
Is the SRMR < .08 ?
Is the CFI > .95 ?
- Mx did this, so 3-5 seem likely wish-list candidates for OpenMx
Are other classic fit indices satisfactory (GFI, TLI, etc.)?
- CFI is returned by summary(model)
Are all correlation residuals < .10 ?
Are all standardized residuals < 1.96 ? [less important in large samples]
- potentially big list… several already calculated
Does the quantile plot of standardized residuals look OK (do the standardized residuals fall along a diagonal line)?
- 9 & 10 are suspect with missing data (if the data are not missing completely at random) FIML estimates of covariances etc. may not match their sample counterparts and residuals could exceed arbitrarily set thresholds for deviation. In addition, item 1 with raw data may be costly to obtain because it requires fitting a model with as many parameters as there are means and covariances. Most studies have at least some missing data and discarding or imputing these increase risk of biased results compared to FIML. It may be worth the loss of certain measures of fit to use FIML.
Are the parameter estimates OK: do they make sense? are they significant?
- R has great graphical capabilities, and if the missing data caveat does not apply, it would be a one- or two-liner to generate the QQ plot.
Are indirect effects statistically significant? [test with bootstrap method]
- Parameter estimates are included in summary()
Do we have sufficient statistical power for the test of the close-fit hypothesis and the test of the not-close-fit hypothesis? [generate script with Preacher’s web site]
Can we argue against equivalent and near-equivalent models?
- likelihood-based confidence interval on the function of parameters of interest - this would be very easy to request and faster than bootstrap (though that can usually be done quite easily as well).
- Is the difference chi-square significant?
- ?mxCompare() compares the fit of models
- Does one of the models have a lower AIC, BIC?
- available in summary(model)