Comprehensive fit indices

No replies
tbates's picture
Offline
Joined: 07/31/2009

Model Fit

  1. Is chi-square non-significant?
  2. Is RMSEA < .05 ?
    • RMSEA is returned by summary(model) for RAM models not using Raw data
  3. Is the lower bound of the 90% CI of the RMSEA < .01 ?
  4. Is the upper bound of the 90% CI of the RMSEA < .10 ?
  5. Is p close non-significant ? [tests the null hypothesis that RMSEA in the population is < .05]
    • Mx did this, so 3-5 seem likely wish-list candidates for OpenMx
  6. Is the SRMR < .08 ?
  7. Is the CFI > .95 ?
    • CFI is returned by summary(model)
  8. Are other classic fit indices satisfactory (GFI, TLI, etc.)?
    • potentially big list… several already calculated
  9. Are all correlation residuals < .10 ?
  10. Are all standardized residuals < 1.96 ? [less important in large samples]
    • 9 & 10 are suspect with missing data (if the data are not missing completely at random) FIML estimates of covariances etc. may not match their sample counterparts and residuals could exceed arbitrarily set thresholds for deviation. In addition, item 1 with raw data may be costly to obtain because it requires fitting a model with as many parameters as there are means and covariances. Most studies have at least some missing data and discarding or imputing these increase risk of biased results compared to FIML. It may be worth the loss of certain measures of fit to use FIML.
  11. Does the quantile plot of standardized residuals look OK (do the standardized residuals fall along a diagonal line)?
    • R has great graphical capabilities, and if the missing data caveat does not apply, it would be a one- or two-liner to generate the QQ plot.
  12. Are the parameter estimates OK: do they make sense? are they significant?
    • Parameter estimates are included in summary()
  13. Are indirect effects statistically significant? [test with bootstrap method]
    • likelihood-based confidence interval on the function of parameters of interest - this would be very easy to request and faster than bootstrap (though that can usually be done quite easily as well).
  14. Do we have sufficient statistical power for the test of the close-fit hypothesis and the test of the not-close-fit hypothesis? [generate script with Preacher’s web site]
  15. Can we argue against equivalent and near-equivalent models?

Model comparison

  1. Is the difference chi-square significant?
    • ?mxCompare() compares the fit of models
  2. Does one of the models have a lower AIC, BIC?
    • available in summary(model)