Interpretation of chi-square goodness-of-fit

2 replies [Last post]
Karin's picture
Offline
Joined: 02/11/2011

I am fitting ACE and ADE families of models and comparing them to the fully saturated model. I am getting results like this:

observed statistics: 2965
estimated parameters: 4
degrees of freedom: 2961
-2 log likelihood: 848.4525
saturated -2 log likelihood: 844.858
number of observations: 2266
chi-square: 3.594549
p: 1

I am a bit confused by the interpretation of the chi-squared statistic and its degrees of freedom. Before, I have seen output from classic Mx where the df for the chi-square statistic would be three for an ACE model, because it had six observed statistics (i.e. four variances and two covariances from the MZ and DZ covariance matrices) and three estimated parameters (not estimating means).

Here, the df are given as 2961 and all of the p-values for the different models round to 1, because of the very high df. This makes the chi-squared test virtually useless for assessing model fit. Am I missing something here? Is there anything else I could do?

Thankyou

Karin

neale's picture
Offline
Joined: 07/31/2009
Probably the wrong df

So the df for the chi-squared should really be the difference between the number of parameters of the saturated model and the number of parameters of the fitted model. If OpenMx is not using that number, then the p-value is incorrect.

mxCompare() might be used to check this out. I suspect that you have 8 parameters in the saturated models (2 x 3 covariance parameters and 2 x 1 mean parameters), so the df for the reported chi-sq should be 4. Indeed it looks like the value of 1 is wrong and this is a bug.

Karin's picture
Offline
Joined: 02/11/2011
mxCompare works

Thanks for the recommendation for mxCompare. That seems to work:

base comparison ep minus2LL df AIC diffLL diffdf p
1 univTwinSat 10 844.8580 2955 -5065.142 NA NA NA
2 univTwinSat univACE 4 848.4525 2961 -5073.547 3.594549 6 0.7313508

Looks like my df is 6....

Thankyou

Karin