Thu, 05/26/2011 - 09:30

Hello,

I have a question on the following: I have a model with 6 IV's, 3 mediators M, and 1 DV. (all observed variables).

Suppose model 1 has all direct effects of the IV's on the M's, and all direct effects of the M's on the DV. All IV's are allowed to correlate.

Model 2 is the same, except that one IV is dropped from the model.

How should I compare these two models to each other? If I understand this correctly, the models are not nested? So should I then use the AIC instead of a Chi sqr difference?

Kind regards, Alexandra

Your models are not nested as you're describing them, so you can't use a likelihood ratio (Chi Square) test. As the data themselves change, AIC isn't valid either. The scale of the likelihood function changes when you remove variables, so you can't make either nested (LR test) or non-nested (AIC, BIC, etc) comparisons.

However, you can specify your model in such a way that they are nested by including the IV you're dropping and specifying no covariances/regressions between that variable and the mediator variables in the model. You'll want to give that variable a mean and a variance. Whether or not the IV in question is allowed to covary with the other IVs in the model: if not, you're fitting a model in which that IV has no relation to anything; if so, you're allowing that IV to covary with other IVs directly and by extension, the mediators and DV indirectly.

Apologies for the late reply.

Thanks a lot, Ryne !

That's kind of a new perspective on things for me. I thought that if a variable does not have any regressions with any other variables in the model, then it is equivalent to not having the variable in the model at all.

Just to make sure I understand things correctly, I have attached to pdf files. So, both model 1 and 2 are 'legitimate' models and I can compare these two as being nested models right ?

TL;DR: Those models look good.

Now for the long version: all of the models in both posts are "legitimate", in that they represent models for a particular process. The second set of models (your pdfs) are able to be compared using some statistic based on a -2 log likelihood because they are fit to the same data. This is a confusing difference between techniques like regression or anova and structural equation modeling. Essentially, all variables are treated as dependent variables in SEM, as all variables (regardless of whether you think of them as IVs, moderators or DVs) are included in the likelihood function. Whereas regression tries to minimize the residual variance in the DV, SEM tries to minimize the difference between the observed covariance matrix and the model implied covariance, which describes all variables. If you delete a variable, you're saying that the misfit for the deleted variable is exactly zero.

The models can be compared via a likelihood ratio test because Model 2 is just Model 1 with a bunch of paths set to zero, i.e. they're nested.

I also feel the need to point out that your Model 2 (which I like, btw) does include an implied covariance between variables 3/4 and variables 5/6/7, albeit an indirect one. The theory you're testing is one with no direct relationship, but you would expect variable 3 and variable 5 to be correlated with an implied covariance of r_13*beta_15 + r_23*beta+25. I think this is a very defensible model, but i don't know what theory you're trying to test. Make sure that this model squares with what question you'd like to answer.