Unable to reproduce MASEM results from a published study

2 replies [Last post]
lind0r's picture
Offline
Joined: 12/27/2013

Hi,

I'd like to reproduce the meta-analytic structural equation modeling (MASEM) results from this study:
soonang[dot]com/wp-content/uploads/2011/04/2007-MISQ-Ang1.pdf

I used the correlation matrix (Table 3, p. 559) as input and specified the paths according to Figure 2 (p. 560).
Additionally, I set the number of observations to 701 (p. 558).
The full openMx code is attached.

The openMx output for the parameter estimates fits the values in Figure 2 quite well.
However, the openMX fit statistics are quite different from the ones in the paper.
openMx: chi square = 2442.81, CFI = 0.27, RMSEA = 0.23
paper: chi square = 160.84, CFI = 0.95, RMSEA = 0.054 (p. 558)
Any idea why these values differ that much?

Thanks and best regards
lind0r

AttachmentSize
partially mediated model.txt2.72 KB
brandmaier's picture
Offline
Joined: 02/04/2010
> The openMx output for the

> The openMx output for the parameter estimates fits the values in Figure 2 quite well.

Indeed, they are close but not the same. I plotted your estimates with Onyx (see attached diagram). For example, the loading from pay to turnover seems quite different. You should be able to reproduce the exact same results. I noticed there is a difference in the degrees-of-freedom between your model and the reported model. Most likely you have not specified the variances of the variables correctly. See also this: http://openmx.psyc.virginia.edu/wiki/example-models

best,
Andreas

brandmaier's picture
Offline
Joined: 02/04/2010
> The openMx output for the

> The openMx output for the parameter estimates fits the values in Figure 2 quite well.

Indeed, they are close but not the same. I plotted your estimates with Onyx (see attached diagram). For example, the loading from pay to turnover seems quite different. You should be able to reproduce the exact same results. My first suggestion is to check the correlation matrix for typos. However, this hardly explains the discrepancy in fit.

best,
Andreas

turnover.png