OpenMx vs. SASV9.2: Standardized Standard Errors Compared

4 replies [Last post]
Offline
Joined: 04/18/2013

Hello Dr. Neale,

My name is Dr. Brandy Rutledge. I have attempted to duplicate the results from OpenMx in SAS for the first example in the Beginner’s Guide for Mx using the path model approach. I was able to duplicate everything except the -2 log likelihood (slightly lower) and the standardized standard errors (very different). I have done a little research on the web and found forums where the standard errors were being discussed a few years ago (2009). I am assuming that these issues have already been fixed. Do you know why the standardized standard errors in SAS using Proc Calis would be different from those that are produced by OpenMx?

Here is my SAS code for the first example in the Beginner’s Guide:

proc calis data=mxtest nobs=500;
path
x1<---G=theta1,
x2<---G=theta2,
x3<---G=theta3,
x4<---G=theta4,
x5<---G=theta5
;
pvar
G=1.0;
run;

Attached are the results from running this model.

Dr. Brandy Rutledge

AttachmentSize
SAS output for Mx Example.txt23 KB
Offline
Joined: 07/31/2009
OpenMx standardized SEs are scaled to agree with unstandardized

Offline, I kicked this problem around as Brandy emailed me directly. In the OpenMx output,
free parameters:
name matrix row col Estimate Std.Error Std.Estimate Std.SE lbound ubound
1 One Factor.A[1,6] A x1 G 0.39715206 0.015549814 0.89130939 0.034897705
2 One Factor.A[2,6] A x2 G 0.50366105 0.018232572 0.93255466 0.033758556
3 One Factor.A[3,6] A x3 G 0.57724129 0.020448473 0.94384668 0.033435279
4 One Factor.A[4,6] A x4 G 0.70277356 0.024011511 0.96236247 0.032880829
5 One Factor.A[5,6] A x5 G 0.79624981 0.026669562 0.97255560 0.032574742
6 One Factor.S[1,1] S x1 x1 0.04081420 0.002812719 0.20556757 0.014166731
7 One Factor.S[2,2] S x2 x2 0.03801998 0.002805792 0.13034181 0.009618944
8 One Factor.S[3,3] S x3 x3 0.04082717 0.003152309 0.10915345 0.008427851
9 One Factor.S[4,4] S x4 x4 0.03938708 0.003408879 0.07385847 0.006392313
10 One Factor.S[5,5] S x5 x5 0.03628714 0.003678565 0.05413560 0.005487931

divding Estimate/Std.Error gives the same value as Std.Estimate/Std.SE. This is a conventional t-value against an estimate being zero. OpenMx has really 'fudged' the Std.SE to agree in this way, because it is known that estimating a function of a parameter g(theta) and its standard error does NOT yield a consistent t-statistic. See http://www.vipbg.vcu.edu/vipbg/Articles/behavior-fitting-1989.pdf p.43-44 for example. This is one of the arguments to avoid using Hessian-based standard errors to judge the significance of parameters. Likelihood-based confidence intervals, and the likelihood-ratio test generally do not have this problem, hence the availability of mxCI() and mxCompare() in OpenMx.

In the end, whether you consider the SAS standardized SE's "correct" or those of OpenMx might be considered a matter of taste. The t-values provided by OpenMx for the standardized estimates agree with those of the unstandardized estimates. I am not sure what is going on with SAS's Standardized SE's but they give t-values that are wildly different, e.g. 11 vs. 14 for a factor loading or 29 vs. 296 (gasp) for the error. Yet this may be correct given the parameter transformation (I've not done the math).

Offline
Joined: 04/19/2011
OpenMx 2.0

OpenMx 2.0 has a new function, `mxStandardizeRAMpaths()`, which gives standard errors that agree pretty closely with what Dr. Rutledge got in SAS:

```                name label matrix row col  Raw.Value  Std.Value      Std.SE
1  One Factor.A[1,6]  <NA>      A  x1   G 0.39715184 0.89130932 0.009723335
2  One Factor.A[2,6]  <NA>      A  x2   G 0.50366062 0.93255456 0.006408558
3  One Factor.A[3,6]  <NA>      A  x3   G 0.57724093 0.94384664 0.005514390
4  One Factor.A[4,6]  <NA>      A  x4   G 0.70277321 0.96236249 0.004007508
5  One Factor.A[5,6]  <NA>      A  x5   G 0.79624933 0.97255562 0.003275473
6  One Factor.S[1,1]  <NA>      S  x1  x1 0.04081418 0.20556769 0.017332999
7  One Factor.S[2,2]  <NA>      S  x2  x2 0.03801998 0.13034199 0.011952660
8  One Factor.S[3,3]  <NA>      S  x3  x3 0.04082716 0.10915353 0.010409477
9  One Factor.S[4,4]  <NA>      S  x4  x4 0.03938702 0.07385844 0.007713351
10 One Factor.S[5,5]  <NA>      S  x5  x5 0.03628708 0.05413557 0.006371159
11 One Factor.S[6,6]  <NA>      S   G   G 1.00000000 1.00000000 0.000000000```

The difference is that now, in 2.0, the necessary calculus is being done to obtain theoretically more-coherent standard errors for the standardized parameter estimates. The derivatives are evaluated numerically, which is why the numDeriv package needs to also be installed to get these standard errors. Incidentally, no standardized values are reported in the 'free parameters' table of `summary()` output any longer.

Definitely, though--as I'm sure Ryne and Dr. Neale would agree--for inferential purposes, likelihood-ratio tests and/or profile-likelihood confidence intervals are generally preferable to these standard errors.

Offline
Joined: 07/31/2009
The math for parameter

The math for parameter transformation is consistent from Estimate to Standard error. For (co)variance paths (S) matrix, all parameters and their standard errors are divided by the product of the model-implied standard deviations of the two variables that go into it. Regressions are multiplied by the model-implied standard deviation of the DV, and divided by the SD of the IV.

While the confidence limits/SEs of linear or non-linear combinations of parameters won't be a simple function of those parameters SEs, simple linear transformations shouldn't affect significance tests. The standard error of X/2 should be SE_X divided by 2.

The standardizeRAM function became what summary uses to standardize RAM models, and is discussed here: