OpenMx vs. SASV9.2: Standardized Standard Errors Compared

3 replies [Last post]
bnrwest's picture
Offline
Joined: 04/18/2013

Hello Dr. Neale,

My name is Dr. Brandy Rutledge. I have attempted to duplicate the results from OpenMx in SAS for the first example in the Beginner’s Guide for Mx using the path model approach. I was able to duplicate everything except the -2 log likelihood (slightly lower) and the standardized standard errors (very different). I have done a little research on the web and found forums where the standard errors were being discussed a few years ago (2009). I am assuming that these issues have already been fixed. Do you know why the standardized standard errors in SAS using Proc Calis would be different from those that are produced by OpenMx?

Here is my SAS code for the first example in the Beginner’s Guide:

proc calis data=mxtest nobs=500;
path
x1<---G=theta1,
x2<---G=theta2,
x3<---G=theta3,
x4<---G=theta4,
x5<---G=theta5
;
pvar
G=1.0;
run;

Attached are the results from running this model.

Thank you in advance,
Dr. Brandy Rutledge

AttachmentSize
SAS output for Mx Example.txt23 KB
neale's picture
Offline
Joined: 07/31/2009
OpenMx standardized SEs are scaled to agree with unstandardized

Offline, I kicked this problem around as Brandy emailed me directly. In the OpenMx output,
free parameters:
name matrix row col Estimate Std.Error Std.Estimate Std.SE lbound ubound
1 One Factor.A[1,6] A x1 G 0.39715206 0.015549814 0.89130939 0.034897705
2 One Factor.A[2,6] A x2 G 0.50366105 0.018232572 0.93255466 0.033758556
3 One Factor.A[3,6] A x3 G 0.57724129 0.020448473 0.94384668 0.033435279
4 One Factor.A[4,6] A x4 G 0.70277356 0.024011511 0.96236247 0.032880829
5 One Factor.A[5,6] A x5 G 0.79624981 0.026669562 0.97255560 0.032574742
6 One Factor.S[1,1] S x1 x1 0.04081420 0.002812719 0.20556757 0.014166731
7 One Factor.S[2,2] S x2 x2 0.03801998 0.002805792 0.13034181 0.009618944
8 One Factor.S[3,3] S x3 x3 0.04082717 0.003152309 0.10915345 0.008427851
9 One Factor.S[4,4] S x4 x4 0.03938708 0.003408879 0.07385847 0.006392313
10 One Factor.S[5,5] S x5 x5 0.03628714 0.003678565 0.05413560 0.005487931

divding Estimate/Std.Error gives the same value as Std.Estimate/Std.SE. This is a conventional t-value against an estimate being zero. OpenMx has really 'fudged' the Std.SE to agree in this way, because it is known that estimating a function of a parameter g(theta) and its standard error does NOT yield a consistent t-statistic. See http://www.vipbg.vcu.edu/vipbg/Articles/behavior-fitting-1989.pdf p.43-44 for example. This is one of the arguments to avoid using Hessian-based standard errors to judge the significance of parameters. Likelihood-based confidence intervals, and the likelihood-ratio test generally do not have this problem, hence the availability of mxCI() and mxCompare() in OpenMx.

In the end, whether you consider the SAS standardized SE's "correct" or those of OpenMx might be considered a matter of taste. The t-values provided by OpenMx for the standardized estimates agree with those of the unstandardized estimates. I am not sure what is going on with SAS's Standardized SE's but they give t-values that are wildly different, e.g. 11 vs. 14 for a factor loading or 29 vs. 296 (gasp) for the error. Yet this may be correct given the parameter transformation (I've not done the math).

Ryne's picture
Offline
Joined: 07/31/2009
The math for parameter

The math for parameter transformation is consistent from Estimate to Standard error. For (co)variance paths (S) matrix, all parameters and their standard errors are divided by the product of the model-implied standard deviations of the two variables that go into it. Regressions are multiplied by the model-implied standard deviation of the DV, and divided by the SD of the IV.

While the confidence limits/SEs of linear or non-linear combinations of parameters won't be a simple function of those parameters SEs, simple linear transformations shouldn't affect significance tests. The standard error of X/2 should be SE_X divided by 2.

The standardizeRAM function became what summary uses to standardize RAM models, and is discussed here:
http://openmx.psyc.virginia.edu/thread/718

neale's picture
Offline
Joined: 07/31/2009
To be mathematically consistent, some more algebra is needed

I don't think we should override the mathematical properties of standardized parameter estimates that are in common use elsewhere (i.e., other packages). At the very least, we should offer both options.

Judging statistical significance of a parameter by dividing its estimate by its standard error (a t-value) can give you different answers depending on how the model is constructed. It is an unfortunate property, but I don't think we should suggest that it doesn't exist by reporting SE's based on the unstandardized parameter estimates. As you and I discussed offline, there are other awkward properties of standardized estimates, such as the t-value being inconsistent depending on where in the model the estimate lies. For example, the standardized estimate of an autoregression parameter may not be the same thing for occasion 1 to occasion 2 as it is for occasion 2 to occasion 3, even though the (unstandardized) parameter has been constrained to be equal in both instances. Yes, there are lots of problems with standardized estimates.

This means that we would have to make a decision as to which instance of a standardized estimate and its standard error should be reported. I don't think this is a difficult decision - it is already made for cases where a parameter (label) appears multiple times, and this is indexed by matrix row and column (as well as label where available). But there is a little bit of matrix calculus to work out (the chain rule essentially) to get the SE's out. Any takers?