Suggestions

Original list from Ken here

Suggestions

  1. Easy boot-based CIs
  2. Implicit CIs on intervals=T
  3. Include z and p for Ho that parameter = 0
  4. Include more fit indices

Done In OpenMx 2.0

  1. Include Fixed Parameters in summary
  2. Include CI for MSEA
  3. Exclude time by default
  4. Include variances in output
  5. Auto labelling
  6. Easy standardized output

4.) Easy boot-based CIs

Having easy to obtain bootstrap confidence interval integration via the boot package would be great. For example, options such as ciBoot=TRUE (and then compute both BCa and percentile) and B=1000 (for the number of bootstrap replications) could be included and the rest of the process done behind the scenes.

Pros

Good suggestion - currently useable with OpenMx but not convenient (esp. comp classic Mx).

Cons

Needs some example code?

Result

Job for helper functions? One now lives on the forums

7.) Implicit CIs on intervals=T

Consider including the confidence intervals whenever intervals=TRUE. Having to specify a new mxCI object separately is awkward (especially since an option seems to exists via intervals=TRUE, even though that must be specified when mxCI is used). I know there was some discussion about how to implement confidence intervals on the Wiki among the team members, but it seems like there is an extra step one has to do here and thus simplification would be great.

Pros

Seems helpful: Intervals=T should add mxCI(everything)

Cons

Don't want this to happen by accident: Might be nicer to give a helpful warning when Intervals= TRUE but is.null(model$intervals)

Work

Result

This is implemented as umx::confint() in umx

8.) Include z and p for Ho that parameter = 0

Including z-values as well as the corresponding p-values for the test of the null hypothesis (that the parameter equals zero) in the output for parameter estimates would be great (and is standard practice in other programs). Or, if not including them by default, a simple option to do this: pvalues=TRUE.

Pros

Cons

Are these estimates inherently biased?

See this paperalso

9.) Include more fit indices

I know some folks don't like fit indices, but others do. It would be nice if there was a way to easily obtain several of the popular fit indices. RMSEA is produced by default, but perhaps some helper function(s) could be used to easily obtain other fit indices.

Pros

Cons

Issues remain for FIML

Result

AIC, CFI and TLI, and RMSEA are in the 1.4 release. What else do people want?

  • your suggestion here

Note: many of these implemented in other packages that can use mxModels as input

Done for OpenMx 2.0

1.) Include Fixed Parameters in summary

I think fixed parameters should be included as part of the output. If someone has only the output, they will not necessarily know what paths were fixed and what those paths were fixed to. It seems it would be easy to handle this be including as part of the output a row that has the name (where "labels" was used) in the mxPath call and the estimate would be the fixed value and the "Std.Error" as NA or something else to denote that it is not computed (because it is fixed).

Pros

Easy for path models and worth doing.

Cons

Not so easy for matrix models: could print all values fixed at values other than zero?

1) I (Mike Neale) don't, for several reasons. First, however, a qualifier: is it proposed that only paths that the user has declared would be reported? There are implicit zero paths all over the place so one would have to assume that the path has been declared in an mxPath() statement. So if we ignore all the implicit ones, there remains a bit of an issue with respect to the Hessian - does this also get padded with zero columns and rows wherever there's a fixed path? How would evaluation of this matrix work - say looking at its eigenvalues? Or do we have a different length parameter estimate vector and derivative/Hessian dimensions? This gets a bit messy.

One advantage I would see is that if one were filling out a table with different estimates in which some of the parameters were fixed in certain models, then the table would be easy to generate. Therefore, some helper function which takes one "Super" model in which all parameters are free, and inserts the values of the "missing" matrix elements would seem viable without incurring complexities described above.

Result

This is provided in OpenMx 2.0 as mxStandardizeRAMpaths which gives separate values for all estimated paths constrained to be equal (rather than just for the first one encountered.

2.) Include CI for RMSEA

Confidence intervals for the RMSEA should be included. This can be done via the MBESS package as:

	require(MBESS)
	ci.rmsea(rmsea=.055, df=35, N=500, conf.level = 0.95)

But, the functions ci.rmsea uses can be pulled out and included in OpenMx so as not to rely on another package.

Pros

Cons

Not scintillating, being simply a function of N and df. Also, FIML of raw data with missing values may not have a consistent "N".

Result

In the beta for OpenMx 2.0

3.) Exclude time by default

I don't see much of a benefit of including the "elapsed time" and all of the calculation timers in the output. Perhaps if someone wants to request it, but it seems unnecessary in general.

Pros

A good space saver:perhaps have a "verbose" toggle which can switch on or off a whole set of non-core reports?

By arranging this as a table, it can be done in just two rows, and left on by default?

Cons

Helpful for time-debugging, so don't remove the possibility of summary calls returning this info.

Result

This an much removed from summary() in OpenMx 2.0 (just say summary(..., verbose = TRUE) to bring it all back.)

5.) Include variances in output

For the summary of the variables that is part of the output, I think the variance or standard deviation should also be included. I think OpenMx calls the R function summary(), which, for whatever reason does not have the variance or standard deviation included in its output (I've always wondered why it didn't include the variance or the standard deviation).

Pros

Should include: Min, Max, Mean, Median, Var and SD and perhaps skew/kurtosis measures

Cons

Do people use this summary info?

Result

Removed from summary() in OpenMx 2.0. Use a function like psych:::describe

6.) Auto labelling

Building some intelligence into the labeling of estimates so that so many labels do not have to be inserted manually would be great. Default values could be the variable name(s) with things like "coef" or "var" or "cor" in front to identify what they are, but the users could specify their own labels.

Pros

Cons

Hard to do this in an intuitive and non-unwieldly way

Need to maintain ability to use NA to force paths to be distinct

Result

Use the umx::umxLabel() helper function.

10.) Easy standardized output

Perhaps this exists but if so I don't know about it and couldn't find anything via a search: consider including an easy way to obtain a standardized solution. For example, for a model fitted to raw data or a covariance matrix, having an option such as standardizedSolution=TRUE would help folks interpret their output. This is different than using a correlation matrix from the outset, as the distribution theory differs for raw vs. standardized data (due to the fact that the process of standardization is based on random variables, namely the variance of the variables). Ideally the appropriate standard errors could be given (RAMONA did this in Systat) for the standardized solution. But, even if only the point estimates were given, that would be helpful.

Pros

Cons

Hard to do for arbitrary models. Done for RAMpath

Result

Included in OpenMx 2.0 as mxStandardizeRAMpaths This also includes correctly calculated S.Es

Also, and initial solution was provided here http://openmx.psyc.virginia.edu/thread/718