Suggestions

Original list from Ken here

Suggestions

  1. Include Fixed Parameters in summary
  2. Include CI for MSEA
  3. Exclude time by default
  4. Easy boot-based CIs
  5. Include variances in output
  6. Auto labelling
  7. Implicit CIs on intervals=T
  8. Include z and p for Ho that parameter = 0
  9. Include more fit indices
  10. Easy standarized output

1.) Include Fixed Parameters in summary

I think fixed parameters should be included as part of the output. If someone has only the output, they will not necessarily know what paths were fixed and what those paths were fixed to. It seems it would be easy to handle this be including as part of the output a row that has the name (where "labels" was used) in the mxPath call and the estimate would be the fixed value and the "Std.Error" as NA or something else to denote that it is not computed (because it is fixed).

Pros

Easy for path models and worth doing.

Cons

Not so easy for matrix models: could print all values fixed at values other than zero?

1) I (Mike Neale) don't, for several reasons. First, however, a qualifier: is it proposed that only paths that the user has declared would be reported? There are implicit zero paths all over the place so one would have to assume that the path has been declared in an mxPath() statement. So if we ignore all the implicit ones, there remains a bit of an issue with respect to the Hessian - does this also get padded with zero columns and rows wherever there's a fixed path? How would evaluation of this matrix work - say looking at its eigenvalues? Or do we have a different length parameter estimate vector and derivative/Hessian dimensions? This gets a bit messy.

One advantage I would see is that if one were filling out a table with different estimates in which some of the parameters were fixed in certain models, then the table would be easy to generate. Therefore, some helper function which takes one "Super" model in which all parameters are free, and inserts the values of the "missing" matrix elements would seem viable without incurring complexities described above.

Result

2.) Include CI for RMSEA

Confidence intervals for the RMSEA should be included. This can be done via the MBESS package as:

	require(MBESS)
	ci.rmsea(rmsea=.055, df=35, N=500, conf.level = 0.95)

But, the functions ci.rmsea uses can be pulled out and included in OpenMx so as not to rely on another package.

Pros

Cons

Not scintillating, being simply a function of N and df. Also, FIML of raw data with missing values may not have a consistent "N".

Result

Now in the cue to be implemented

3.) Exclude time by default

I don't see much of a benefit of including the "elapsed time" and all of the calculation timers in the output. Perhaps if someone wants to request it, but it seems unnecessary in general.

Pros

A good space saver:perhaps have a "verbose" toggle which can switch on or off a whole set of non-core reports?

By arranging this as a table, it can be done in just two rows, and left on by default?

Cons

Helpful for time-debugging, so don't remove the possibility of summary calls returning this info.

Result

4.) Easy boot-based CIs

Having easy to obtain bootstrap confidence interval integration via the boot package would be great. For example, options such as ciBoot=TRUE (and then compute both BCa and percentile) and B=1000 (for the number of bootstrap replications) could be included and the rest of the process done behind the scenes.

Pros

Good suggestion - currently useable with OpenMx but not convenient (esp. comp classic Mx).

Cons

Needs some example code?

Result

Job for helper functions?

5.) Include variances in output

For the summary of the variables that is part of the output, I think the variance or standard deviation should also be included. I think OpenMx calls the R function summary(), which, for whatever reason does not have the variance or standard deviation included in its output (I've always wondered why it didn't include the variance or the standard deviation).

Pros

Should include: Min, Max, Mean, Median, Var and SD and perhaps skew/kurtosis measures

Cons

Do people use this summary info?

Result

6.) Auto labelling

Building some intelligence into the labeling of estimates so that so many labels do not have to be inserted manually would be great. Default values could be the variable name(s) with things like "coef" or "var" or "cor" in front to identify what they are, but the users could specify their own labels.

Pros

Cons

Hard to do this in an intuitive and non-unweildly way

Need to maintain ability to use NA to force paths to be distinct

Result

Job for helper functions

7.) Implicit CIs on intervals=T

Consider including the confidence intervals whenever intervals=TRUE. Having to specify a new mxCI object separately is awkward (especially since an option seems to exists via intervals=TRUE, even though that must be specified when mxCI is used). I know there was some discussion about how to implement confidence intervals on the Wiki among the team members, but it seems like there is an extra step one has to do here and thus simplification would be great.

Pros

Seems helpful: Intervals=T should add mxCI(everything)

Cons

Don't want this to happen by accident

Work

Result

8.) Include z and p for Ho that parameter = 0

Including z-values as well as the corresponding p-values for the test of the null hypothesis (that the parameter equals zero) in the output for parameter estimates would be great (and is standard practice in other programs). Or, if not including them by default, a simple option to do this: pvalues=TRUE.

Pros

Cons

Are these estimates inherently biased?

See this paper
also

9.) Include more fit indices

I know some folks don't like fit indices, but others do. It would be nice if there was a way to easily obtain several of the popular fit indices. RMSEA is produced by default, but perhaps some helper function(s) could be used to easily obtain other fit indices.

Pros

Cons

Issues remain for FIML

Result

CFI and TLI are in the 1.1 release. What else do people want?

  • your suggestion here

10.) Easy standarized output

Perhaps this exists but if so I don't know about it and couldn't find anything via a search: consider including an easy way to obtain a standardized solution. For example, for a model fitted to raw data or a covariance matrix, having an option such as standarizedSolution=TRUE would help folks interpret their output. This is different than using a correlation matrix from the outset, as the distribution theory differers for raw vs. standardized data (due to the fact that the process of standardization is based on random variables, namely the variance of the variables). Ideally the appropriate standard errors could be given (RAMONA did this in Systat) for the standardized solution. But, even if only the point estimates were given, that would be helpful.

Pros

Cons

Hard to do for arbitrary models. Done for RAMpath

Result

Check out http://openmx.psyc.virginia.edu/thread/718