checkpointing & CIs

3 replies [Last post]
smedland's picture
Offline
Joined: 08/04/2009

Hi
Would it be possible if you are running a model with checkpointing & CI to get it to check point when it hits a solution for an upper or lower CI?
Currently if you running checkpointing & CI it checkpoints when it hits the 'solution' - regardless of the minutes/iterations since the last checkpoint
If it could do the same thing when it finds a CI it would be a great help
thanks
Sarah

mspiegel's picture
Offline
Joined: 07/31/2009
No problem. I checked in a

No problem. I checked in a change to the subversion repository that writes to the checkpoint file when confidence interval calculations are performed. To preserve backwards-compatibility with the existing checkpoint file format, I took advantage of the date and time field (which is a string) and use that column to display which checkpoint is displayed.

For each confidence interval calculation, the last row of that subset of the table represents the optimizer state at the conclusion of the calculation.

iterations	timestamp	objective	"a"	"c"	"e"	"mean"
18 "Oct 26 2012 03:28:41 PM" 4067.66290   0.78569   0.00000   0.41599  21.39293
18 "common.A[0, 0] begin lower interval" NA NA NA NA NA 
54 "Oct 26 2012 03:28:41 PM"   0.54836  -0.74049  -0.17506   0.42290  21.39249
54 "common.A[0, 0] begin upper interval" NA NA NA NA NA 
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "Oct 26 2012 03:28:41 PM"  -0.68390   0.82700   0.00000   0.40948  21.39305
82 "common.C[0, 0] begin lower interval" NA NA NA NA NA 
89 "Oct 26 2012 03:28:41 PM"   0.00000   0.78215  -0.00000   0.39373  21.39110
89 "Oct 26 2012 03:28:41 PM"   0.00000   0.78215  -0.00000   0.39373  21.39110
89 "common.C[0, 0] begin upper interval" NA NA NA NA NA 
137 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "Oct 26 2012 03:28:41 PM"  -0.05222   0.75618  -0.22859   0.41812  21.39236
138 "common.E[0, 0] begin lower interval" NA NA NA NA NA 
231 "Oct 26 2012 03:28:41 PM"   0.15375   0.79682  -0.00000  -0.39211  21.39306
231 "common.E[0, 0] begin upper interval" NA NA NA NA NA 
263 "Oct 26 2012 03:28:41 PM"  -0.19561   0.77294   0.00000   0.44229  21.39279

neale's picture
Offline
Joined: 07/31/2009
Really cool

It turns out that this information can be really useful, because it returns all the other parameter estimates at the conclusion of optimization to find a confidence interval! These estimates can help diagnose 'weird' or unexpected behavior (such as sign-flipping) when estimating confidence intervals.

smedland's picture
Offline
Joined: 08/04/2009
wonderful!!! looks

wonderful!!! looks great
thanks
Sarah