snow and snowfall now optional

3 replies [Last post]
mspiegel's picture
Offline
Joined: 07/31/2009

As of revision 808 in the subversion repository, snow and snowfall are now optional packages. If they have been loaded prior to running a model, then independent submodels will be executed in parallel. Otherwise sequential execution occurs. We can update the installer in the next closed beta release (0.1.4) to not download these packages.

tbates's picture
Offline
Joined: 07/31/2009
In the kind of setup we have

In the kind of setup we have with a supermodel whose objective is the sum of its sub-model's objectives, and submodels which do algebra on matrices in the supermodel, is there any scope for parallelisation?

neale's picture
Offline
Joined: 07/31/2009
If I understand you

If I understand you correctly, there should be some scope but at a finer granularity than if the submodels (aka groups in Mx1 parlance) are independent. So, it would be possible to evaluate the submodels separately for each set of parameter estimates, and to combine the threads on the pass back to the optimizer with the sum of the fit functions of each submodel. Indeed, this could significantly speed up single-group jobs simply by cutting the dataset into as many subsets as there are processors, and creating a separate group for each.

tbates's picture
Offline
Joined: 07/31/2009
Yes, that's what I

Yes, that's what I meant.
Certainly having each group automatically evaluate on a separate core if available passing the fit functions to the optimizer where they need to be aggregated would be a massive speed up.

Certainly seeing just 7 of 8 cores running at 0% is silicon sadness :-)