[Ncep.list.nems.announce] Feedback requested on replacing the NEMS build system.

Rocky Dunlap - NOAA Affiliate rocky.dunlap at noaa.gov
Tue Feb 7 16:56:35 UTC 2017

Since NEMS is a "unified" system, a big question here is what goes under
NEMS and what goes under each component and what goes under each app. Per
Gerhard's comment, a very nice feature of the NEMSAppBuilder is that if I
want to pull in an existing component into my app, I do not really have to
know how to build it--that was figured out once by a developer and that
knowledge is then put into the NEMSAppBuilder under the build_xxx()

Now, per Sam's point, having to update the NEMS trunk when the build of a
single component changes is not ideal. Instead, what about a hybrid
approach where:
- the build system framework is under NEMS  (something like NEMSAppBuilder)
- component-specific builds are under each component  (something like

The NEMSAppBuilder could then pull in the individual pieces from each
component--they would need to be in standard locations with standard
parameters, etc.  NEMS would only need to be updated if the build system
framework itself changed, which should not be too often.

Ideally, the build_xxx() piece could ALSO be used standalone, e.g., to
build a library or executable from a single component.

Keep in mind that as more things move out to the app or component level,
the less "unified" NEMS becomes. If everything goes back to separate builds
of separate components and apps, then eventually you will have a bunch of
separate systems. My understanding is the point of NEMS is to move away
from that.


On Tue, Feb 7, 2017 at 9:27 AM, Gerhard Theurich <theurich at sourcespring.net>

> Hi,
> Just for the immediate, I am wondering what the problem is with
> NEMSAppBuilder on WCOSS? Has anybody looked into it? I thought Patrick
> was building the coupled system on WCOSS, so it does surprise me that
> Moorthi cannot get GSM standalone working.
> Most important for the long term that I see is a _maintained_ system.
> Any system that is not maintained will break, especially if it has to
> cover a wide range of complex cases that keep changing (e.g. new
> components coming in). A maintained system will be able to change
> according to changing needs.
> One fundamental concept of the NEMSAppBuilder was to provide a single
> place where the build recipes of all the various components are kept.
> That is why it is part of the NEMS infrastructure: any app that uses a
> specific model component (even if it is a different revision of that
> component) should be able to just tell the AppBuilder that it wants to
> build that certain component, not have to know how to do this. (In the
> current AppBuilder this is implemented via the build_xxx() routines in
> the script. E.g. build_cice() will tell the system how to build CICE.
> These routines are typically just a few lines long and use a few
> variables to get the job done.)
> Another basic concept was that the configuration for an app should be
> provided by a very simple file, telling the app builder what components
> are to be built and linked together. Those are the *.appBuilder files.
> Obviously the implementation details, including how maybe a GUI is
> implemented could change, but I do think the fundamental concepts and
> how they are divided between app layer and NEMS layer are still valid.
> -Gerhard
> On 02/07/2017 06:04 AM, Shrinivas Moorthi wrote:
> > I can't compile standalone GSM on wcoss.  How do I do it without
> > blackbox (appbuilder)?
> > Thanks
> > Moorthi
> >
> > On 02/07/2017 08:46 AM, Samuel Trahan - NOAA Affiliate wrote:
> >> Hi all,
> >>
> >> The NEMS build system, NEMSAppBuilder, is causing a lot of problems,
> >> especially for the atmospheric model developers.  We plan on replacing
> >> the build system.  What do you want out of the new build system?  What
> >> do you dislike about the old one?
> >>
> >> Some questions to ponder:
> >>
> >> - How should the build system work internally?  Shell script?  Make?
> >> Cmake?
> >>
> >> - How should you run the build system?  Shell script?  Run "make?"
> >> Run a GUI?
> >>
> >> - When do we replace the build system?  Do we do it now, and risk
> >> breaking coupled systems? Do we wait until we can test it with the
> >> coupled applications?
> >>
> >> Feedback thus far:
> >>
> >> 1. It is difficult to navigate the 1900 line NEMSAppBuilder to figure
> >> out how to change the build commands.
> >>
> >> 2. Some users want to be able to manually build the NEMS without
> >> running an overarching script.
> >>
> >> 3. Some users want a simple graphical interface to select components
> >> and build the NEMS.  (Yes, there are users that want this.)
> >>
> >> 4. Sometimes, components compile with options that are incompatible
> >> with the linking options.  This is because each component has its own
> >> configuration system.  This causes problems, as we saw with FV3.
> >>
> >> 5. The logic for building a component is in the NEMS framework level.
> >> Any time a component's build system changes, the NEMS framework has to
> >> be updated, breaking applications that use older versions of the
> >> component. This forces applications to use non-trunk versions of the
> >>
> >> 6.  The NEMS/src/configure script contains application-specific
> >> logic.  This also forces applications to use non-trunk versions of the
> >> NEMS.
> >>
> >> Sincerely,
> >> Sam Trahan
> >>
> >>
> >> _______________________________________________
> >> Ncep.list.nems.announce mailing list
> >> Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
> >> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.
> list.nems.announce
> >
> >
> > --
> > Dr. Shrinivas Moorthi
> > Research Meteorologist
> > Global Climate and Weather Modeling Branch
> > Environmental Modeling Center / National Centers for Environmental
> Prediction
> > 5830 University Research Court - (W/NP23), College Park MD 20740 USA
> > Tel:(301)683-3718
> >
> >
> >
> > _______________________________________________
> > Ncep.list.nems.announce mailing list
> > Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
> > https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.
> list.nems.announce
> >
> _______________________________________________
> Ncep.list.nems.announce mailing list
> Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.nems.announce/attachments/20170207/c283fe97/attachment.html 

More information about the Ncep.list.nems.announce mailing list