[Ncep.list.nems.announce] Feedback requested on replacing the NEMS build system.

Tue Feb 7 17:35:40 UTC 2017

Sam,

I understand. Just that it makes building a unified system that spans 
atm-only to complex coupled systems very difficult. Further there is 
also a mandate to the larger community. And it wasn't like the old 
(pre-AppBuilder) system was all that great. E.g. folks could not just 
take it to a different system and expect to get it to build unless you 
had good knowledge of the system - and that was just for atm-only. 
Considering all of the different coupled systems at EMC, the atm-only 
development, and the larger community (e.g. represented in the SAWG), I 
am honestly scared of how the unanimous consent will look like (if even 
achievable). But I know you know all that, too.

-Gerhard

On 02/07/2017 09:21 AM, Samuel Trahan - NOAA Affiliate wrote:
> Gerhard,
>
> The ideas behind the NEMSAppBuilder are fundamentally sound.
> Unfortunately, the underlying implementation of it and the surrounding
> system (conf files, module files, nems/src/configure) make it hard to
> develop and maintain.  The largest problem is not a technical issue, it
> is a problem with the EMC corporate culture.
>
> The atmospheric standalone developers refuse to use or maintain the
> NEMSAppBuilder, which is making it difficult for the coupled model
> developers to update their atmospheric component.  Unfortunately, EMC
> has no command structure (except on paper) and developers are free to do
> as they please.  That means we need unanimous consent about the NEMS
> build and test systems in order to move forward.
>
> Sincerely,
> Sam Trahan
>
> On Tue, Feb 7, 2017 at 11:27 AM, Gerhard Theurich
> <theurich at sourcespring.net <mailto:theurich at sourcespring.net>> wrote:
>
>     Hi,
>
>     Just for the immediate, I am wondering what the problem is with
>     NEMSAppBuilder on WCOSS? Has anybody looked into it? I thought Patrick
>     was building the coupled system on WCOSS, so it does surprise me that
>     Moorthi cannot get GSM standalone working.
>
>     Most important for the long term that I see is a _maintained_ system.
>     Any system that is not maintained will break, especially if it has to
>     cover a wide range of complex cases that keep changing (e.g. new
>     components coming in). A maintained system will be able to change
>     according to changing needs.
>
>     One fundamental concept of the NEMSAppBuilder was to provide a single
>     place where the build recipes of all the various components are kept.
>     That is why it is part of the NEMS infrastructure: any app that uses a
>     specific model component (even if it is a different revision of that
>     component) should be able to just tell the AppBuilder that it wants to
>     build that certain component, not have to know how to do this. (In the
>     current AppBuilder this is implemented via the build_xxx() routines in
>     the script. E.g. build_cice() will tell the system how to build CICE.
>     These routines are typically just a few lines long and use a few
>     variables to get the job done.)
>
>     Another basic concept was that the configuration for an app should be
>     provided by a very simple file, telling the app builder what components
>     are to be built and linked together. Those are the *.appBuilder files.
>     Obviously the implementation details, including how maybe a GUI is
>     implemented could change, but I do think the fundamental concepts and
>     how they are divided between app layer and NEMS layer are still valid.
>
>     -Gerhard
>
>     On 02/07/2017 06:04 AM, Shrinivas Moorthi wrote:
>      > I can't compile standalone GSM on wcoss.  How do I do it without
>      > blackbox (appbuilder)?
>      > Thanks
>      > Moorthi
>      >
>      > On 02/07/2017 08:46 AM, Samuel Trahan - NOAA Affiliate wrote:
>      >> Hi all,
>      >>
>      >> The NEMS build system, NEMSAppBuilder, is causing a lot of problems,
>      >> especially for the atmospheric model developers.  We plan on
>     replacing
>      >> the build system.  What do you want out of the new build
>     system?  What
>      >> do you dislike about the old one?
>      >>
>      >> Some questions to ponder:
>      >>
>      >> - How should the build system work internally?  Shell script?  Make?
>      >> Cmake?
>      >>
>      >> - How should you run the build system?  Shell script?  Run "make?"
>      >> Run a GUI?
>      >>
>      >> - When do we replace the build system?  Do we do it now, and risk
>      >> breaking coupled systems? Do we wait until we can test it with the
>      >> coupled applications?
>      >>
>      >> Feedback thus far:
>      >>
>      >> 1. It is difficult to navigate the 1900 line NEMSAppBuilder to
>     figure
>      >> out how to change the build commands.
>      >>
>      >> 2. Some users want to be able to manually build the NEMS without
>      >> running an overarching script.
>      >>
>      >> 3. Some users want a simple graphical interface to select components
>      >> and build the NEMS.  (Yes, there are users that want this.)
>      >>
>      >> 4. Sometimes, components compile with options that are incompatible
>      >> with the linking options.  This is because each component has
>     its own
>      >> configuration system.  This causes problems, as we saw with FV3.
>      >>
>      >> 5. The logic for building a component is in the NEMS framework
>     level.
>      >> Any time a component's build system changes, the NEMS framework
>     has to
>      >> be updated, breaking applications that use older versions of the
>      >> component. This forces applications to use non-trunk versions of
>     the NEMS.
>      >>
>      >> 6.  The NEMS/src/configure script contains application-specific
>      >> logic.  This also forces applications to use non-trunk versions
>     of the
>      >> NEMS.
>      >>
>      >> Sincerely,
>      >> Sam Trahan
>      >>
>      >>
>      >> _______________________________________________
>      >> Ncep.list.nems.announce mailing list
>      >> Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
>     <mailto:Ncep.list.nems.announce at lstsrv.ncep.noaa.gov>
>      >>
>     https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce
>     <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce>
>      >
>      >
>      > --
>      > Dr. Shrinivas Moorthi
>      > Research Meteorologist
>      > Global Climate and Weather Modeling Branch
>      > Environmental Modeling Center / National Centers for
>     Environmental Prediction
>      > 5830 University Research Court - (W/NP23), College Park MD 20740 USA
>      > Tel:(301)683-3718
>      >
>      >
>      >
>      > _______________________________________________
>      > Ncep.list.nems.announce mailing list
>      > Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
>     <mailto:Ncep.list.nems.announce at lstsrv.ncep.noaa.gov>
>      >
>     https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce
>     <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce>
>      >
>     _______________________________________________
>     Ncep.list.nems.announce mailing list
>     Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
>     <mailto:Ncep.list.nems.announce at lstsrv.ncep.noaa.gov>
>     https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce
>     <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce>
>
>