<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
I think there are many ways this could be done and I'm not going to
advocate a particular solution,<br>
but CESM has had to deal with the same issues. The approach taken
by CESM is similar to what Rocky<br>
suggested. There is a central build script that calls the
"build_xxx" of each component and that<br>
build_xxx lives within the component. There are standard names and
locations for those so the central<br>
script can find them. In addition, in CESM all components use the
same Makefile to build. So the central build,<br>
depending on the app, calls the appropriate component build_xxx
routines and those execute a gmake call that<br>
all share a common Makefile with several variables defined that
allow each component to customize the<br>
build as needed. A common Makefile is not actually required but I
believe you also cannot allow each model<br>
to build with arbitrary options, the modules and some compile
settings need to be common.<br>
<br>
CESM has taken the same approach to setting up the "namelist" and
other input data. A central script<br>
calls into the "buildnml_xxx" for each component and that generates
the namelist (or equivalent text)<br>
input files as well as creates a list of "binary" input files needed
to run the model. Again, there are several<br>
common variables and conventions but the script that does most of
this work lives in each component.<br>
<br>
It is important to separate the "central" scripts from the
"component" scripts cleanly and that is not always<br>
easy to do, especially as they need to be flexible and extensible.
But as much as possible, that separation<br>
should be clear and the requirements on either side of that
separation should be clearly defined. I also<br>
agree that keeping it simple is better for most users and to keep
maintenance lower. But, there is also <br>
likely to be a point where the scripts work well enough and meet
user requirements well enough that<br>
most users won't need to know the details of the implementation.
Ultimately, I think that's where you want<br>
to end up. A clearly implemented set of scripts that the users
don't ultimately care much about.<br>
<br>
To provide just a few more details. As I said, the CESM build uses
a common Makefile. It also has some<br>
other common tools that the Makefile uses like a tool to build the
dependencies on the fly, that all components<br>
can use. Also, the component scripts that build the namelist for
each component also generate a list of<br>
input files needed. We then have a tool that checks that all those
file exist and are ready to use. If not,<br>
the job will report a problem before running. That's been very
handy. <br>
<br>
thanks,<br>
<br>
tony.......<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 2/7/17 8:56 AM, Rocky Dunlap - NOAA
Affiliate wrote:<br>
</div>
<blockquote
cite="mid:CAL19TQjJ5ZuLbeweoaCCRvEXaNrKhUyA-MJ-=vkhhWbFcajD_Q@mail.gmail.com"
type="cite">
<div dir="ltr">Since NEMS is a "unified" system, a big question
here is what goes under NEMS and what goes under each component
and what goes under each app. Per Gerhard's comment, a very nice
feature of the NEMSAppBuilder is that if I want to pull in an
existing component into my app, I do not really have to know how
to build it--that was figured out once by a developer and that
knowledge is then put into the NEMSAppBuilder under the
build_xxx() routines.
<div><br>
</div>
<div>Now, per Sam's point, having to update the NEMS trunk when
the build of a single component changes is not ideal. Instead,
what about a hybrid approach where:</div>
<div>- the build system framework is under NEMS (something like
NEMSAppBuilder)</div>
<div>- component-specific builds are under each component
(something like build_xxx())</div>
<div><br>
</div>
<div>The NEMSAppBuilder could then pull in the individual pieces
from each component--they would need to be in standard
locations with standard parameters, etc. NEMS would only need
to be updated if the build system framework itself changed,
which should not be too often.</div>
<div><br>
</div>
<div>Ideally, the build_xxx() piece could ALSO be used
standalone, e.g., to build a library or executable from a
single component.</div>
<div> </div>
<div>Keep in mind that as more things move out to the app or
component level, the less "unified" NEMS becomes. If
everything goes back to separate builds of separate components
and apps, then eventually you will have a bunch of separate
systems. My understanding is the point of NEMS is to move away
from that.</div>
<div><br>
</div>
<div>Rocky</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Feb 7, 2017 at 9:27 AM, Gerhard
Theurich <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:theurich@sourcespring.net" target="_blank">theurich@sourcespring.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Just for the immediate, I am wondering what the problem is
with<br>
NEMSAppBuilder on WCOSS? Has anybody looked into it? I
thought Patrick<br>
was building the coupled system on WCOSS, so it does
surprise me that<br>
Moorthi cannot get GSM standalone working.<br>
<br>
Most important for the long term that I see is a
_maintained_ system.<br>
Any system that is not maintained will break, especially if
it has to<br>
cover a wide range of complex cases that keep changing (e.g.
new<br>
components coming in). A maintained system will be able to
change<br>
according to changing needs.<br>
<br>
One fundamental concept of the NEMSAppBuilder was to provide
a single<br>
place where the build recipes of all the various components
are kept.<br>
That is why it is part of the NEMS infrastructure: any app
that uses a<br>
specific model component (even if it is a different revision
of that<br>
component) should be able to just tell the AppBuilder that
it wants to<br>
build that certain component, not have to know how to do
this. (In the<br>
current AppBuilder this is implemented via the build_xxx()
routines in<br>
the script. E.g. build_cice() will tell the system how to
build CICE.<br>
These routines are typically just a few lines long and use a
few<br>
variables to get the job done.)<br>
<br>
Another basic concept was that the configuration for an app
should be<br>
provided by a very simple file, telling the app builder what
components<br>
are to be built and linked together. Those are the
*.appBuilder files.<br>
Obviously the implementation details, including how maybe a
GUI is<br>
implemented could change, but I do think the fundamental
concepts and<br>
how they are divided between app layer and NEMS layer are
still valid.<br>
<span class="HOEnZb"><font color="#888888"><br>
-Gerhard<br>
</font></span>
<div class="HOEnZb">
<div class="h5"><br>
On 02/07/2017 06:04 AM, Shrinivas Moorthi wrote:<br>
> I can't compile standalone GSM on wcoss. How do I
do it without<br>
> blackbox (appbuilder)?<br>
> Thanks<br>
> Moorthi<br>
><br>
> On 02/07/2017 08:46 AM, Samuel Trahan - NOAA
Affiliate wrote:<br>
>> Hi all,<br>
>><br>
>> The NEMS build system, NEMSAppBuilder, is
causing a lot of problems,<br>
>> especially for the atmospheric model
developers. We plan on replacing<br>
>> the build system. What do you want out of the
new build system? What<br>
>> do you dislike about the old one?<br>
>><br>
>> Some questions to ponder:<br>
>><br>
>> - How should the build system work internally?
Shell script? Make?<br>
>> Cmake?<br>
>><br>
>> - How should you run the build system? Shell
script? Run "make?"<br>
>> Run a GUI?<br>
>><br>
>> - When do we replace the build system? Do we
do it now, and risk<br>
>> breaking coupled systems? Do we wait until we
can test it with the<br>
>> coupled applications?<br>
>><br>
>> Feedback thus far:<br>
>><br>
>> 1. It is difficult to navigate the 1900 line
NEMSAppBuilder to figure<br>
>> out how to change the build commands.<br>
>><br>
>> 2. Some users want to be able to manually build
the NEMS without<br>
>> running an overarching script.<br>
>><br>
>> 3. Some users want a simple graphical interface
to select components<br>
>> and build the NEMS. (Yes, there are users that
want this.)<br>
>><br>
>> 4. Sometimes, components compile with options
that are incompatible<br>
>> with the linking options. This is because each
component has its own<br>
>> configuration system. This causes problems, as
we saw with FV3.<br>
>><br>
>> 5. The logic for building a component is in the
NEMS framework level.<br>
>> Any time a component's build system changes,
the NEMS framework has to<br>
>> be updated, breaking applications that use
older versions of the<br>
>> component. This forces applications to use
non-trunk versions of the NEMS.<br>
>><br>
>> 6. The NEMS/src/configure script contains
application-specific<br>
>> logic. This also forces applications to use
non-trunk versions of the<br>
>> NEMS.<br>
>><br>
>> Sincerely,<br>
>> Sam Trahan<br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> Ncep.list.nems.announce mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:Ncep.list.nems.announce@lstsrv.ncep.noaa.gov">Ncep.list.nems.announce@<wbr>lstsrv.ncep.noaa.gov</a><br>
>> <a moz-do-not-send="true"
href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce"
rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.nems.announce</a><br>
><br>
><br>
> --<br>
> Dr. Shrinivas Moorthi<br>
> Research Meteorologist<br>
> Global Climate and Weather Modeling Branch<br>
> Environmental Modeling Center / National Centers
for Environmental Prediction<br>
> 5830 University Research Court - (W/NP23), College
Park MD 20740 USA<br>
> Tel:<a moz-do-not-send="true"
href="tel:%28301%29683-3718" value="+13016833718">(301)683-3718</a><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Ncep.list.nems.announce mailing list<br>
> <a moz-do-not-send="true"
href="mailto:Ncep.list.nems.announce@lstsrv.ncep.noaa.gov">Ncep.list.nems.announce@<wbr>lstsrv.ncep.noaa.gov</a><br>
> <a moz-do-not-send="true"
href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce"
rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.nems.announce</a><br>
><br>
______________________________<wbr>_________________<br>
Ncep.list.nems.announce mailing list<br>
<a moz-do-not-send="true"
href="mailto:Ncep.list.nems.announce@lstsrv.ncep.noaa.gov">Ncep.list.nems.announce@<wbr>lstsrv.ncep.noaa.gov</a><br>
<a moz-do-not-send="true"
href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce"
rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.nems.announce</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Ncep.list.nems.announce mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Ncep.list.nems.announce@lstsrv.ncep.noaa.gov">Ncep.list.nems.announce@lstsrv.ncep.noaa.gov</a>
<a class="moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce">https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce</a>
</pre>
</blockquote>
<br>
</body>
</html>