<div dir="ltr"><div>Hi all,</div><div><br></div><div>I have gotten some excellent feedback on the new NEMS build system and made the necessary changes. I'm waiting for one FV3-MOM6-CICE5 commit that will go in soon. Then I'll start run the big, multi-machine, multi-app test, one last time before committing. I'm hoping to commit Friday afternoon, but it depends in part on queue time and on the timing of the FV3-MOM6-CICE5 commit.<br></div><div><br></div><div>You can expect another build system commit in a week or so, to add some portability changes.</div><div><br></div><div>Sincerely,</div><div>Sam Trahan<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, 9 Oct 2018 at 13:00, Samuel Trahan - NOAA Affiliate <<a href="mailto:samuel.trahan@noaa.gov">samuel.trahan@noaa.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>NEMS developers,</div><div><br></div><div>There are two commits coming into the NEMS git master soon. We're expecting to commit them early next week, hopefully Monday.<br></div><div><br></div><div>1. Support for new ESMF 8 beta snapshots which break backward compatibility to ESMF 7.x and earlier. This requires adding an #ifdef in NEMS code that triggers on the ESMF version. The NEMS build system cannot support this, so we need a new NEMS build system.</div><div><br></div><div>2. A new NEMS build system. It re-expresses the NEMSAppBuilder in GNU Make.</div><div><br></div><div>We're working on merging change #1 into change #2, and we plan on committing them at the same time. <br></div><div><br></div><div>The remainder of this email details the new NEMS build system.</div><div><br></div><div>Sincerely,</div><div>Sam Trahan<br></div><div><br></div><div><b><span style="color:rgb(0,0,255)">---------------------------------------------------------------------------------------------------------------<b><span style="color:rgb(0,0,255)">-----------------------------------------------</span></b></span></b><br></div></div><div dir="ltr"><br></div><div style="text-align:center"><b><font size="4"><i>The New NEMS Build System</i></font></b></div><div style="text-align:center"><font size="4"><i><br></i></font></div><div style="text-align:center"><font size="4"><i>Overview</i></font><br></div><div dir="ltr"><br></div><div>This is not the next-generation build system that will save the world; it is just a cleaned-up version of the old one, re-expressed in GNU Make. The NEMSAppBuilder GUI is gone, but other than this, the front-end is backward-compatible. You will not notice any changes to the build system if you're just building or running the NEMS or its Supported apps via the supported methods. Specifically, the NEMSCompsetRun, NEMSAppBuilder, NEMSfv3gfs/tests, and NEMSGSM/oldtests will all work just like they did before. This change is mostly to benefit those of us that do need to access or modify the internals of the NEMS build system. It is intended to be easier to work with and easier to port than the old one.<br></div><div><br></div><div>You will only be impacted by this commit if you have your own build system, if you use a component that is not in a Supported app, or if you intend to modify or port the build system. For example, if you want to add components or change the component build rules, you will need to know the internals of the new system. Also, the NEMS/src/configure and NEMS/src/makefile are gone. The new build system moves the logic of NEMSAppBuilder into a top-level GNUmakefile and src/incmake/*.mk files for details of the build. A description of the step-by-step logic, and purpose of the lower-level makefiles, can be found in the top-level GNUmakefile.<br></div><div dir="ltr"><br></div><div>The build system is only fully tested for Supported apps and their target platforms, listed below. That means some actively-developed components and platforms are not tested. All components are present in the build system, and were converted from shell to GNU make. The platform logic should be able to handle any arbitrary platform, but only certain platforms are tested. We will push to the git master before the non-Supported features are tested, so you can expect a few more commits over the coming weeks to fix any non-Supported features.</div><div><br></div><div>I would like to re-iterate that the NEMSAppBuilder GUI mode is gone forever, by request of EMC management. There are superior GUI build systems and Integrated Development Environments (IDE) available. I prefer Emacs, but saner people seem to like Eclipse, which can be run on most (all?) NOAA clusters. Now that the build system is written in Make, your favorite IDE will be better able to understand it.<br></div><div><br></div><div>Read on for details...<br></div><div><br></div><div><br></div><div style="text-align:center"><font size="4"><i>Where to get the New NEMS Build System</i></font><br></div><div dir="ltr"><br><div>You can find the new build system in the "rewrite-appbuilder" branches of these apps:<br></div><div><br></div><div>git clone -b rewrite-appbuilder gerrit:NEMSfv3gfs<br>git clone -b rewrite-appbuilder gerrit:EMC_GSM-MOM5-CICE5<br>git clone -b rewrite-appbuilder gerrit:EMC_NEMSGSM<br>git clone -b rewrite-appbuilder gerrit:EMC_FV3-WW3<br>git clone -b rewrite-appbuilder gerrit:EMC_ATM-WW3<br>git clone -b rewrite-appbuilder gerrit:EMC_HYCOM-GSM-CICE<br>git clone -b rewrite-appbuilder gerrit:EMC_FV3-MOM6-CICE5</div><div><br></div><div>The NEMS branch is rewrite-appbuilder for all apps except GSM-MOM5-CICE5. That app has fallen behind and cannot use the latest NEMS. There is a rewrite-appbuilder-GSM-MOM5-CICE5 branch for that app's NEMS submodule.</div><div><br></div><div><br></div><div style="text-align:center"><font size="4"><i>What is Tested, Untested, or Removed?</i></font><br></div><div><br></div><div>This is what has been tested and confirmed to work:<br></div><div><br></div><div><div> - theia.intel, t/u jet, WCOSS Phase 1 & 2, WCOSS Cray, GAEA C3</div><div> - These apps: NEMSfv3gfs, GSM-MOM5-CICE5, NEMSGSM, FV3-WW3, ATM-WW3, HYCOM-GSM-CICE, and FV3-MOM6-CICE5</div><div> - These components: FV3, FMS, GSM, MOM5, MOM6, CICE5, WW3, HYCOM<br></div><div> - NEMSAppBuilder</div><div> - NEMSCompsetRun</div><div> - tests/compile.sh</div><div> - tests/rt.sh</div><div> - direct build via "cd NEMS ; make ... options ..."<br></div><div> - Building FV3 with CCPP, but not running it.<br></div><br></div><div>This is being tested now:<br></div><div><br></div> - NEMSfv3gfs on WCOSS Dell -- we have no NEMSfv3gfs regression test system here yet, and no other apps target this platform<br></div><div></div><div dir="ltr"><div><div> - NEMSGSM/oldtests/rt.sh -- works, except for GOCART<br></div><div><div></div></div> - GSM GOCART support<br></div><div dir="ltr"><div><div></div></div></div><br><div>This is untested:</div><div><br></div> - Running CCPP to test if its build is correct</div> - PGI & GNU compilers<br><div dir="ltr"><div><div> - Mac OS/X<br></div><div> - Clusters outside of NOAA (Odin, Cheyenne, etc.)</div> - These components' build rules have been converted to the new build system but are untested: DATM, GSDCHEM, IMP, IPE, KISS, LIS, NOAH, NOAHMP, WRFHYDRO.<br></div> - Apps that are not Supported (other than GSM-MOM5-CICE5)</div><div dir="ltr"><br></div><div>This is intentionally removed:</div><div><br></div><div> - GUI NEMSAppBuilder. This is permanently removed by request of EMC management. May it rest in peace.<br></div><div></div><div><br></div><div>This is moved or renamed:<br></div><div><br></div><div> - NEMS/src/configure is gone; configuration is done via "make configure"<br></div><div> - NEMS/src/makefile is now NEMS/src/GNUmakefile</div><div> - NEMS/src/ENS_Cpl/makefile is now NEMS/src/ENS_Cpl/GNUmakefile<br></div><div> - NEMS/src/conf/configure.nems.NUOPC is not in the repository. It is now automatically generated from <a href="http://configure.nems.NUOPC.in" target="_blank">configure.nems.NUOPC.in</a>, the component_*.mk files, and the component list.<br></div><div dir="ltr"><div><br></div><div><br></div><div><br></div><div style="text-align:center"><font size="4"><i>Major Features and Notable Changes</i></font></div><div><font size="4"><i><br></i></font></div><div><font size="4"><i><br></i></font></div><div style="text-align:left"><font size="4"><i>Adding Components and Modifying Component Build Rules<br></i></font></div><div><i><br></i></div><div>One of the goals of this system is to simplify adding and modifying components. For this reason, the components each have their own file with build rules:</div><div><br></div><div> NEMS/src/incmake/component_FOO.mk</div><div><br></div><div>for a given component FOO. To add a new component, you just have to make a new component_FOO.mk file for it. The file will be automatically used if the component is in the list of components to build (Makefile $(COMPONENTS) variable), and ignored otherwise. Hence, two components can have conflicting logic as long as the components are not built together in the one executable.<br></div><div><br></div><div>There is one more file for specifying dependencies between components, such as building FMS before FV3 and MOM6:</div><div><br></div><div> NEMS/src/incmake/<a href="http://dependencies.mk" target="_blank">dependencies.mk</a></div><div><br></div><div><br></div><div><font size="4"><i>Application-Level Overrides</i></font><br></div><div><br></div><div>Another big goal is to let apps easily override logic in the NEMS build system without having to maintain a separate NEMS branch. For example, adding components, modifying component build rules, or changing platform configuration logic. <br></div><div><br></div><div>This is accomplished by:</div><div><br></div><div>1. If a component_FOO.mk is in the application-level conf/ directory, it overrides any NEMS/src/incmake/component_FOO.mk. New components or updated build logic can be added in that manner.</div><div><br></div><div>2. Three makefile fragments are loaded from the application level (if present) to override logic and add application-specific build rules:</div><div><br></div><div> - conf/<a href="http://before_components.mk" target="_blank">before_components.mk</a> - overrides platform decisions and provides defaults for component configuration</div><div> - conf/<a href="http://after_components.mk" target="_blank">after_components.mk</a> - overrides component configuration decisions</div><div> - conf/<a href="http://after_everything.mk" target="_blank">after_everything.mk</a> - overrides anything else; sourced last, just before build rules are executed</div><div><br></div><div>For example, the NEMSfv3gfs adds the tests/*.exe files as dependencies of the "build" rule. That way, a "make build" will generate the tests/*.exe file without any extra "cp" commands previously present in compile.sh.<br></div><div><br></div><div>3. Makefile variables that can override platform and component logic. Documenting this properly is too much for an email.</div><div><br></div><div>4. The *.appBuilder files are still supported (make app=standaloneFV3)<br></div><div><br></div><div><br></div><div><font size="4"><i>Automated Tests</i></font></div><div><font size="4"><font size="2"> <br></font></font></div>The NEMS/src/incmake/<a href="http://tests.mk" target="_blank">tests.mk</a> is able to run automated tests (as in cmake, scons, or autotools) to determine basic aspects of the platform and environment. Presently, it only detects case sensitivity of the filesystem, and the ESMF version. A file NEMS/src/conf/<a href="http://test-results.mk" target="_blank">test-results.mk</a> is created with results of the tests.<br> <div><font size="4"><i><br></i></font></div><div><font size="4"><i>Build Without a Wrapper</i></font><br></div><div><br></div><div>It is easy to build directly, without assistance of NEMSAppBuilder or compile.sh:</div><div><br></div><div>cd NEMS</div><div>make -j 8 COMPONENTS=FV3,MOM6,CICE build</div><br><div>The -j 8 is optional, and will be passed down automatically to the underlying components. However, only one component will be built at a time even if -j is specified. Parallel build of components can work, but is explicitly disabled via ".NOTPARALLEL:" in NEMS/GNUmakefile because the build log is totally unreadable when building multiple components at once.</div><div><br></div><div>Note that component dependencies are handled automatically. If you specify MOM6 or FV3, then "FMS" will be added (if absent) and will be at the beginning of the list. If you ask for CCPP and FV3, CCPP will be built first regardless of the order you specify. The logic in NEMS/src/incmake/<a href="http://dependencies.mk" target="_blank">dependencies.mk</a> adds components or changes the order in which they're built. The individual component_*.mk contains logic to detect an impossible component list (such as CCPP without a supported atmospheric component).<br></div><div><br></div><div><br></div><div><font size="4"><i>Cleaning Components and NEMS</i></font></div><div><br></div><div>Unlike the NEMSAppBuilder, the NEMS/GNUmakefile will not clean before building. You have to explicitly tell it to clean, just like in any makefile. There are two levels of cleaning:<br></div><div><br></div><div> make ... options ... -k clean # delete intermediate files</div><div> make ... options ... -k distclean # also delete targets (libraries, modules, executables, etc.)<br></div><div><br></div><div>The "-k" option is important when cleaning. It ensures Make will run all of the "clean" or "distclean" rules, even if one fails. Some of the components' underlying build systems will fail to clean themselves, and their "clean" rule will have non-zero status. Without the "-k", Make will stop cleaning as soon as one component's "clean" or "distclean" rule aborts. For the components in the Supported apps, I've put in workarounds for bugs I've found in the components' clean rules. However, there is no 100% guarantee that each component will clean itself properly.</div><div><br></div><div>If you want to be much more thorough, there is an aptly-named build target, "armageddon." It will attempt to revert to the repository state, delete all untracked files and directories (even "ignored" ones), discard local changes, and force a new checkout of all submodules. This is done recursively from the application level on down. Some types of submodule conflicts can prevent armageddon because the "git submodule" implementation may refuse to switch branches if conflicting files are present. Use this feature with great care, lest you discard what you meant to keep.<br></div><div><br></div><div><br></div><div><font size="4"><i>Fine-Grained Build</i></font><br></div><div><br></div><div>The build is split into multiple steps via Make rules. These steps can be run directly to execute just one part of the build or configuration process.<br></div><div><br></div><div>Components can be cleaned, distcleaned, and built individually:</div><div><br></div><div> make COMPONENTS=FV3,MOM6,CICE5 distclean_MOM6</div><div> make COMPONENTS=FV3,MOM6,CICE5 build_MOM6</div><div><br></div><div>The NEMS source and executable can be cleaned independently of the components:<br></div><div><br></div><div> make COMPONENTS=FV3,MOM6,CICE5 clean_NEMS # delete intermediate files<br></div><div> make COMPONENTS=FV3,MOM6,CICE5 distclean_NEMS # also delete targets<br></div></div><div dir="ltr"><br></div><div dir="ltr">You can configure the NEMS without building anything:</div><div dir="ltr"><br></div><div dir="ltr"> make COMPONENTS=FV3,MOM6,CICE5 configure</div><div dir="ltr"><br></div><div>You can test the selected modulefile (or lack thereof) without building anything:</div><div><br></div><div> make COMPONENTS=FV3,MOM6,CICE5 module_test</div><div><br></div><div>Or you could get information about what would be built, without actually building or configuring anything:<br></div><div><br></div><div> make COMPONENTS=FV3,MOM6,CICE5 # No build target.</div><div><br></div><div>This runs the internal makefile logic, without executing any Make rules. The automated tests are not done via make rules, so they'll still be run. This is the default action if no target is specified, and so a help message is included in the output.<br></div><div><br></div></div></div>
</blockquote></div>