[Ncep.list.nems.announce] NEMS Upcoming commits: NEMS Ticket #174 & GSM #102: to the NEMS, GSM, NMM, phys, NEMSmodulefiles and chem trunks
Eugene Mirvis - NOAA Affiliate
eugene.mirvis at noaa.gov
Tue Aug 9 23:30:16 UTC 2016
Greetings,
At this moment, NEMS components' gatekeepers have completed the order of
previous planned commits on NEMS components trunks and after some machines
got fixed now too, I'd like to finalize a commit of the isolated atmos
physics and chem (gocart), described in previous e-mail (see below), NEMS's
ticket #174 and GSM ticket #102.
:
After sycned of my branches with latest commits, and all good RTs on 2
machines, the reintegration process with NEMS trunk seems to have no
problem at this step (see a dry-run below):
[Eugene.Mirvis at g10a2 nemstrunk]$ svn merge --dry-run --reintegrate
https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/nemstrunk_NEMSLegacyMD_ph2
--- Merging differences between repository URLs into '.':
U tests/Compile_theia.log
U tests/Compile_wcoss.log
U tests/tests/nmm_mvg_nests_gens
U tests/tests/nmm_cntrl
U tests/tests/nmm_reg_timesr
U tests/tests/gfs_slg_stochy
U tests/tests/nmm_reg_rst_nio_debug
U tests/tests/nmm_reg_hur
U tests/tests/nmm_reg_filt
U tests/tests/nmm_nests
U tests/tests/nmm_2way_nests
U tests/tests/nmm_reg_filt_zombie
U tests/tests/nmm_2way_nests_debug
U tests/RegressionTests_theia.log
U tests/RegressionTests_wcoss.log
U tests
U src/configure
U src/atmos/share/makefile
D src/chem
D src/atmos/share/idea_composition.f
D src/atmos/phys
D src/atmos/share/mersenne_twister.f
U src/atmos/makefile
U src/atmos
U src/conf/modules.nems.Wcoss_ESMF_700_nmmb
U src/conf/configure.nems.Theia.intel_gsm
U src/conf/configure.nems.Theia.intel_nmmb
U src/conf/configure.nems.Wcoss.intel_gsm
U src/conf/configure.nems.Wcoss.intel_nmmb
A src/conf/modules.nems.Theia_ESMF_700_gsm
U src/conf/modules.nems.Wcoss_ESMF_700_gsm
U src/conf/configure.nems.Theia.intel
A src/conf/modules.nems.Theia_ESMF_700_nmmb
D src/conf/modules.nems.Theia_ESMF_7_gsm
D src/conf/modules.nems.Theia_ESMF_7_nmmb
A src/detect_machine1.sh
A src/detect_machine.sh
U src/makefile
U src
U .
tests/tests/ changes are unplanned, result of the WCT linmit changes only
and occured becouse of the extrime slowness of Theia this weekend. These
changes have no impact on the performance or testing, but increase
wallclocktime limits only. Are Ok on testing for both machines.
All regression tests passed OK on Theia and WCOSS now , the successful
logs are located @
https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/nemstrunk_NEMSLegacyMD_ph2/tests/
I'd like to thank Raghu and RDHPCS sys admins who cooperated to speedup RT
during the latest Theia H/W troubles.
The main difference between last description (below) and current commit is
-- the rest of the atmos/phys is located now under gsm/gsmphys project and
"integared atmos phys" external linked to that.
This restructuring passed all tests as well, including 35 NMMB tests.
The NMMB code managers! (Ratko/Tom/Dusan),
Please consider the merge of my little changes of the nmmb makefile (when
compiled within NEMS) to be adapted to nmmb trunk.
I spoke to Tom and Dusan on that already, and it sounds like there are ok
with that - it's only 1 minor parameter to be introduced.
Because I have no access now to nmmb project, I've drafted this change @ my
nems branch:
"
https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/nmmb_NEMSLegacyMD_ph2
"
There is very a very little change in nmmb makefile, but I hope you'll find
it acceptable and can reintegrate for me simultaneously with my NEMS trunk
commit:
[Eugene.Mirvis at g10a2 nmm]$ svn merge --dry-run --reintegrate
https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/nmmb_NEMSLegacyMD_ph2
--- Merging differences between repository URLs into '.':
C makefile
U .
Summary of conflicts:
Text conflicts: 1
That small change would allow to use unified pre-build phys on both
machines, if you will decide to choose so, by changing on parameter in
configuration file
to PHYS_MODE=prebld.
There are following advantages to do so as we can foresee:
- —to share physics across all models within and outside NEMS framework.
- —to be able run physics within a single column model and work with
R2O,
- —To utilize PHYS releases in multi- physics & multi- model ensembles,
- —to develop a standalone new Common Community Physics Package (CCPP),
- —to use a shared physics driver for multiple or unified models, and to
- —enable the outside community to readily contribute physics schemes.
Please contact me if any questions.
Thanks,
-Eugene
On Tue, Jul 12, 2016 at 11:55 AM, Eugene Mirvis - NOAA Affiliate <
eugene.mirvis at noaa.gov> wrote:
>
>
> On Tue, Jul 12, 2016 at 11:48 AM, Eugene Mirvis - NOAA Affiliate <
> eugene.mirvis at noaa.gov> wrote:
>
>> ==
>> NEMS TRAC Ticket #174:New NEMS features: the standalone integrated
>> pre-builds of Phys and Chem libraries. ==
>>
>> ----
>>
>> 1. The current set of the upcoming commits for the NEMS trac ticket #174
>> provides a standalone capabilities for NEMS isolated physics and chemistry
>> separately.
>>
>> 2. The phys and chem library sources and build system will be located now
>> as the sub-projects of nceplibs in EMC repository, inheriting all NEMS
>> development history as a regular nceplibs sub-projects, contain branches,
>> tags and the trunks, tagged currently as v10.0.10, accordingly of the NEMS
>> restructuring history.
>>
>> 3. Both: phys and chem are using the same configuration NEMS procedure,
>> therefore, each has NEMS conf directory as the external links to the nems
>> conf disrectory for now. That secures consistency of configurations between
>> phys, chem, and NEMS framework.
>>
>> 4. A dual built capability is provided for the both phys and chem
>> libraries:1) as the stand-alone installations 2) the builts within a
>> traditional NEMS structure for phys and chem development process.
>>
>> 5. That selection is provided via the two external parameters: $PHYS_MODE
>> and $CHEM_MODE.
>> If these parameters are set as “compile ”the built system assumes full
>> compilation if such library. In this case a directory chem source should
>> have an external link to chem at the “src” while the phys dir an external
>> link at src/atmos level, defined at the subversion properties at NEMSLegacy
>> externals. By default, both will be set to “compile” by now, as currently
>> agreed with component code manager and will be assigned and are located in
>> configuration files in conf directory for Theia and WCOSS.
>>
>> 6. If these options are set to use of the pre-built libraries on
>> operational and R&D EMC machines are needed to be exercised, $ PHYS_MODE
>> and/or $CHEM_MODE the value of these parameters must be changed to any
>> other than “compile”. In this case, NEMS’s makefile will be acting
>> accordingly to use pre-built libraries on each machine and the source code
>> within NEMS structure is not necessary to be checked out.
>>
>> 7. The pre-built PHYS and CHEM libraries locations :
>>
>>
>> The pre-built libraries are available on Theia & WCOSS at the following
>> locations:
>> On Theia:
>> /scratch3/NSEPDEV/nceplibs/dev/
>> phys, chem, phys_full and chem_full
>> The builds of phys and chem contain the pre-builds with an option of
>> GOCART_MODE=stub while phys_full and chem_full contain pre-builds with the
>> GOCART_MODE=full.
>>
>> On WCOSS, the locations of the similar builts are at:
>> /usrx/local/mceplibs/dev/
>> phys, chem, phys_full and chem_full consecutively.
>>
>> You can link against each of these libraries directly but you have to
>> change the values of PHYS_MODE and CHEM_MODE from the “compile” to, perhaps
>> "prebuild" in the NEMS conf/configure.nems.Theia* and
>> conf/configure.nems.WCOSS* files. That will automatically assume in the
>> NEMS makefile the pre-built location paths are activated on each site
>> .
>> 8. The chem and phys modulefiles______________________
>> In order to conveniently manipulate with different options of use, for
>> installation procedure and for regression testing; and also to support the
>> NCEPLIBS computing environment flexibility and version standards, phys and
>> chem modulefiles have been created. These modulefiles are also residing
>> under EMC repositiory as the NEMSmodulefiles sub-project of the nceplibs
>> project and are also available on Theia and WCOSS currently under:
>> /scratch3/NCEPDEV/nceplibs/dev/NEMS modulefiles,
>> and
>> /usrx/local/nceplibs/dev/NEMS modulefiles, consecutively, until an
>> operational implementation of the phys and chem @ nwprod will occure
>> Also at the repository :
>> https://svnemc.ncep.noaa.gov/projects/nceplibs/Modules/NEMSmodulefiles/
>>
>> 9. Phys and Chem installation and version control
>> procedures________________________
>> The installation of the phys and chem are simplified to :
>> a) Checking out need version, dev branch, tag or the phys trunk of the
>> development
>> b) gmake clean
>> c) running ./Install script
>> Commit procedure to the phys and chem trunks should be address to the
>> phys and chem code manager.
>>
>> 10. Expected file changes in reintegration:
>> 10.1. CHEM:
>> [Eugene.Mirvis at t14a1 chem_trunk]$ svn merge --dry-run --reintegrate
>> https://svnemc.ncep.noaa.gov/projects/nceplibs/chem/
>> branches/Euge/nemstrunk_NEMSLegacyMD
>> --- Merging differences between repository URLs into '.':
>> A ESMFVersionDefine_ESMF_NUOPC.h
>> A configure
>> A Compile_theia.log
>> A ESMFConvenienceMacros.h
>> A InstallM
>> A Modulefiles_chem
>> A Modulefiles_chem/Modulefiles_phys
>> A Modulefiles_chem/chem
>> A Modulefiles_chem/chem/10.0.7
>> A Modulefiles_chem/chem/10.0.1
>> A Modulefiles_chem/chem/10.0.10
>> A Modulefiles_chem/chem/build10.0.10
>> A Modulefiles_chem/chem/10.0.2
>> A Compile_wcoss.log
>> A Install
>> A detect_machine1.sh
>> A clean.h
>> A makefile_std1
>> A makefile_std2
>> U makefile
>> U .
>> No textual conflicts are expected
>>
>> 10.2 PHYS reintegration_____________________________:
>>
>> Phys directory contains 2 externals for now (that reflects what was
>> insisted by the MESO group and accepted by M.Iredell for now)
>> [Eugene.Mirvis at t14a1 phys]$ svn propget svn:externals
>> conf https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/
>> nemstrunk_NEMSLegacyMD/src/conf
>> nmmbphys https://svnemc.ncep.noaa.gov/projects/nmmb/trunk/nmmbphys
>>
>> Reintegration:
>> [Eugene.Mirvis at t14a1 phys_trunk]$ svn merge --dry-run --reintegrate
>> https://svnemc.ncep.noaa.gov/projects/nceplibs/phys/
>> branches/Euge/ncepphys_trunk_NEMSLegacy_1physMD
>> --- Merging differences between repository URLs into '.':
>> A nmmbphys_Cln
>> A share
>> A share/mkDepends.pl
>> A Install
>> A ESMFVersionDefine_ESMF_NUOPC.h
>> A Compile_theia.log
>> A include
>> A include/ESMFVersionDefine_ESMF_NUOPC.h
>> A include/ESMFVersionDefine.h
>> A include/configure.nems_univ
>> A include/ESMFVersionLogic.h
>> A include/NCEP_base.mk
>> A include/ESMFVersionDefine_ESMF_6.3.0rp1API.h
>> A include/NUOPC.inc
>> A include/ESMFVersionDefine_ESMF_3.h
>> A include/kind.inc
>> A Compile_wcoss.log
>> A Install0
>> A nmmbphys_Get
>> U makefile
>> A configure
>> A detect_machine1.sh
>> A makefile_TODO
>> A mersenne_twister.f
>> A makefile_std1
>> A idea_composition.f
>> A Install_gsm
>> A Install_nmm
>> U .
>>
>> No textual conflicts are expected
>>
>> 10.3 NEMS_reintegration:
>> [Eugene.Mirvis at t14a1 nems_trunk]$ svn merge --dry-run --reintegrate
>> https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/
>> nemstrunk_NEMSLegacyMD
>> --- Merging differences between repository URLs into '.':
>> U tests/Compile_theia.log
>> U tests/rt.conf
>> U tests/RegressionTests_theia.log
>> U src/configure
>> U src/atmos/share/makefile
>> D src/chem
>> D src/atmos/share/idea_composition.f
>> D src/atmos/phys
>> D src/atmos/share/mersenne_twister.f
>> U src/atmos/makefile
>> U src/conf/modules.nems.Wcoss_ESMF_700_nmmb
>> U src/conf/configure.nems.Theia.intel
>> U src/conf/configure.nems.Theia.intel_gsm
>> U src/conf/configure.nems.Theia.intel_nmmb
>> U src/conf/configure.nems.Wcoss.intel_gsm
>> U src/conf/configure.nems.Wcoss.intel_nmmb
>> A src/conf/modules.nems.Theia_ESMF_700_gsm
>> A src/conf/modules.nems.Theia_ESMF_700_nmmb
>> U src/conf/modules.nems.Wcoss_ESMF_700_gsm
>> D src/conf/modules.nems.Theia_ESMF_7_gsm
>> D src/conf/modules.nems.Theia_ESMF_7_nmmb
>> A src/detect_machine1.sh
>> A src/detect_machine.sh
>> U src/makefile
>> U src
>> U .
>>
>> No textual or tree conflicts
>> 10.4 GSM_reintegration
>> [Eugene.Mirvis at t14a1 gsm_trunk]$ svn merge --dry-run --reintegrate
>> https://svnemc.ncep.noaa.gov/projects/gsm/branches/Euge/
>> gfs_NEMSLegacy1physMD
>> --- Merging differences between repository URLs into '.':
>> U module_GFS_GRID_COMP_stub.F90
>> U module_GFS_INTEGRATE.F90
>> U phys/makefile
>> U phys
>> U libutil/makefile
>> U libutil
>> U module_GFS_GRID_COMP.F90
>> U module_GOCART_ROUTINES_stub.F90
>> U module_GFS_CORE_SETUP.F90
>> U module_GOCART_ROUTINES.F90
>> U dyn/gfs_dynamics_initialize_slg_mod.f
>> U dyn/makefile
>> U dyn
>> UU makefile
>> U module_GFS_INTERNAL_STATE.F90
>> U .No text conflicts are expected or result change
>>
>> 10. 5
>> [Eugene.Mirvis at t14a1 nmmb_trunk_reint]$ svn merge --dry-run
>> --reintegrate https://svnemc.ncep.noaa.gov/projects/nems/branches/euge/
>> nmmbtrunk_MD
>> --- Merging differences between repository URLs into '.':
>> U nmmbphys/makefile
>> U makefile
>> No text conflicts expected
>>
>> After that, src/atmos/phys and src/chem. Are going to be deleted from
>> NEMS and introduced on NEMSLegacy level as the subversion external
>> properties as the following:
>>
>>
>> [Eugene.Mirvis at t14a1 NEMSLegacy_1physT_MDg]$ svn propget svn:externals
>> NEMS https://svnemc.ncep.noaa.gov/
>> projects/nems/branches/euge/nemstrunk_NEMSLegacyMD
>> NEMS/src/atmos/gsm https://svnemc.ncep.noaa.gov/
>> projects/gsm/branches/Euge/gfs_NEMSLegacy1physMD
>> NEMS/src/atmos/nmm https://svnemc.ncep.noaa.gov/
>> projects/nems/branches/euge/nmmbtrunk_MD
>> NEMS/src/atmos/phys https://svnemc.ncep.noaa.gov/
>> projects/nceplibs/phys/branches/Euge/ncepphys_trunk_NEMSLegacy_1physMD
>> NEMS/src/chem https://svnemc.ncep.noaa.gov/
>> projects/nceplibs/chem/branches/Euge/nemstrunk_NEMSLegacyMD
>>
>> For now
>> Or after the commit:
>> NEMS https://svnemc.ncep.noaa.gov/projects/nems/trunk
>> NEMS/src/atmos/gsm https://svnemc.ncep.noaa.gov/projects/gsm
>> NEMS/src/atmos/nmm https://svnemc.ncep.noaa.gov/projects/nems/trunk
>> NEMS/src/atmos/phys https://svnemc.ncep.noaa.gov/
>> projects/nceplibs/phys/trunk
>> NEMS/src/chem https://svnemc.ncep.noaa.gov/
>> projects/nceplibs/chem/
>>
>> With associated revisions on the head of the trunks
>>
>> 11. Working (tested) directories
>> All 4 full sets of the 45 regression tests have been PASSED now.
>> There are 2 working directories have been tested on each machines, tested
>> with the options a) to compile phys and chem within traditional structure
>> $PHYS_MODE=compile, $CHEM_MODE=compile ( are default for now) and
>> b) tested with the pre-build phys, chem., phys_full and chem._full
>> libraries :
>> With the phys and chem. prebuild options PHYS_MODE=compile
>> CHEM_MODE=compile:
>> On Theia:
>> PHYS_MODE=compile
>> CHEM_MODE=compile
>>
>> /scratch4/NCEPDEV/nems/noscrub/Eugene.Mirvis/NEMSLegacy_1physT_MDg
>>
>> On WCOSS __________:
>> PHYS_MODE=compile
>> CHEM_MODE=compile
>> /gpfs/td1/emc/global/noscrub/Eugene.Mirvis/NEMS/NEMSLegacy_1physT_MDg
>>
>> With the phys and chem. prebuild options:
>> PHYS_MODE=prebuild
>> CHEM_MODE=prebuild:
>> On Theia:
>> /scratch4/NCEPDEV/nems/noscrub/Eugene.Mirvis/NEMSLegacy_1physT_MD_prebld
>> On WCOSS :
>> /gpfs/td1/emc/global/noscrub/Eugene.Mirvis/NEMS/NEMSLegacy_
>> 1physT_MDg_prebld
>> All full regression tests have been passed in all 4 directories. No
>> results changes.
>>
>> GSM , NMMB code manages and NEMS team please observe upcoming changes in
>> 24 hrs and contact me if you have any questions
>>
>> Thank you,
>>
>> -Eugene
>>
>> ---
>> Eugene Mirvis, *Tech Lead,*
>> Senior Computational Scientist, IMSG @ EMC
>> National Centers for Environmental Prediction,
>> National Oceanic and Atmospheric Administration
>> 5830 University Research Ct., NCWCP rm 2183
>> College Park, MD 20740
>> Ph. 301.683.3809
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.nems.announce/attachments/20160809/841534e3/attachment-0001.html
More information about the Ncep.list.nems.announce
mailing list