[Ncep.list.fv3-announce] SLURM workflow updates now in global-workflow master!

[Kate Friedman - NOAA Federal]

All,
Thank you for your patience! The SLURM updates for the global-workflow are
now in its master! Thank you to everyone who worked hard to test and
resolve any final issues this week! Some additional action items were
produced during the final testing phase so look for that mentioned below
and in upcoming commits. Please let me know if you have any issues with the
workflow after this commit.

One very important note is that some developers (our workflow team
included) noticed differences in SLURM runs done before and after this
week's Theia maintenance. These differences are NOT seen between repeated
runs done after the maintenance with the exact same inputs and settings.
These differences are very small (butterflies) but will be discussed with
the machine admins. Any needed fixes will be implemented ASAP.

SLURM workflow updates for R&D machines
<https://vlab.ncep.noaa.gov/redmine/news/767>

SLURM workflow updates for R&D machines (Theia - no impact on WCOSS)

Redmine Issue: https://vlab.ncep.noaa.gov/redmine/issues/58894

Summary of changes:

   - env/THEIA.env - change launcher format for SLURM
   - modulefiles/module_base.theia - updates to add slurm prod_util module
   - ush/rocoto/rocoto.py - added partition statements
   - ush/rocoto/setup_expt.py - set icsdir option requirement to false
   - ush/rocoto/setup_workflow.py - add SLURM checks for Theia parts
   - ush/rocoto/workflow_utils.py - add SLURM checks for Theia parts
   - util/sorc/compile_gfs_util_wcoss.sh - small fixes
   - updated fit2obs jobcard for SLURM

Things still to address in later commits:

   - Downstream jobs (e.g. gempak) are not yet supported/tested on R&D
   machines or under SLURM

Caveats:

   - Differences have been observed between SLURM tests done before and
   after May 7th Theia maintenance. This will be investigated and any
   necessary fixes will be tested and committed to the global-workflow master
   ASAP. Please report any similar behavior observed!

How to incorporate changes into your own copy. Your options:

   1. Sync merge the global-workflow master into your branch
   (preferred/advised option)
   2. Apply changes via patch:

   See changes in following file on Theia:


   /scratch4/NCEPDEV/global/save/glopara/utilities/global-workflow_slurm_master.diff

   Apply changes to your branch while within clone:

   git apply --reject --whitespace=fix
   /scratch4/NCEPDEV/global/save/glopara/utilities/global-workflow_slurm_master.diff


Kate Friedman (formerly Howard)
NOAA/NWS/NCEP/EMC Engineering and Implementation Branch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20190510/427f51e1/attachment.html