[Ncep.list.nems.announce] impending NEMS emergency commit

[Samuel Trahan - NOAA Affiliate]

Hi all,

The commit is in.  As always, please let me know if you have any problems
with it.

Sincerely,
Sam Trahan

On Wed, Aug 1, 2018 at 4:42 PM, Samuel Trahan - NOAA Affiliate <
samuel.trahan at noaa.gov> wrote:

> Hi all,
>
> I'm going to commit an emergency NEMS change from the move-produtil
> branch; see below for details.  We're running the regression tests now on
> Theia and the WCOSS production machine now.  I'll commit the very moment
> they finish.
>
>
> Get ready to delve into the details of git submodules...
>
>
> Some users are having trouble with the latest NEMS commit, which switched
> to a submodule for produtil within NEMS.  Specifically, when you "git
> checkout" an older version of NEMS, the implementation of "git submodule"
> refuses to replace the produtil submodule with the embedded produtil
> scripts from older versions.  It is upset that you're askign it to replace
> the submodule in tests/produtil/ush with files.  Thus, it reports a
> conflict and does not continue until you delete the submodule.  This is a
> deliberate design (flaw) in the implementation of "git submodule".
>
> You can see the error if you:
>
> git clone NEMS
> cd NEMS
> git submodule update --init --recursive
> git checkout cee1f5cc2c50e089015c9251a4d17670530b1057
> git submodule update --init --recursive
>
> The last command will fail.  You can work around this by deleting
> tests/produtil/ush before the "git checkout" of the older version of NEMS.
>
> I'm about to commit a change with a better workaround for this git design
> (flaw).  I'm going to move the produtil submodule to a new directory that
> does not exist in earlier revisions (tests/produtil/NCEPLIBS-pyprodutil).
> You can see the effect of this change by telling "git clone" to checkout
> the move-produtil branch:
>
> git clone NEMS -b move-produtil
> cd NEMS
> git submodule update --init --recursive
> git checkout cee1f5cc2c50e089015c9251a4d17670530b1057
> git submodule update --init --recursive
>
> You will get a warning about the tests/produtil/NCEPLIBS-pyprodutil not
> being empty, and that directory will still exist, but everything will
> function as expected.
>
> There is one thing that still won't work after this change.  If you check
> out the revision that is presently the master of NEMS (the problematic one)
> then you won't be able to switch away from it without first doing an "rm
> -rf tests/produtil/ush".  All other revisions will be fine.
>
> If this becomes a major issue for some people, I can retroactively edit
> that revision via a "git rebase."  I would rather avoid that because it
> will cause problems for people who already have a clone on disk (git won't
> know the remote hash no longer matches the local one).  It will also lose
> commit history, which is less than ideal.
>
> Because we need this commit immediately, as in "right this moment," for
> the FV3 GFS parallel work, we're going through an unusual test process:
>
> 1. The aforementioned git commands were run on Tide, Luna, Mars, Jet,
> Gaea, and Theia to ensure the workaround works.
>
> 2. The NEMSfv3gfs "fv3_control" test was run on Jet.
>
> 3. The full suite of NEMS regression tests is being run on Theia.
>
> 4. The NEMSfv3gfs regression test is being run on Luna (production WCOSS)
> by emc.glopara.
>
> Gaea is skipped because of the data destruction issue.  Other WCOSS tests
> are skipped because we don't want to overuse the production whitespace.  We
> only run one test on Jet because hfv3gfs is near quota and thus the full
> suite of tests would probably fail.  Besides, nearly all of Jet is blocked
> with reservations, so the full suite may take a full day to get through.
>
> Sincerely,
> Sam Trahan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.nems.announce/attachments/20180801/2498f03a/attachment-0001.html