[Ncep.list.emc.fv3gfs_tickets] FV3GFS Ticket #11: Cycling workflow.

Sat Mar 11 19:00:08 UTC 2017

Yes, please point out what was decided by me that was not discussed with
you or with others.
I am open to change.
Rahul

On Sat, Mar 11, 2017 at 1:55 PM, Fanglin Yang - NOAA Affiliate <
fanglin.yang at noaa.gov> wrote:

> Rahul,
> You made a lot of unilateral changes without discussing with me.  I have
> to make some corrections to make it work on wcoss. You need to use the
> super package to understand what is really working.  KEEPDATA is used
> incurrent package, not my invention. Talk to me offline. Without vrfy fcst
> arch we cannot do anything at the moment.
>
> Fanglin
>
> On Sat, Mar 11, 2017 at 12:07 PM FV3GFS Trac Ticket <
> ncep.list.emc.fv3gfs_tickets at noaa.gov> wrote:
>
>> #11: Cycling workflow.
>>
>> -------------------------+----------------------------------
>> ---------------
>>
>>   Reporter:              |      Owner:  rahul.mahajan@…
>>
>>   rahul.mahajan@…        |     Status:  assigned
>>
>>       Type:              |  Milestone:
>>
>>   enhancement            |    Version:
>>
>>   Priority:  major       |   Keywords:
>>
>>  Component:  component1  |  workflow,scripting,cycling,utilities
>>
>> Resolution:              |
>>
>> -------------------------+----------------------------------
>> ---------------
>>
>>
>>
>> Comment (by rahul.mahajan@…):
>>
>>
>>
>>  Replying to [comment:23 fanglin.yang@…]:
>>
>>  > '''Merge Rahul Mahajan's branch to the trunk'''
>>
>>  >
>>
>>  Fanglin,
>>
>>
>>
>>  Thank you for merging the branch into the [source:trunk].
>>
>>
>>
>>  I have some reservations on some of the changes that were made without
>>
>>  giving me a chance to review and comment on them.
>>
>>  Some are simply property changes, but some are script/source code changes
>>
>>  and have not been reviewed or commented. Unilateral decisions are never a
>>
>>  good idea.
>>
>>
>>
>>  I will highlight some of them here:
>>
>>
>>
>>  1. We discussed extensively to make the {{{exglobal_fcst_fv3gfs.sh.
>> ecf}}}
>>
>>  machine independent. Anything machine related, such as loading modules,
>> or
>>
>>  setting {{{MPICH}}} related variables should be outside of this script.
>>
>>  You agreed then, but now you seem to have rolled back on that.
>>
>>
>>
>>  2. We agreed that the {{{exglobal_fcst_fv3gfs.sh.ecf}}} should be setup
>>
>>  for cycling and can be modified for stand-alone or forecast-only runs.
>>
>>  What you have added is logic for one or the other. I think we can do
>>
>>  better than that.
>>
>>
>>
>>  3. I have no objections to the FIX directory names, I used the name
>>
>>  {{{FIX_DIR}}} because that was the standard in the {{{GFS}}}, if you want
>>
>>  to call it {{{FIX_FV3}}}, I am Ok with that.
>>
>>
>>
>>  4. You added a variable {{{$KEEPDATA}}}, which is in different than the
>>
>>  logic followed in the existing GFS that if the {{{$DATA}}} directory
>>
>>  exists, keep it, otherwise cleanup on exit. I am not opposed to this and
>>
>>  if this is the standard you wish to follow, it should be communicated so
>>
>>  that it is implemented elsewhere.
>>
>>
>>
>>  5. There are several more changes that I did not make in my branch
>>
>>  {{{source:branches/Rahul.Mahajan/EXP-cyc EXP-cyc}}} that are made as
>> part
>>
>>  of this {source:trunk} re-integration. They should be via their own
>>
>>  branches, or atleast be documented in a ticket somewhere before re-
>>
>>  integration to the [source:trunk], and reviewed. This should be a
>>
>>  forbidden practice.
>>
>>
>>
>>  6. As you know, the GSI repository holds a lot of the cycling scripts,
>> and
>>
>>  I have modified '''all''' of them with the standard that was in place
>>
>>  before, with the new directory structure and file name conventions. As I
>>
>>  have mentioned in the past, and re-iterate here, am more than willing to
>>
>>  work with you to make the necessary changes that work towards unification
>>
>>  of the EMC parallel workflow with the NCO workflow as the end goal. With
>>
>>  the FV3 implementation, there should be no 2 ways (EMC or NCO) of doing
>>
>>  things, that is my goal. Kindly guide me if I am misled to believe this
>> is
>>
>>  what I should be working towards.
>>
>>
>>
>>  7. For now, I will update my branch [source:branches/Rahul.
>> Mahajan/EXP-cyc
>>
>>  EXP-cyc] currently at r89242 with the [source:trunk] at r89554, and
>>
>>  discuss with you on how to address the issues outlined in this comment.
>>
>>
>>
>>  8. I have deliberately not used {{{fcst.sh}}}, {{{post.sh}}}, and
>>
>>  {{{vrfy.sh}}}, and to the extent none of the analysis or ensemble drivers
>>
>>  either because there is no point in updating these scripts if we move
>>
>>  towards the {{{JJOBS}}} way of doing our parallels. This should be
>> handled
>>
>>  by the workflow team anyways with input from you, ofcourse. As I
>> mentioned
>>
>>  earlier, I finished the last of the analysis related {{{exglobal}}}
>>
>>  scripts and will now move towards a cycling capability with the
>>
>>  {{{JJOBS}}} as the target.
>>
>>
>>
>>  Thanks,
>>
>>  Rahul.
>>
>>
>>
>>
>>
>>  > Rahul's branch is
>>
>>  https://svnemc.ncep.noaa.gov/projects/fv3gfs/branches/Rahul.Mahajan/EXP-
>>
>>  cyc/.  Please see https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11 for
>>
>>  the details of his changes.
>>
>>  >
>>
>>  > 1.  Created a tag of the current trunk
>>
>>  https://svnemc.ncep.noaa.gov/projects/fv3gfs/tags/fv3gfs_v3_20170307/
>>
>>  >
>>
>>  > 2.  Created a copy of Rahul's branch
>>
>>  https://svnemc.ncep.noaa.gov/projects/fv3gfs/branches/fanglin.yang/EXP-
>>
>>  cyc/, made further changes to a few scripts to ensure the package still
>>
>>  works on CRAY.  fv3gfs_nc2nemsio.sh, fv3gfs_remap.sh, JGFS_POST.sh, pend,
>>
>>  psub, fcst.sh, post.sh, vrfy.sh and arch.sh have also been updated to
>>
>>  follow the new directory structure.
>>
>>  >
>>
>>  > 3. Reintegrate working branch back to the trunk
>>
>>  >
>>
>>  > SURGE-slogin1 > svn -m "reintegarte Rahul branch back to the trunk"
>>
>>  commit
>>
>>  > Sending        .
>>
>>  > Sending        gdas.v15.0.0
>>
>>  > Sending        gdas.v15.0.0/jobs
>>
>>  > Sending        gdas.v15.0.0/jobs/JGDAS_ANALYSIS_HIGH
>>
>>  > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_INFLATE_RECENTER
>>
>>  > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_INNOVATE_OBS
>>
>>  > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_SELECT_OBS
>>
>>  > Sending        gdas.v15.0.0/scripts
>>
>>  > Sending        gdas.v15.0.0/ush
>>
>>  > Sending        gfs.v15.0.0
>>
>>  > Sending        gfs.v15.0.0/jobs
>>
>>  > Sending        gfs.v15.0.0/scripts
>>
>>  > Sending        gfs.v15.0.0/ush
>>
>>  > Sending        gfs_workflow.v15.0.0
>>
>>  > Sending        gfs_workflow.v15.0.0/para
>>
>>  > Sending        gfs_workflow.v15.0.0/para/bin
>>
>>  > Sending        gfs_workflow.v15.0.0/para/bin/pend
>>
>>  > Sending        gfs_workflow.v15.0.0/para/bin/psub
>>
>>  > Sending        gfs_workflow.v15.0.0/para/exp_fv3gfs/para_config
>>
>>  > Sending        gfs_workflow.v15.0.0/para/exp_fv3gfs/submit_fv3gfs.sh
>>
>>  > Sending        gfs_workflow.v15.0.0/para/jobs
>>
>>  > Sending        gfs_workflow.v15.0.0/para/jobs/JGFS_POST.sh
>>
>>  > Sending        gfs_workflow.v15.0.0/para/jobs/arch.sh
>>
>>  > Sending        gfs_workflow.v15.0.0/para/jobs/fcst.sh
>>
>>  > Sending        gfs_workflow.v15.0.0/para/jobs/vrfy.sh
>>
>>  > Sending        gfs_workflow.v15.0.0/para/ush
>>
>>  > Sending        global_shared.v15.0.0
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/docs/Release_Notes.global_shared_gsm.v14.1.0.txt
>>
>>  > Sending        global_shared.v15.0.0/fix/fix_am
>>
>>  > Adding         global_shared.v15.0.0/parm/parm_fv3diag/diag_table_da
>>
>>  > Adding
>>
>>  global_shared.v15.0.0/parm/parm_fv3diag/variable_table_da.txt
>>
>>  > Sending        global_shared.v15.0.0/scripts
>>
>>  > Sending        global_shared.v15.0.0/scripts/exglobal_analysis.sh.ecf
>>
>>  > Sending        global_shared.v15.0.0/scripts/
>> exglobal_fcst_fv3gfs.sh.ecf
>>
>>  > Sending        global_shared.v15.0.0/scripts/exglobal_fcst_nems.sh.ecf
>>
>>  > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/Makefile
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/calc_increment.fd/Makefile.conf.cray
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/calc_increment.fd/Makefile.conf.theia
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/calc_increment.fd/constants.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/kinds.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/main.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/calc_increment.fd/namelist_def.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/global_chgres.fd
>>
>>  > Sending        global_shared.v15.0.0/sorc/orog.fd/ml01rg2.x
>>
>>  > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.cray
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.jet
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.theia
>>
>>  > Sending        global_shared.v15.0.0/sorc/
>> regrid_nemsio.fd/Makefile_cray
>>
>>  > Sending        global_shared.v15.0.0/sorc/
>> regrid_nemsio.fd/constants.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/fv3_interface.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/gfs_nems_interface.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/interpolation_interface.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/kinds.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/main.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/mpi_interface.f90
>>
>>  > Sending
>>
>>  global_shared.v15.0.0/sorc/regrid_nemsio.fd/namelist_def.f90
>>
>>  > Sending        global_shared.v15.0.0/sorc/
>> regrid_nemsio.fd/physcons.f90
>>
>>  > Sending        global_shared.v15.0.0/ush
>>
>>  > Adding         global_shared.v15.0.0/ush/fv3gfs_calc_increment.sh
>>
>>  > Sending        global_shared.v15.0.0/ush/fv3gfs_chgres.sh
>>
>>  > Sending        global_shared.v15.0.0/ush/fv3gfs_driver_chgres.sh
>>
>>  > Sending        global_shared.v15.0.0/ush/fv3gfs_nc2nemsio.sh
>>
>>  > Sending        global_shared.v15.0.0/ush/fv3gfs_regrid_nemsio.sh
>>
>>  > Sending        global_shared.v15.0.0/ush/fv3gfs_remap.sh
>>
>>  > Transmitting file data ...................
>>
>>  > Committed revision 89554.
>>
>>  >
>>
>>
>>
>> --
>>
>> Ticket URL: <https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11#comment:
>> 24>
>>
>> fv3gfs <https://svnemc.ncep.noaa.gov/trac/fv3gfs>
>>
>> NGGPS FV3GFS Development
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.emc.fv3gfs_tickets/attachments/20170311/fc90cc97/attachment-0001.html 