[Ncep.list.emc.fv3gfs_tickets] FV3GFS Ticket #11: Cycling workflow.

FV3GFS Trac Ticket ncep.list.emc.fv3gfs_tickets at noaa.gov
Sat Mar 11 17:07:27 UTC 2017

#11: Cycling workflow.
  Reporter:              |      Owner:  rahul.mahajan@…
  rahul.mahajan@…        |     Status:  assigned
      Type:              |  Milestone:
  enhancement            |    Version:
  Priority:  major       |   Keywords:
 Component:  component1  |  workflow,scripting,cycling,utilities
Resolution:              |

Comment (by rahul.mahajan@…):

 Replying to [comment:23 fanglin.yang@…]:
 > '''Merge Rahul Mahajan's branch to the trunk'''

 Thank you for merging the branch into the [source:trunk].

 I have some reservations on some of the changes that were made without
 giving me a chance to review and comment on them.
 Some are simply property changes, but some are script/source code changes
 and have not been reviewed or commented. Unilateral decisions are never a
 good idea.

 I will highlight some of them here:

 1. We discussed extensively to make the {{{exglobal_fcst_fv3gfs.sh.ecf}}}
 machine independent. Anything machine related, such as loading modules, or
 setting {{{MPICH}}} related variables should be outside of this script.
 You agreed then, but now you seem to have rolled back on that.

 2. We agreed that the {{{exglobal_fcst_fv3gfs.sh.ecf}}} should be setup
 for cycling and can be modified for stand-alone or forecast-only runs.
 What you have added is logic for one or the other. I think we can do
 better than that.

 3. I have no objections to the FIX directory names, I used the name
 {{{FIX_DIR}}} because that was the standard in the {{{GFS}}}, if you want
 to call it {{{FIX_FV3}}}, I am Ok with that.

 4. You added a variable {{{$KEEPDATA}}}, which is in different than the
 logic followed in the existing GFS that if the {{{$DATA}}} directory
 exists, keep it, otherwise cleanup on exit. I am not opposed to this and
 if this is the standard you wish to follow, it should be communicated so
 that it is implemented elsewhere.

 5. There are several more changes that I did not make in my branch
 {{{source:branches/Rahul.Mahajan/EXP-cyc EXP-cyc}}} that are made as part
 of this {source:trunk} re-integration. They should be via their own
 branches, or atleast be documented in a ticket somewhere before re-
 integration to the [source:trunk], and reviewed. This should be a
 forbidden practice.

 6. As you know, the GSI repository holds a lot of the cycling scripts, and
 I have modified '''all''' of them with the standard that was in place
 before, with the new directory structure and file name conventions. As I
 have mentioned in the past, and re-iterate here, am more than willing to
 work with you to make the necessary changes that work towards unification
 of the EMC parallel workflow with the NCO workflow as the end goal. With
 the FV3 implementation, there should be no 2 ways (EMC or NCO) of doing
 things, that is my goal. Kindly guide me if I am misled to believe this is
 what I should be working towards.

 7. For now, I will update my branch [source:branches/Rahul.Mahajan/EXP-cyc
 EXP-cyc] currently at r89242 with the [source:trunk] at r89554, and
 discuss with you on how to address the issues outlined in this comment.

 8. I have deliberately not used {{{fcst.sh}}}, {{{post.sh}}}, and
 {{{vrfy.sh}}}, and to the extent none of the analysis or ensemble drivers
 either because there is no point in updating these scripts if we move
 towards the {{{JJOBS}}} way of doing our parallels. This should be handled
 by the workflow team anyways with input from you, ofcourse. As I mentioned
 earlier, I finished the last of the analysis related {{{exglobal}}}
 scripts and will now move towards a cycling capability with the
 {{{JJOBS}}} as the target.


 > Rahul's branch is
 cyc/.  Please see https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11 for
 the details of his changes.
 > 1.  Created a tag of the current trunk
 > 2.  Created a copy of Rahul's branch
 cyc/, made further changes to a few scripts to ensure the package still
 works on CRAY.  fv3gfs_nc2nemsio.sh, fv3gfs_remap.sh, JGFS_POST.sh, pend,
 psub, fcst.sh, post.sh, vrfy.sh and arch.sh have also been updated to
 follow the new directory structure.
 > 3. Reintegrate working branch back to the trunk
 > SURGE-slogin1 > svn -m "reintegarte Rahul branch back to the trunk"
 > Sending        .
 > Sending        gdas.v15.0.0
 > Sending        gdas.v15.0.0/jobs
 > Sending        gdas.v15.0.0/jobs/JGDAS_ANALYSIS_HIGH
 > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_INFLATE_RECENTER
 > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_INNOVATE_OBS
 > Sending        gdas.v15.0.0/jobs/JGDAS_ENKF_SELECT_OBS
 > Sending        gdas.v15.0.0/scripts
 > Sending        gdas.v15.0.0/ush
 > Sending        gfs.v15.0.0
 > Sending        gfs.v15.0.0/jobs
 > Sending        gfs.v15.0.0/scripts
 > Sending        gfs.v15.0.0/ush
 > Sending        gfs_workflow.v15.0.0
 > Sending        gfs_workflow.v15.0.0/para
 > Sending        gfs_workflow.v15.0.0/para/bin
 > Sending        gfs_workflow.v15.0.0/para/bin/pend
 > Sending        gfs_workflow.v15.0.0/para/bin/psub
 > Sending        gfs_workflow.v15.0.0/para/exp_fv3gfs/para_config
 > Sending        gfs_workflow.v15.0.0/para/exp_fv3gfs/submit_fv3gfs.sh
 > Sending        gfs_workflow.v15.0.0/para/jobs
 > Sending        gfs_workflow.v15.0.0/para/jobs/JGFS_POST.sh
 > Sending        gfs_workflow.v15.0.0/para/jobs/arch.sh
 > Sending        gfs_workflow.v15.0.0/para/jobs/fcst.sh
 > Sending        gfs_workflow.v15.0.0/para/jobs/vrfy.sh
 > Sending        gfs_workflow.v15.0.0/para/ush
 > Sending        global_shared.v15.0.0
 > Sending
 > Sending        global_shared.v15.0.0/fix/fix_am
 > Adding         global_shared.v15.0.0/parm/parm_fv3diag/diag_table_da
 > Adding
 > Sending        global_shared.v15.0.0/scripts
 > Sending        global_shared.v15.0.0/scripts/exglobal_analysis.sh.ecf
 > Sending        global_shared.v15.0.0/scripts/exglobal_fcst_fv3gfs.sh.ecf
 > Sending        global_shared.v15.0.0/scripts/exglobal_fcst_nems.sh.ecf
 > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/Makefile
 > Sending
 > Sending
 > Sending
 > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/kinds.f90
 > Sending        global_shared.v15.0.0/sorc/calc_increment.fd/main.f90
 > Sending
 > Sending        global_shared.v15.0.0/sorc/global_chgres.fd
 > Sending        global_shared.v15.0.0/sorc/orog.fd/ml01rg2.x
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile
 > Sending
 > Sending
 > Sending
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile_cray
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/constants.f90
 > Sending
 > Sending
 > Sending
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/kinds.f90
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/main.f90
 > Sending
 > Sending
 > Sending        global_shared.v15.0.0/sorc/regrid_nemsio.fd/physcons.f90
 > Sending        global_shared.v15.0.0/ush
 > Adding         global_shared.v15.0.0/ush/fv3gfs_calc_increment.sh
 > Sending        global_shared.v15.0.0/ush/fv3gfs_chgres.sh
 > Sending        global_shared.v15.0.0/ush/fv3gfs_driver_chgres.sh
 > Sending        global_shared.v15.0.0/ush/fv3gfs_nc2nemsio.sh
 > Sending        global_shared.v15.0.0/ush/fv3gfs_regrid_nemsio.sh
 > Sending        global_shared.v15.0.0/ush/fv3gfs_remap.sh
 > Transmitting file data ...................
 > Committed revision 89554.

Ticket URL: <https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11#comment:24>
fv3gfs <https://svnemc.ncep.noaa.gov/trac/fv3gfs>
NGGPS FV3GFS Development

More information about the Ncep.list.emc.fv3gfs_tickets mailing list