[Ncep.list.emc.fv3gfs_tickets] FV3GFS Ticket #11: Cycling workflow.
FV3GFS Trac Ticket
ncep.list.emc.fv3gfs_tickets at noaa.gov
Sat Mar 11 17:07:27 UTC 2017
#11: Cycling workflow.
-------------------------+-------------------------------------------------
Reporter: | Owner: rahul.mahajan@…
rahul.mahajan@… | Status: assigned
Type: | Milestone:
enhancement | Version:
Priority: major | Keywords:
Component: component1 | workflow,scripting,cycling,utilities
Resolution: |
-------------------------+-------------------------------------------------
Comment (by rahul.mahajan@…):
Replying to [comment:23 fanglin.yang@…]:
> '''Merge Rahul Mahajan's branch to the trunk'''
>
Fanglin,
Thank you for merging the branch into the [source:trunk].
I have some reservations on some of the changes that were made without
giving me a chance to review and comment on them.
Some are simply property changes, but some are script/source code changes
and have not been reviewed or commented. Unilateral decisions are never a
good idea.
I will highlight some of them here:
1. We discussed extensively to make the {{{exglobal_fcst_fv3gfs.sh.ecf}}}
machine independent. Anything machine related, such as loading modules, or
setting {{{MPICH}}} related variables should be outside of this script.
You agreed then, but now you seem to have rolled back on that.
2. We agreed that the {{{exglobal_fcst_fv3gfs.sh.ecf}}} should be setup
for cycling and can be modified for stand-alone or forecast-only runs.
What you have added is logic for one or the other. I think we can do
better than that.
3. I have no objections to the FIX directory names, I used the name
{{{FIX_DIR}}} because that was the standard in the {{{GFS}}}, if you want
to call it {{{FIX_FV3}}}, I am Ok with that.
4. You added a variable {{{$KEEPDATA}}}, which is in different than the
logic followed in the existing GFS that if the {{{$DATA}}} directory
exists, keep it, otherwise cleanup on exit. I am not opposed to this and
if this is the standard you wish to follow, it should be communicated so
that it is implemented elsewhere.
5. There are several more changes that I did not make in my branch
{{{source:branches/Rahul.Mahajan/EXP-cyc EXP-cyc}}} that are made as part
of this {source:trunk} re-integration. They should be via their own
branches, or atleast be documented in a ticket somewhere before re-
integration to the [source:trunk], and reviewed. This should be a
forbidden practice.
6. As you know, the GSI repository holds a lot of the cycling scripts, and
I have modified '''all''' of them with the standard that was in place
before, with the new directory structure and file name conventions. As I
have mentioned in the past, and re-iterate here, am more than willing to
work with you to make the necessary changes that work towards unification
of the EMC parallel workflow with the NCO workflow as the end goal. With
the FV3 implementation, there should be no 2 ways (EMC or NCO) of doing
things, that is my goal. Kindly guide me if I am misled to believe this is
what I should be working towards.
7. For now, I will update my branch [source:branches/Rahul.Mahajan/EXP-cyc
EXP-cyc] currently at r89242 with the [source:trunk] at r89554, and
discuss with you on how to address the issues outlined in this comment.
8. I have deliberately not used {{{fcst.sh}}}, {{{post.sh}}}, and
{{{vrfy.sh}}}, and to the extent none of the analysis or ensemble drivers
either because there is no point in updating these scripts if we move
towards the {{{JJOBS}}} way of doing our parallels. This should be handled
by the workflow team anyways with input from you, ofcourse. As I mentioned
earlier, I finished the last of the analysis related {{{exglobal}}}
scripts and will now move towards a cycling capability with the
{{{JJOBS}}} as the target.
Thanks,
Rahul.
> Rahul's branch is
https://svnemc.ncep.noaa.gov/projects/fv3gfs/branches/Rahul.Mahajan/EXP-
cyc/. Please see https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11 for
the details of his changes.
>
> 1. Created a tag of the current trunk
https://svnemc.ncep.noaa.gov/projects/fv3gfs/tags/fv3gfs_v3_20170307/
>
> 2. Created a copy of Rahul's branch
https://svnemc.ncep.noaa.gov/projects/fv3gfs/branches/fanglin.yang/EXP-
cyc/, made further changes to a few scripts to ensure the package still
works on CRAY. fv3gfs_nc2nemsio.sh, fv3gfs_remap.sh, JGFS_POST.sh, pend,
psub, fcst.sh, post.sh, vrfy.sh and arch.sh have also been updated to
follow the new directory structure.
>
> 3. Reintegrate working branch back to the trunk
>
> SURGE-slogin1 > svn -m "reintegarte Rahul branch back to the trunk"
commit
> Sending .
> Sending gdas.v15.0.0
> Sending gdas.v15.0.0/jobs
> Sending gdas.v15.0.0/jobs/JGDAS_ANALYSIS_HIGH
> Sending gdas.v15.0.0/jobs/JGDAS_ENKF_INFLATE_RECENTER
> Sending gdas.v15.0.0/jobs/JGDAS_ENKF_INNOVATE_OBS
> Sending gdas.v15.0.0/jobs/JGDAS_ENKF_SELECT_OBS
> Sending gdas.v15.0.0/scripts
> Sending gdas.v15.0.0/ush
> Sending gfs.v15.0.0
> Sending gfs.v15.0.0/jobs
> Sending gfs.v15.0.0/scripts
> Sending gfs.v15.0.0/ush
> Sending gfs_workflow.v15.0.0
> Sending gfs_workflow.v15.0.0/para
> Sending gfs_workflow.v15.0.0/para/bin
> Sending gfs_workflow.v15.0.0/para/bin/pend
> Sending gfs_workflow.v15.0.0/para/bin/psub
> Sending gfs_workflow.v15.0.0/para/exp_fv3gfs/para_config
> Sending gfs_workflow.v15.0.0/para/exp_fv3gfs/submit_fv3gfs.sh
> Sending gfs_workflow.v15.0.0/para/jobs
> Sending gfs_workflow.v15.0.0/para/jobs/JGFS_POST.sh
> Sending gfs_workflow.v15.0.0/para/jobs/arch.sh
> Sending gfs_workflow.v15.0.0/para/jobs/fcst.sh
> Sending gfs_workflow.v15.0.0/para/jobs/vrfy.sh
> Sending gfs_workflow.v15.0.0/para/ush
> Sending global_shared.v15.0.0
> Sending
global_shared.v15.0.0/docs/Release_Notes.global_shared_gsm.v14.1.0.txt
> Sending global_shared.v15.0.0/fix/fix_am
> Adding global_shared.v15.0.0/parm/parm_fv3diag/diag_table_da
> Adding
global_shared.v15.0.0/parm/parm_fv3diag/variable_table_da.txt
> Sending global_shared.v15.0.0/scripts
> Sending global_shared.v15.0.0/scripts/exglobal_analysis.sh.ecf
> Sending global_shared.v15.0.0/scripts/exglobal_fcst_fv3gfs.sh.ecf
> Sending global_shared.v15.0.0/scripts/exglobal_fcst_nems.sh.ecf
> Sending global_shared.v15.0.0/sorc/calc_increment.fd/Makefile
> Sending
global_shared.v15.0.0/sorc/calc_increment.fd/Makefile.conf.cray
> Sending
global_shared.v15.0.0/sorc/calc_increment.fd/Makefile.conf.theia
> Sending
global_shared.v15.0.0/sorc/calc_increment.fd/constants.f90
> Sending global_shared.v15.0.0/sorc/calc_increment.fd/kinds.f90
> Sending global_shared.v15.0.0/sorc/calc_increment.fd/main.f90
> Sending
global_shared.v15.0.0/sorc/calc_increment.fd/namelist_def.f90
> Sending global_shared.v15.0.0/sorc/global_chgres.fd
> Sending global_shared.v15.0.0/sorc/orog.fd/ml01rg2.x
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.cray
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.jet
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile.conf.theia
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/Makefile_cray
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/constants.f90
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/fv3_interface.f90
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/gfs_nems_interface.f90
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/interpolation_interface.f90
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/kinds.f90
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/main.f90
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/mpi_interface.f90
> Sending
global_shared.v15.0.0/sorc/regrid_nemsio.fd/namelist_def.f90
> Sending global_shared.v15.0.0/sorc/regrid_nemsio.fd/physcons.f90
> Sending global_shared.v15.0.0/ush
> Adding global_shared.v15.0.0/ush/fv3gfs_calc_increment.sh
> Sending global_shared.v15.0.0/ush/fv3gfs_chgres.sh
> Sending global_shared.v15.0.0/ush/fv3gfs_driver_chgres.sh
> Sending global_shared.v15.0.0/ush/fv3gfs_nc2nemsio.sh
> Sending global_shared.v15.0.0/ush/fv3gfs_regrid_nemsio.sh
> Sending global_shared.v15.0.0/ush/fv3gfs_remap.sh
> Transmitting file data ...................
> Committed revision 89554.
>
--
Ticket URL: <https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/11#comment:24>
fv3gfs <https://svnemc.ncep.noaa.gov/trac/fv3gfs>
NGGPS FV3GFS Development
More information about the Ncep.list.emc.fv3gfs_tickets
mailing list