[Ncep.list.fv3-announce] fv3gfs release beta test
Dusan Jovic
dusan.jovic at noaa.gov
Fri May 12 20:04:27 UTC 2017
Thanks, Sam.
Unloading the modules is indeed something you can't do with a simple
text file containing "module load ..." lines.
But I do not understand the reasons 2. and 3. File modules.nems is not
supposed to be load-ed but sourced from either interactive shell (bash
or tcsh) or from a shell script, which can be either *sh or *csh. Why
would anyone "source /etc/profile" from module.nems file, which only
contains simple commands. Interactive shell or a shell script must be
properly configured to load modules, which is also the case with
modulefies and that's why you have module-setup scripts.
Anyway, thank you again for the explanation.
Dusan
On 05/12/2017 03:10 PM, Samuel Trahan - NOAA Affiliate wrote:
> The reasons for moving to a modulefile:
>
> 1. You can unload the modulefile.
>
> 2. The same file works in all shells. Previously, if you used a tcsh
> login, you could not load modules.nems. This was a major problem for
> some users.
>
> 3. It prevents adding "source /etc/profile" to the modulefile, which
> causes problems if /etc/profile has already been sourced.
>
> There is now a pair of shell include files,
> NEMS/src/conf/module-setup.*sh.inc, which know how to obtain the
> module command and purge modules on seven clusters and five shells.
> We also have a test system for it in NEMS/tests/modulecmd/. That
> replaces "source /etc/profile" and "module purge."
>
> Sincerely,
> Sam Trahan
>
> On Fri, May 12, 2017 at 1:59 PM, Dusan Jovic <dusan.jovic at noaa.gov
> <mailto:dusan.jovic at noaa.gov>> wrote:
>
> Can some of you 'module' experts please explain why is writing
> real modulefiles (a file that starts with #%Module) and then
> loading that file with 'module load' better then writing a text
> file with just a bunch of module load commands which is then
> "sourced" from a build or run scripts? We used to use the second
> approach in "the old NEMS" for many years and never had any problem.
>
> What are the advantages and are there any disadvantages?
>
> Thanks,
>
> Dusan
>
>
> On 05/12/2017 12:40 PM, Jun Wang - NOAA Affiliate wrote:
>> Jim,
>>
>> Sam is working on that, we hope to make it to this release. Thanks.
>>
>> Jun
>>
>> On Fri, May 12, 2017 at 12:37 PM, James Rosinski - NOAA Affiliate
>> <james.rosinski at noaa.gov <mailto:james.rosinski at noaa.gov>> wrote:
>>
>> Jun and others;
>>
>> I noticed there is a build capability for fv3gfs on jet, but
>> not any run instructions or runjob_jet.sh for jet. Is jet run
>> capability intended, or is that for later?
>>
>> Regards,
>> Jim Rosinski
>>
>>
>>
>> On Fri, May 12, 2017 at 10:16 AM, Rusty Benson - NOAA Federal
>> <rusty.benson at noaa.gov <mailto:rusty.benson at noaa.gov>> wrote:
>>
>> Eugene,
>>
>> Responding only to point 4, what options do you see that
>> are Cray compilation flags?
>>
>> Rusty
>> --
>> Rusty Benson, PhD
>> Modeling Systems Group
>> NOAA Geophysical Fluid Dynamics Lab
>> Princeton, NJ
>>
>> On Fri, May 12, 2017 at 11:58 AM, Eugene Mirvis
>> <eugene.mirvis at noaa.gov <mailto:eugene.mirvis at noaa.gov>>
>> wrote:
>>
>> Gerard,
>>
>> Just several points to clarify.
>>
>> 1. NEMS practice to call modulefiles what is
>> actually the scripts (calling module commands),
>> that require mostly bash env in order to source and
>> keep environment was always non standard use.
>>
>> Therefore, if the developers will take Sam's advice
>> and make real modulefiles (starting with #%Module)
>> from the script, "export" and other scripting works
>> wouldn't make sense, while Module util commands and
>> Tcl/Tk will work.
>>
>> 2. There is another dilemma - to keep needed
>> environment and change within a workflow env. change.
>> btw, *
>> *unsetenv, append-path, prepend-path and remove-path
>> module commands are very useful for controlling that,
>> but you have to keep
>> $LOADEDMODULES, and $MODULE PATH in consistent order.
>>
>> 3. Speaking of which,
>> module purge
>> and
>> module switch
>> are very useful to unload application driven modules.
>> a/ You just have to do that not any moment, but
>> before apps module is loaded, then, unload
>> <appsModulefile> - not purge, but unload.
>> b/ On Crays, the are some internal dependencies
>> inside of PrgEnv. So you have to keep all "module use
>> <knowns>" to recover, otherwise you might find
>> "module not found"
>>
>> 4.
>> Compiling on Theia, I'm just wondering why Cray's
>> compilation flags are utilized... allover:
>> See for instance
>> ...
>> *mpiifort -I/apps/netcdf/4.3.0-intel/include
>> -fno-alias -auto -safe-cray-ptr -ftz -assume byterecl
>> -nowarn -sox -align array64byte -i4 -real-size 64
>> -no-prec-div -no-prec-sqrt
>> -xCORE-AVX2**-qno-opt-dynamic-align* -O2 -debug
>> minimal -fp-model source -qoverride-limits
>> -qopt-prefetch=3 -qopenmp
>> -I/apps/esmf/7.0.0/intel/intelmpi/mod/modO/Linux.intel.64.in
>> <http://Linux.intel.64.in>telmpi.default
>> -I/apps/esmf/7.0.0/intel/intelmpi/include
>> -I/apps/netcdf/4.3.0-intel/include -IENS_Cpl -I.
>> -I/scratch4/NCEPDEV/global/noscrub/Eugene.Mirvis/fv3gfs.v0beta/FV3/nems_dir
>> -c module_MEDIATOR_methods.f90
>> ...
>>
>> Thanks,
>> -Eugene
>> On 5/12/2017 3:27 AM, Samuel Trahan - NOAA Affiliate
>> wrote:
>>> Gerard, The "module load module.free-nctools" is
>>> failing because that file contains commands that are
>>> not valid in a modulefile. You can have as many
>>> "module" commands as you want in a modulefile, but
>>> you cannot have bash code. Also, it needs to start
>>> with #%Module export A=B # bad setenv A B
>>> prepend-path A (something) No "source /etc/profile"
>>> No "source /.../init/bash" No "module purge" because
>>> that causes infinite loops on some platforms. (The
>>> module is trying to unload itself.) Obtaining the
>>> "module" command and purging modules has to be done
>>> before loading the module. We have a pair of scripts
>>> (one csh, one sh) that do that on all NOAA machines.
>>> They are aware of csh, tcsh, ksh, bash, and sh; and
>>> run the correct initialization for each. The master
>>> copies are in NEMS/src/conf/module-setup.*sh.inc
>>> (csh and sh versions). Those scripts work on all
>>> NOAA machines and have a test suite inside
>>> NEMS/tests. man 4 modulefile
>>> http://modules.sourceforge.net/c/modulefile.html
>>> <http://modules.sourceforge.net/c/modulefile.html>
>>>
>>>
>>> Sincerely,
>>> Sam Trahan
>>>
>>> On Thu, May 11, 2017 at 5:01 PM, Gerard Ketefian - NOAA Affiliate <
>>> gerard.ketefian at noaa.gov
>>> <mailto:gerard.ketefian at noaa.gov>> wrote:
>>>
>>>> Hi Jun,
>>>>
>>>> Yes, after the missing "cp" fix, I (and Jim too) still had a problem with
>>>> this line in runjob_theia.sh:
>>>>
>>>> module load module.fre-nctools
>>>>
>>>> This syntax (where you specify a file name after "module load") is not
>>>> supported in our environment. So what we did was first change the above
>>>> line to:
>>>>
>>>> . ./module.fre-nctools
>>>>
>>>> to just source the module.fre-nctools file. Also, we changed the contents
>>>> of the file ${BASE_DIR}/fv3gfs.v0beta/release/v0/modulefiles/fv3gfs/fre-nctools.theia
>>>> to:
>>>>
>>>> module load impi/5.1.2.150 <http://5.1.2.150>
>>>> module load netcdf/4.3.0
>>>> module load hdf5/1.8.14
>>>> export HDF5_DIR=$HDF5
>>>> export NETCDF_DIR=$NETCDF
>>>> export LIBRARY_PATH=${LIBRARY_PATH}:${NETCDF}/lib:${HDF5}/lib
>>>>
>>>> (Btw, should LIBRARY_PATH be changed to LD_LIBRARY_PATH in the last line?)
>>>> With this, the remapping worked without any error messages. When I
>>>> compared the output with the baseline run, all the files were the same
>>>> EXCEPT the three 1deg files from the remapping. I don't know why those
>>>> were differnt. They weren't exactly the same but had almost the same
>>>> sizes.
>>>>
>>>> It would be nice to be able to use the syntax
>>>>
>>>> module load module.fre-nctools
>>>>
>>>> in our environment as well. When I do a "module help", I don't see this
>>>> syntax as an option (where you're specifying a file name that contains
>>>> several commands). Is there a way to get this to work on theia?
>>>>
>>>> Thanks,
>>>> Gerard
>>>>
>>>>
>>>> On Thu, May 11, 2017 at 7:00 AM, Jun Wang - NOAA Affiliate <
>>>> jun.wang at noaa.gov <mailto:jun.wang at noaa.gov>> wrote:
>>>>
>>>>> Gerard,
>>>>>
>>>>> With the fix last night, do you still have problem load the module
>>>>> module.fre-nctools?
>>>>>
>>>>> Thanks for pointing out the forecast time specified in runjob_theia.sh, I
>>>>> changed it to be consistent with the baseline.
>>>>>
>>>>> Another fix on the forecast executable is Jim Abeles are also committed
>>>>> to the tag.
>>>>>
>>>>> I am planning to commit all the changes made to the temporary beta tag to
>>>>> trunk later today for Sam to add jet extension, please send me any
>>>>> fix/suggestion. Thanks to all who are doing testing.
>>>>>
>>>>>
>>>>> Jun
>>>>>
>>>>> On Thu, May 11, 2017 at 5:03 AM, Gerard Ketefian - NOAA Affiliate <
>>>>> gerard.ketefian at noaa.gov
>>>>> <mailto:gerard.ketefian at noaa.gov>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> With Ligia's hints and Jun's last fix, I was able to complete the run
>>>>>> but not the remap. I think the remap fails because some modules don't get
>>>>>> loaded properly.
>>>>>>
>>>>>> To get the remap to also work, I replaced the following line in
>>>>>> runjob_theia.sh
>>>>>>
>>>>>> module load module.fre-nctools
>>>>>>
>>>>>> with the following block (copied and modified from the file
>>>>>> module.fre-nctools):
>>>>>>
>>>>>> module load impi/5.1.2.150 <http://5.1.2.150>
>>>>>> module load netcdf/4.3.0
>>>>>> module load hdf5/1.8.14
>>>>>> export HDF5_DIR=$HDF5
>>>>>> export NETCDF_DIR=$NETCDF
>>>>>> export LIBRARY_PATH=${LIBRARY_PATH}:${NETCDF}/lib:${HDF5}/lib
>>>>>>
>>>>>> This change should allow the 1deg remapped netcdf files to be generated.
>>>>>>
>>>>>> When I do the comparison of the sample run's netcdf files with baseline,
>>>>>> there is a about factor of 5 difference (the sample run files being
>>>>>> larger). This is because there are only 8 output times in the baseline
>>>>>> files but 40 in the run output.
>>>>>>
>>>>>> Gerard
>>>>>>
>>>>>>
>>>>>> On Wed, May 10, 2017 at 9:56 PM, Jun Wang - NOAA Affiliate <
>>>>>> jun.wang at noaa.gov <mailto:jun.wang at noaa.gov>> wrote:
>>>>>>
>>>>>>> Ligia,
>>>>>>>
>>>>>>> Thanks for the feedback. The suggestion on instruction is put in
>>>>>>> readme.txt. It is found that "cp " is missing in line 124 in
>>>>>>> runjob_theia.sh. I committed the changes to the beta test tag:
>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t
>>>>>>> <https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t>
>>>>>>> ags/fv3gfs.v0beta
>>>>>>>
>>>>>>> Please check again. The results differences will need further
>>>>>>> investigation. Thanks.
>>>>>>>
>>>>>>> Jun
>>>>>>>
>>>>>>> On Wed, May 10, 2017 at 10:28 PM, Ligia Bernardet - NOAA Affiliate <
>>>>>>> ligia.bernardet at noaa.gov
>>>>>>> <mailto:ligia.bernardet at noaa.gov>> wrote:
>>>>>>>
>>>>>>>> Folks,
>>>>>>>>
>>>>>>>> Here is some feedback
>>>>>>>>
>>>>>>>>
>>>>>>>> *About the instructions*
>>>>>>>>
>>>>>>>> 1. Minor typo. The word "trunk" should be removed: Four executable
>>>>>>>> files will be created under fv3gfs.v0beta/*trunk*/NEMS/exe
>>>>>>>> 2. runjob_theia.sh: Non-NCEPDEV people need to change directories
>>>>>>>> DATA and ROTDIR to an area they can write to
>>>>>>>> 3. diff_baseline.sh:
>>>>>>>> 1. It would be helpful to tell users to add arguments
>>>>>>>> to diff_baseline.sh to set resolution and machine.
>>>>>>>> 2. Non-NCEPDEV people need to change directory dir1 to location
>>>>>>>> of their output
>>>>>>>>
>>>>>>>> *Outcome*
>>>>>>>> It seems I was able to get through the forecast but failed in remap.
>>>>>>>> Problem seems related to loading modules, I did not fully investigate yet.
>>>>>>>> Output is in /scratch4/BMC/gmtb/Ligia.Berna
>>>>>>>> rdet/fv3gfs.v0beta/release/v0/exp
>>>>>>>>
>>>>>>>> When running diff, NetCDF files differ from the baseline. I noticed
>>>>>>>> the file sizes are different (mine are larger than the baseline).
>>>>>>>>
>>>>>>>> Ligia
>>>>>>>>
>>>>>>>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>>>>>>>> exp/../modulefiles/fv3gfs/fre-nctools.theia module.fre-nctools
>>>>>>>>
>>>>>>>> /var/spool/torque/mom_priv/jobs/23566207.bqs3.SC <http://23566207.bqs3.SC>: line 126:
>>>>>>>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>>>>>>>> exp/../modulefiles/fv3gfs/fre-nctools.theia: *Permission denied*
>>>>>>>>
>>>>>>>> + module load module.fre-nctools
>>>>>>>>
>>>>>>>> ++ /apps/lmod/lmod/libexec/lmod bash load module.fre-nctools
>>>>>>>>
>>>>>>>> Lmod has detected the following error: The following module(s) are
>>>>>>>> unknown:
>>>>>>>>
>>>>>>>> "module.fre-nctools"
>>>>>>>>
>>>>>>>> On Wed, May 10, 2017 at 5:47 PM, James Rosinski - NOAA Affiliate <
>>>>>>>> james.rosinski at noaa.gov
>>>>>>>> <mailto:james.rosinski at noaa.gov>> wrote:
>>>>>>>>
>>>>>>>>> Hi Jun;
>>>>>>>>>
>>>>>>>>> I am about to head home for the day, but here are my comments so far,
>>>>>>>>> after following the instructions for theia:
>>>>>>>>>
>>>>>>>>> o The builds of models and remap codes completed successfully. One
>>>>>>>>> suggestion might be to have the user specify 32 or 64-bit, and nh vs. hydro
>>>>>>>>> in order to cut down compilation time by a factor of 4.
>>>>>>>>>
>>>>>>>>> o The batch job attempting to run the model failed. Relevant lines in
>>>>>>>>> err_theia are:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *+ cd /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900+
>>>>>>>>> /bin/cp -p
>>>>>>>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp/../../../NEMS/exe/fv3_gfs_nh.prod.32bit.x
>>>>>>>>> /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900/.+ -prepend-rank
>>>>>>>>> -np 288 ./fv3_gfs_nh.prod.32bit.x+ ERR=127+ export ERR+ err=127*
>>>>>>>>> Looks like somehow "mpirun" was not found (note there is nothing in
>>>>>>>>> front of "-prepend-rank"). FYI I use csh for my login shell--not sure if
>>>>>>>>> this is behind the error. I had no modules loaded when submitting the job.
>>>>>>>>>
>>>>>>>>> If you'd like to examine the output you should have read access to
>>>>>>>>> it here on theia:
>>>>>>>>>
>>>>>>>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp
>>>>>>>>>
>>>>>>>>> More info tomorrow...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Jim Rosinski
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, May 10, 2017 at 2:55 PM, Jun Wang - NOAA Affiliate <
>>>>>>>>> jun.wang at noaa.gov <mailto:jun.wang at noaa.gov>> wrote:
>>>>>>>>>
>>>>>>>>>> Dear all,
>>>>>>>>>>
>>>>>>>>>> I was noticed that some directory names in the readme.txt are not
>>>>>>>>>> correct. I just updated the tag version, please let me know if you have any
>>>>>>>>>> further questions. Thanks.
>>>>>>>>>>
>>>>>>>>>> Jun
>>>>>>>>>>
>>>>>>>>>> On Wed, May 10, 2017 at 4:39 PM, Jun Wang - NOAA Affiliate <
>>>>>>>>>> jun.wang at noaa.gov <mailto:jun.wang at noaa.gov>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Rusty,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for quick feedback. Today we just had a VLAB meeting on how
>>>>>>>>>>> to provide information for the public release. Vijay mentioned that EMC
>>>>>>>>>>> will be setting up an FV3GFS community web page through VLAB, some basic
>>>>>>>>>>> document will be provided there. A formal instruction on how to get release
>>>>>>>>>>> code, and to compile and run experiment will be on that web page too. For
>>>>>>>>>>> questions/feedback, a forum will be set up for users to post questions and
>>>>>>>>>>> to provide answers &feedback, the purpose is that all the developers will
>>>>>>>>>>> see the questions/answers, it is suggested not to send questions/feedback
>>>>>>>>>>> to any individual's personal email (If people receive questions from users,
>>>>>>>>>>> we suggest that they post the questions along with their answers to the
>>>>>>>>>>> forum). The readme.txt is a temporary solution to get the testing started,
>>>>>>>>>>> it may be changed in the final release.
>>>>>>>>>>>
>>>>>>>>>>> Kate Howard (kate.howard at noaa.gov
>>>>>>>>>>> <mailto:kate.howard at noaa.gov>) is working on the VLAB fv3gfs
>>>>>>>>>>> web page, she can add the gfdl fv3gfs support email on the web page too, if
>>>>>>>>>>> you have any fv3 document for general developers, please send to her.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Jun
>>>>>>>>>>>
>>>>>>>>>>> On Wed, May 10, 2017 at 4:02 PM, Rusty Benson - NOAA Federal <
>>>>>>>>>>> rusty.benson at noaa.gov
>>>>>>>>>>> <mailto:rusty.benson at noaa.gov>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Jun and Vijay,
>>>>>>>>>>>>
>>>>>>>>>>>> In the readme.txt Q&A, you mention where to get help. Has there
>>>>>>>>>>>> been any thought to putting together a single email for tracking all
>>>>>>>>>>>> questions/requests that can be used as a basis for creating a knowledgebase
>>>>>>>>>>>> via a wiki or other forum? By segmenting FV3 and physics support, I think
>>>>>>>>>>>> we are missing an opportunity for personnel to get exposure to and learn
>>>>>>>>>>>> about system pieces for which they may not necessarily be responsible.
>>>>>>>>>>>>
>>>>>>>>>>>> If we do go the route of a single support email, we have an
>>>>>>>>>>>> existing email for FV3 support which could be used as an alias member.
>>>>>>>>>>>> Otherwise, we would want to publish the email inside of the readme.txt and
>>>>>>>>>>>> not have individual team members being contacted directly
>>>>>>>>>>>>
>>>>>>>>>>>> <oar.gfdl.fvgfs_support at noaa.gov>
>>>>>>>>>>>> <mailto:oar.gfdl.fvgfs_support at noaa.gov>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Rusty
>>>>>>>>>>>> --
>>>>>>>>>>>> Rusty Benson, PhD
>>>>>>>>>>>> Modeling Systems Group
>>>>>>>>>>>> NOAA Geophysical Fluid Dynamics Lab
>>>>>>>>>>>> Princeton, NJ
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 10, 2017 at 2:08 PM, Jun Wang - NOAA Affiliate <
>>>>>>>>>>>> jun.wang at noaa.gov <mailto:jun.wang at noaa.gov>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The following email is for people who are willing to do beta
>>>>>>>>>>>>> testing for the fv3gfs May 15 release. Please ignore the email if you are
>>>>>>>>>>>>> not going to run the test.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The svn tag for beta testing is located at:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/
>>>>>>>>>>>>> <https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/>
>>>>>>>>>>>>> tags/fv3gfs.v0beta
>>>>>>>>>>>>>
>>>>>>>>>>>>> The instruction file on how to get and compile the code and to
>>>>>>>>>>>>> run an experiment is at:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t
>>>>>>>>>>>>> <https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t>
>>>>>>>>>>>>> ags/fv3gfs.v0beta/release/v0/readme.txt
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please follow the instructions to see if you can run an
>>>>>>>>>>>>> experiment.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Ncep.list.fv3-announce mailing list
>>>>>>>>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>>>>>>>>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>>>>>>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list
>>>>>>>>>>>>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list>.
>>>>>>>>>>>>> fv3-announce
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Ncep.list.fv3-announce mailing list
>>>>>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>>>>>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>>>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list
>>>>>>>>>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list>.
>>>>>>>>>> fv3-announce
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ncep.list.fv3-announce mailing list
>>>>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>>>>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list
>>>>>>>>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list>.
>>>>>>>>> fv3-announce
>>>>>>>>>
>>>>>>>>>
>>>>>> --
>>>>>> Gerard Ketefian
>>>>>> Research Scientist
>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>>>> 325 Broadway
>>>>>> Boulder, CO 80305
>>>>>> phone:303-497-6209 <tel:%28303%29%20497-6209> <(303)%20497-6209>
>>>>>>
>>>>>>
>>>> --
>>>> Gerard Ketefian
>>>> Research Scientist
>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>> 325 Broadway
>>>> Boulder, CO 80305
>>>> phone:303-497-6209 <tel:%28303%29%20497-6209>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ncep.list.fv3-announce mailing list
>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>>>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce>
>>>>
>>>>
>>
>> --
>> EUGENE MIRVIS, Tech Lead,
>> Senior Computational Scientist, IMSG @
>> Global Climate & Weather Modeling Branch of
>> NOAA/NCEP Environmental Modeling Center
>> NCWCP, rm 2183
>> 5830 University Research Ct.
>> College Park, MD 20740
>> Ph.301.683.3809 <tel:%28301%29%20683-3809>
>>
>>
>> _______________________________________________
>> Ncep.list.fv3-announce mailing list
>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce>
>>
>>
>>
>> _______________________________________________
>> Ncep.list.fv3-announce mailing list
>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ncep.list.fv3-announce mailing list
>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce>
>
>
>
> _______________________________________________
> Ncep.list.fv3-announce mailing list
> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
> <mailto:Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov>
> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
> <https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20170512/902ff0b8/attachment-0001.html
More information about the Ncep.list.fv3-announce
mailing list