<div dir="ltr"><div><div><div><div><div>The reasons for moving to a modulefile:<br><br></div>1. You can unload the modulefile.<br><br></div>2. The same file works in all shells. Previously, if you used a tcsh login, you could not load modules.nems. This was a major problem for some users.<br><br></div>3. It prevents adding "source /etc/profile" to the modulefile, which causes problems if /etc/profile has already been sourced.<br><br></div><div>There is now a pair of shell include files, NEMS/src/conf/module-setup.*sh.inc, which know how to obtain the module command and purge modules on seven clusters and five shells. We also have a test system for it in NEMS/tests/modulecmd/. That replaces "source /etc/profile" and "module purge."<br></div><div><br></div>Sincerely,<br></div>Sam Trahan<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 12, 2017 at 1:59 PM, Dusan Jovic <span dir="ltr"><<a href="mailto:dusan.jovic@noaa.gov" target="_blank">dusan.jovic@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_6702000289726040642moz-cite-prefix">
<p>Can some of you 'module' experts please explain why is writing
real modulefiles (a file that starts with #%Module) and then
loading that file with 'module load' better then writing a text
file with just a bunch of module load commands which is then
"sourced" from a build or run scripts? We used to use the second
approach in "the old NEMS" for many years and never had any
problem.<br>
</p>
<p>What are the advantages and are there any disadvantages?</p>
<p>Thanks,</p>
<p>Dusan</p><div><div class="h5">
<br>
On 05/12/2017 12:40 PM, Jun Wang - NOAA Affiliate wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Jim,<br>
<br>
</div>
Sam is working on that, we hope to make it to this release.
Thanks.<br>
<br>
</div>
Jun<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May 12, 2017 at 12:37 PM, James
Rosinski - NOAA Affiliate <span dir="ltr"><<a href="mailto:james.rosinski@noaa.gov" target="_blank">james.rosinski@noaa.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>Jun and others;<br>
<br>
</div>
I noticed there is a build capability for fv3gfs on
jet, but not any run instructions or runjob_jet.sh for
jet. Is jet run capability intended, or is that for
later?<br>
<br>
</div>
Regards,<br>
</div>
Jim Rosinski<br>
<div>
<div><br>
<br>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span>On Fri, May
12, 2017 at 10:16 AM, Rusty Benson - NOAA
Federal <span dir="ltr"><<a href="mailto:rusty.benson@noaa.gov" target="_blank">rusty.benson@noaa.gov</a>></span>
wrote:<br>
</span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>
<div dir="ltr">Eugene,
<div><br>
</div>
<div>Responding only to point 4, what
options do you see that are Cray
compilation flags?<br>
</div>
</div>
</span>
<div class="gmail_extra"><span><br clear="all">
<span>
<div>
<div class="m_6702000289726040642m_4349914553471657777m_11006847854449668gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div style="font-size:12.8px">Rusty</div>
<div style="font-size:12.8px">--</div>
<div style="font-size:12.8px">Rusty
Benson, PhD</div>
<div style="font-size:12.8px">Modeling
Systems Group</div>
<div style="font-size:12.8px">NOAA
Geophysical Fluid Dynamics
Lab</div>
<div style="font-size:12.8px">Princeton,
NJ</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</span></span>
<div>
<div class="m_6702000289726040642m_4349914553471657777h5">
<div class="gmail_quote">
<div>
<div class="m_6702000289726040642h5">On Fri, May 12, 2017
at 11:58 AM, Eugene Mirvis <span dir="ltr"><<a href="mailto:eugene.mirvis@noaa.gov" target="_blank">eugene.mirvis@noaa.gov</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>
<div class="m_6702000289726040642h5">
<div class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-cite-prefix">Gerard,<br>
<br>
Just several points to
clarify.<br>
<br>
1. NEMS practice to call
modulefiles what is actually
the scripts (calling module
commands),<br>
that require mostly bash env
in order to source and keep
environment was always non
standard use.<br>
<br>
Therefore, if the developers
will take Sam's advice and
make real modulefiles
(starting with #%Module) from
the script, "export" and other
scripting works <br>
wouldn't make sense, while
Module util commands and
Tcl/Tk will work.<br>
<br>
2. There is another dilemma -
to keep needed environment and
change within a workflow env.
change.<br>
btw, <b><br>
</b>unsetenv, append-path,
prepend-path and remove-path
module commands are very
useful for controlling that,
but you have to keep<br>
<font size="-1">$LOADEDMODULES,
and $MODULE PATH in
consistent order.<br>
</font><br>
3. Speaking of which<font size="-1">, <br>
module purge <br>
and<br>
module switch <br>
are very useful to unload
application driven modules.<br>
a/ You just have to do that
not any moment, but before
apps module is loaded, then,
unload
<appsModulefile> - not
purge, but unload. <br>
</font>b/ On Crays, the are
some internal dependencies
inside of PrgEnv. So you have
to keep all "module use
<knowns>" to recover,
otherwise you might find
"module not found"<br>
<br>
4. <br>
Compiling on Theia, I'm just
wondering why Cray's
compilation flags are
utilized... allover: <br>
See for instance<br>
...<br>
<b>mpiifort
-I/apps/netcdf/4.3.0-intel/inc<wbr>lude
-fno-alias -auto
-safe-cray-ptr -ftz -assume
byterecl -nowarn -sox -align
array64byte -i4 -real-size
64 -no-prec-div
-no-prec-sqrt -xCORE-AVX2</b><b>
-qno-opt-dynamic-align</b>
-O2 -debug minimal -fp-model
source -qoverride-limits
-qopt-prefetch=3 -qopenmp
-I/apps/esmf/7.0.0/intel/intel<wbr>mpi/mod/modO/<a href="http://Linux.intel.64.in" target="_blank">Linux.intel.64.in</a><wbr>telmpi.default
-I/apps/esmf/7.0.0/intel/intel<wbr>mpi/include
-I/apps/netcdf/4.3.0-intel/inc<wbr>lude
-IENS_Cpl -I.
-I/scratch4/NCEPDEV/global/nos<wbr>crub/Eugene.Mirvis/fv3gfs.v0be<wbr>ta/FV3/nems_dir
-c module_MEDIATOR_methods.f90<br>
...<br>
<br>
Thanks,<br>
-Eugene<br>
On 5/12/2017 3:27 AM, Samuel
Trahan - NOAA Affiliate wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<pre><div><div class="m_6702000289726040642h5">Gerard,
The "module load module.free-nctools" is failing because that file contains
commands that are not valid in a modulefile. You can have as many "module"
commands as you want in a modulefile, but you cannot have bash code. Also,
it needs to start with #%Module
export A=B # bad
setenv A B
prepend-path A (something)
No "source /etc/profile"
No "source /.../init/bash"
No "module purge" because that causes infinite loops on some platforms.
(The module is trying to unload itself.)
Obtaining the "module" command and purging modules has to be done before
loading the module. We have a pair of scripts (one csh, one sh) that do
that on all NOAA machines. They are aware of csh, tcsh, ksh, bash, and sh;
and run the correct initialization for each. The master copies are in
NEMS/src/conf/module-setup.*sh<wbr>.inc (csh and sh versions). Those scripts
work on all NOAA machines and have a test suite inside NEMS/tests.
man 4 modulefile
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="http://modules.sourceforge.net/c/modulefile.html" target="_blank">http://modules.sourceforge.net<wbr>/c/modulefile.html</a></div></div>
Sincerely,
Sam Trahan
On Thu, May 11, 2017 at 5:01 PM, Gerard Ketefian - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:gerard.ketefian@noaa.gov" target="_blank">gerard.ketefian@noaa.gov</a>> wrote:
</pre>
<blockquote type="cite">
<div>
<div class="m_6702000289726040642h5">
<pre>Hi Jun,
Yes, after the missing "cp" fix, I (and Jim too) still had a problem with
this line in runjob_theia.sh:
module load module.fre-nctools
This syntax (where you specify a file name after "module load") is not
supported in our environment. So what we did was first change the above
line to:
. ./module.fre-nctools
to just source the module.fre-nctools file. Also, we changed the contents
of the file ${BASE_DIR}/fv3gfs.v0beta/rele<wbr>ase/v0/modulefiles/fv3gfs/fre-<wbr>nctools.theia
to:
module load impi/<a href="http://5.1.2.150" target="_blank">5.1.2.150</a>
module load netcdf/4.3.0
module load hdf5/1.8.14
export HDF5_DIR=$HDF5
export NETCDF_DIR=$NETCDF
export LIBRARY_PATH=${LIBRARY_PATH}:$<wbr>{NETCDF}/lib:${HDF5}/lib
(Btw, should LIBRARY_PATH be changed to LD_LIBRARY_PATH in the last line?)
With this, the remapping worked without any error messages. When I
compared the output with the baseline run, all the files were the same
EXCEPT the three 1deg files from the remapping. I don't know why those
were differnt. They weren't exactly the same but had almost the same
sizes.
It would be nice to be able to use the syntax
module load module.fre-nctools
in our environment as well. When I do a "module help", I don't see this
syntax as an option (where you're specifying a file name that contains
several commands). Is there a way to get this to work on theia?
Thanks,
Gerard
On Thu, May 11, 2017 at 7:00 AM, Jun Wang - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:jun.wang@noaa.gov" target="_blank">jun.wang@noaa.gov</a>> wrote:
</pre>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_6702000289726040642h5">
<pre>Gerard,
With the fix last night, do you still have problem load the module
module.fre-nctools?
Thanks for pointing out the forecast time specified in runjob_theia.sh, I
changed it to be consistent with the baseline.
Another fix on the forecast executable is Jim Abeles are also committed
to the tag.
I am planning to commit all the changes made to the temporary beta tag to
trunk later today for Sam to add jet extension, please send me any
fix/suggestion. Thanks to all who are doing testing.
Jun
On Thu, May 11, 2017 at 5:03 AM, Gerard Ketefian - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:gerard.ketefian@noaa.gov" target="_blank">gerard.ketefian@noaa.gov</a>> wrote:
</pre>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_6702000289726040642h5">
<pre>Hi all,
With Ligia's hints and Jun's last fix, I was able to complete the run
but not the remap. I think the remap fails because some modules don't get
loaded properly.
To get the remap to also work, I replaced the following line in
runjob_theia.sh
module load module.fre-nctools
with the following block (copied and modified from the file
module.fre-nctools):
module load impi/<a href="http://5.1.2.150" target="_blank">5.1.2.150</a>
module load netcdf/4.3.0
module load hdf5/1.8.14
export HDF5_DIR=$HDF5
export NETCDF_DIR=$NETCDF
export LIBRARY_PATH=${LIBRARY_PATH}:$<wbr>{NETCDF}/lib:${HDF5}/lib
This change should allow the 1deg remapped netcdf files to be generated.
When I do the comparison of the sample run's netcdf files with baseline,
there is a about factor of 5 difference (the sample run files being
larger). This is because there are only 8 output times in the baseline
files but 40 in the run output.
Gerard
On Wed, May 10, 2017 at 9:56 PM, Jun Wang - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:jun.wang@noaa.gov" target="_blank">jun.wang@noaa.gov</a>> wrote:
</pre>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_6702000289726040642h5">
<pre>Ligia,
Thanks for the feedback. The suggestion on instruction is put in
readme.txt. It is found that "cp " is missing in line 124 in
runjob_theia.sh. I committed the changes to the beta test tag:
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/t</a>
ags/fv3gfs.v0beta
Please check again. The results differences will need further
investigation. Thanks.
Jun
On Wed, May 10, 2017 at 10:28 PM, Ligia Bernardet - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:ligia.bernardet@noaa.gov" target="_blank">ligia.bernardet@noaa.gov</a>> wrote:
</pre>
</div>
</div>
<blockquote type="cite">
<pre>Folks,
Here is some feedback
*About the instructions*
1. Minor typo. The word "trunk" should be removed: Four executable
files will be created under fv3gfs.v0beta/*trunk*/NEMS/exe
2. runjob_theia.sh: Non-NCEPDEV people need to change directories
DATA and ROTDIR to an area they can write to
3. diff_baseline.sh:
1. It would be helpful to tell users to add arguments
to diff_baseline.sh to set resolution and machine.
2. Non-NCEPDEV people need to change directory dir1 to location
of their output
*Outcome*
It seems I was able to get through the forecast but failed in remap.
Problem seems related to loading modules, I did not fully investigate yet.
Output is in /scratch4/BMC/gmtb/Ligia.Berna
rdet/fv3gfs.v0beta/release/v0/<wbr>exp
When running diff, NetCDF files differ from the baseline. I noticed
the file sizes are different (mine are larger than the baseline).
Ligia
/scratch4/BMC/gmtb/Ligia.Berna<wbr>rdet/fv3gfs.v0beta/release/v0/
exp/../modulefiles/fv3gfs/fre-<wbr>nctools.theia module.fre-nctools
/var/spool/torque/mom_priv/job<wbr>s/<a href="http://23566207.bqs3.SC" target="_blank">23566207.bqs3.SC</a>: line 126:
/scratch4/BMC/gmtb/Ligia.Berna<wbr>rdet/fv3gfs.v0beta/release/v0/
exp/../modulefiles/fv3gfs/fre-<wbr>nctools.theia: *Permission denied*
+ module load module.fre-nctools
++ /apps/lmod/lmod/libexec/lmod bash load module.fre-nctools
Lmod has detected the following error: The following module(s) are
unknown:
"module.fre-nctools"
On Wed, May 10, 2017 at 5:47 PM, James Rosinski - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:james.rosinski@noaa.gov" target="_blank">james.rosinski@noaa.gov</a>> wrote:
</pre>
<blockquote type="cite">
<pre>Hi Jun;
I am about to head home for the day, but here are my comments so far,
after following the instructions for theia:
o The builds of models and remap codes completed successfully. One
suggestion might be to have the user specify 32 or 64-bit, and nh vs. hydro
in order to cut down compilation time by a factor of 4.
o The batch job attempting to run the model failed. Relevant lines in
err_theia are:
*+ cd /scratch4/NCEPDEV/stmp3/James.<wbr>Rosinski/C96fv3gfs2016092900+
/bin/cp -p
/scratch3/BMC/gsd-hpcs/rosinsk<wbr>i/fv3gfs.v0beta/release/v0/exp<wbr>/../../../NEMS/exe/fv3_gfs_nh.<wbr>prod.32bit.x
/scratch4/NCEPDEV/stmp3/James.<wbr>Rosinski/C96fv3gfs2016092900/.<wbr>+ -prepend-rank
-np 288 ./fv3_gfs_nh.prod.32bit.x+ ERR=127+ export ERR+ err=127*
Looks like somehow "mpirun" was not found (note there is nothing in
front of "-prepend-rank"). FYI I use csh for my login shell--not sure if
this is behind the error. I had no modules loaded when submitting the job.
If you'd like to examine the output you should have read access to
it here on theia:
/scratch3/BMC/gsd-hpcs/rosinsk<wbr>i/fv3gfs.v0beta/release/v0/exp
More info tomorrow...
Regards,
Jim Rosinski
On Wed, May 10, 2017 at 2:55 PM, Jun Wang - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:jun.wang@noaa.gov" target="_blank">jun.wang@noaa.gov</a>> wrote:
</pre>
<div>
<div class="m_6702000289726040642h5">
<blockquote type="cite">
<pre>Dear all,
I was noticed that some directory names in the readme.txt are not
correct. I just updated the tag version, please let me know if you have any
further questions. Thanks.
Jun
On Wed, May 10, 2017 at 4:39 PM, Jun Wang - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:jun.wang@noaa.gov" target="_blank">jun.wang@noaa.gov</a>> wrote:
</pre>
<blockquote type="cite">
<pre>Rusty,
Thanks for quick feedback. Today we just had a VLAB meeting on how
to provide information for the public release. Vijay mentioned that EMC
will be setting up an FV3GFS community web page through VLAB, some basic
document will be provided there. A formal instruction on how to get release
code, and to compile and run experiment will be on that web page too. For
questions/feedback, a forum will be set up for users to post questions and
to provide answers &feedback, the purpose is that all the developers will
see the questions/answers, it is suggested not to send questions/feedback
to any individual's personal email (If people receive questions from users,
we suggest that they post the questions along with their answers to the
forum). The readme.txt is a temporary solution to get the testing started,
it may be changed in the final release.
Kate Howard (<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:kate.howard@noaa.gov" target="_blank">kate.howard@noaa.gov</a>) is working on the VLAB fv3gfs
web page, she can add the gfdl fv3gfs support email on the web page too, if
you have any fv3 document for general developers, please send to her.
Thanks.
Jun
On Wed, May 10, 2017 at 4:02 PM, Rusty Benson - NOAA Federal <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:rusty.benson@noaa.gov" target="_blank">rusty.benson@noaa.gov</a>> wrote:
</pre>
<blockquote type="cite">
<pre>Hi Jun and Vijay,
In the readme.txt Q&A, you mention where to get help. Has there
been any thought to putting together a single email for tracking all
questions/requests that can be used as a basis for creating a knowledgebase
via a wiki or other forum? By segmenting FV3 and physics support, I think
we are missing an opportunity for personnel to get exposure to and learn
about system pieces for which they may not necessarily be responsible.
If we do go the route of a single support email, we have an
existing email for FV3 support which could be used as an alias member.
Otherwise, we would want to publish the email inside of the readme.txt and
not have individual team members being contacted directly
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-rfc2396E" href="mailto:oar.gfdl.fvgfs_support@noaa.gov" target="_blank"><oar.gfdl.fvgfs_support@noaa.g<wbr>ov></a>
Rusty
--
Rusty Benson, PhD
Modeling Systems Group
NOAA Geophysical Fluid Dynamics Lab
Princeton, NJ
On Wed, May 10, 2017 at 2:08 PM, Jun Wang - NOAA Affiliate <
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:jun.wang@noaa.gov" target="_blank">jun.wang@noaa.gov</a>> wrote:
</pre>
<blockquote type="cite">
<pre>Dear all,
The following email is for people who are willing to do beta
testing for the fv3gfs May 15 release. Please ignore the email if you are
not going to run the test.
The svn tag for beta testing is located at:
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/</a>
tags/fv3gfs.v0beta
The instruction file on how to get and compile the code and to
run an experiment is at:
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/t</a>
ags/fv3gfs.v0beta/release/v0/r<wbr>eadme.txt
Please follow the instructions to see if you can run an
experiment.
Thanks.
Jun
______________________________<wbr>_________________
Ncep.list.fv3-announce mailing list
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a>
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list</a>.
fv3-announce
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre>______________________________<wbr>_________________
Ncep.list.fv3-announce mailing list
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a>
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list</a>.
fv3-announce
</pre>
</blockquote>
<pre>______________________________<wbr>_________________
Ncep.list.fv3-announce mailing list
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a>
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list</a>.
fv3-announce
</pre>
</div>
</div>
</blockquote>
</blockquote>
</blockquote>
<pre>--
Gerard Ketefian
Research Scientist
NOAA/OAR/ESRL/GSD/EMB, R/GSD1
325 Broadway
Boulder, CO 80305
phone: <a href="tel:%28303%29%20497-6209" value="+13034976209" target="_blank">303-497-6209</a> <(303)%20497-6209>
</pre>
</blockquote>
</blockquote>
<span>
<pre>--
Gerard Ketefian
Research Scientist
NOAA/OAR/ESRL/GSD/EMB, R/GSD1
325 Broadway
Boulder, CO 80305
phone: <a href="tel:%28303%29%20497-6209" value="+13034976209" target="_blank">303-497-6209</a>
______________________________<wbr>_________________
Ncep.list.fv3-announce mailing list
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a>
<a class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>fv3-announce</a>
</pre>
</span></blockquote>
<span class="m_6702000289726040642HOEnZb"><font color="#888888"> <span class="m_6702000289726040642m_4349914553471657777m_11006847854449668HOEnZb"><font color="#888888"> </font></span></font></span></blockquote>
<span class="m_6702000289726040642HOEnZb"><font color="#888888"><span class="m_6702000289726040642m_4349914553471657777m_11006847854449668HOEnZb"><font color="#888888">
<p><br>
</p>
<pre class="m_6702000289726040642m_4349914553471657777m_11006847854449668m_-1807567512764868220moz-signature" cols="72">--
EUGENE MIRVIS, Tech Lead,
Senior Computational Scientist, IMSG @
Global Climate & Weather Modeling Branch of
NOAA/NCEP Environmental Modeling Center
NCWCP, rm 2183
5830 University Research Ct.
College Park, MD 20740
Ph. <a href="tel:%28301%29%20683-3809" value="+13016833809" target="_blank">301.683.3809</a></pre>
</font></span></font></span></div>
<span class="m_6702000289726040642HOEnZb"><font color="#888888">
<br>
</font></span><span>______________________________<wbr>_________________<br>
Ncep.list.fv3-announce mailing list<br>
<a href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a><br>
<a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>fv3-announce</a><br>
<br>
</span></blockquote>
</div>
<br>
</div>
</div>
</div>
<span>
<br>
______________________________<wbr>_________________<br>
Ncep.list.fv3-announce mailing list<br>
<a href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a><br>
<a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>fv3-announce</a><br>
<br>
</span></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="m_6702000289726040642mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Ncep.list.fv3-announce mailing list
<a class="m_6702000289726040642moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a>
<a class="m_6702000289726040642moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.fv3-announce</a>
</pre>
</blockquote>
<p><br>
</p>
</div></div></div>
<br>______________________________<wbr>_________________<br>
Ncep.list.fv3-announce mailing list<br>
<a href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov">Ncep.list.fv3-announce@lstsrv.<wbr>ncep.noaa.gov</a><br>
<a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.fv3-announce</a><br>
<br></blockquote></div><br></div>