<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">How can we speed up compiling
"GFS_restart"? It takes for ever.<br>
Thanks<br>
Moorthi<br>
On 05/12/2017 01:52 PM, Samuel Trahan - NOAA Affiliate wrote:<br>
</div>
<blockquote
cite="mid:CAJ3i2We66uxMpa_JyRJ=xHdhg-nEOHC1Z_sEFoQDbcC1TH=YPA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>Eugene,<br>
<br>
</div>
That flag selects the intel processor architecture to
target. <br>
<br>
</div>
On Jet, we target two by using -axavx which creates two code
paths: one for the old processors and one for the new. It
does take a while to compile, though.<br>
<br>
</div>
Sincerely,<br>
</div>
Sam Trahan<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May 12, 2017 at 1:50 PM, Eugene
Mirvis <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:eugene.mirvis@noaa.gov" target="_blank">eugene.mirvis@noaa.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Rusty,<br>
<br>
It may be standard.<br>
However, we implemented "-xcore-AVX2" by the vendor
recommendation to make compatible build for both the
sandybridge and haswell.<br>
As far as I know we never implemented that for NEMS on any
other machines but on WCOSS-cray for this purpose. with ftn
wrapper. Not mpiifort.<br>
<br>
-safe-cray-ptr yes, it tells the compiler that Cray
pointers do not alias (that is, do not specify sharing
memory with) other variables.<br>
Do we have any?<br>
<br>
I just think that it happened because of the conf structure
hierarchy.<br>
Now, if it has been done on Theia on purpuse - please let me
know.<br>
<br>
Thanks,<br>
-Eugene
<div class="HOEnZb">
<div class="h5"><br>
<br>
<br>
<br>
<br>
<br>
<br>
On 5/12/2017 1:30 PM, Rusty Benson - NOAA Federal wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Eugene,<br>
<br>
These are standard Intel compiler options and have no
relationship to a<br>
particular vendor's system. "-xcore-AVX2" describes
the instruction set<br>
support you are requesting from the compiler and will
generate instructions<br>
suitable for the latest *well family of processors
from Intel.<br>
<br>
"-safe-cray-ptr" deals with a Fortran extension known
as "Cray pointers"<br>
and states that your use of them is unique and it is
safe to optimize code<br>
containing these objects.<br>
<br>
Rusty<br>
--<br>
Rusty Benson, PhD<br>
Modeling Systems Group<br>
NOAA Geophysical Fluid Dynamics Lab<br>
Princeton, NJ<br>
<br>
On Fri, May 12, 2017 at 1:23 PM, Eugene Mirvis <<a
moz-do-not-send="true"
href="mailto:eugene.mirvis@noaa.gov" target="_blank">eugene.mirvis@noaa.gov</a>><br>
wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Rusty,<br>
<br>
For instance the flags "-xCORE-AVX2" and
"-safe-cray-ptr" we use only on<br>
Crays on the certain purposes.<br>
<br>
-Eugene<br>
<br>
On 5/12/2017 12:16 PM, Rusty Benson - NOAA Federal
wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Eugene,<br>
<br>
Responding only to point 4, what options do you
see that are Cray<br>
compilation flags?<br>
<br>
Rusty<br>
--<br>
Rusty Benson, PhD<br>
Modeling Systems Group<br>
NOAA Geophysical Fluid Dynamics Lab<br>
Princeton, NJ<br>
<br>
On Fri, May 12, 2017 at 11:58 AM, Eugene Mirvis
<<a moz-do-not-send="true"
href="mailto:eugene.mirvis@noaa.gov"
target="_blank">eugene.mirvis@noaa.gov</a>><br>
wrote:<br>
<br>
Gerard,<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
Just several points to clarify.<br>
<br>
1. NEMS practice to call modulefiles what is
actually the scripts<br>
(calling module commands),<br>
that require mostly bash env in order to source
and keep environment was<br>
always non standard use.<br>
<br>
Therefore, if the developers will take Sam's
advice and make real<br>
modulefiles (starting with #%Module) from the
script, "export" and other<br>
scripting works<br>
wouldn't make sense, while Module util commands
and Tcl/Tk will work.<br>
<br>
2. There is another dilemma - to keep needed
environment and change<br>
within<br>
a workflow env. change.<br>
btw,<br>
unsetenv, append-path, prepend-path and
remove-path module commands are<br>
very useful for controlling that, but you have
to keep<br>
$LOADEDMODULES, and $MODULE PATH in consistent
order.<br>
<br>
3. Speaking of which,<br>
module purge<br>
and<br>
module switch<br>
are very useful to unload application driven
modules.<br>
a/ You just have to do that not any moment, but
before apps module is<br>
loaded, then, unload <appsModulefile> -
not purge, but unload.<br>
b/ On Crays, the are some internal dependencies
inside of PrgEnv. So you<br>
have to keep all "module use <knowns>" to
recover, otherwise you might<br>
find "module not found"<br>
<br>
4.<br>
Compiling on Theia, I'm just wondering why
Cray's compilation flags are<br>
utilized... allover:<br>
See for instance<br>
...<br>
*mpiifort -I/apps/netcdf/4.3.0-intel/inc<wbr>lude
-fno-alias -auto<br>
-safe-cray-ptr -ftz -assume byterecl -nowarn
-sox -align array64byte -i4<br>
-real-size 64 -no-prec-div -no-prec-sqrt
-xCORE-AVX2**<br>
-qno-opt-dynamic-align* -O2 -debug minimal
-fp-model source<br>
-qoverride-limits -qopt-prefetch=3 -qopenmp
-I/apps/esmf/7.0.0/intel/<br>
intelmpi/mod/modO/Linux.intel.<wbr>64.intelmpi.default<br>
-I/apps/esmf/7.0.0/intel/intel<wbr>mpi/include
-I/apps/netcdf/4.3.0-intel/inc<br>
lude<br>
-IENS_Cpl -I. -I/scratch4/NCEPDEV/global/nos<br>
crub/Eugene.Mirvis/fv3gfs.v0be<wbr>ta/FV3/nems_dir<br>
-c module_MEDIATOR_methods.f90<br>
...<br>
<br>
Thanks,<br>
-Eugene<br>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
</div>
</div>
<div class="HOEnZb">
<div class="h5">
-- <br>
EUGENE MIRVIS, Tech Lead,<br>
Senior Computational Scientist, IMSG @<br>
Global Climate & Weather Modeling Branch of<br>
NOAA/NCEP Environmental Modeling Center<br>
NCWCP, rm 2183<br>
5830 University Research Ct.<br>
College Park, MD 20740<br>
Ph. 301.683.3809<br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Ncep.list.fv3-announce mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov">Ncep.list.fv3-announce@lstsrv.ncep.noaa.gov</a>
<a class="moz-txt-link-freetext" href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce">https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce</a>
</pre>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Dr. Shrinivas Moorthi
Research Meteorologist
Modeling and Data Assimilation Branch
Environmental Modeling Center / National Centers for Environmental Prediction
5830 University Research Court - (W/NP23), College Park MD 20740 USA
Tel:(301)683-3718</pre>
</body>
</html>