[Ncep.list.fv3-announce] fv3gfs release beta test

[Rusty Benson - NOAA Federal]

Eugene,

There is code inside of FMS that relies on Cray pointers because Fortran
pointers don't have tne needed functionality for the way they are being
used.

Cray recommended it because you are in a cross-compiling environment and
you need to tell the compiler to target a specific instruction set for best
performance.

Rusty
--
Rusty Benson, PhD
Modeling Systems Group
NOAA Geophysical Fluid Dynamics Lab
Princeton, NJ

On Fri, May 12, 2017 at 1:50 PM, Eugene Mirvis <eugene.mirvis at noaa.gov>
wrote:

> Rusty,
>
> It may be standard.
>  However, we implemented "-xcore-AVX2" by the vendor recommendation to
> make compatible build for both the sandybridge and haswell.
> As far as I know we never implemented that for NEMS on any other machines
> but on WCOSS-cray for this purpose. with ftn wrapper. Not mpiifort.
>
> -safe-cray-ptr  yes, it tells the compiler that Cray pointers do not alias
> (that is, do not specify sharing memory with) other variables.
> Do we have any?
>
> I just think that it happened  because of the conf structure hierarchy.
> Now, if it has been done on Theia on purpuse - please let me know.
>
> Thanks,
> -Eugene
>
>
>
>
>
>
> On 5/12/2017 1:30 PM, Rusty Benson - NOAA Federal wrote:
>
>> Eugene,
>>
>> These are standard Intel compiler options and have no relationship to a
>> particular vendor's system.  "-xcore-AVX2" describes the instruction set
>> support you are requesting from the compiler and will generate
>> instructions
>> suitable for the latest *well family of processors from Intel.
>>
>> "-safe-cray-ptr" deals with a Fortran extension known as "Cray pointers"
>> and states that your use of them is unique and it is safe to optimize code
>> containing these objects.
>>
>> Rusty
>> --
>> Rusty Benson, PhD
>> Modeling Systems Group
>> NOAA Geophysical Fluid Dynamics Lab
>> Princeton, NJ
>>
>> On Fri, May 12, 2017 at 1:23 PM, Eugene Mirvis <eugene.mirvis at noaa.gov>
>> wrote:
>>
>> Rusty,
>>>
>>> For instance the flags "-xCORE-AVX2" and "-safe-cray-ptr" we use only on
>>> Crays on the certain purposes.
>>>
>>> -Eugene
>>>
>>> On 5/12/2017 12:16 PM, Rusty Benson - NOAA Federal wrote:
>>>
>>> Eugene,
>>>>
>>>> Responding only to point 4, what options do you see that are Cray
>>>> compilation flags?
>>>>
>>>> Rusty
>>>> --
>>>> Rusty Benson, PhD
>>>> Modeling Systems Group
>>>> NOAA Geophysical Fluid Dynamics Lab
>>>> Princeton, NJ
>>>>
>>>> On Fri, May 12, 2017 at 11:58 AM, Eugene Mirvis <eugene.mirvis at noaa.gov
>>>> >
>>>> wrote:
>>>>
>>>> Gerard,
>>>>
>>>>> Just several points to clarify.
>>>>>
>>>>> 1. NEMS practice  to call modulefiles what is actually the scripts
>>>>> (calling module commands),
>>>>> that require mostly bash env in order to source and keep environment
>>>>> was
>>>>> always non standard use.
>>>>>
>>>>> Therefore,  if the developers will take Sam's advice and make real
>>>>> modulefiles (starting with #%Module) from the script, "export" and
>>>>> other
>>>>> scripting works
>>>>> wouldn't make sense, while Module util commands and Tcl/Tk  will work.
>>>>>
>>>>> 2. There is another dilemma - to keep needed environment and change
>>>>> within
>>>>> a workflow env. change.
>>>>> btw,
>>>>> unsetenv, append-path, prepend-path and remove-path  module commands
>>>>> are
>>>>> very useful for controlling that, but you have to keep
>>>>> $LOADEDMODULES, and $MODULE PATH in consistent order.
>>>>>
>>>>> 3. Speaking of which,
>>>>> module purge
>>>>> and
>>>>> module switch
>>>>> are very useful to unload application driven modules.
>>>>> a/ You just have to do that not any moment, but before apps module is
>>>>> loaded, then, unload <appsModulefile> - not purge, but unload.
>>>>> b/ On Crays, the are some internal dependencies  inside of PrgEnv. So
>>>>> you
>>>>> have to keep all "module use <knowns>"  to recover, otherwise you might
>>>>> find "module not found"
>>>>>
>>>>> 4.
>>>>> Compiling on Theia, I'm just wondering why Cray's compilation flags are
>>>>> utilized... allover:
>>>>> See for  instance
>>>>> ...
>>>>> *mpiifort -I/apps/netcdf/4.3.0-intel/include -fno-alias -auto
>>>>> -safe-cray-ptr -ftz -assume byterecl -nowarn -sox -align array64byte
>>>>> -i4
>>>>> -real-size 64 -no-prec-div -no-prec-sqrt -xCORE-AVX2**
>>>>> -qno-opt-dynamic-align* -O2 -debug minimal -fp-model source
>>>>> -qoverride-limits -qopt-prefetch=3 -qopenmp -I/apps/esmf/7.0.0/intel/
>>>>> intelmpi/mod/modO/Linux.intel.64.intelmpi.default
>>>>> -I/apps/esmf/7.0.0/intel/intelmpi/include
>>>>> -I/apps/netcdf/4.3.0-intel/inc
>>>>> lude
>>>>> -IENS_Cpl -I.  -I/scratch4/NCEPDEV/global/nos
>>>>> crub/Eugene.Mirvis/fv3gfs.v0beta/FV3/nems_dir
>>>>> -c module_MEDIATOR_methods.f90
>>>>> ...
>>>>>
>>>>> Thanks,
>>>>> -Eugene
>>>>>
>>>>>
> --
> EUGENE MIRVIS, Tech Lead,
> Senior Computational Scientist, IMSG @
>  Global Climate & Weather Modeling Branch of
>   NOAA/NCEP Environmental Modeling Center
>             NCWCP,  rm  2183
>      5830 University Research Ct.
>         College Park, MD 20740
>             Ph. 301.683.3809
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20170512/9ed52303/attachment-0001.html