[Ncep.list.fv3-announce] fv3gfs release beta test

Samuel Trahan - NOAA Affiliate samuel.trahan at noaa.gov
Fri May 12 17:39:39 UTC 2017


Rusty,

Yes, I have also noticed Jet MVAPICH2 to be problematic when placing
ranks.  I have not worked on optimizing the FV3 on Jet yet; I only made it
function.  Once we get closer to the point of running FV3 workflows on Jet,
we can focus more on optimization.

Sincerely,
Sam Trahan

On Fri, May 12, 2017 at 1:31 PM, Rusty Benson - NOAA Federal <
rusty.benson at noaa.gov> wrote:

> I don't know how much experience people have with Jet, but please be aware
> there is a mpirun wrapper that supersedes the package version (impi,
> mvapich, etc).  I point this out because the behavior is very different
> than the experience on Theia, where one uses the package version mpirun
> directly.
>
> We at GFDL have found Jet's wrapper to be particularly bothersome when
> trying to run with OpenMP as we place/space MPI-ranks explicitly as part of
> the mpirun request and internally manage threading via namelist options.
>
> Rusty
> --
> Rusty Benson, PhD
> Modeling Systems Group
> NOAA Geophysical Fluid Dynamics Lab
> Princeton, NJ
>
> On Fri, May 12, 2017 at 12:40 PM, Jun Wang - NOAA Affiliate <
> jun.wang at noaa.gov> wrote:
>
>> Jim,
>>
>> Sam is working on that, we hope to make it to this release. Thanks.
>>
>> Jun
>>
>> On Fri, May 12, 2017 at 12:37 PM, James Rosinski - NOAA Affiliate <
>> james.rosinski at noaa.gov> wrote:
>>
>>> Jun and others;
>>>
>>> I noticed there is a build capability for fv3gfs on jet, but not any run
>>> instructions or runjob_jet.sh for jet. Is jet run capability intended, or
>>> is that for later?
>>>
>>> Regards,
>>> Jim Rosinski
>>>
>>>
>>>
>>> On Fri, May 12, 2017 at 10:16 AM, Rusty Benson - NOAA Federal <
>>> rusty.benson at noaa.gov> wrote:
>>>
>>>> Eugene,
>>>>
>>>> Responding only to point 4, what options do you see that are Cray
>>>> compilation flags?
>>>>
>>>> Rusty
>>>> --
>>>> Rusty Benson, PhD
>>>> Modeling Systems Group
>>>> NOAA Geophysical Fluid Dynamics Lab
>>>> Princeton, NJ
>>>>
>>>> On Fri, May 12, 2017 at 11:58 AM, Eugene Mirvis <eugene.mirvis at noaa.gov
>>>> > wrote:
>>>>
>>>>> Gerard,
>>>>>
>>>>> Just several points to clarify.
>>>>>
>>>>> 1. NEMS practice  to call modulefiles what is actually the scripts
>>>>> (calling module commands),
>>>>> that require mostly bash env in order to source and keep environment
>>>>> was always non standard use.
>>>>>
>>>>> Therefore,  if the developers will take Sam's advice and make real
>>>>> modulefiles (starting with #%Module) from the script, "export" and other
>>>>> scripting works
>>>>> wouldn't make sense, while Module util commands and Tcl/Tk  will work.
>>>>>
>>>>> 2. There is another dilemma - to keep needed environment and change
>>>>> within a workflow env. change.
>>>>> btw,
>>>>> unsetenv, append-path, prepend-path and remove-path  module commands
>>>>> are very useful for controlling that, but you have to keep
>>>>> $LOADEDMODULES, and $MODULE PATH in consistent order.
>>>>>
>>>>> 3. Speaking of which,
>>>>> module purge
>>>>> and
>>>>> module switch
>>>>> are very useful to unload application driven modules.
>>>>> a/ You just have to do that not any moment, but before apps module is
>>>>> loaded, then, unload <appsModulefile> - not purge, but unload.
>>>>> b/ On Crays, the are some internal dependencies  inside of PrgEnv. So
>>>>> you have to keep all "module use <knowns>"  to recover, otherwise you might
>>>>> find "module not found"
>>>>>
>>>>> 4.
>>>>> Compiling on Theia, I'm just wondering why Cray's compilation flags
>>>>> are utilized... allover:
>>>>> See for  instance
>>>>> ...
>>>>> *mpiifort -I/apps/netcdf/4.3.0-intel/include -fno-alias -auto
>>>>> -safe-cray-ptr -ftz -assume byterecl -nowarn -sox -align array64byte -i4
>>>>> -real-size 64 -no-prec-div -no-prec-sqrt -xCORE-AVX2*
>>>>> * -qno-opt-dynamic-align* -O2 -debug minimal -fp-model source
>>>>> -qoverride-limits -qopt-prefetch=3 -qopenmp -I/apps/esmf/7.0.0/intel/intel
>>>>> mpi/mod/modO/Linux.intel.64.in <http://linux.intel.64.in/>telmpi.default
>>>>> -I/apps/esmf/7.0.0/intel/intelmpi/include
>>>>> -I/apps/netcdf/4.3.0-intel/include -IENS_Cpl -I.
>>>>> -I/scratch4/NCEPDEV/global/noscrub/Eugene.Mirvis/fv3gfs.v0beta/FV3/nems_dir
>>>>> -c module_MEDIATOR_methods.f90
>>>>> ...
>>>>>
>>>>> Thanks,
>>>>> -Eugene
>>>>>
>>>>
> _______________________________________________
> Ncep.list.fv3-announce mailing list
> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20170512/0a45c9c4/attachment.html 


More information about the Ncep.list.fv3-announce mailing list