[Ncep.list.fv3-announce] fv3gfs release beta test

Thu May 11 09:03:47 UTC 2017

Hi all,

With Ligia's hints and Jun's last fix, I was able to complete the run but
not the remap.  I think the remap fails because some modules don't get
loaded properly.

To get the remap to also work, I replaced the following line in
runjob_theia.sh

module load module.fre-nctools

with the following block (copied and modified from the file
module.fre-nctools):

module load impi/5.1.2.150
module load netcdf/4.3.0
module load hdf5/1.8.14
export HDF5_DIR=$HDF5
export NETCDF_DIR=$NETCDF
export LIBRARY_PATH=${LIBRARY_PATH}:${NETCDF}/lib:${HDF5}/lib

This change should allow the 1deg remapped netcdf files to be generated.

When I do the comparison of the sample run's netcdf files with baseline,
there is a about factor of 5 difference (the sample run files being
larger).  This is because there are only 8 output times in the baseline
files but 40 in the run output.

Gerard

On Wed, May 10, 2017 at 9:56 PM, Jun Wang - NOAA Affiliate <
jun.wang at noaa.gov> wrote:

> Ligia,
>
> Thanks for the feedback.  The suggestion on instruction is put in
> readme.txt. It is found that "cp " is missing  in line 124 in
> runjob_theia.sh. I committed the changes to the beta test tag:
> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/
> tags/fv3gfs.v0beta
>
> Please check again. The results differences will need further
> investigation. Thanks.
>
> Jun
>
> On Wed, May 10, 2017 at 10:28 PM, Ligia Bernardet - NOAA Affiliate <
> ligia.bernardet at noaa.gov> wrote:
>
>> Folks,
>>
>> Here is some feedback
>>
>>
>> *About the instructions*
>>
>>    1. Minor typo. The word "trunk" should be removed: Four executable
>>    files will be created under fv3gfs.v0beta/*trunk*/NEMS/exe
>>    2. runjob_theia.sh: Non-NCEPDEV people need to change directories
>>    DATA and ROTDIR to an area they can write to
>>    3. diff_baseline.sh:
>>       1. It would be helpful to tell users to add arguments
>>       to diff_baseline.sh to set resolution and machine.
>>       2. Non-NCEPDEV people need to change directory dir1 to location of
>>       their output
>>
>> *Outcome*
>> It seems I was able to get through the forecast but failed in remap.
>> Problem seems related to loading modules, I did not fully investigate yet.
>> Output is in /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>> exp
>>
>> When running diff, NetCDF files differ from the baseline. I noticed the
>> file sizes are different (mine are larger than the baseline).
>>
>> Ligia
>>
>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>> exp/../modulefiles/fv3gfs/fre-nctools.theia module.fre-nctools
>>
>> /var/spool/torque/mom_priv/jobs/23566207.bqs3.SC: line 126:
>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>> exp/../modulefiles/fv3gfs/fre-nctools.theia: *Permission denied*
>>
>> + module load module.fre-nctools
>>
>> ++ /apps/lmod/lmod/libexec/lmod bash load module.fre-nctools
>>
>> Lmod has detected the following error: The following module(s) are
>> unknown:
>>
>> "module.fre-nctools"
>>
>> On Wed, May 10, 2017 at 5:47 PM, James Rosinski - NOAA Affiliate <
>> james.rosinski at noaa.gov> wrote:
>>
>>> Hi Jun;
>>>
>>> I am about to head home for the day, but here are my comments so far,
>>> after following the instructions for theia:
>>>
>>> o The builds of models and remap codes completed successfully. One
>>> suggestion might be to have the user specify 32 or 64-bit, and nh vs. hydro
>>> in order to cut down compilation time by a factor of 4.
>>>
>>> o The batch job attempting to run the model failed. Relevant lines in
>>> err_theia are:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *+ cd /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900+
>>> /bin/cp -p
>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp/../../../NEMS/exe/fv3_gfs_nh.prod.32bit.x
>>> /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900/.+ -prepend-rank
>>> -np 288 ./fv3_gfs_nh.prod.32bit.x+ ERR=127+ export ERR+ err=127*
>>> Looks like somehow "mpirun" was not found (note there is nothing in
>>> front of "-prepend-rank"). FYI I use csh for my login shell--not sure if
>>> this is behind the error. I had no modules loaded when submitting the job.
>>>
>>> If  you'd like to examine the output you should have read access to it
>>> here on theia:
>>>
>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp
>>>
>>> More info tomorrow...
>>>
>>> Regards,
>>> Jim Rosinski
>>>
>>>
>>> On Wed, May 10, 2017 at 2:55 PM, Jun Wang - NOAA Affiliate <
>>> jun.wang at noaa.gov> wrote:
>>>
>>>> Dear all,
>>>>
>>>> I was noticed that some directory names in the readme.txt are not
>>>> correct. I just updated the tag version, please let me know if you have any
>>>> further questions. Thanks.
>>>>
>>>> Jun
>>>>
>>>> On Wed, May 10, 2017 at 4:39 PM, Jun Wang - NOAA Affiliate <
>>>> jun.wang at noaa.gov> wrote:
>>>>
>>>>> Rusty,
>>>>>
>>>>> Thanks for quick feedback. Today we just had a VLAB meeting on how to
>>>>> provide information for the public release. Vijay mentioned that EMC will
>>>>> be setting up an FV3GFS community web page through VLAB, some basic
>>>>> document will be provided there. A formal instruction on how to get release
>>>>> code, and to compile and run experiment will be on that web page too. For
>>>>> questions/feedback,  a forum will be set up for users to post questions and
>>>>> to provide answers &feedback, the purpose is that all the developers will
>>>>> see the questions/answers, it is suggested not to send questions/feedback
>>>>> to any individual's personal email (If people receive questions from users,
>>>>> we suggest that they post the questions along with their answers to the
>>>>> forum). The readme.txt is a temporary solution to get the testing started,
>>>>> it may be changed in the final release.
>>>>>
>>>>> Kate Howard (kate.howard at noaa.gov) is working on the VLAB fv3gfs web
>>>>> page, she can add the gfdl fv3gfs support email on the web page too, if you
>>>>> have any fv3 document for general developers, please send to her.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> Jun
>>>>>
>>>>> On Wed, May 10, 2017 at 4:02 PM, Rusty Benson - NOAA Federal <
>>>>> rusty.benson at noaa.gov> wrote:
>>>>>
>>>>>> Hi Jun and Vijay,
>>>>>>
>>>>>> In the readme.txt Q&A, you mention where to get help.  Has there been
>>>>>> any thought to putting together a single email for tracking all
>>>>>> questions/requests that can be used as a basis for creating a knowledgebase
>>>>>> via a wiki or other forum?  By segmenting FV3 and physics support, I think
>>>>>> we are missing an opportunity for personnel to get exposure to and learn
>>>>>> about system pieces for which they may not necessarily be responsible.
>>>>>>
>>>>>> If we do go the route of a single support email, we have an existing
>>>>>> email for FV3 support which could be used as an alias member.  Otherwise,
>>>>>> we would want to publish the email inside of the readme.txt and not have
>>>>>> individual team members being contacted directly
>>>>>>
>>>>>> <oar.gfdl.fvgfs_support at noaa.gov>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Rusty
>>>>>> --
>>>>>> Rusty Benson, PhD
>>>>>> Modeling Systems Group
>>>>>> NOAA Geophysical Fluid Dynamics Lab
>>>>>> Princeton, NJ
>>>>>>
>>>>>> On Wed, May 10, 2017 at 2:08 PM, Jun Wang - NOAA Affiliate <
>>>>>> jun.wang at noaa.gov> wrote:
>>>>>>
>>>>>>> Dear all,
>>>>>>>
>>>>>>> The following email is for people who are willing to do beta testing
>>>>>>> for the fv3gfs May 15 release. Please ignore the email if you are not going
>>>>>>> to run the test.
>>>>>>>
>>>>>>> The svn tag for beta testing is located at:
>>>>>>>
>>>>>>>  https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/
>>>>>>> tags/fv3gfs.v0beta
>>>>>>>
>>>>>>> The instruction file on how to get and compile the code and to run
>>>>>>> an experiment is at:
>>>>>>>
>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t
>>>>>>> ags/fv3gfs.v0beta/release/v0/readme.txt
>>>>>>>
>>>>>>> Please follow the instructions to see if you can run an experiment.
>>>>>>>
>>>>>>>  Thanks.
>>>>>>>
>>>>>>>
>>>>>>> Jun
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ncep.list.fv3-announce mailing list
>>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>>>>> fv3-announce
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ncep.list.fv3-announce mailing list
>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>> fv3-announce
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ncep.list.fv3-announce mailing list
>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>>>
>>>
>>
>

-- 
Gerard Ketefian
Research Scientist
NOAA/OAR/ESRL/GSD/EMB, R/GSD1
325 Broadway
Boulder, CO 80305
phone: 303-497-6209
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20170511/69bc837a/attachment.html 