[Ncep.list.fv3-announce] fv3gfs release beta test
Samuel Trahan - NOAA Affiliate
samuel.trahan at noaa.gov
Thu May 11 18:48:47 UTC 2017
Hi,
There were some questions about porting the version 0 release to other
systems. At present, that is not easy to do. The release has some
dependencies on NCEP software that has no standardized or automated build
systems. Also, some code has only been tested on the Intel compiler. The
v0 release scripts are somewhat ad hoc; there are platform-specific
assumptions in several locations.
Clearly this is something we would like to fix in future releases.
Sincerely,
Sam Trahan
On Thu, May 11, 2017 at 9:00 AM, Jun Wang - NOAA Affiliate <
jun.wang at noaa.gov> wrote:
> Gerard,
>
> With the fix last night, do you still have problem load the module
> module.fre-nctools?
>
> Thanks for pointing out the forecast time specified in runjob_theia.sh, I
> changed it to be consistent with the baseline.
>
> Another fix on the forecast executable is Jim Abeles are also committed to
> the tag.
>
> I am planning to commit all the changes made to the temporary beta tag to
> trunk later today for Sam to add jet extension, please send me any
> fix/suggestion. Thanks to all who are doing testing.
>
>
> Jun
>
> On Thu, May 11, 2017 at 5:03 AM, Gerard Ketefian - NOAA Affiliate <
> gerard.ketefian at noaa.gov> wrote:
>
>> Hi all,
>>
>> With Ligia's hints and Jun's last fix, I was able to complete the run but
>> not the remap. I think the remap fails because some modules don't get
>> loaded properly.
>>
>> To get the remap to also work, I replaced the following line in
>> runjob_theia.sh
>>
>> module load module.fre-nctools
>>
>> with the following block (copied and modified from the file
>> module.fre-nctools):
>>
>> module load impi/5.1.2.150
>> module load netcdf/4.3.0
>> module load hdf5/1.8.14
>> export HDF5_DIR=$HDF5
>> export NETCDF_DIR=$NETCDF
>> export LIBRARY_PATH=${LIBRARY_PATH}:${NETCDF}/lib:${HDF5}/lib
>>
>> This change should allow the 1deg remapped netcdf files to be generated.
>>
>> When I do the comparison of the sample run's netcdf files with baseline,
>> there is a about factor of 5 difference (the sample run files being
>> larger). This is because there are only 8 output times in the baseline
>> files but 40 in the run output.
>>
>> Gerard
>>
>>
>> On Wed, May 10, 2017 at 9:56 PM, Jun Wang - NOAA Affiliate <
>> jun.wang at noaa.gov> wrote:
>>
>>> Ligia,
>>>
>>> Thanks for the feedback. The suggestion on instruction is put in
>>> readme.txt. It is found that "cp " is missing in line 124 in
>>> runjob_theia.sh. I committed the changes to the beta test tag:
>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t
>>> ags/fv3gfs.v0beta
>>>
>>> Please check again. The results differences will need further
>>> investigation. Thanks.
>>>
>>> Jun
>>>
>>> On Wed, May 10, 2017 at 10:28 PM, Ligia Bernardet - NOAA Affiliate <
>>> ligia.bernardet at noaa.gov> wrote:
>>>
>>>> Folks,
>>>>
>>>> Here is some feedback
>>>>
>>>>
>>>> *About the instructions*
>>>>
>>>> 1. Minor typo. The word "trunk" should be removed: Four executable
>>>> files will be created under fv3gfs.v0beta/*trunk*/NEMS/exe
>>>> 2. runjob_theia.sh: Non-NCEPDEV people need to change directories
>>>> DATA and ROTDIR to an area they can write to
>>>> 3. diff_baseline.sh:
>>>> 1. It would be helpful to tell users to add arguments
>>>> to diff_baseline.sh to set resolution and machine.
>>>> 2. Non-NCEPDEV people need to change directory dir1 to location
>>>> of their output
>>>>
>>>> *Outcome*
>>>> It seems I was able to get through the forecast but failed in remap.
>>>> Problem seems related to loading modules, I did not fully investigate yet.
>>>> Output is in /scratch4/BMC/gmtb/Ligia.Berna
>>>> rdet/fv3gfs.v0beta/release/v0/exp
>>>>
>>>> When running diff, NetCDF files differ from the baseline. I noticed the
>>>> file sizes are different (mine are larger than the baseline).
>>>>
>>>> Ligia
>>>>
>>>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>>>> exp/../modulefiles/fv3gfs/fre-nctools.theia module.fre-nctools
>>>>
>>>> /var/spool/torque/mom_priv/jobs/23566207.bqs3.SC: line 126:
>>>> /scratch4/BMC/gmtb/Ligia.Bernardet/fv3gfs.v0beta/release/v0/
>>>> exp/../modulefiles/fv3gfs/fre-nctools.theia: *Permission denied*
>>>>
>>>> + module load module.fre-nctools
>>>>
>>>> ++ /apps/lmod/lmod/libexec/lmod bash load module.fre-nctools
>>>>
>>>> Lmod has detected the following error: The following module(s) are
>>>> unknown:
>>>>
>>>> "module.fre-nctools"
>>>>
>>>> On Wed, May 10, 2017 at 5:47 PM, James Rosinski - NOAA Affiliate <
>>>> james.rosinski at noaa.gov> wrote:
>>>>
>>>>> Hi Jun;
>>>>>
>>>>> I am about to head home for the day, but here are my comments so far,
>>>>> after following the instructions for theia:
>>>>>
>>>>> o The builds of models and remap codes completed successfully. One
>>>>> suggestion might be to have the user specify 32 or 64-bit, and nh vs. hydro
>>>>> in order to cut down compilation time by a factor of 4.
>>>>>
>>>>> o The batch job attempting to run the model failed. Relevant lines in
>>>>> err_theia are:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *+ cd /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900+
>>>>> /bin/cp -p
>>>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp/../../../NEMS/exe/fv3_gfs_nh.prod.32bit.x
>>>>> /scratch4/NCEPDEV/stmp3/James.Rosinski/C96fv3gfs2016092900/.+ -prepend-rank
>>>>> -np 288 ./fv3_gfs_nh.prod.32bit.x+ ERR=127+ export ERR+ err=127*
>>>>> Looks like somehow "mpirun" was not found (note there is nothing in
>>>>> front of "-prepend-rank"). FYI I use csh for my login shell--not sure if
>>>>> this is behind the error. I had no modules loaded when submitting the job.
>>>>>
>>>>> If you'd like to examine the output you should have read access to it
>>>>> here on theia:
>>>>>
>>>>> /scratch3/BMC/gsd-hpcs/rosinski/fv3gfs.v0beta/release/v0/exp
>>>>>
>>>>> More info tomorrow...
>>>>>
>>>>> Regards,
>>>>> Jim Rosinski
>>>>>
>>>>>
>>>>> On Wed, May 10, 2017 at 2:55 PM, Jun Wang - NOAA Affiliate <
>>>>> jun.wang at noaa.gov> wrote:
>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> I was noticed that some directory names in the readme.txt are not
>>>>>> correct. I just updated the tag version, please let me know if you have any
>>>>>> further questions. Thanks.
>>>>>>
>>>>>> Jun
>>>>>>
>>>>>> On Wed, May 10, 2017 at 4:39 PM, Jun Wang - NOAA Affiliate <
>>>>>> jun.wang at noaa.gov> wrote:
>>>>>>
>>>>>>> Rusty,
>>>>>>>
>>>>>>> Thanks for quick feedback. Today we just had a VLAB meeting on how
>>>>>>> to provide information for the public release. Vijay mentioned that EMC
>>>>>>> will be setting up an FV3GFS community web page through VLAB, some basic
>>>>>>> document will be provided there. A formal instruction on how to get release
>>>>>>> code, and to compile and run experiment will be on that web page too. For
>>>>>>> questions/feedback, a forum will be set up for users to post questions and
>>>>>>> to provide answers &feedback, the purpose is that all the developers will
>>>>>>> see the questions/answers, it is suggested not to send questions/feedback
>>>>>>> to any individual's personal email (If people receive questions from users,
>>>>>>> we suggest that they post the questions along with their answers to the
>>>>>>> forum). The readme.txt is a temporary solution to get the testing started,
>>>>>>> it may be changed in the final release.
>>>>>>>
>>>>>>> Kate Howard (kate.howard at noaa.gov) is working on the VLAB fv3gfs
>>>>>>> web page, she can add the gfdl fv3gfs support email on the web page too, if
>>>>>>> you have any fv3 document for general developers, please send to her.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>>
>>>>>>> Jun
>>>>>>>
>>>>>>> On Wed, May 10, 2017 at 4:02 PM, Rusty Benson - NOAA Federal <
>>>>>>> rusty.benson at noaa.gov> wrote:
>>>>>>>
>>>>>>>> Hi Jun and Vijay,
>>>>>>>>
>>>>>>>> In the readme.txt Q&A, you mention where to get help. Has there
>>>>>>>> been any thought to putting together a single email for tracking all
>>>>>>>> questions/requests that can be used as a basis for creating a knowledgebase
>>>>>>>> via a wiki or other forum? By segmenting FV3 and physics support, I think
>>>>>>>> we are missing an opportunity for personnel to get exposure to and learn
>>>>>>>> about system pieces for which they may not necessarily be responsible.
>>>>>>>>
>>>>>>>> If we do go the route of a single support email, we have an
>>>>>>>> existing email for FV3 support which could be used as an alias member.
>>>>>>>> Otherwise, we would want to publish the email inside of the readme.txt and
>>>>>>>> not have individual team members being contacted directly
>>>>>>>>
>>>>>>>> <oar.gfdl.fvgfs_support at noaa.gov>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Rusty
>>>>>>>> --
>>>>>>>> Rusty Benson, PhD
>>>>>>>> Modeling Systems Group
>>>>>>>> NOAA Geophysical Fluid Dynamics Lab
>>>>>>>> Princeton, NJ
>>>>>>>>
>>>>>>>> On Wed, May 10, 2017 at 2:08 PM, Jun Wang - NOAA Affiliate <
>>>>>>>> jun.wang at noaa.gov> wrote:
>>>>>>>>
>>>>>>>>> Dear all,
>>>>>>>>>
>>>>>>>>> The following email is for people who are willing to do beta
>>>>>>>>> testing for the fv3gfs May 15 release. Please ignore the email if you are
>>>>>>>>> not going to run the test.
>>>>>>>>>
>>>>>>>>> The svn tag for beta testing is located at:
>>>>>>>>>
>>>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/
>>>>>>>>> tags/fv3gfs.v0beta
>>>>>>>>>
>>>>>>>>> The instruction file on how to get and compile the code and to run
>>>>>>>>> an experiment is at:
>>>>>>>>>
>>>>>>>>> https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/t
>>>>>>>>> ags/fv3gfs.v0beta/release/v0/readme.txt
>>>>>>>>>
>>>>>>>>> Please follow the instructions to see if you can run an
>>>>>>>>> experiment.
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jun
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ncep.list.fv3-announce mailing list
>>>>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>>>>>>> fv3-announce
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ncep.list.fv3-announce mailing list
>>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>>>> fv3-announce
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ncep.list.fv3-announce mailing list
>>>>> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>>> fv3-announce
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Gerard Ketefian
>> Research Scientist
>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>> 325 Broadway
>> Boulder, CO 80305
>> phone: 303-497-6209 <(303)%20497-6209>
>>
>>
>
> _______________________________________________
> Ncep.list.fv3-announce mailing list
> Ncep.list.fv3-announce at lstsrv.ncep.noaa.gov
> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.fv3-announce
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.fv3-announce/attachments/20170511/e8868c7a/attachment-0001.html
More information about the Ncep.list.fv3-announce
mailing list