<div dir="ltr">George,<div><br></div><div>One suggestion I would make regarding the Cray system is to run the test case multiple times and compare the data to verify run-to-run reproducibility. With some of the advanced Intel compiler options, you can lose the guarantee of run-to-run reproducibility - especially when the advanced vector instruction sets are in use and dynamic memory alignment is enabled.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="font-size:12.8px">Rusty</div><div style="font-size:12.8px">--</div><div style="font-size:12.8px">Rusty Benson, PhD</div><div style="font-size:12.8px">Modeling Systems Group</div><div style="font-size:12.8px">NOAA Geophysical Fluid Dynamics Lab</div><div style="font-size:12.8px">Princeton, NJ</div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Wed, Apr 5, 2017 at 1:07 PM, FV3GFS Trac Ticket <span dir="ltr"><<a href="mailto:ncep.list.emc.fv3gfs_tickets@noaa.gov" target="_blank">ncep.list.emc.fv3gfs_tickets@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">#12: Changes for nests<br>
--------------------------+---<wbr>-------------------------<br>
Reporter: tom.black@… | Owner: george.gayno@…<br>
Type: enhancement | Status: accepted<br>
Priority: major | Milestone:<br>
Component: component1 | Version:<br>
Resolution: | Keywords: nest<br>
--------------------------+---<wbr>-------------------------<br>
<br>
Comment (by george.gayno@…):<br>
<br>
As mentioned in comment:8, the build on Theia uses "-O3" and on Cray uses<br>
"-O0". I used Grads to view the output orography files created on each<br>
machine. I only checked tiles 1 and 7. And only the 'rlat', 'rlon',<br>
'slmsk', 'land_frac', 'orog_filt' and 'orog_raw' records.<br>
<br>
For tile 1, only the 'land_frac' record was different. Only one point was<br>
different. This difference was 'floating point'.<br>
<br>
For tile 7 (the nest), the 'land_frac' record had differences between<br>
-.002 and -.001. The 'orog_raw' and 'orog_filt' records had differences<br>
between -1.5 and 0.9. All differences were confined to a single column in<br>
the middle of the grid.<br>
<br>
--<br>
Ticket URL: <<a href="https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/12#comment:11" rel="noreferrer" target="_blank">https://svnemc.ncep.noaa.gov/<wbr>trac/fv3gfs/ticket/12#comment:<wbr>11</a>><br>
fv3gfs <<a href="https://svnemc.ncep.noaa.gov/trac/fv3gfs" rel="noreferrer" target="_blank">https://svnemc.ncep.noaa.gov/<wbr>trac/fv3gfs</a>><br>
NGGPS FV3GFS Development<br>
</blockquote></div><br></div>