<div dir="ltr"><div><div>Moorthi,<br><br></div>I just got this error yesterday. Please see the job submit line, you should have &quot;aprun... executable&quot;, without aprun (cray) you will get that error. <br><br></div>Jun<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 9, 2017 at 12:54 PM, Shrinivas Moorthi <span dir="ltr">&lt;<a href="mailto:shrinivas.moorthi@noaa.gov" target="_blank">shrinivas.moorthi@noaa.gov</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <div class="m_-7061746746540679015moz-cite-prefix">Does anyone know what might cause the
      following error right off the bat?<br>
      &quot;FATAL from PE    92: mpp_sync_self: size_recv does not match of
      data received<br>
      <br>
      Rank 145 [Tue May  9 16:38:33 2017] [c7-0c2s6n0] Fatal error in
      PMPI_Wait: Message truncated, error stack:<br>
      PMPI_Wait(186)................<wbr>...: MPI_Wait(request=0xbf3e7c4,
      status=0xb30ac68) failed<br>
      MPIR_Wait_impl(79)............<wbr>...:<br>
      MPID_nem_gni_lmt_start_recv(<wbr>1580): Message from rank 89 and tag 2
      truncated; 33528 bytes received but buffer size is 5880&quot;<br>
      Thanks<br>
      Moorthi<div><div class="h5"><br>
      On 05/09/2017 11:48 AM, Dusan Jovic wrote:<br>
    </div></div></div><div><div class="h5">
    <blockquote type="cite">
      
      <div class="m_-7061746746540679015moz-cite-prefix">Jun,<br>
        <br>
        Here&#39;s what I get when I try to checkout that branch:<br>
        <br>
        slogin2:/gpfs/hps/emc/meso/<wbr>save/Dusan.Jovic/FV3&gt; <b>svn co
          <a class="m_-7061746746540679015moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/branches/fv3.v0release" target="_blank">https://svnemc.ncep.noaa.gov/<wbr>projects/nems/apps/NEMSfv3gfs/<wbr>branches/fv3.v0release</a></b><br>
        <p>A    fv3.v0release/parm<br>
          A    fv3.v0release/parm/model_<wbr>configure.IN<br>
        </p>
        <p>. . . .</p>
        <p>A    fv3.v0release/compsets/fv3.<wbr>input<br>
          A    fv3.v0release/standaloneFV3.<wbr>appBuilder<br>
           U   fv3.v0release<br>
          <br>
          Fetching external item into &#39;<b>fv3.v0release/NEMS</b>&#39;:<br>
          A    fv3.v0release/NEMS/exe<br>
          A    fv3.v0release/NEMS/exe/<wbr>mkDepends.pl<br>
        </p>
        <p> . . . . <br>
        </p>
        <p>A    fv3.v0release/NEMS/<wbr>NEMSAppBuilder<br>
          A    fv3.v0release/NEMS/<wbr>OldCompsetRun<br>
           U   fv3.v0release/NEMS<br>
          <br>
          Fetching external item into &#39;<b>fv3.v0release/NEMS/tests/<wbr>produti</b>l&#39;:<br>
          A    fv3.v0release/NEMS/tests/<wbr>produtil/ush<br>
          A    fv3.v0release/NEMS/tests/<wbr>produtil/ush/testgen.py<br>
          . . . .</p>
        <p>A    fv3.v0release/NEMS/tests/<wbr>produtil/ush/forcetest.py<br>
          A    fv3.v0release/NEMS/tests/<wbr>produtil/ush/testme.py<br>
          Checked out external at revision 90581.<br>
          <br>
          Checked out revision 90943.<br>
          <font color="#ff0000">svn: warning: W200000: Error handling
            externals definition for &#39;<b>fv3.v0release/FV3</b>&#39;:<br>
            svn: warning: W175013: Unable to connect to a repository at
            URL &#39;<b><a class="m_-7061746746540679015moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/fv3/trunk" target="_blank">https://svnemc.ncep.noaa.gov/<wbr>projects/fv3/trunk</a></b>&#39;<br>
          </font>Checked out revision 92536.</p>
        <p>Again, is this somehow related to last week&#39;s problems with
          the svn server?</p>
        <p>Dusan<br>
        </p>
        <br>
        On 05/09/2017 11:22 AM, Jun Wang - NOAA Affiliate wrote:<br>
      </div>
      <blockquote type="cite">
        <div dir="ltr">
          <div>
            <div>
              <div>Dusan,<br>
                <br>
              </div>
              Thanks for looking at the release readme file! The readme
              file is prepared for the targeted release, which is a 
              final nemsfv3gfs tag, currently it does not exist yet.
              Please check out the current branch instead:<br>
              <br>
              <b><a class="m_-7061746746540679015gmail-m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/branches/fv3.v0release/release/v0/readme.txt" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/b<wbr>ranches/fv3.v0release</a><br>
                <br>
              </b></div>
            then following the readme file to see if you can run the
            code on theia and wcoss, please let me know if you find any
            mistakes/errors. Thank you very much for the first person
            for beta testing!<br>
            <br>
            <br>
          </div>
          Jun<b><br>
          </b></div>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Tue, May 9, 2017 at 11:10 AM,
            Dusan Jovic <span dir="ltr">&lt;<a href="mailto:dusan.jovic@noaa.gov" target="_blank">dusan.jovic@noaa.gov</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <div class="m_-7061746746540679015m_6373680597087819793moz-cite-prefix">Is
                  this readme.txt file the latest version available?<br>
                  <br>
                  <b><a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/branches/fv3.v0release/release/v0/readme.txt" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/b<wbr>ranches/fv3.v0release/release/<wbr>v0/readme.txt</a></b><b><br>
                  </b><br>
                  If it is then the URL of the v0 repository is either
                  wrong or I am not allowed to check it out (is this
                  related to recent issues with our svn server?)<br>
                  <br>
                  <pre>1.  check out release version at:
      <a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/tags/fv3_release.v0" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/t<wbr>ags/fv3_release.v0</a>, 

    %svn co <a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/tags/fv3_release.v0" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/t<wbr>ags/fv3_release.v0</a>
    %cd fv3_release.v0


svn co <a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/tags/fv3_release.v0" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/t<wbr>ags/fv3_release.v0</a>
svn: E170000: URL &#39;<a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/tags/fv3_release.v0" target="_blank">https://svnemc.ncep.noaa.gov/<wbr>projects/nems/apps/NEMSfv3gfs/<wbr>tags/fv3_release.v0</a>&#39; doesn&#39;t exist

</pre><span class="m_-7061746746540679015HOEnZb"><font color="#888888">
      Dusan</font></span><div><div class="m_-7061746746540679015h5">

      

      

      On 05/03/2017 03:45 PM, FV3GFS Trac Ticket wrote:

    </div></div></div><div><div class="m_-7061746746540679015h5">
    <blockquote type="cite">
      <pre>#15: FV3 May 15 Release
------------------------------<wbr>+------------------------
  Reporter:  samuel.trahan@…  |      Owner:  somebody
      Type:  task             |     Status:  new
  Priority:  critical         |  Milestone:  milestone1
 Component:  component1       |    Version:
Resolution:                   |   Keywords:  release
------------------------------<wbr>+------------------------

Comment (by jun.wang@…):

 Following Vijay&#39;s suggestion, a nemsfv3 branch is created at:

 <b><a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-freetext" href="https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSfv3gfs/branches/fv3.v0release" target="_blank">https://svnemc.ncep.noaa.gov/p<wbr>rojects/nems/apps/NEMSfv3gfs/b<wbr>ranches/fv3.v0release</a></b>

 This branch will contain required script and instruction to run the test
 cases Fanglin set up.

 A new directory release/v0 is added under fv3.v0release, following files
 are copied from Fanglin&#39;s fv3gfs branch: FV3GFS_V0_RELEASE:

 exec  exp  modulefiles  parm  readme.txt  scripts  sorc  ush

 where:
 exec: hold fregrid executables
 exp: contains build.sh and runjob_cray.sh, runjob_theia.sh, run script to
 start the experiment
 modulefiles : contain fregrid model files
 readme.txt: instructions on running release cases
 scripts: exglobal_fcst_nemsfv3gfs.sh forecast scripts to set up
 namelist,etc.
 sorc: contain fregrid source code
 ush: utility script to run remapping

 A new script runjob_theia.sh is created and run the test cases on theia.

--
Ticket URL: <a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-rfc2396E" href="https://svnemc.ncep.noaa.gov/trac/fv3gfs/ticket/15#comment:2" target="_blank">&lt;https://svnemc.ncep.noaa.gov/<wbr>trac/fv3gfs/ticket/15#comment:<wbr>2&gt;</a>
fv3gfs <a class="m_-7061746746540679015m_6373680597087819793moz-txt-link-rfc2396E" href="https://svnemc.ncep.noaa.gov/trac/fv3gfs" target="_blank">&lt;https://svnemc.ncep.noaa.gov/<wbr>trac/fv3gfs&gt;</a>
NGGPS FV3GFS Development
</pre>
    </blockquote>
    <p>

    </p>
  </div></div></div>

</blockquote></div>
</div>



</blockquote><p>
</p>


</blockquote>
<p>
</p></div></div><span class="HOEnZb"><font color="#888888"><pre class="m_-7061746746540679015moz-signature" cols="72">-- 
Dr. Shrinivas Moorthi
Research Meteorologist
Global Climate and Weather Modeling Branch
Environmental Modeling Center / National Centers for Environmental Prediction
5830 University Research Court - (W/NP23), College Park MD 20740 USA
Tel:<a href="tel:(301)%20683-3718" value="+13016833718" target="_blank">(301)683-3718</a></pre></font></span></div></blockquote></div><br></div>