<div dir="ltr"><div>IWSB on-call is still investigating issues with this.<br><br></div>NCEP/SDM<br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On 11 March 2017 at 18:53, Paul Kirkwood - NOAA Federal <span dir="ltr"><<a href="mailto:paul.kirkwood@noaa.gov" target="_blank">paul.kirkwood@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">2017.03.11:</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">SR RFC's are reporting missing Q3 data.</div></div><div class="gmail_extra"><br clear="all"><div><div class="m_4663868955915422822gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><font size="2" face="courier new, monospace"><br></font><div><div><font size="2" face="monospace, monospace">Thanks,</font></div><div><font size="2" face="monospace, monospace">Paul Kirkwood - NOAA Federal</font></div><div><font size="2" face="monospace, monospace">Chief, Technology Infusion Branch (TIB)</font></div><div><font size="2" face="monospace, monospace">Science and Technology Services Division (STSD)</font></div><div><font size="2" face="monospace, monospace">Southern Region Headquarters</font></div><div><font size="2" face="monospace, monospace">Email: <a href="mailto:Paul.Kirkwood@noaa.gov" target="_blank">Paul.Kirkwood@noaa.gov</a></font></div><div><font size="2" face="monospace, monospace">Phone: <span style="background-image:initial;background-repeat:initial"><a href="tel:(682)%20703-3745" value="+16827033745" target="_blank">682.703.3745</a></span></font></div><div><font size="2" face="monospace, monospace">Web: <a href="http://weather.gov/srh" target="_blank">http://weather.gov/srh</a></font></div><div><font size="2" face="monospace, monospace">"Science Is Organized Knowledge" - Library of Congress</font></div></div><div><font size="2" face="courier new, monospace"><br></font></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Sat, Mar 11, 2017 at 10:22 AM, Daryl Herzmann <span dir="ltr"><<a href="mailto:akrherz@iastate.edu" target="_blank">akrherz@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
I am noticing an uptick in missing files today. Are known troubles<br>
still afoot? Here's a printout of some missing files in CPRK in the<br>
past 24 hours for the products I care about :)<br>
<br>
MISS cprk 2017-03-10T22:00Z RadarOnly_QPE_01H<br>
MISS cprk 2017-03-11T07:00Z RadarOnly_QPE_01H<br>
MISS cprk 2017-03-11T12:00Z RadarOnly_QPE_01H<br>
MISS cprk 2017-03-11T13:00Z RadarOnly_QPE_01H<br>
MISS cprk 2017-03-10T19:26Z PrecipFlag<br>
MISS cprk 2017-03-10T22:00Z PrecipFlag<br>
MISS cprk 2017-03-11T02:00Z PrecipFlag<br>
MISS cprk 2017-03-11T04:10Z PrecipFlag<br>
MISS cprk 2017-03-11T07:00Z PrecipFlag<br>
MISS cprk 2017-03-11T09:54Z PrecipFlag<br>
MISS cprk 2017-03-11T12:00Z PrecipFlag<br>
MISS cprk 2017-03-11T13:00Z PrecipFlag<br>
MISS cprk 2017-03-10T21:00Z SeamlessHSR<br>
MISS cprk 2017-03-10T22:00Z SeamlessHSR<br>
MISS cprk 2017-03-10T23:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T02:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T04:16Z SeamlessHSR<br>
MISS cprk 2017-03-11T07:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T09:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T12:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T13:00Z SeamlessHSR<br>
MISS cprk 2017-03-11T14:00Z SeamlessHSR<br>
MISS cprk 2017-03-10T19:26Z PrecipRate<br>
MISS cprk 2017-03-10T19:56Z PrecipRate<br>
MISS cprk 2017-03-10T22:00Z PrecipRate<br>
MISS cprk 2017-03-11T01:04Z PrecipRate<br>
MISS cprk 2017-03-11T02:00Z PrecipRate<br>
MISS cprk 2017-03-11T03:32Z PrecipRate<br>
MISS cprk 2017-03-11T03:46Z PrecipRate<br>
MISS cprk 2017-03-11T04:10Z PrecipRate<br>
MISS cprk 2017-03-11T07:00Z PrecipRate<br>
MISS cprk 2017-03-11T07:26Z PrecipRate<br>
MISS cprk 2017-03-11T07:46Z PrecipRate<br>
MISS cprk 2017-03-11T12:00Z PrecipRate<br>
MISS cprk 2017-03-11T12:40Z PrecipRate<br>
MISS cprk 2017-03-11T13:00Z PrecipRate<br>
MISS cprk 2017-03-11T13:20Z PrecipRate<br>
MISS cprk 2017-03-11T14:48Z PrecipRate<br>
<br>
<br>
daryl<br>
<div><div class="m_4663868955915422822h5"><br>
On Fri, Mar 10, 2017 at 1:58 PM, Derek VanPelt - NOAA Affiliate<br>
<<a href="mailto:derek.vanpelt@noaa.gov" target="_blank">derek.vanpelt@noaa.gov</a>> wrote:<br>
> MRMS customers,<br>
><br>
> The MRMS application endured latencies, and produced some incomplete<br>
> products over ~3 hours starting this morning @1530Z.<br>
><br>
> We were able to identify the issue quickly, but it took longer than expected<br>
> to enact a full fix.<br>
><br>
> Apologies. I believe we have learned from this to the point that we should<br>
> be able to avoid future issues. We will continue enhanced monitoring<br>
> throughout the afternoon.<br>
><br>
> Derek<br>
><br>
> On Fri, Mar 10, 2017 at 2:51 PM, Karen Cooper - NOAA Affiliate<br>
> <<a href="mailto:karen.cooper@noaa.gov" target="_blank">karen.cooper@noaa.gov</a>> wrote:<br>
>><br>
>> Hi Steve,<br>
>><br>
>> I'm including the NCEP IDP support group here, as this involves them.<br>
>><br>
>> It does appear that from about 15:46 to 18:30 that a lot of the products<br>
>> that are created by stitching the tiles were missing data. This is from<br>
>> looking at the College Park system (<a href="http://vm-lnx-mrms-convert1a.ncep.noaa.gov" rel="noreferrer" target="_blank">vm-lnx-mrms-convert1a.ncep.no<wbr>aa.gov</a><br>
>> specfically).<br>
>><br>
>> I'm not sure what would have caused this, but it also looks like there<br>
>> delays in the processing of some data. Our processing has timing controls<br>
>> to keep data for a short time -- typically twice the frequency of the<br>
>> product, but it will disregard data that is too old so that we mitigate the<br>
>> risk of including old data, as it could be misleading to analysts.<br>
>><br>
>> I'm going to defer to the NCO team as to what may have caused the slowdown<br>
>> during that time.<br>
>><br>
>> There are multiple CREFs some being done as 2D products which are not<br>
>> tiled, and some done from 3D tiled data, so depending on which one you were<br>
>> looking at there might not have been tiles missing.<br>
>><br>
>><br>
>> On Fri, Mar 10, 2017 at 11:48 AM, Steven Lack - NOAA Federal<br>
>> <<a href="mailto:steven.a.lack@noaa.gov" target="_blank">steven.a.lack@noaa.gov</a>> wrote:<br>
>>><br>
>>> Karen,<br>
>>><br>
>>> It looks like starting after 1530 or so we started dropping tiles and we<br>
>>> have had intermittent data coming in since then. This has been a problem<br>
>>> with at least RALA and ET-18, somehow CREF looks okay. This is both on our<br>
>>> AWIPS2 feed and our NIDS feed for NAWIPS.<br>
>>><br>
>>> Thanks for any info,<br>
>>><br>
>>> Steve<br>
>>><br>
>>> --<br>
>>> Steven A. Lack, Ph.D.<br>
>>> Techniques Development Meteorologist<br>
>>> Aviation Weather Center<br>
>>> 7220 NW 101st Terrace<br>
>>> Kansas City, MO 64153<br>
>>><br>
>>> office: <a href="tel:816-584-7281" value="+18165847281" target="_blank">816-584-7281</a><br>
>>> cell: <a href="tel:303-882-9395" value="+13038829395" target="_blank">303-882-9395</a><br>
>>><br>
>>> The views or opinions expressed in this email do not necessarily reflect<br>
>>> those of NOAA or the United States Government.<br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> “The presence of those seeking the truth is infinitely to be preferred to<br>
>> the presence of those who think they’ve found it."<br>
>> -- Terry Prachett<br>
>><br>
>> ------------------------------<wbr>-------------<br>
>> <a href="mailto:Karen.Cooper@noaa.gov" target="_blank">Karen.Cooper@noaa.gov</a><br>
>><br>
>> Phone#: <a href="tel:405-325-6456" value="+14053256456" target="_blank">405-325-6456</a><br>
>> Cell: <a href="tel:405-834-8559" value="+14058348559" target="_blank">405-834-8559</a><br>
>> National Severe Storms Laboratory<br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> Ncep.list.idp_support mailing list<br>
>> <a href="mailto:Ncep.list.idp_support@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.idp_support@lstsrv.n<wbr>cep.noaa.gov</a><br>
>> <a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp_support" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>idp_support</a><br>
>><br>
><br>
><br>
><br>
> --<br>
> Derek Van Pelt<br>
> IDP Onboarding<br>
> NOAA/NCEP/NCO<br>
><br>
</div></div>> ______________________________<wbr>_________________<br>
> Ncep.list.idp.mrms.users mailing list<br>
> <a href="mailto:Ncep.list.idp.mrms.users@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.idp.mrms.users@lstsr<wbr>v.ncep.noaa.gov</a><br>
> <a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp.mrms.users" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>idp.mrms.users</a><br>
><br>
<br>
______________________________<wbr>_________________<br>
Ncep.list.idp.mrms.users mailing list<br>
<a href="mailto:Ncep.list.idp.mrms.users@lstsrv.ncep.noaa.gov" target="_blank">Ncep.list.idp.mrms.users@lstsr<wbr>v.ncep.noaa.gov</a><br>
<a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp.mrms.users" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.g<wbr>ov/mailman/listinfo/ncep.list.<wbr>idp.mrms.users</a><br>
</blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
Ncep.list.idp.mrms.users mailing list<br>
<a href="mailto:Ncep.list.idp.mrms.users@lstsrv.ncep.noaa.gov">Ncep.list.idp.mrms.users@<wbr>lstsrv.ncep.noaa.gov</a><br>
<a href="https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp.mrms.users" rel="noreferrer" target="_blank">https://www.lstsrv.ncep.noaa.<wbr>gov/mailman/listinfo/ncep.<wbr>list.idp.mrms.users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Senior Duty Meteorologist<br>NOAA/NWS/NCEP/NCO/OMB<br></div></div></div>
</div>