[Ncep.list.idp.mrms.users] [Ncep.list.idp_support] MRMS issue today

Derek VanPelt - NOAA Affiliate derek.vanpelt at noaa.gov
Fri Mar 10 19:58:37 UTC 2017


MRMS customers,

The MRMS application endured latencies, and produced some incomplete
products over ~3 hours starting this morning @1530Z.

We were able to identify the issue quickly, but it took longer than
expected to enact a full fix.

Apologies.  I believe we have learned from this to the point that we should
be able to avoid future issues.  We will continue enhanced monitoring
throughout the afternoon.

Derek

On Fri, Mar 10, 2017 at 2:51 PM, Karen Cooper - NOAA Affiliate <
karen.cooper at noaa.gov> wrote:

> Hi Steve,
>
> I'm including the NCEP IDP support group here, as this involves them.
>
> It does appear that from about 15:46 to 18:30 that a lot of the products
> that are created by stitching the tiles were missing data.   This is from
> looking at the College Park system (vm-lnx-mrms-convert1a.ncep.noaa.gov
> specfically).
>
> I'm not sure what would have caused this, but it also looks like there
> delays in the processing of some data.   Our processing has timing controls
> to keep data for a short time -- typically twice the frequency of the
> product, but it will disregard data that is too old so that we mitigate the
> risk of including old data, as it could be misleading to analysts.
>
> I'm going to defer to the NCO team as to what may have caused the slowdown
> during that time.
>
> There are multiple CREFs some being done as 2D products which are not
> tiled, and some done from 3D tiled data, so depending on which one you were
> looking at there might not have been tiles missing.
>
>
> On Fri, Mar 10, 2017 at 11:48 AM, Steven Lack - NOAA Federal <
> steven.a.lack at noaa.gov> wrote:
>
>> Karen,
>>
>> It looks like starting after 1530 or so we started dropping tiles and we
>> have had intermittent data coming in since then.  This has been a problem
>> with at least RALA and ET-18, somehow CREF looks okay.  This is both on our
>> AWIPS2 feed and our NIDS feed for NAWIPS.
>>
>> Thanks for any info,
>>
>> Steve
>>
>> --
>> Steven A. Lack, Ph.D.
>> Techniques Development Meteorologist
>> Aviation Weather Center
>> 7220 NW 101st Terrace
>> Kansas City, MO 64153
>>
>> office: 816-584-7281 <(816)%20584-7281>
>> cell:  303-882-9395 <(303)%20882-9395>
>>
>> The views or opinions expressed in this email do not necessarily reflect
>> those of NOAA or the United States Government.
>>
>
>
>
> --
> “The presence of those seeking the truth is infinitely to be preferred to
> the presence of those who think they’ve found it."
>  -- Terry Prachett
>
> -------------------------------------------
> Karen.Cooper at noaa.gov
>
> Phone#:  405-325-6456 <(405)%20325-6456>
> Cell:   405-834-8559 <(405)%20834-8559>
> National Severe Storms Laboratory
>
>
> _______________________________________________
> Ncep.list.idp_support mailing list
> Ncep.list.idp_support at lstsrv.ncep.noaa.gov
> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp_support
>
>


-- 
Derek Van Pelt
IDP Onboarding
NOAA/NCEP/NCO
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.idp.mrms.users/attachments/20170310/97e00ea4/attachment.html 


More information about the Ncep.list.idp.mrms.users mailing list