[Ncep.list.idp.mrms.users] OPEN - TID: 809080283 - IDP - URGENT: Paducah radar data not being used in MRMS (since at least12z)?

Wed Sep 12 16:34:05 UTC 2018

Hello All,

Here at SPoRT, we run an experimental land surface model product over CONUS at ~3km resolution.  We utilize the MRMS hourly gauge-corrected radar QPE product in the 3 most recent days for driving the soil model.  We saw the same type of polygon blockage pattern in our web graphics from Saturday 8 September, as the remnants of tropical storm Gordon traversed western Kentucky/southern Illinois.  The degraded MRMS product clearly affected out output quality as well.  (see an animation of hourly near-surface soil moisture with MRMS contours overlaid from 8 September at https://weather.msfc.nasa.gov/cgi-bin/basicLooper.pl?category=lis_CONUS&initialize=first&regex=rsoim0-10_20180908*.gif).

Sincerely,
Jonathan Case @ NASA/SPoRT Center

From: ncep.list.idp.mrms.users-bounces at lstsrv.ncep.noaa.gov <ncep.list.idp.mrms.users-bounces at lstsrv.ncep.noaa.gov> On Behalf Of Kenneth Howard - NOAA Federal
Sent: Sunday, September 9, 2018 9:56 AM
To: NOAA SDM <sdm at noaa.gov>
Cc: Holly Reckel - NOAA Federal <holly.reckel at noaa.gov>; Kevin Conaty <kevin.conaty at noaa.gov>; _NWS NCF Supervisors <nws.ncf.supervisors at noaa.gov>; Carrie Langston <carrie.langston at noaa.gov>; Shellene Cope - NOAA Federal <shellene.cope at noaa.gov>; Jeff Brogden - NOAA Affiliate <jeff.brogden at noaa.gov>; Patrick O'Reilly <patrick.oreilly at noaa.gov>; Jian Zhang <jian.zhang at noaa.gov>; ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov; Susan Murphy <susan.murphy at noaa.gov>; NCO <toc.nwstg at noaa.gov>; _NWS ER OHRFC <nws.er.ohrfc at noaa.gov>; Curt Janota <curt.janota at noaa.gov>; Alan Gerard - NOAA Federal <alan.e.gerard at noaa.gov>; Vanessa Martinez - NOAA Affiliate <vanessa.martinez at noaa.gov>
Subject: Re: [Ncep.list.idp.mrms.users] OPEN - TID: 809080283 - IDP - URGENT: Paducah radar data not being used in MRMS (since at least12z)?

From the NSSL perspective the best option to handle the situation would be to remove KPAH from MRMS processes, and let other neighboring radars (KLSX,KVWX, KHPX, and KNQA) fill in.  Yes, there would be some slight degradation of products in the KPAH area but this may be significantly better then the current 'hole',

For MRMS v12, we'll have an automated ARJ failure identification scheme and a corresponding QC mitigation to avoid the issue being currently experienced.

Our apology this is currently occurring in the MRMS operational system.

Ken

On Sun, Sep 9, 2018 at 6:29 AM, NOAA SDM <sdm at noaa.gov<mailto:sdm at noaa.gov>> wrote:
To all,

A summary of what IDP Onboarding has provided concerning this urgent ticket.

This is a recap of this morning's phone conversations.  Initially we thought this was an MRMS ingest issue so we restarted processing on KPAH radar (vm-lnx-mrms-sr17a). This is before we got the email saying it is a hardware failure at the site.    Since some of the data is coming in for the KPAH radar,  MRMS does not know it has to fill in from the other radars.   In order for them to fill in we need to remove KPAH from MRMS completely. This requires a config file change and a restart of MRMS processing on that VM.   The problem with this is that it would cause WFO Paducah  to lose the local radars products for KPAH which are still coming in. Based on this I understand you do not want us to make the change and we will stand down.

Randy

On 9 September 2018 at 11:53, NCO <toc.nwstg at noaa.gov<mailto:toc.nwstg at noaa.gov>> wrote:
Ticket Properties
Ticket Number: 809080283
Opened: 2018-09-08 19:54:10
Status: OPEN
Category: IDP

Description: URGENT:   Paducah radar data not being used in MRMS (since at least12z)?

2018-09-09 11:52:28 GMT At 1146z, Mark F, e-mailed further clarification:
But even with the PAH down for maintenance UFN, this shouldn't leave a hole in the product. Radars go down throughout the country fairly often, and when this happens, MRMS data is filled in from surrounding radars. But with the PAH outage, this fill-in from other radars is not happening like it usually does, and THIS is where the problem exists -- where the MRMS product is created. I hope this clarifies things. //JOHN//

===================
2018-09-09 11:37:31 GMT [NCO-MCL] NCO called the Lead Forecaster at PAH and he reported that the issue is with the Radar itself. Since there is weather in the area, technician would like to wait until weather cleared before working on the radar further.

===================
2018-09-09 11:32:06 GMT At 1120z, On-Call IDP Sys Eng, Vanessa called with update that she is researching to see what options may exist, if any, at the systems level, to enhance MRMS output. At this time, it remains uncertain. Vanessa is also going to consult with Senior IDP Eng for additional input. //JOHN//

===================
2018-09-09 11:01:42 GMT At 1050z, NCO-Tech Control consulted with SDM, and then called IDP oOn-Call System Eng, Vanessa M. Informed Vanessa this ticket has been upgraded to URGENT, based on input from Sr Meteorologist from Wilmington OH, RFC, and we are interested in determining what type of action, if any, can be performed on MRMS system, to allow whole to be filled by surrounding Radar's, short of Radar repair outside of RADAR repair, at KPAH. Vanessa will take a look to determine what actions can be performed and will report back. //JOHN//

===================
2018-09-09 10:44:08 GMT At 1028z, received e-mail update from RFC Senior Meteorologist, Mark, F:
Well, I don't know who Scott talked to, but it IS critical to OHRFC!! We've got the biggest event we've had in a very long time and this is affecting one of our most important products that multiple agencies depend on. How can this not be considered critical??! //JOHN//

===================
2018-09-08 20:54:42 GMT 2021Z - Received email notification from the SDM: NCF called and also reported the same to the SDM. There are no MRMS issues in BB. Sent an E-mail to IDP on boarding support. 2007Z - Talked to Scott at NCF and he reported that after receiving feedback from several RFC sites it was deemed non-critical and could wait until Monday.
(SEC)

===================
2018-09-08 20:02:46 GMT 1857Z - Received email notification from Ray Davis: i there,
I bet that someone has already reported this, but just in case... The KPAH dual-pole radar products seem to be missing, and MRMS seems to be preventing the surrounding radars to fill in the void. The problem appears to have started sometime after 12z this morning. Could you check into this?

The KPAH hole didn't appear after we processed our grids up through 12z, but it appears that MRMS is now backfilling previously saved grids with bad data. The Following FTM was issued from the WFO in reference to this site: KPAH 062106 FTMPAH Message Date: Sep 06 2018 21:10:57 THE KPAH RADAR IS OPERATIONAL WITH DEGRADED DUAL POL PRODUCTS. DUAL POL PRODUCTS SHOULD BE USED WITH CAUTION. REPAIRS WILL BE MADE TO THE RADAR EARLY NEXT WEEK.

(SEC)

===================

--
Senior Duty Meteorologist
NOAA/NWS/NCEP/NCO/OMB

_______________________________________________
Ncep.list.idp.mrms.users mailing list
Ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov<mailto:Ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov>
https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.idp.mrms.users

--
Kenneth W. Howard
Program Manager and Research Meteorologist,
Stormscale Hydro Meteorology Group (SHMET), WRDD
National Severe Storms Laboratory
120 David L Boren Blvd, Norman, OK 73072
Phone: (405) 535 <tel:(405)%20325-6485> 9863        Email: kenneth.howard at noaa.gov<mailto:jian.zhang at noaa.gov>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.idp.mrms.users/attachments/20180912/056e189a/attachment-0001.html 