[Ncep.list.idp.mrms.users] MRMS LDM Users: Transition to Boulder

Thu Aug 11 15:00:57 UTC 2016

Hello,

>From the NCEP point of view, the MRMS LDM feed is operational.

As for the data latencies. We have done a few things in the past week and
there are some unknown problems. We have been running out of College Park
since late 2014. This has been working fairly well, with intermittent
problems caused by network and system problems out of our direct control.
To help build a stronger system, we have established an alternate site in
Boulder. On 8/4, we moved the MRMS application to be primary out of the
Boulder servers. Concurrent with that, NCO started noticing slow network
responses for other systems and data movement, including data off of the
WCOSS supercomputer. This is all actively being worked on to find and fix
the problem. To add to the problems, on Tuesday of this week, there was a
major fiber cut that caused delivery of data from the WCOSS to Boulder to
slow down so much it was unusable. So, we failed MRMS back to College Park
late on Tuesday afternoon/evening. The fiber cut has been repaired, but the
slowdown from last week persists.

My team at NCO is on this thread and we will try to do a better job of
keeping everyone informed about issues like this.

Scott

Scott Jacobs
NOAA/NWS/NCEP/NCO/IDSB
301-683-3910

On Thu, Aug 11, 2016 at 10:31 AM, David Welch - NOAA Federal <
david.welch at noaa.gov> wrote:

> Mike:
>
> Almost all of the RFCs would argue that it is an operational product and
> most RFCs are using it as a base field for generating the hourly QPEs. For
> the most part it has been a fairly stable dissemination process with us
> pulling it in via LDM through our Regions, but would be nice to have it on
> the SBN at some point.
>
> David
>
> On Thu, Aug 11, 2016 at 9:22 AM, Mike Dross <mdross at wright-weather.com>
> wrote:
>
>> As a general question ref MRMS.
>>
>> I am not sure if it is considered an operational product at this point?
>> We are testing the output for possible inclusion with some mission critical
>> apps in support of power generation plants/nuclear emergency response.
>>
>> Has it been formally adopted into the suite of operationally supported
>> applications of NCEP/NWS?  If so, is there mechanism, that when we see
>> issues, such as erroneous data/artifacts  we can report that ?
>>
>> Thanks,
>>
>> -Mike
>>
>> On Thu, Aug 11, 2016 at 9:07 AM, David Welch - NOAA Federal <
>> david.welch at noaa.gov> wrote:
>>
>>> All/NCEP:
>>>
>>> Is there a way to elevate the status of the ticket with NCEP to
>>> critical? Overnight, the grid dissemination has been sporadic, missing
>>> 01-05z and sporadic during the day yesterday also, and the grids that have
>>> been making it in are coming in late (up to 46 past the hour), whereas it
>>> was normally coming in at 04 past the hour. The Hydo applications on
>>> A2/Edex run at 24 past the hour and causes artifacts in the QPEs produced
>>> overnight when the grids are missing because of latency issues and is
>>> causing extra QC work during what is a significant rainfall event.
>>>
>>> Thanks,
>>> David
>>>
>>>
>>> On Thu, Aug 11, 2016 at 7:54 AM, Paul Kirkwood - NOAA Federal <
>>> paul.kirkwood at noaa.gov> wrote:
>>>
>>>> Do we know what changed?  We tested all this out and we had
>>>> connectivity.  Seems odd that it would change.  Since I am out of the
>>>> office, I have added Doug and Tony if you need help in diagnosing the
>>>> problem.
>>>>
>>>> On Wednesday, August 10, 2016, Scott Jacobs - NOAA Federal <
>>>> scott.jacobs at noaa.gov> wrote:
>>>>
>>>>> Laurie,
>>>>>
>>>>> They can report the problem to the IDP Support mailing list included
>>>>> on this thread. I also put in an OpsNet ticket and they did an RC to allow
>>>>> all the Regions to connect to our Boulder servers. So, maybe a ticket with
>>>>> OpsNet to check and fix the firewalls.
>>>>>
>>>>> Scott
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Scott Jacobs
>>>>> NOAA/NWS/NCEP/NCO/IDSB
>>>>> 301-683-3910
>>>>>
>>>>>
>>>>> On Wed, Aug 10, 2016 at 11:16 AM, Laurie Hogan - NOAA Federal <
>>>>> laurie.hogan at noaa.gov> wrote:
>>>>>
>>>>>> Scott,
>>>>>> Hi.
>>>>>> FYI.
>>>>>> I'm on a list server group that includes the RFCs.  There was
>>>>>> discussion trend started yesterday where there were several CR and SR that
>>>>>> were discussion issues with the receipt of the Q3, xmrg format MRMS files.
>>>>>> There didn't seem to be receipt issues by ER RFCs.  I don't recall WR
>>>>>> RFCs chiming in.  The RFCs that noticed issues talked about the issues
>>>>>> starting on 8/4.
>>>>>>
>>>>>> Is there a ticketing system that they should be reporting to, or help
>>>>>> desk that they can contact directly?  Or since they receive these second
>>>>>> hand through their regions, they should do something different?
>>>>>>
>>>>>> Thanks.
>>>>>> Laurie
>>>>>>
>>>>>> On Mon, Aug 1, 2016 at 8:02 AM, Scott Jacobs - NOAA Federal <
>>>>>> scott.jacobs at noaa.gov> wrote:
>>>>>>
>>>>>>> MRMS LDM Customers,
>>>>>>>
>>>>>>> As we told you in June, NCO has set up a mirror facility in
>>>>>>> Boulder, CO, for the Integrated Dissemination Program.  This
>>>>>>> facility in Boulder will be running the same IDP applications as the
>>>>>>> current NCO facility in College Park, MD, including the
>>>>>>> Multi-Radar/Multi-Sensor (MRMS) application developed by NSSL.
>>>>>>>
>>>>>>> On Thursday, August 4, 2016 at 1500 UTC, we will be switching to the
>>>>>>> Boulder servers as the primary distribution point. We plan on running on
>>>>>>> the Boulder servers for the foreseeable future. Therefore, please ensure
>>>>>>> you have the following configuration in your LDM:
>>>>>>>
>>>>>>> Your firewall should accept both:
>>>>>>>
>>>>>>>    - CURRENT MRMS LDM server: 140.90.98.15 (
>>>>>>>    *mrms-ldmout.cprk.ncep.noaa.go**v*)
>>>>>>>    - NEW Boulder MRMS LDM server: 140.172.138.50
>>>>>>>    *(mrms-ldmout.bldr.ncep.noaa.go**v*)
>>>>>>>
>>>>>>> Additionally, your LDM configuration should request from LDM server *mrms-ldmout.ncep.noaa.gov
>>>>>>> <http://mrms-ldmout.ncep.noaa.gov>*.
>>>>>>>
>>>>>>> Once the firewall rules are updated and the LDM request line is
>>>>>>> changed to *mrms-ldmout.ncep.noaa.gov
>>>>>>> <http://mrms-ldmout.ncep.noaa.gov> *no further changes should be
>>>>>>> required on your side.
>>>>>>> We ask that if you have problems on August 4, please send an email
>>>>>>> to *ncep.list.idp_support at noaa.gov*. We will be monitoring the
>>>>>>> servers and will respond to any issues as soon as possible.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Scott
>>>>>>>
>>>>>>>
>>>>>>> Scott Jacobs
>>>>>>> NOAA/NWS/NCEP/NCO/IDSB
>>>>>>> 301-683-3910
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ncep.list.idp.mrms.users mailing list
>>>>>>> Ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov
>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>>>>> idp.mrms.users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Paul Kirkwood - NOAA Federal (from iPad)
>>>> Chief, Technology Infusion Branch (TIB)
>>>> Science and Technology Services Division (STSD)
>>>> Southern Region Headquarters
>>>> Email: Paul.Kirkwood at noaa.gov
>>>> Phone: 817-978-1100 x.145
>>>> http://www.srh.noaa.gov/
>>>> "Science Is Organized Knowledge" - Library of Congress
>>>>
>>>> _______________________________________________
>>>> Ncep.list.idp.mrms.users mailing list
>>>> Ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov
>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>>> idp.mrms.users
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ncep.list.idp.mrms.users mailing list
>>> Ncep.list.idp.mrms.users at lstsrv.ncep.noaa.gov
>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.
>>> idp.mrms.users
>>>
>>>
>>
>>
>> --
>> This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity to whom they are
>> addressed. If you have received this email in error please notify the
>> system manager. This message contains confidential information and is
>> intended only for the individual named. If you are not the named addressee
>> you should not disseminate, distribute or copy this e-mail. Please notify
>> the sender immediately by e-mail if you have received this e-mail by
>> mistake and delete this e-mail from your system. If you are not the
>> intended recipient you are notified that disclosing, copying, distributing
>> or taking any action in reliance on the contents of this information is
>> strictly prohibited.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.idp.mrms.users/attachments/20160811/50da05e0/attachment.html 