[Ncep.list.idp.mrms.users] MRMS outages on October 3, 2017

[Derek VanPelt - NOAA Affiliate]

MRMS customers,

Our data center in College Park had a networking problem yesterday and
these problems continue into today.  When the network problems occurred we
lost much of our access to the internet, and unfortunately lost our ability
to connect to our list serve.  As you may have noticed, the message
detailing the problems with MRMS and the action taken (moving active
operations to the BLDR site) was held up until after some of the problems
were resolved.

The rough time line of these events is that we started experiencing
problems with out MRMS feed at ~14Z, and were able to move the active site
to BLDR at ~16Z.  Issues with the DNS server delayed the propagation of all
of our customers to Boulder until as late as ~18Z in some cases.

The College Park instance of MRMS continues to operated in a severely
degraded mode, with multiple products missing or completely unavailable, we
are continuing to work to restore College Park, but for now those of you
who depend on the local College Park instance may still be seeing outages.

As of late yesterday afternoon all Bouilder MRMS services should have been
restored.  A lot of time and effort is being spent to understand why the
network failed yesterday, and how to prevent these occurrences in the
future.

Please let us know if you have any further issues today.

Thank you,

Derek

-- 
Derek Van Pelt
IDP Onboarding
NOAA/NCEP/NCO
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/ncep.list.idp.mrms.users/attachments/20171004/e6f9e4b0/attachment.html