routing issue, TX Datacenter

A couple of links are flapping and seems to be main datacenter core switch issue it is being checked by multiple engineers and our staff are on standby as well.
 
I am here at the datacenter updating staff by phone as I have updates.

There was a non customer impacting maintenance tonight that didn't go well and that took down some core fiber links somehow. This didn't set to well with me we'll work that out after all resolved however. We will now insure we get notice of such even if "non impacting" due to this.
 
Things are coming up, I will be following up on the matter and working to get a report, and also work on internal solutions to prevent but this will not be immediate
 
Still mostly down I am on 2 phones right now trying to call every escalation I have. This is unacceptable maintenance window for such matters.

I really am fuming and have to watch my words right now.
 
They are on call with the hardware vendor now to resolve the matter, this is a bad situation as all the fiber passes these systems and they are down.
 
Seeing a lot of up, down, up down, up, down...waiting for the links to stay live. The vendor is working on it now.
 
Onsite techs, and multiple engineer teams working on the problem with the fiber gear.
 
I have my devices recharged a bit and heading back to the datacenter where hopefully I can provide more up to date info on status.
 
Up for about 10-12 minutes now without a down, but not an all clear signal given yet.
 
ok they just called me back, keeping an eagle eye on the gear but things should be good for all, no links have flapped in about 25 minutes now.
 
Looks good, not quite ready to call it resolved but no packetloss at all, no flapping links, and looks all routes working.
 
There have been no more issues with this, we are still working to get more details on the failure.
 
You must log in or register to reply here.
Back
Top