Apparently they issued a major maintenance notice and did not include us in the notice list to take precautions against problems....we are working it out over the phone now.
the window was for 15 minutes, which they have just passed now. But not up yet on the fiber.
The maintenance was for a switch update, and it has failed they are dispatching a technician to the datacenter to check it. it looks like at least 20 more minutes.
just to note: this is NOT our equipment that has an issue, this is a core fiber uplink that is in the carrier cage at the datacenter. There are other providers in the datacenter down as well due to this issue.
They are well beyond their 15 minutes of scheduled so even if we had gotten notice, this would have been well outside of it. I am staying on the phone with them till we are up, initially they wanted a callback but it did not come in 5 minutes, so I am staying on to get updates on the fly.
i know you are wanting an ETA, even our own staff is, however this is all unplanned at this point and ETAs are not available. I will give up updates as I have them.
ok update:
The bootstrapper is not loading on the hardware platform, they are working to roll everything back.
Other companies are riding on this fiber as well, that was my fear. I have been told by them they are getting new flash memory in the unit and reloading now.
I don't have further updates yet. I feel so powerless in cases like this. It is very bad, we do realize that. Made worse by the fact that we were not given a notice of this.
manager just came on and talked to me and said that the compact flash card that the OS is stored on it not loading properly and that is what is making the delay.
Re: Network issue - major fiber maintenance - GONE BAD
I am about to make a followup call and have an escalation on this matter to have an engineer followup on this and how it has happened. but the good news is that all is up right now.
Re: Network issue - major fiber maintenance - GONE BAD
I am absolutely livid right now, they never told me last night this was a two night event, and they are rebooting more routers across the network for a security issue.
I have a SCRENSHOT of the notice we were supposed to be sent, but not send, which is after the fact now. And they are doing maintenance again tonight.