Win5 went down at 2:11am according to my monitoring service. I waited 10 minutes and submitted a ticket at 2.22am.
Why did it take a whole hour to acknowledge there was a problem?
Why did it take over 2 and a half hours to fix it?
Thanks for fixing it, but this is another major outage in my book and I’m not very happy. I guess I should be thankful it happened in the dead on night.
The network card failed at 2:15AM
Issue identified by 2:35AM
We tried to use IP recreator of HSphere IPs. Our efforts were abandoned at 2:55AM
Shared IP assigned by 3:00AM. 90% of sites came back after this.
Took another 5 to 10 to reassign dedicated IPs and the remaining 10% sites were up
Your ticket:
Created: Dec 11, 2004 2:28:25 AM
Last Mod: Dec 11, 2004 3:16:40 AM
Subject: Win5 Down
Your ticket was updated at 3:16AM with the issue stated as “resolved”. You created your ticket less than 1 hour before that time.
Win5 was definately not down for 2 hours and 30 minutes. We got everything up as fast as humanly possible. This was a hardware failure and its the first time we have ever seen a network card failure
Seems to me you need to complain loudly to PSoft to get their IP recreator to actually work. That should shave off an hour of recovery time should it ever occur again..
[QUOTE=Logan]
Yet another problem with HSphere and PSoft that makes me worry about your dependance on them.
[/QUOTE]
The issue was not caused by HSphere/PSOFT. A tool that could reduce the outage time was not functioning. However such a tool is not available with other win cps such as Helm.
The issue was caused by an unexpected network card failure. I believe your sites were on a shared IP so they were not affected by the extended downtime dedicated IP customers faced.