HVM4 - resolved

Stephen

Stephen

US Operations
Staff member
It seems with the recent issue a couple of HD may have had some issues, as we are seeing one dropped from RAID this morning and the other is giving some Bad sector errors.

We are working on this at top priority, and will move them to other drives without this issue.
 
only 2VMs are down at the moment, we have disabled websitepanel control of the node at the moment while we work to get this resolved. ALL VMs will need to migrate to new storage and have a downtime for that process, if you have snapshots it WILL take longer than not having snapshots. We again do not recommend snapshots be used, this is one of the reasons.
 
ok, now another is doing BSOD on booting, so the issue is getting worse and all may have to go down, we were trying to do it at least in a phased manner, but may not be possible.
 
ok, they all must go down while we correct the issue, it is getting worse trying to keep them live. We have emailed each client on a new ticket.

We have both RAID1 drive copies to work from, one is already out and trying to work on it is being beyond slow, the other we were trying to run the remaining servers from live and it was not working well.
 
Because we are dealing with two mirrors drives that split a couple hours ago due to bad sectors, we have a lot of repair options here, but looking at about 5 hours ETA to getting some up in my best level estimate.
 
Ouch, this is quite bad over 200 bad sectors so far in the scan, and over 200 repaired, but it is taking quite a long time. hoping we hit a patch were it is working better ASAP.
 
as of now 14% of scan is done with 904 bad sectors. The objective right now is to let it finish and get data off the drive ASAP! The good news is the 904 bad sectors have all been repaired.
 
28% done now with 950 found and repaired, so finally a big jump in progress.
 
94.18% done with a total of 1008 bad sectors, all repaired, we are hoping to be able to get some machines online very soon, but we need to move them to new home quickly as well.
 
Moved the drive back to the server and made an attempt to run live off it, but it is weak, so we are doing a copy now to other RAIDs to bring live.
 
Very frustrating here, one is about to be up, but others still having some issues we are working to resolve. Seems like the sectors are going bad faster than they are being used...
 
The drive that dropped offline about 45 minutes before the 2nd one started having problems is looking to have less bad sectors than the one that stayed online longer. We are working to get data off this one at a priority now over the other drive. About to have a 2nd one on it is copying to the new array now, once it is copied we'll bring it up, 45 min ETA on it, others still a major work in progress.
 
Progress, as slow as it is going, is still happening, there are currently 3 offline VPS that we are working on.
 
Having trouble getting complete copies of any of these 3 VHD files, we are working on alternative ways now.
 
down to 2 now....This is another bad day, but We're going to make the best of it!
 
Just to provide a final update, we got one up last night on final place, all others up.

One we had to bring up non RAID and working to move it now but it is up and running.

both drives has 8 HARD bad sectors, unrepairable and not remapping with any software. They were in the same region and both had the same issues, there were other bad sectors as well but we managed to repair them without issue.
 
You must log in or register to reply here.
Back
Top