We were doing a RAID rebuild on the node for these servers today, and that’s why I was in the datacenter when the MSSQL6 server went down.
The RAID was going well in rebuild but seems that another drive in the RAID set failed. Fortunately it was a RAID1 and we can very likely get it back up rather soon, but I am going back onsite to do so now (Was working mssql6 remote now)
However we will also likely need to schedule a maintenance period to move these as a drive going offline is a bad sign of impending failure.
We’ve decided we are going to take course of action to copy the servers off the one that went offline to a newly built RAID and not risk it going offline again on another rebuild attempt.
going to continue in a single thread for web2, web7, and mail2
We started copying the files and again the drive went off, so we are sure that a RAID rebuild was not going to get us anywhere with this, since it never would have rebuilt. We’re now working on alternative options and doing restore from backups.
MAIL2 is LIVE. (from live data, not from restore)
Web7 is up and down some, data is restored but the web server is crashing quite often. We’re working on getting the configurations reloaded so it stabilizes. Web2 is still restoring, other services are all back up and running well other than web7 web service crashes, and the web2 restoration still in progress.
Web2 is LIVE. Web7 still having some struggles being worked out.