Mail server is inaccessible due to some problem.
Secondary mail server is already up, so mails will not be lost.
We will provide an update soon here about the issue.
We are currently examining hard disk. We will spend 5-10 minutes to recover it. If not, we have a brand new hard disk. We will install it. We will then restore from yesterdays backup. It will take 4-5 hrs.
Just to let customers know, the backup mail server is online so no incoming email is being lost
It appears that the Harddirve/RAID system on the mail server has failed. We are trying to recover that system. If it is not possible, expected recovery time is about 4 to 5 hours as Atul outlined.
Hard drives are officially dead.
We are proceeding with a complete recovery
We have an excellent team that is dealing with the recovery right now so we request customers to remain calm at let our team recovery from this failure as quickly as possible
Server has been reinstalled
Qmail and HSphere and going to be installed shortly. After that, directory recreation and data recovery will start.
Data restoration is underway
1.5 hours to 3 hours is project recovery time from this moment
current ETA is between 1 to 2 hours.
We are extremely appologetic for this incident. We as a hosting company did do quite a bit (much more than alot of hosts) to ensure 100% email reliability including a backup server and top-class hardware
But a RAID failure is the worst thing that can happen to a host. We will however be looking towards a hot-backup solution for our mail server with active replication to ensure that in future, recovery time is much much shorter
We are entering the final phases of recovery and there may be a few qmail restarts in the process.
Mail fully recovered.
All email missed during this period is going to be piped back to the primary mail server from the backup server within 15 to 20min
The emails from the backup server are being piped over shortly.
Please do not turn on/off mail services now. No need