Dated: may 04, 2006
Mail3 has been experiencing occasional spikes in I/O activity and load to very high levels causing the server to stop responding to new POP3 connections for short periods (around 5 minutes each) sporadically in the last 48 hours. We've been working to isolate potential causes for these sudden load increases and have determined the best option for us to prevent these issues from happening in the future is to give the server more CPU and I/O capacity to deal with these spikes.
In that light, we have decided to transparently migrate Mail3 to a new Mail3 server that is a Dual, Dual Core Opteron with twice the I/O and disk space capacity than the current server.
We understand customers may be asking why we didn't anticipate this issue. This has been a failure on our part. However, Mail3 had no signs of any issues from as recent as last week. No new customers have been added to this server in the last several months. However, certain software updates and increased swapping on the server in the last 48 hours has been causing these issues
The migration will take place over the next 24 hours. We expect only a minor 15 minute outage during this entire process towards the early morning hours (Eastern standard time, May 6th). The exact time would be announced precisely as we move further into this rolling migration
While the migration takes place, we continue to monitor the current Mail3 server and are acting to prevent any more paralyzing I/O spikes...
Mail3 has been experiencing occasional spikes in I/O activity and load to very high levels causing the server to stop responding to new POP3 connections for short periods (around 5 minutes each) sporadically in the last 48 hours. We've been working to isolate potential causes for these sudden load increases and have determined the best option for us to prevent these issues from happening in the future is to give the server more CPU and I/O capacity to deal with these spikes.
In that light, we have decided to transparently migrate Mail3 to a new Mail3 server that is a Dual, Dual Core Opteron with twice the I/O and disk space capacity than the current server.
We understand customers may be asking why we didn't anticipate this issue. This has been a failure on our part. However, Mail3 had no signs of any issues from as recent as last week. No new customers have been added to this server in the last several months. However, certain software updates and increased swapping on the server in the last 48 hours has been causing these issues
The migration will take place over the next 24 hours. We expect only a minor 15 minute outage during this entire process towards the early morning hours (Eastern standard time, May 6th). The exact time would be announced precisely as we move further into this rolling migration
While the migration takes place, we continue to monitor the current Mail3 server and are acting to prevent any more paralyzing I/O spikes...