Category: "Chicago DC"

Metis Server - Unplanned Outage (05/25/2010) (RESOLVED)

May 25, 2010 at 3:06 PM

While resolving some service issues on the Metis server today we encountered a problem that required the server to be rebooted immediately, during the reboot it was determined that there was filesystem corruption on the primary hard drive, and we are waiting on a filesystems check to complete at this time. The estimated total downtime will be around 90 minutes, which leaves approximately 30 minutes for the remainder of the filesystems check. If this timeframe is to be extended we will update this notice here immediately.

UPDATE:

We have determined there are further problems to the filesystems on the Metis server which require an additional filesystems check and repair, we do not have an exact ETA for resolution at this time, however we expect between 30 minutes and two hours to complete the remaining filesystems repairs.

UPDATE:

The filesystem repairs to the Metis server have been completed at this time, and service is coming back online now for users on the Metis server.

Chicago DC - Planned Maintenance Window (04/28/2010-04/30/2010) (RESOLVED)

April 19, 2010 at 2:04 AM

Greetings fellow geeks!

All servers located in our Chicago, IL data center location will be undergoing a brief maintenance period that will require approximately 15-30 minutes of downtime per server.

The maintenance will consist of moving servers to a different location within the same data center. This move will lay the groundwork for some of our major service enhancements that will begin next month.

We have tentatively scheduled this maintenance period between the dates of 04/28/2010 and 04/30/2010. The move will happen between the hours of 12 AM and 3 AM CST on those dates.

Further updates regarding this maintenance will be posted here.

UPDATE (04/27/2010 @ 11:35 PM CST):

The previously announced maintenance window for the Chicago DC servers has been scheduled between 12:00AM and 3:00AM CST Friday morning (04/30/2010). The downtime per server should be between 15-30 minutes.

We look forward to continuing improvements for our services, stay tuned for many exciting announcements following the server relocation at the Chicago DC!

UPDATE (04/30/2010 @ 02:32 AM CST):

It looks like our initial estimate of 15-30 minutes downtime per server was a little optimistic, at this time we expect service to be restored within 1 hour to all Chicago servers. We apologize for the extended maintenance period and would like to thank everyone for their patience.

UPDATE (04/30/2010 @ 05:50 AM CST):

The maintenance for the Chicago DC servers has completed as of this time, note for users on Node3 you may notice an additional 10-15 minute outage as your quotas are recalculated, we apologize for this inconvenience. Once again we would like to thank everyone for their patience throughout the maintenance this morning.