What happened:

At approximately 5:00AM PST ServerCanyon was alerted to severe performance degradation. The cause as to why is still unknown, but for transparency here is a timeline of events:

• 5:30AM, We rebooted our primary server
• 5:45AM, After realizing a severe problem was preventing the server from booting, the failover server kicked into gear and began to download client backups
•  6:45AM, 80% of affected clients had been activated on the backup server
• 7:30AM, We got on the phone with our domain registrar, as the changes to our name servers “A” record were about 45min overdue.
• 7:40AM, Data recovery began on the old server.
• 8:00AM, All accounts had been brought back online, changes since the last offsite backup began to sync.

What we are doing to prevent this from happening in the future:

• We increased our in-datacenter backup provision by 50%
• All accounts will now be backed up daily (5 day retention) and weekly (4 week retention)
• Over the course of the next several weeks we will be moving clients to a pool of IP addresses that can be routed to new servers
• We are moving to a more reliable domain registrar

If any customer discovers any issues with their website, please contact support immediately!