Server Canyon - Reimagining Web Infrastructure

Category Archives

9 Articles

Updating default PHP version

by judahnator

The coming months mark the end of life for two popular versions of PHP, which the vast majority of the sites running with Server Canyon use.

What does this mean for me?

If your website is running with PHP versions 5.6 or 7.0, their end of life is coming up. See the Supported Versions page on the PHP website for more information. On January 1st, 2019 those versions of PHP will no longer be available to use, and you will be required to switch to a newer version.

How do I prepare for this update?

If you are like most of the Server Canyon clients you are probably using WordPress. Luckily WordPress supports the latest and greatest versions of PHP. The important thing to note is if your site relies on third party plugins and themes, you want to make sure they are up to date. If they are not your site may not be usable.

How can I tell if my site is PHP 7.1+ ready?

This is the easy bit. Here are the instructions:

  1. Simply log into your websites cPanel account by visiting “yourwebsite.com/cpanel” and entering your login information. If you do not know your login information, reach out to Server Canyon via our support page.
  2. Scroll down to the “MultiPHP Manager” link and go to that page.
  3. Select all of the domains you wish to change the default PHP version on, select the appropriate version from the dropdown menu, and click “Apply.”
  4. If your website still works, congrats! You are all set! If not, revert the changes then be sure your plugins and themes are up to date before trying again. If you need help you can always contact support.

What happens if I take no action?

On December 12th, 2018 all websites that have not manually set their default PHP version to any of PHP 7.1, 7.2, or 7.3 (which will release soon) will be automatically updated to PHP 7.1. If your site breaks during this update you may revert to either PHP 5.6 or 7.0 to troubleshoot.

On January 1st, 2019, versions 5.6 and 7.0 will no longer be available. It is important to make sure your website is up to date before this date to prevent any chance of downtime.

Can I get help with my update?

Yes! Server Canyon is partnering up with Modular Industries to help clients who would rather not deal with any of this stay up to date.

A one-time checkup for your site to make sure it is PHP 7.1 ready will cost a flat rate of $75. If your site requires updates to be ready Modular Industries will make a custom plan for your website to help bring it up to date. For more information visit their website!

4 views

Upgrades Completed

by judahnator

Following up the previous post, which can be found here, all upgrades have completed successfully with less downtime than initially expected.

We are now using the latest and most secure versions of all our software, keeping our customers security a top priority!

0 views

Notice of Scheduled Upgrades

by judahnator

Server Canyon will be installing critical software updates, some of which are security updates, early Saturday (2/25/2017) morning, starting around 7:00AM Chicago Time.

The duration of the downtime is expected to be between 10 and 15 minutes.

What we are doing to prevent loss of data:
The upgrade will be comprised of three steps.

  1. Backups of all accounts will be verified before attempting the upgrade
  2. A live system image will be captured. This way if anything goes wrong, we can roll back to a running snapshot.
  3. The new software will be installed, the kernel will be updated, and the server will reboot

Once all that is done service will continue like normal.

Look forward to many new features in the coming months that will make not only hosting, but billing and upgrades even smoother!

–EDIT–
Upgrades have been completed. See the blog post here.

0 views

Unscheduled Downtime Update

There was some downtime last night. This was caused by a problem with an upstream provider. Although there was nothing we could do but wait, here is a quick timeline of events:

  • At approximately 9:40PM PST we got a notification that our primary server was under attack, and that attack mitigation was automatically put into effect.
    Due to information obtained throughout the incident, we believe this was either a false alarm or unrelated to the problem.
  • At approximately 10:10PM PST I called our provider. They explained they had some electrical problems in their BHS datacenter, and that many dedicated servers and almost all VPS systems had to be taken offline. They said they had dispatched a team, and had either just finished the repairs or were nearly done.
  • At approximately 10:25PM PST I called our service provider again. They confirmed that the electrical problem had been resolved. Servers were coming back online, and that I was in que. ETA to online was yet unknown.
  • At approximately 12:15AM PST the server came back online. There were networking issues. The machine could detect the failover IP addresses, but the primary (which was listed as still under attack) was not detected. There was no network access. Web services were still offline.
  • I got off the phone with our provider again at about 12:40AM PST. They said the networking problem might be caused by ongoing electrical issues. Some switches had gone down, and needed replaced.
  • Over the course of the next 2 hours, the server had rebooted a few times. I called our provider again at 2:30 a.m. PST, they said that they are actively working on the problem. The recovery time was estimated at two and a half hours.
  • Approximately 5:15 a.m., I got off the phone with our service provider. They said they had restarted all VPSs however they could not ping approximately 7000 some of them. The estimated another few hours.
  • I got first response from the server at about 6:15 a.m. It was back online, but with severely degraded performance.

Over the course of the next several hours, performance and response times returned to normal and the mail server began to get caught up. Everything went back to normal by 10:00 AM PST.

Here is a quote directly from the service provider (and a rough translation below), with more information of the incident:

 

Bonjour,
A 10h40pm (4h40 heure française) nous avons constaté un défaut d’alimentation
de 2 salles d’hébergement, C et D, à BHS4, sur 54 baies. Il s’agit de
baies C1-18 et D1-36. Dans ces baies nous avons le PCI/VPS et les SD.
L’origine du problème est un gros cours-circuit dans le bus qui alimente les
baies D1-18. Le court-circuit a mis en défaut l’onduleur T04D. L’équipe
d’électriciens s’est dépêchée sur place pour vérifier l’etat de l’onduleur,
de lignes. L’onduleur a été remis en route. Les baies C1-18 ont été réalimenté
0h30am (6h30). Le remplacement du bus a pris 2H30. A 3h30am (9h30) les baies
D1-36 ont été réalimentée à nouveau.
Nous travaillons sur la mise en route de tous les services qui ne sont pas
encore UP.
D’autres informations vont suivre.

Octave

 

Hello,
At 10h40pm (4:40 French time) we found a power failure in
2 facilities hosting, C and D, BHS4, of 54 bays. It is
C1-18 and D1-36. In these bays we have the PCI / VPS and SD.

The origin of the problem is a large current circuit in the bus that feeds bays
D1-18. The short circuit has faulted the T04D inverter. The team
electricians was dispatched to check the state of the inverter,
lines. The UPS has been restarted. C1-18 were brought online
0h30am (6:30 am). The replacement bus took 2:30. At 3h30am (9.30) bays
D1-36 were brought online again.

We are working on the initiation of all services that are not
online.

Further information will follow.

Octave

 

An important note:

This is a one in a million event. The chances of a catastrophic and cascading failure of this sort are incredibly small. Once the dust and fog of this event have been cleared, I will work closely with Server Canyons provider to harden our infrastructure against future events of this kind. If no progress can be made with this provider, I will not hesitate to leave them for another company.

It is good to know that more often than not, you will learn much more through failure than success. This does not excuse the fact that this event has happened. However, I believe all affected parties have learned a lot from this event!

 

TL;DR:
Were sorry

0 views

Expanding Our Network

by judahnator 0 Comments

Earlier today we received another block of IP addresses from our main service provider.

It is just a small step, but any great adventure must be taken one step at a time!

2 views

Scheduled Maintenance Notice

by judahnator 0 Comments

On the first of February we will apply some software updates to our core infrastructure.

We do not anticipate any downtime, however the increased network load may result in file-transfers and downloads to be slower than normal. The entire process from beginning to end should take no more than 20 minutes.

If any details change, we will put out another post when we have more information.

0 views

Load Spike

by judahnator 0 Comments

At approximately 10:30 AM PST we experienced a momentary load spike, resulting in several websites being slow to respond or timing out.

Upon investigation, it appeared to be a bot trying to gain unauthorized access to the primary server. Security measures and automated attack mitigation services picked up around 10:35 AM PST and traffic slowly died down to normal levels by 11:00 AM PST. We will continue to closely monitor our infrastructure for any further problems.

No data was lost and there was no unauthorized access granted. Services are running smoothly again.

5 views

Billing Errors

by judahnator 0 Comments

Earlier today we discovered that a few clients had been invoiced incorrect amounts for hosting.

The problem was discovered after some customers were notified that a new invoice was available. After resolving the problem new invoice notifications were sent out.
It is important to note that no clients were actually billed because of the error.

As of 9:00PM Tonight (PST), all the green lights are happily blinking again!

1 view