Simplifying our hosting infrastructure

Over the past decade, we at Makalu have tended to host everything ourselves. We leased several dedicated servers, hosted all our websites and apps, hosted and managed our own email and operated our own DNS. Recently, we’ve been looking at ways of simplifying.

What's the problem with self-hosting?

While self-hosting offers dedicated server resources and lots of flexibility, there are some downsides:

• System administrator time is expensive.
• Managing the systems means dealing with UNIX—which basically excludes everybody but the system administrator (and some developers) from making changes.
• There's a lot of moving parts in UNIX and configuration involves text files, so you're likely to run into problems related to the fact that a human is turning the knobs. (For example, someone popular once linked to my WordPress blog and even though I was running a caching plugin the site fell over. Why? Because the permissions on the directory the cache plugin writes to were broken. And precisely at that moment, our system administrator was paddling down a river somewhere.)

Simplifying

As part of our simplification process, we’ve:

1. Moved most of our Rails apps to Heroku. They've made the process of deploying and managing Rails apps so simple even I can do it, and their free plan provides enough resources for most of our projects. And given their recently announced ability to auto-scale resources, we may just be able to move our big apps there soon, too.
2. Moved all of our static and WordPress sites to Dreamhost.
3. Moved most of our domains to Dreamhost, and DNS to either Dreamhost, OpenSRS or CloudFlare (more on CloudFlare in a bit).

Pretty much the only things we’re left self-managing is our email, and a few complex Rails apps.

Shared hosting?

A decade ago, I would have never considered hosting anything serious with a shared hosting provider like Dreamhost. Yeah, the costs were low, but you’d better hope your site doesn’t get popular or that you need any support.

Boy have times changed! Having moved most of our domains and WordPress sites to Dreamhost, here’s what I’ve found:

• Value. $10 per month gets you hosting of as many domains as you like, including web sites, email accounts, databases, etc.
• Customer support. The customer support is superb. I've called on them three times in online chat, and each time got somebody who absolutely knew their stuff and could solve my problems.
• Usable management interface. Although the Dreamhost people aren't going to win any visual design awards, they definitely have a pretty good handle on usability design. I've found their management interface to be quite usable, especially considering the breadth and complexity of services it provides. (The quality of the management interface is, in fact, why I discounted the first company I tried, Bluehost. Whereas Dreamhost have developed their own management interface, Bluehost use the (open source?) "cPanel". I took one look at cPanel, became nauseous and asked for a refund.)
• Convenience features. Dreamhost makes it easy to do some things that required a bit of Apache trickery when operating our own servers. For example, in Dreamhost, I setup automatic redirecting of matt.makalumedia.com and thisux.com (my old blog domains) to dafacto.com (my new blog domain).
• One-click installs. Dreamhost offers a number of one-click-installs, making it so easy to setup a new WordPress site.
• Automatic updating. Although I've not confirmed it yet, I've been told that Dreamhost will automatically update WordPress blogs as soon as new versions are released. You can't imagine the number of times our WordPress sites have been hacked because we waited a bit too before getting around to manually updating!

All in all, in my opinion, Dreamhost represents an astounding value.

But is it suitable for a high-traffic site?

As happy as I am with Dreamhost’s offerings, there still remains the fact that your PHP-based WordPress site is sharing server resources with lots of other hosting customers. So if you happen to get fireballed or linked to by someone popular, you run the risk of having your site go down.

There’s two counter-measures you can take, that pretty much eliminate this as a potential problem, and allow low-cost hosting to remain a perfectly feasible option:

1. WordPress Caching. I use the W3 Total Cache plugin.
2. Run your site behind CloudFlare.

CloudFlare

CloudFlare is an amazing service, introduced to me by a colleage of mine in Makalu. It “sits in front of your website”, and so it doesn’t matter what technology your site uses—WordPress, Rails, Drupal, whatever.

What do I mean by “sits in front of your site?”

To use CloudFlare, you have to let it manage your domain’s DNS so that via DNS, it can route traffic to your website through itself. Once that’s setup, it provides the following as part of its free plan:

1. Automatic caching of your website content, including distribution of your static assets to a CDN.
2. Selective performance enhancements, including things like auto-minification of your HTML, CSS and JavaScript.
3. CloudFlare will continue serving your cached content, even if your website goes down.
4. Automatic blocking of certain threats—spammers, bots, etc.
5. Analytics data.

And apart from all this, the user experience is great, e.g. the quality of their DNS management is the best I’ve seen:

Moving your DNS to CloudFlare might sound tricky, but it’s not:

1. You add your domain to your account at CloudFlare.
2. CloudFlare scans your current DNS, importing all the records (it also gives you a change to make adjustments if needed).
3. CloudFlare then provides you with two CloudFlare DNS server host names.
4. Visit your domain registrar and change the DNS servers from whatever they are currently, to those two specified by CloudFlare.
5. Shortly thereafter, you'll get an email from CloudFlare saying they've detected that your DNS has been updated, and that your CloudFlare services are active.

In addition, CloudFlare will create some default sub-domains, like “ssh.domain.com” and “mysql.domain.com”, providing you with access to certain services that bypass the CloudFlare system.

A natural question is that if your WordPress site is running behind CloudFlare, should you still run a caching plugin? The answer is apparently “yes”, as recommended by CloudFlare. In fact, they point out that the W3 Total Cache plugin even integrates with the CloudFlare service.

Done and dusted

Although for the time being we’ll still maintain at least one dedicated server, our overall hosting infrastructure has been greatly simplified, we’ve reduced our costs and have made management of our majority of systems available to a broader range of people in the company.

Enjoy this article? — You can find similar content via the category and tag links below.

• Categories — Technology
• Tags — Geek

Questions or comments? — Feel free to email me using the contact form below, or reach out on Twitter.