Just under two years ago, my SuccessWhale Twitter client was gaining new users at a steady rate and, as I noticed with alarm, was about to blow through my then-limited bandwidth allowance.
I’ve since relocated all my web stuff to Dreamhost, taking advantage of their unlimited bandwidth offering to plow through 10 GB and more a month. But now I’m coming up against the last remaining limit of my shared hosting - memory usage.
Both Westminster Hubble, which constantly crawls MPs’ social networks and RSS feeds, and an increasingly complex SuccessWhale, churn through a ton of memory. I don’t have a nice scary graph for this one, but at peak times, I’d estimate that my web server kills over half my PHP processes due to excess memory use. That means Only Dreaming basically goes down, while SuccessWhale throws errors around if it even loads at all.
It looks like I’m left taking the expensive plunge of moving my hosting to a VPS rather than a shared solution, which is a jump I’m nervous to make, especially since none of my web properties make me any money. Most worrying of all is that VPS prices tend to vary by available memory, and I don’t actually know how much memory all my stuff would take up if it were allowed free rein. And nor do I have any way of finding out, bar jumping ship to a VPS and taking advantage of free trial weeks.
So, dear lazyweb, do you have any experience with this sort of thing? And can anyone reccommend a good (cheap!) VPS host that fulfils the following criteria:
-
LAMP stack with “P” being both PHP and Python (or *BSD instead of Linux)
-
Full shell access
-
Unlimited (or at least 100 GB) bandwidth
-
Unlimited (or at least 10 GB) disk space
-
At least 20 MySQL databases
-
IMAP mailboxes & mail forwarding
I’ve been recommended linode by a friend which seems great for tinkering, though the price scales up rapidly with RAM use and I’m not sure I want to deal with the hassle of setting up Apache, MySQL etc. by myself. And there’s Dreamhost’s own offering, which would be virtually zero-hassle to switch to, but probably isn’t the cheapest around.
So, citizens of the interweb, I seek your advice!
Comments
I'm on WebFaction, but that was because they give you sufficient control that I could run Haskell, Yesod + PostGres (or Redis)* on it without having to hassle them, rather than any impressive price structure or high limits.
Perhaps the flipside of the coin is to look at optimising memory usage?
WebFaction only offer shared hosting, though unlike Dreamhost (AFAICS, anyway) you can at least buy more memory allowance up to a limit.
Optimising memory usage would be nice - I've never paid attention to it before, so I don't really know what's possible or how to do it. My gut feeling is that there's not really anywhere it can be optimised, which is a sure sign that I don't know what I'm talking about :P Are there any common ways of profiling PHP memory usage?
out of interest, how many concurrent users are you averaging? is it posible to work out a 'per user' load?
this sort of sounds like a stack-exchange question.
Google Analytics suggests we run at around 30 users per day, but it's pretty difficult to judge how many files are served to them, as only index.php has the Analytics code -- I don't get to see how many times column.php is requested, for example. (And for the sake of speed, it probably ought to stay that way.)
Analytics suggests users spend on average 12 minutes per session on the site (no idea how it calculates that), but each user may have multiple sessions over the course of the day.
It gets more complicated still, because not every column causes equal load -- a Twitter Home Timeline column is much easier to crunch data for than my "Mentions & Notifications" column for my five accounts :)
I think I'd have to add some custom analytics code within SuccessWhale to get exactly the stats we're interested in. And you're right, it could well be Stack Exchange material.
in which case it is definatly worth doing some profiling, etc. otherwise who is to say that adding more RAM will really make any difference? and learning how to profile php code is probably a useful skill to have. :)
What's your upper limit on memory usage?
I don't really know that much about running a website, the internals PHP interpreter, or indeed profiling anything that isn't Java or Haskell. But 30 visitors/day with an average of 12mins/visit sounds like there surely must be lots of scope for improvement.
Interested to see what you end up doing with this. :)
That's the tricky thing with shared hosting, there is no hard limit on memory use. Individual processes are killed if they use too much, but the limit is a factor of not only what else you have running (e.g. other PHP scripts) but also what the other people hosting on that server are using.
It's particularly frustrating as there's no way to gauge how responsive a website will be in the long run. As a case in point, when I wrote this post, SW and Only Dreaming were really suffering with around 50% of requests timing out or failing at peak times. Today, it's much better, and I'm not noticing any changes in my own stats. That probably means it's load on someone else's site contributing to my problems.