I did not set out to run a controlled experiment. I set out to make five slow sites less embarrassing. But Google’s announcement this morning turned the work I did last month into something more useful than I intended, and the reason is a single clause most people are going to skip past.
Amit Singhal and Matt Cutts confirmed today that site speed is now part of the ranking algorithm. Two qualifications came with it. Fewer than 1% of queries are affected. And the signal applies only to visitors searching in English on Google.com.
My five sites rank in Swedish, on the local Google. They sit outside the signal that launched. And they moved anyway.
What was actually done
Five sites, between 6,000 and 9,000 pages each, all sitting on ordinary shared hosting. Webmaster Tools flagged every one of them as 40–80% slower than average. Measured by hand, the home pages took five to six seconds.
They were moved onto virtual private servers. Alongside the move: gzip compression, page caching on the one site running WordPress, a switch off Apache, PHP with an opcode cache, and a pass over the MySQL databases where the indexes turned out to be doing far less work than they should have been. Home pages came out at 0.4 to 1.2 seconds.
Two weeks after the move, all five sites showed an average of 15% more pages in Google’s index, and on four of the five, a large number of keywords had climbed out of the twenties and thirties into the top ten. Several more weeks on, the gain has held. This is not a re-crawl wobble settling back down.
The explanation I nearly published
The draft I had sitting in front of me said: slow sites are penalised, these sites were slow, they are now fast, the penalty lifted. It fits. It also flatters a position I had already taken publicly back in November, which should have been the first warning sign.
If the speed signal is not switched on for Swedish queries, it cannot have lifted anything. So the explanation has to live somewhere else, and it does - in the number I treated as a footnote at the time.
Fifteen per cent more indexed pages.
Rankings are an outcome. Indexation is a precondition. A page that is not in the index does not rank badly, it does not rank. On a site of 8,000 pages, 15% is roughly 1,200 pages that went from not existing in search results to existing in them.
Crawl budget is the mechanism
Googlebot spends a finite amount of time and a finite number of requests on your site per day. Google has not published a number, but the behaviour is hard to read any other way. How much of your site it gets through inside that allowance depends on how fast you answer.
The arithmetic is not subtle. A site that returns a page in five seconds and a site that returns one in half a second do not get the same number of pages crawled in the same window. That is an order of magnitude. On a 300-page brochure site it makes no practical difference, the crawler laps the whole thing several times a week regardless. On an 8,000-page site it is the difference between the whole structure being crawled regularly and the tail never being reached at all.
That was exactly the position these five sites were in, and I did not see it until this morning. Deep category pages and old articles were being visited so rarely that they never established themselves in the index. Cut the response time by a factor of ten and the crawler gets all the way round.
The ranking jump follows from the same thing, one step further out. A site that is fully crawled has its whole internal link structure visible. Link equity from the home page can distribute the way the architecture intends. Before the move it was distributing across half a site.
Where this argument is weak
I changed several things at once, so I cannot fully separate the effects. The sites also moved from shared IP addresses to their own, and that alone could account for some of the movement. I have no way to isolate it now, which is the weak joint in the whole measurement.
What I can say is that the indexation figure points one way, and that the mechanism I first wanted to credit is, on Google’s own account, not active for these queries.
What today’s announcement is worth
For an English-language site competing on Google.com, the speed signal is real from today and worth acting on. For everyone else it is a preview. Both groups will read the same headlines this week, and I expect a good deal of speed optimisation to be sold over the next few months on the strength of an announcement that explicitly does not apply to the buyer’s market.
The irritating part is that the recommendation is still correct. It is the reasoning that is wrong. If you run a large site, cut your response times, and do it now, not because Google hands you a ranking bonus for being quick, but because a slow server is holding a meaningful share of your pages outside the index entirely. That is a far more expensive problem than a few positions.
Which changes the order of work. Do not start by shaving milliseconds off the home page, it is the most frequently crawled page you own and the least of your problem. Start by establishing how many of your pages are actually in the index against how many you have published. Then look at response times on the deep pages: categories, product pages, archived articles. That is where the crawler runs out.
And change one variable at a time. I moved hosts, swapped web servers, added caching and rebuilt database indexes over a single weekend, and four weeks later I cannot tell you which one did the work. I expected that to bother me less than it does.
Has anyone else measured indexed pages before and after a hosting move? I would like to know whether 15% is typical or whether these five were unusually badly served to begin with. If you have the numbers, I would be glad to hear from you.