The Fresh Egg blog
Latest digital marketing news
Over at SEOmoz a while back, Rebecca mentioned (while here on her SEO Internship training) how Google was incorrectly losing the homepage, and only the homepage, of many .COM Top Level Domains (TLDs) from UK only searches. Even although the domains were hosted entirely on UK IP addresses, Google was having some issues that caused the homepage to vanish when the search radio button was selected to show Pages from UK only.
Well, a contact at Google had a word with some of the folks in the relevant team there and reported back that they had identified the issue, and that it should be fixed within a few days. That was a couple of weeks ago now.
Today, I noted a very similar phenomenon occurring with a .co.uk domain - a Country Code TLD (ccTLD). The homepage of www.dialaphone.co.uk shows up clearly in a search for phone performed on Google UK where results from the web are selected. However, select the radio button for 'pages from the UK' for the same search and they vanish. Note: You may not be able to duplicate those searches, as this issue does seem to affect only certain datacenters.
This is not at all a new domain, and apart from adding a blog recently, no drastic changes have been made to the site. It is a ccTLD for the UK, and hosted on a UK IP address, so should have no reason to ever vanish from UK only results no matter which method of detection is used. So why is it suffering the same issues as some of those erroneously vanishing .com domains?
It needed further investigation, so the first thing was to run all of the standard checks on the domain, IP block, etc. Nothing new there, but I did run a TraceRoute check via my trusty old SamSpade tools.
Somewhere along the Hops in a traceroute I find the following:
18.104.22.168 31ms 29ms 23ms TTL: 0 (ge-2-15.c01.londen02.uk.wh.verio.net probable bogus rDNS: No DNS)
Not an entirely uncommon thing to find in a traceroute, but something that I've seen on many of the affected .com domains I investigated.
Checking further we confirm again:
The hop via 22.214.171.124 is claiming to be ge-1-15.c00.londen02.uk.wh.verio.net (it's reverse DNS) however ge-1-15.c00.londen02.uk.wh.verio.net does not exist.
host 126.96.36.199 188.8.131.52.IN-ADDR.ARPA domain name pointer ge-1-15.c00.londen02.uk.wh.verio.net
host ge-1-15.c00.londen02.uk.wh.verio.netÂ Host not found.
This is the one common feature I have found so far in all the examples of homepages vanishing from Google's results for UK only pages. All have a hop in the Traceroute that gives no rDNS result. Feel free to check the route for yourself on some of the .com domains that vanish such as madmash.com and tubstore.com and you will likely find the same issue.
Make no mistake, even if this is the cause, this is still a bug in Google. A ccTLD should not need to have a UK IP address anyway.
There is no reason why rDNS issues should only and solely affect the homepage either, as happens when you do a site:www.madmash.com or a site:www.tubstore.com search with and without the 'pages from UK only' option selected. This is one symptom I can't duplicate with a ccTLD as site:www.dialaphone.co.uk still shows their homepage.
It is quite a complex issue, and hard to nail down. In the meantime I have advised Dial-a-phone to get on to Verio about the issue of the failing rDNS and see if it can be fixed at that end anyway.
I advise any UK based ecommerce sites experiencing similar issues of disappearing from UK only search results to also investigate the traceroute for the site. The Sam Spade tool is free from www.samspade.org and very useful, not only for investigating these traceroutes and DNS issues, but also for examining the HTTP Server response codes, looking for errors, or checking whether a redirect uses a temporary 302 header, or a permanently moved 301 header correctly.
For those without an affected UK website among their client lists, want to buy a new phone?