David Kendall

In case you didn’t read my original article, I have taken over the SEO of a blacklisted website and been told by a Google official that my best bet is to use Google Webmaster Tools including sitemaps. Yesterday I set up a robots.txt file to inform the search engines of all of the dead pages left over in the index of the old site. I also created a sitemap xml file and submitted to the webmaster tools website application.

The SEO of a blacklisted website day 1

OK, here we are at day2. Logging into the Google Webmaster Tools website it tells me that the robots.txt file was found last night whilst I was happily tucked up in bed. The fools who originally ‘optimised’ the website used an automated tool that created many gateway pages, the majority of which had been cached by Google, one of the main reasons for the blacklisting. My robots.txt file tells Google that these pages no longer exist and should not be listed.

This is good news, the first step to gaining an official entry is to remove the bad pages. Google obviously paid attention to these ‘bad’ pages.

OK, so the robots.txt file was accessed, what about the sitemap.xml file? This field tells Google about all crawlable pages within the website. Click the sitemaps tab and yes, success, the sitemaps.xml file was found and read. All of the url I listed within the sitemaps.xml file were submitted to Google.

My next port of call was to see whether the bad pages were still in Google’s index and the good pages from my sitemaps.xml file had been added. Choosing ‘index stats’ and ‘indexed pages in your website’ on the Google webmaster tools application displays every page of your website google has in it’s index.

Well, hats off to Google. All of the bad pages have now been removed from Google’s index and all pages from my sitemaps.xml file have been listed. You can easily find out which pages of a website are listed on google by typing into the Google search bar

site:www.yourwebsite.com

So we are only on day 2 and things are going swimmingly. We created robots.txt and sitemaps.xml files to tell Google to remove bad pages from the index and list the good. Google promptly did this task for us overnight (I am in the UK).

Tomorrow I will check the keywords of the website and see whether we have made any inroads into rising up the search engine rankings.