NoIndex and NoFollow Tags

Other feeds

What's New

[ 0 comments ]

I wanted to allow search engines to follow links from a certain page, but not to actually index that page. Obviously I knew that I could use the noindex and follow metatags, but most webpages that I could find said that search engines didn't recognise these tags. Using the robots.txt file would be no good, since that just stops robots from indexing the site. Anyway, it seems that the major 4 search engines DO recognise the metatags:

Google

Yahoo

MSN

Ask

While looking for this information, I was amused to find that MSN suggested a novel way of stopping its search bot from visiting a page. MSN suggested (http://search.msn.com.sg/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_RemoveSite.htm) removing the page from your website!

The only things that I need to find out is how to stop search engines indexing my homepage twice. There are two ways of getting to my homepage:

http://www.robdickson.co.uk
http://www.robdickson.co.uk/index.php

Obviously both of these links go to exactly the same page, but they often get indexed twice. I can't use the noindex tag, because that will stop both links from being indexed. I could try including:

User-agent: *
Disallow: /index.php

in my robots.txt file, but this seems a bit risky to me. I think I'll just allow the search engines to index it twice.

[ 0 comments ]

Bookmark with: