While crawling through a website, search engine spiders and bots can create a lot of issues which are not actually present. They might create issues on duplicate content, not consider important pages and write them off as junk, and many such problems which might become a real headache for search engine optimization (SEO) experts. There have been instances where Google bot has provided PageRank to unimportant pages such as subscribe, register, etc. and even moved important pages to the Google supplemental index. All such incidents point towards the importance to guide these search engine spiders and bots and minimize the recurrence of similar issues. As a result, the SEO industry has come up with working with ‘noindex’ and ‘nofollow’ tags in order to guide these bots appropriately.

Pages with ‘noindex’ tags indicate to the search engine spiders that these pages are not meant for indexing. Similarly, pages with ‘nofollow’ tags mean that those pages are not to be passed with PageRank through a specific link to the page that it is linked with. However, the usage of these tags has not been very wise and more often than not they have created more problems rather than helping solve page indexing related issues.

One of the common mistakes made while using these tags is their combined usage. Often, you will find ‘noindex’ tags are combined with ‘nofollow’ tags, which tend to confuse the search engine spiders. While one tag indicates not to index the page the other means that the bot is not supposed to follow the link. This combination is not required.

Using a ‘noindex’ tag alone is sufficient if you do not want a page to get indexed. For instance, if you have a page with similar content like another, you may want to use the ‘noindex’ tag to avoid duplicate content issues. However, you might still want the page to pass PageRank. Using the ‘nofollow’ link will prevent this from happening. The ‘nofollow meta robots tag’ is similar to that of ‘nofollow’ tags. However, the ‘nofollow meta robots tag has its effect at a higher level or which can be said as the page level. This means when the search engine spiders find ‘nofollow meta robots tag’ they are not supposed to follow any of the links present in the page while the ‘nofollow’ tag points towards a specific link in the page.

It is advisable to use ‘nofollow’ tags in circumstances when you are doubtful about the link quality or page that appears through the link is difficult to crawl or causing duplicate content issues. You can also use these tags when the link is sponsored by someone or if you do not want the page to pass PageRank in order to implement any specific strategy.

India based Search Engine Optimization Firm Convonix has helped websites all over the world to rank on top search result pages across major search engines. Convonix Inc offers customized Search Engine Marketing Services including SEO and PPC campaign management service to help websites make the most of their online property.

The above article is Copyright of Convonix, Inc. This article may be copied for electronic publication on a blog, newsletter, or website, provided it is re-published in its original form, with the author information, copyright, and hyperlinks intact.