Google, Yahoo & Microsoft Unite On “Canonical Tag” To Reduce Duplicate Content Clutter

This can be a problem for a couple of reasons.

* Less of the site may get crawled. Search engine crawlers use a limited amount of bandwidth on each site (based on numerous factors). If the crawler only is able to crawl 100 pages of your site in a single visit, you want it to be 100 unique pages, not 10 pages 10 times each.
* Each page may not get full link credit. If a page has 10 URLs that point to it, then other sites can link to it 10 different ways. One link to each URL dilutes the value the page could have if all 10 links pointed to a single URL.

Using the new canonical tag

Specify the canonical version using a tag in the head section of the page as follows:

< link rel="canonical" href="http://www.example.com/product.php?item=swedish-fish">

That’s it!

* You can only use the tag on pages within a single site (subdomains and subfolders are fine).
* You can use relative or absolute links, but the search engines recommend absolute links.

This tag will operate in a similar way to a 301 redirect for all URLs that display the page with this tag.

* Links to all URLs will be consolidated to the one specified as canonical.
* Search engines will consider this URL a “strong hint” as to the one to crawl and index.

Canonical URL best practices

The search engines use this as a hint, not as a directive, (Google calls it a “suggestion that we honor strongly”) but are more likely to use it if the URLs use best practices, such as:

* The content rendered for each URL is very similar or exact
* The canonical URL is the shortest version
* The URL uses easy to understand parameter patterns (such as using ? and %)

Can this be abused by spammers? They might try, but Matt Cutts of Google told me that the same safeguards that prevent abuse by other methods (such as redirects) are in place here as well, and that Google reserves the right to take action on sites that are using the tag to manipulate search engines and violate search engine guidelines.

For instance, this tag will only work with very similar or identical content, so you can’t use it to send all of the link value from the less important pages of your site to the more important ones.

If tags conflict (such as pages point to each other as canonical, the URL specified as canonical redirects to a non-canonical version, or the page specified as canonical doesn’t exist), search engines will sort things out just as they do now, and will determine which URL they think is the best canonical version.

The tag in action

This tag will most often be useful in the case of multiple URLs pointing at the same page, but might also be used when multiple versions of a page exist. For instance, wikia.com is using the tag for previous revisions of a page. Both http://watchmen.wikia.com/index.php?title=Comedian%27s_badge&diff=4901&oldid=4819 and http://watchmen.wikia.com/index.php?title=Comedian%27s_badge&diff=5401&oldid=4901reference the latest version of the article (http://watchmen.wikia.com/wiki/Comedian%27s_badge) as the canonical.

The search engines stress that it’s still important to build good URL structure and also note that if you aren’t able to implement this tag, they’ll still keep the processes they have now to determine the canonical. For instance, at SMX West on Tuesday, Maile Ohye of Google explained how Google can detect patterns in URLs if they use standard parameters. For instance, with these URLs:

* http://www.example.com/buffy?cat=spike
* http://www.example.com/buffy?cat=spike&sort=evil
* http://www.example.com/buffy?cat=spike&sort=good

Maile explained that Google can detect (particularly when looking at patterns across the site) that the sort parameter may order the page differently, but that the URLs with the sort parameter display the same content as the shorter URL (http://www.example.com/buffy?cat=spike).

While it’s rare for the search engines to join forces, this isn’t the first time they’ve come together on a standard. In November 2006, they came together to support sitemaps.org. And in June 2008 they announced a standard set of robots.txt directives. Matt Cutts of Google and Nathan Buggia of Microsoft told me that they want to help reduce the clutter on the web, and make things easier for searchers as well as site owners.

This new tag won’t completely solve duplicate issues on the web, but it should help make things quite a bit easier particuarly for ecommerce sites, who likely need all the help they can get in the current economic conditions. Site owners have been asking for help with these issues for a really long time so this should be a greatly welcomed addition.

Postscript by Barry Schwartz:

The search engines will be talking about this news at the Ask the Search Engines panel at SMX West. We will be blogging this panel live at the Search Engine Roundtable.

Google "Directory Submission" Update

While Google has condemned buying and selling links that pass PageRank, they’ve encouraged listing in paid directories like Yahoo for years. It seems that era may have come to an end earlier today. The following bullet points have been removed from Google’s Webmaster Guidelines Webmaster Help Center*

• “Have other relevant sites link to yours.”
• “Submit your site to relevant directories such as the Open Directory Project and Yahoo!, as well as to other industry-specific expert sites.”

Does this recent move reflect a renewed emphasis on rooting out paid links passing PageRank and/or low quality links by Google?

*As mentioned, the bullet points above have been removed from the US version of Google’s Webmaster Help Center. Other versions may not yet reflect this change.

Overdoing Search Engine Optimization

Search engine optimization is not a bad thing. In deed, SEO is a good thing because, at least in theory, doing SEO helps the cream rise to the top.

However, if you overdo SEO for your website, your website will be crowded with SEO friendly design only such as webpages are full with text links, text, text, text, text links, alt meta tags, alt meta tags, alt meta tags, alt meta tags, and more.

Your website will get the good results for search engines for sure. But what you need from those search engine search results are “potential visitors and customers” or not?

Yes, visitors will like the easy navigated websites which are friendly to their eyed too. Don’t amaze them and chase them out of your websites with overdoing search engine optimization (Overdo SEO) because you will be the loser instead.

Great tips to ranking in Google for a HIGHLY competitive keyword

Every one of us wants to rank high in Google. You must focus on backlink building to have good serps. But backlink building has to be done with great care - not to get penalized.

Here are some simple tips which will definetely help:

1. Don't buy (or sell) sitewide links for serps. Buy links from individual pages*.

2. Don't look for high PR links only - start your campaign for a new site with PR 0-1 and slowly move up each week as you see your rankings improve.

3. Control link placement speed. Remember of "Too many links at once" Google filter.

4. Control the number of outbound links on your pages and on pages where you place your links, avoid link-farms, links to gambling, pills, etc.

5. A link from a relevant website is excellent, but a link from a quality site of other niche or other language can also boost your ranking - so don't ignore such links.

6. Make links look natural. Use different link anchors, promote not only your main page, but your deep pages too.

7. Don't do reciprocal linkexchange.

* - Links from individual pages are far more effective when speaking of improving serps, not PR. They are more natural and organic. Sitewide links can easily be filtered.
Selling sitewide links also means loosing lots of money - if your site has 1000 of PR2-3 pages, you could earn hundreds of $$$ a month just from selling ONE outbound link per page, compared to some $20-30 for selling a sitewide link (now, when it has become possible). What do you think?

Using this simple technique, many our sites has received an excellent rank in Google in just 1 month for HIGHLY competitive keywords. If you are interested, I can tell how we did it.

Just remember, that your site's ranking depends on your efforts and skills, it doesn't depend on your site's PR etc. (see our PR if you don't believe it).

If you are too stupid to use a computer you might try giving SEO advice

Now that everybody and their uncle is an expert in search engine optimization you get to read some really off the wall advice. I have collected a few gems that had me laughing and reproduced them below. Most of these are from Yahoo Answers and Live QnA but you can find answers like these almost everywhere.

These are all from the last few months and I have corrected the spelling, grammar and replaced any links with .

Q. What is cloaking in SEO?
A. Cloaking is the speed process for your cpu. You can over cloak your cpu but be aware of the overheating, the biggest problem of over cloaking is that it overheats the cpu.

Q. Seo stuff, what are some basic pointers?
A. My friend I have the perfect website for you. It includes over 1000 links to free advertising websites including free directories, free search engine submissions, free viral marketing, free top keywords and much, much more also investing in the big daddy search engine and program hoppers will prove very profitable.

Q. If your site is a cooking site but you add your site as a link to sports related forums, would your site be penalized or banned from the search engines?
A. It is legal to point from one topic to other, you will just get a lower rank but it is better than nothing.

Q. Can someone explain how sub-domains can impact search engine optimization?
A. If you want top ranking in MSN just spam with sub-domains.

Q. Why does page rank fall?
A. The number of hits you get and the frequency that your site is updated changes your ratings.

Q. What does “omitted results” mean e.g. “In order to show you the most relevant results, we have omitted some entries very similar to the 103 already displayed”?
A. It means that there are search results that have been left out.

Q. What is SEO?
A. South East Organization.

Q. Can an expert tell me why Google is not updating our website?
A. No it doesn’t update, you have to resubmit it.

Q. Why are SEO Consultants too expensive for webmasters?
A. I personally used a firm that did a 250,000 site submissions for my site, it worked great.

Q. What are SEO and SEM and how do they differ?
A. SEO typically includes keyword research, density balancing, tagging, linking strategy and website submission. With SEM you can buy advertising to get to the top.

Q. Can any of you SEO experts recommend a good link exchange site that works?
A. I’ve found 45 different link exchange services and the two that I think work the best are and .

Q. Can anyone suggest any SEO tips for my website?
A. I checked your website and one thing is missing, a Visitor’s Guest Book where visitors can insert a message describing their business and website.

Q. What are some SEO tips?
A. The most important item is meta tags, very good keywords and keyword density.

Q. What is the best way to get a website to appear on the first page of search engines, can I do it myself?
A. My experience is that trying to do it yourself does not give the best results. My website has been online for about 5 years now and on average I get about 11 unique hits per day and my meta tags are in order.

Q. How do you get removed from a search result?
A. You don’t. Whatever embarrassing thing you may have committed in a public forum is indelibly etched.

How do I add my website to search engines like Google, Yahoo, AOL?
A. From my understanding, your website will pop up in search engines based on how many people visit your website but how people visit your website if it’s not in a search engine, ironic right?