2/17/2009

Google, Yahoo & Microsoft Unite On “Canonical Tag” To Reduce Duplicate Content Clutter

This can be a problem for a couple of reasons.

* Less of the site may get crawled. Search engine crawlers use a limited amount of bandwidth on each site (based on numerous factors). If the crawler only is able to crawl 100 pages of your site in a single visit, you want it to be 100 unique pages, not 10 pages 10 times each.
* Each page may not get full link credit. If a page has 10 URLs that point to it, then other sites can link to it 10 different ways. One link to each URL dilutes the value the page could have if all 10 links pointed to a single URL.

Using the new canonical tag

Specify the canonical version using a tag in the head section of the page as follows:

< link rel="canonical" href="http://www.example.com/product.php?item=swedish-fish">

That’s it!

* You can only use the tag on pages within a single site (subdomains and subfolders are fine).
* You can use relative or absolute links, but the search engines recommend absolute links.

This tag will operate in a similar way to a 301 redirect for all URLs that display the page with this tag.

* Links to all URLs will be consolidated to the one specified as canonical.
* Search engines will consider this URL a “strong hint” as to the one to crawl and index.

Canonical URL best practices

The search engines use this as a hint, not as a directive, (Google calls it a “suggestion that we honor strongly”) but are more likely to use it if the URLs use best practices, such as:

* The content rendered for each URL is very similar or exact
* The canonical URL is the shortest version
* The URL uses easy to understand parameter patterns (such as using ? and %)

Can this be abused by spammers? They might try, but Matt Cutts of Google told me that the same safeguards that prevent abuse by other methods (such as redirects) are in place here as well, and that Google reserves the right to take action on sites that are using the tag to manipulate search engines and violate search engine guidelines.

For instance, this tag will only work with very similar or identical content, so you can’t use it to send all of the link value from the less important pages of your site to the more important ones.

If tags conflict (such as pages point to each other as canonical, the URL specified as canonical redirects to a non-canonical version, or the page specified as canonical doesn’t exist), search engines will sort things out just as they do now, and will determine which URL they think is the best canonical version.

The tag in action

This tag will most often be useful in the case of multiple URLs pointing at the same page, but might also be used when multiple versions of a page exist. For instance, wikia.com is using the tag for previous revisions of a page. Both http://watchmen.wikia.com/index.php?title=Comedian%27s_badge&diff=4901&oldid=4819 and http://watchmen.wikia.com/index.php?title=Comedian%27s_badge&diff=5401&oldid=4901reference the latest version of the article (http://watchmen.wikia.com/wiki/Comedian%27s_badge) as the canonical.

The search engines stress that it’s still important to build good URL structure and also note that if you aren’t able to implement this tag, they’ll still keep the processes they have now to determine the canonical. For instance, at SMX West on Tuesday, Maile Ohye of Google explained how Google can detect patterns in URLs if they use standard parameters. For instance, with these URLs:

* http://www.example.com/buffy?cat=spike
* http://www.example.com/buffy?cat=spike&sort=evil
* http://www.example.com/buffy?cat=spike&sort=good

Maile explained that Google can detect (particularly when looking at patterns across the site) that the sort parameter may order the page differently, but that the URLs with the sort parameter display the same content as the shorter URL (http://www.example.com/buffy?cat=spike).

While it’s rare for the search engines to join forces, this isn’t the first time they’ve come together on a standard. In November 2006, they came together to support sitemaps.org. And in June 2008 they announced a standard set of robots.txt directives. Matt Cutts of Google and Nathan Buggia of Microsoft told me that they want to help reduce the clutter on the web, and make things easier for searchers as well as site owners.

This new tag won’t completely solve duplicate issues on the web, but it should help make things quite a bit easier particuarly for ecommerce sites, who likely need all the help they can get in the current economic conditions. Site owners have been asking for help with these issues for a really long time so this should be a greatly welcomed addition.


Postscript by Barry Schwartz:

The search engines will be talking about this news at the Ask the Search Engines panel at SMX West. We will be blogging this panel live at the Search Engine Roundtable.

11 comments:

John said...

oakley sunglasses wholesale
coach factory outlet
ugg boots for men
uggs on sale
coach factory outlet
christian louboutin
tods outlet
gucci handbags
gucci shoes
michael kors outlet online
uggs boots
coach outlet store online
coach factory outlet
michael kors outlet
michael kors handbags
ugg boots
celine handbags
cheap oakleys
ugg boots
cheap oakleys
michael kors bag
tory burch outlet
coach outlet online
ugg boots
adidas superstar
michael kors outlet
canada goose
michael kors outlet online
abercrombie
20151215yuanyuan

dong dong23 said...

oakley sunglasses
tory burch boots
ray ban sunglasses uk
replica watches
oakley outlet
air max 90
coach outlet store online
ray ban sunglasses
kobe 9
asics outlet
oakley canada
louis vuitton
air jordan shoes
louboutin shoes
gucci outlet online
prada outlet
replica rolex watches
michael kors handbags
michael kors uk
christian louboutin outlet
tory burch outlet online
christian louboutin shoes
coach factory outlet online
polo ralph lauren
ralph lauren outlet
cheap jordan shoes
fit flops
discount jordans
michael kors outlet online sale
coach outlet online
toms wedges
louis vuitton outlet
hollister clothing
louis vuitton outlet
cheap ray ban sunglasses
coach outlet online
toms wedges
louis vuitton bags
michael kors outlet
michael kors outlet clearance
20164.5wengdongdong

dong dong23 said...

oakley sunglasses
tory burch boots
ray ban sunglasses uk
replica watches
oakley outlet
air max 90
coach outlet store online
ray ban sunglasses
kobe 9
asics outlet
oakley canada
louis vuitton
air jordan shoes
louboutin shoes
gucci outlet online
prada outlet
replica rolex watches
michael kors handbags
michael kors uk
christian louboutin outlet
tory burch outlet online
christian louboutin shoes
coach factory outlet online
polo ralph lauren
ralph lauren outlet
cheap jordan shoes
fit flops
discount jordans
michael kors outlet online sale
coach outlet online
toms wedges
louis vuitton outlet
hollister clothing
louis vuitton outlet
cheap ray ban sunglasses
coach outlet online
toms wedges
louis vuitton bags
michael kors outlet
michael kors outlet clearance
20164.5wengdongdong

Zhenhong Bao said...

kate spade uk
coach factory outlet
lebron james shoes
kate spade handbags
coach outlet
ray ban sunglasses
coach handbags outlet
thomas sabo uk
michael kors handbags
ray ban sunglasses
reebok trainers
christian louboutin uk
true religion outlet uk
michael kors outlet online
longchamp pliage
bottega veneta outlet online
longchamp outlet
links of london jewellery
tory burch outlet
mcm outlet
coach outlet online
ferragamo outlet
rolex watches for sale
michael kors online outlet
polo ralph lauren
oakley sunglasses
chrome hearts outlet
prada outlet
michael kors outlet uk
longchamp pliage
ray-ban sunglasses
tory burch shoes
swarovski outlet
oakley canada
longchamp handbags
20160509zhenhong

mmjiaxin said...

adidas nmd runner
http://www.jordanretro.uk
tiffany and co
michael kors handbags
cheap oakley sunglasses
cheap air jordans
tiffany and co outlet
tiffany & co
http://www.cheapairjordan.uk
ray ban sunglasses
cheap basketball shoes
cheap authentic jordans
nfl jerseys
michael kors outlet
michael kors factory outlet
http://www.huarachesshoes.uk
fitflops clearance
http://www.tiffanyand.co.uk
Cheap Oakley Sunglasses
michael kors outlet
gg

Hua Cai said...

hermes birkin bag
vans outlet
nike tn pas cher
chaussure louboutin
air max 2015
giuseppe zanotti shoes
hollister uk
coach outlet online
louis vuitton handbags
michael kors handbags
hermes belt for sale
nike air force 1
prada sunglasses
true religion outlet uk
true religion jeans
louis vuitton bags
ralph lauren outlet
nike roshe run
longchamp handbags
true religion jeans
polo ralph lauren
coach outlet online
air max 90
hollister
mizuno shoes
air jordan shoes for sale
cartier watches for sale
swarovski crystal
fitflops uk
michael kors outlet online
herve leger outlet
converse shoes
christian louboutin shoes
ralph lauren pas cher
ghd uk
20160722caihuali

Hua Cai said...

hermes birkin bag
vans outlet
nike tn pas cher
chaussure louboutin
air max 2015
giuseppe zanotti shoes
hollister uk
coach outlet online
louis vuitton handbags
michael kors handbags
hermes belt for sale
nike air force 1
prada sunglasses
true religion outlet uk
true religion jeans
louis vuitton bags
ralph lauren outlet
nike roshe run
longchamp handbags
true religion jeans
polo ralph lauren
coach outlet online
air max 90
hollister
mizuno shoes
air jordan shoes for sale
cartier watches for sale
swarovski crystal
fitflops uk
michael kors outlet online
herve leger outlet
converse shoes
christian louboutin shoes
ralph lauren pas cher
ghd uk
20160722caihuali

chenlina said...

los angeles clippers jerseys
miami heat jerseys
rolex submariner
michael kors handbags
jordan 4
adidas shoes
ugg canada
ray ban sunglasses uk
louis vuitton outlet
gucci outlet
roshe run
timberland boots
kd 8 shoes
nike roshe run
ray ban sunglasses
tory burch handbags
toms shoes outlet
polo ralph lauren outlet
oakley sunglasses outlet
christian louboutin
ugg uk
coach outlet
coach factory outlet
uggs on sale
mont blanc pens for sale
louis vuitton
canada goose jackets
michael kors outlet
adidas uk
tods shoes
nike nfl jerseys
nba jerseys
oakley outlet
ed hardy clothing
true religion jeans
uggs
michael kors canada
lebron james shoes
air jordans
cheap uggs
chenlina20160817

John said...

canada goose sale
cheap ugg boots
birkenstock sale
michael kors outlet online
ugg outlet
yeezy boost
jordans
cheap nfl jerseys
polo ralph lauren outlet online
cheap ugg boots
2016107yuanyuan

Unknown said...

cheap nhl jerseys
michael kors uk
replica watches
michael kors outlet
colts jerseys
cleveland cavaliers
replica rolex
ecco shoes outlet
hermes belt
cheap nfl jerseys

qqq said...

ultra boost
michael kors uk
toms shoes outlet
adidas yeezy
birkenstock outlet
dallas cowboys jerseys
pandora outlet
fitflops
nike air max outlet
adidas outlet
170802yueqin

Live Page Popularity