Not all webmasters are aware that search engines view each URL as a different page, even it has the same content or is different from the other URL by ending with a slash or if it preceeded by a www or not. To serve only one version of the page to get the deserved traffic, you'll need to remember a couple of things about fixing URL issues.
The simplest matter is having both versions of a domain, the one with www and the one without (smart developers even call this URL canonicalization issue, but I believe - or hope - there has to be an easier term for this). The search engines view them as two separate sites with identical content and regard links pointing to them accordingly. Though they each get their own share of traffic, you can get more traffic by having just one version, because the links would be pointing to one site, thus boosting its search engine rankings.
To redirect a site from the non-www version to the www version, you can insert the following code in your .htaccess file:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^yoursite\.com [nc]
RewriteRule (.*) http://www.yoursite.com/$1 [R=301,L]
It'll only work for Unix platforms with Apache and mod_rewrite enabled. If it doesn't work after refreshing the page, make sure you do have Unix hosting and contact your support to check if they have mod_rewrite enabled. If they don't, I'd suggest switching hosting.
Read more about using 301 (permanent) redirects with .htaccess.
Though it is technically easy to understand that www and non-www sites are different, how about understanding that pages with and without slashes in the end of the URL are different, too, according to the search engines. That's why you only need to have one version of those.
If you have a WordPress blog, then a simple Permalink Redirect plugin will help. I have recently installed it on my site and it works like a charm.
For other websites, you'll need to check your Apache version with your hosting and go to the according section of the Apache manual. Here are the links:
To me, the code samples in the guide look the same. I also didn't manage to make them work - probably due to conflicts with WordPress, or something. You can discuss the topic or ask questions about the trailing slash issue at Cre8asite Forums.
Suppose you have pages with the same content on your article and you want to only have one page, but don't want to lose the link weight or decrease the experience on your website. Then you need to 301 redirect (probably a PHP one) one page to another.
A 301 redirect (as well as .htaccess redirect) can be very helpful in solving duplicate content and broken links (if you have links pointing to a page that doesn't exist) and is mostly the only cure to the problem.
If you have a blog or just serve RSS feeds to your visitors, you should consider blocking them from the robots through the robots.txt file, because it'll help filter out duplicate content from their indices (feeds have the same content as your main site, remember?).
A good start could be inserting the following code in the robots.txt file:
User-agent: *
Disallow: /feed/
Disallow: /feed/atom/
Disallow: /feed/rss/
Disallow: /wp-
Alternatively, you can try using the following code to block spiders from indexing feeds for all pages:
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Read more about duplicate content and feeds at The Van Blog - be sure to check the comments, too.
Note: the above code seems to work for GoogleBot only (or for other bots, supporting the * wildcard.
Let me know if you have a better solution for blocking duplicate content with robots.txt.
Update: if you have blocked the "/#comment" URL as advised earlier, please unblock it. Otherwise, you'll be conducting a "how to live without the search engines" experiment. Although interesting, you don't want to block your homepage from the spiders. (You won't notice a sharp decrease in traffic, though, you'll see your pages without meta description and capitalized titles in the SERPs instead).
When developing your website, you need to remember a couple of things to keep your content more unique (both to the people and the search engines):
Read more on how you can get rid of duplicate content on the Google's Official Blog.
Keeping your site content unique, you will get more visitors from the search engines, because all your incoming link weight will be associated with a single page, thus making it more visible. Partly, correct indexing will help get your real content pages out of the supplemental index. That being said, don't be obsessed with supplemental results. Focus on your customers.
By knowing how you can keep your site unique, you can easily increase your chances of getting more search engine visitors. Granted, this process is fairly easy and can be done within a day with the right education. So, you are still not looking at your site? Here's what you should look at:
Enjoy, and may Google be with you.
Use experience and skills of a reliable SEO and a web designer I trust to get more sales from your website.
Comments
Post new comment