XML sitemaps are used to help search engine crawlers do a better job of finding and prioritizing all of your content. It doesn’t guarantee that your pages will be indexed, but it does assist the crawlers in determining what content you don’t want indexed, and what content you would like to see a higher priority placed on.
There are several web-based xml sitemap generators, and in particular a very good WordPress plugin that will do the job for you and make it a plug and forget affair.
The important things to keep in mind when setting up your XML sitemap are 1) figuring out what content you do not want indexed, and 2) figuring out how to include your content so that the pages you really want showing up in the search results do.
There are pages on your site that you do not want to include in your XML sitemap. It’s great having a lot of content to submit to the search engines, but you want search engine results to return content, not your Contact page.
Some of the content you may not want showing up in search results:
- Contact form pages
- About pages
- Poll results pages
- Links pages
- Users online pages
- Category pages
- Archive pages
- Comments pages
- Disclaimers and disclosure policy pages
Keep a watchful eye on your polls, some poll plugins generate duplicate pages! One of the surefire ways to incur Google’s wrath.
What do you want to have the search engines return in search results? Your content or the comments? Content.
Starting to get the idea? Say you’ve targeted the keyword schnobblegrueber. When someone searches schnobblegrueber on Google, you want them to land on search results that send them to articles you have written that are relevant to schnobblegrueber. This is where the category and archive sections can cause a problem if you are submitting them to be indexed.
You want people to enter on:
yoursite.com/category/schnobblegrueber/ways-to-increase-your-sites-schnobblegrueber
as opposed to:
yoursite.com/category/schnobblegrueber/3
or:
yoursite.com/2007/10/12/ways-to-increase-your-sites-schnobblegrueber
The problem with 2 of the 3 above urls?
- yoursite.com/category/schnobblegrueber/3 is constantly changing if you write more articles in that category. You don’t want people to get a search result that takes them to this page on your site, because the schnobblegrueber content may not be there anymore. Now they bounce off to somewhere where they can get what they were looking for, where if your search result would have returned the article page they might have stuck around for some pageviews and ad clicks.
- yoursite.com/2007/10/12/ways-to-increase-your-sites-schnobblegrueber creates duplicate content on the search engines. They don’t like that, and it’s confusing to have two urls for one piece of content.
You probably do want the search engines to display your category pages, but remember that Google won’t only end up displaying the first page in your category listing but /2 /3 /4 ad infinitum as well. Set them to have a lower priority than your individual article entries.
Starting to get the idea? You want the search engines to like you and think you’re one of the places they should send people. 1200 entries in your XML sitemap may look impressive, but often 500 entries that direct the search engines to what you really want people to see is much more effective. Avoid duplicate content, don’t submit items that people aren’t going to be searching for, prioritize how you want the search engines to display your content, and you have just taken another step toward increasing your serps.
