Breaking News!
Google is coming out with a new tag called “unavailable_after” which will allow people to tell Google when a particular page will no longer be available for crawling. For instance, if you have a special offer on your site that expires on a particular date, you might want to use the unavailable_after tag to let Google know when to stop indexing it. Or perhaps you write articles that are free for a particular amount of time, but then get moved to a paid-subscription area of your site. Unavailable_after is the tag for you! Pretty neat stuff!
So, what does this mean for search marketers?
If you are actively adding content to your website that is only useful to your users to a certain date, you will now be able to let the Googlebot know to stop indexing this content beyond your chosen date. Here is an example of this meta tag from Search Engine Land:
META NAME=”GOOGLEBOT” CONTENT=”unavailable_after: 23-Jul-2007 18:00:00 EST”
Once Google crawls the page and sees this Meta tag, it will take about a day for the page to be removed from the search results. Also this tag is only supported with web search. To remove the page completely from Google, including the cache copies, use the Google removal tool.
Google has also added support to control access to non-HTML documents that can’t have meta tags in them for blocking, such as PDF files, audio, xls documents and so. This is through a new X-Robots-Tag directive issued via the HTTP Header used to serve the file.
This can help you to maintain control over your content. If you have offers with expiration dates (daily, weekly, monthly, etc.), you can now allow your new offer to be the primary focus for your visitors from Google looking for your latest offers.
Google is giving you more control over your content and what (and how long) it is provided by Google to users. This doesn’t mean that it will be easier, actually it will be more complicated. So content management will be more difficult, but more valuable.
What are your thoughts?
Have you tried the new meta tag?
Related Posts >>
- Hot Topics Around the Company Start Page?
- Windows Vista – New Launch Date
- AskCity – New Local Search
- Microsoft Webmaster Portal is Live in Beta
- Google Searches Patents
