Be Careful Where You Click

by Stephen Pitts on February 2, 2007

This week, the Yahoo Search Blog announced new additions to SiteExplorer. Among new extras like a badge for your site, Yahoo has added a few useful and one powerful additions:
1. Detailed Authentication Errors
2. Site Authentication Using META Tags
3. Delete URLs

The most powerful, by far, is the Delete URLs:

“For your authenticated sites, you can now delete any URLs from the index. Simply locate the URL in Site Explorer and click on the ‘Delete URL’ button. The URL and all its subpaths will be deleted shortly thereafter. This is meant to work in conjunction with the robots.txt file while providing greater responsiveness. Please continue to use the robots.txt protocol to ensure that our crawler does not crawl pages you want to keep out of our index.”

This looks to be a potentially damaging addition to a useful tool.

Here are what others are saying:
Marketing Pilgrim:

“The feature I am most interested in and also worried about is the new “delete URL” feature. It is literally a disaster waiting to happen. There is zero verification other than being logged into the proper Yahoo account to delete an entire site from the Yahoo index.”

Search Engine Land:

“The virtual meta robots tag concept is important. No, you do not have to have an actual meta robots tag set to noindex on the pages you want to remove. Nor do you need to have a robots.txt file blocking pages. Delete URL will work instead of either of these to keep pages out. It will also work in addition to them.”

What do you think of the new additions to this tool?

Previous post:

Next post: