“…the only time you should let search engines index the output of an aggregator is when that’s the only place the feeds appear as HTML. Otherwise, it’s duplicate content, and that’s going to wind up hurting someone, whether it’s you or the source.”
I think a debate is due about the Irish Blog aggregator’s policy of indexing our feeds. I know a tiny amount about aggregating blogs, but of course not as much as Roger Galligan, for whom I voted in the Irish Blog Awards for Best Contribution to the Irish Bloggersphere, which he deservedly won. But I wonder if the site is over-reaching itself.
It is my belief that the responsible setting for search engine robots in feed aggregators is “noindex, follow”. Gregarius, the open source aggregator, decided to set this as the default option, persuaded by the arguments in the essay quoted above.
I like the Irish Blogs aggregator, I really do. But I’ve noticed recently that posts I have made on various subjects turn up in Google search, but the accompanying link is to the irishblogs.ie archive. They are my words, but people clicking on them are taken to the aggregator first, because search engines don’t like to duplicate posts, and irishblogs.ie has a higher pagerank score than my humble blog – no doubt helped by all of us willing to sign up and add our feed and make it a popular site. This makes increasing my blog’s pageranking more difficult, because my posts do not show up as much in Google. Irishblogs.ie, however, is very likely to become increasingly more authoritative, at my expense, in terms of pagerank – and that means, as well as losing visitors to my blog, adsense income. (Which is peanuts for me at the moment, like most of us I imagine – but I can’t see it improving, now.)
Since Irish Blogs has begun to move from a voluntary effort with a lot of goodwill, to a commercial enterprise with sponsors, I think that some accountability is called for, some clear terms of service that I can take or leave. Planet Journals has a robots setting of “noindex”. Why can’t Irish Blogs?
An aggregator should only, in my opinion, syndicate the feeds of its contributors so that they can all be read in one place: a browser or newsreader. It should not, I believe, muscle in on the indexing of its contributory blog posts in search engines. I did not give irishblogs.ie my explicit permission for that when I signed up, and I do not believe it is reasonable or standard practice.
Roger, over to you…
{ 5 } Comments
To prove my point, if you are on the post page you can see, below the comments, trackbacks to this post. Blogger call them “backlinks” and they use Google Search to find blog posts that reference an URL. If you look at the one from Damien Mulley, entitled “Should IrishBlogs.ie and Journals.ie stop indexing blog posts?” and click on it, it will take you not to Damien’s blog… but to irishblogs.ie.
BTW, just to clarify, Damien, journals.ie is not guilty of this, it’s only irishblogs.ie that is doing this.
mmm It irritates me a little alright when the result shows in Google above my own blog.
I’ve added a poll at the Irish Blogs Yahoo! group if anyone would care to express their opinion on this.
I think a nofollow, noindex policy is the policy that respects all interests concerned. Setting that into the aggies is no guarantee that a blog with lower PR will supercede the higher PR standing of an Irish aggregator for a category or a meme when queried through a major search engine.
I agree that one can’t guarantee anything when it comes to the mysteries of PR. But if aggregators demonstrate they are aware of the problem and are doing their best not to divert traffic from individual blog posts to cached duplicates, then I have no beef with them.
BTW – I think its great if an aggregator’s category page does well in a search for a particular meme. A category page listing various posts from Irish blogs on a topic is great, and should do very well.
Anyway irishblogs.ie have announced via the Irish Blogs mailing list on Yahoo! that changes are rolling out along the lines discussed in this post, which is great.
Post a Comment