An Observation, about How My Blog Gets Spidered

If a blog was just to consists of a bunch of Web-pages, then one side-effect of that would be, that the blogger would add pages on a daily basis, but that his latest pages would not be spidered by the major search engines, potentially for a long time.

If we use ‘’ as our blogging engine, then one feature is, the very first time we add a new posting, doing so is announced or broadcast, to a service that lets search engines know, that a posting has been added to the blog. But one way in which this service does not work, is to reannounce, every time an existing posting has been edited by the blogger. This is fair, because bloggers like me may edit postings as many as 10 or even 20 times eventually. Reannouncing these edits would put an unfair burden on a free service.

But as it happens, the way this service works can have side-effects for me. For example, one of my recent postings links to the following URL:

The problem with this URL is, that in the original posting, it ended with ‘.html’, and not with ‘.pdf’ . I edited that posting after I had first created it, to use a ‘…pdf’ URL, instead of an ‘…html’ URL. What tends to happen is that Google will spider my new postings within seconds of their being created, while certain other search-engines will take maybe a few hours to spider the same posting. And this can entirely be a performance issue, with each search engine. But unfortunately for me, this suggests that Google caught a version of this earlier posting, that contains a broken link, just because the ‘…html’ URL no longer exists on my server. And if the posting contains any apparent, broken links, obviously, the search engine penalizes their ranking.

So that may be one reason, why the posting in question, has not received as many clicks, as it should according to how often other postings of mine receive clicks.