Robot Exclusion Enforcement

Posted on by
1

With thanks to Paul Ford for permission to write a sequel to the original “Robot Exclusion Protocol” short story. I hope I haven’t done the original any disfavours.

I was on a job when I saw it. A couple of feet tall, made of metal, with camera lenses for eyes and a few dozen gripping arms. A Googlebot! If it wasn’t for the stakeout you wouldn’t have caught me in that part of town. It certainly wasn’t a suitable neighbourhood for a Googlebot. I pulled my coat more closely around me, hunkered down further behind the bins in case it decided to index me and blow my cover, and settled down to watch.
Continue reading →

Over A Third Of My Visitors Are Bad Bots!

Posted on by
Reply

I’m starting on the long, long learning curve of blocking bad bots. I have a long way to go yet but just wondered what the work I had done so far had revealed.

The site running the most up to date code was not a huge site. A long-standing, decidely niche site, it used to generate 1000-2000 visitors a day depending on the day of the week. Currently running far below that due to me repeatedly messing up refactoring the bot blocking and not testing sufficiently before putting it live (it’s only a hobby site, lack of time meant it was easier to debug by promoting to live. It all went horribly wrong but it was at least a deliberate choice). The most up to date code is not particularly clever yet. I’m only blocking bots stupid enough to try and masquerade as spiders that can be identified with roundtrip dns, and bots not quite that stupid but still stupid enough not to be able to manage sessions!
Continue reading →

For 2008: Backup, Backup, Backup!

Posted on by
Reply
Lifebelt

Still looking for a New Year’s resolution. SugarRae’s latest post coincides with events in our house.

Shortly before xmas my wife’s computer failed to boot up, failing the disk check every time.

Like SugarRae, my wife has irreplacable files on her computer, years of personal photos, files related to her worek and her studying. With great trepidation I ran the tool the error messages were suggesting I run but were also saying were virtually guaranteed to lose some data.. Continue reading →

Telephone Numbers On The Mobile Web

Posted on by
Reply
Laptop and Phone

This lunchtime I had cause to try and find the phone number for a local restaurant. Rather than pay for directory enquiries I opted to do a Google search on my mobile (I have a flat rate data plan). Of course, I could only do this because I happened to have paper and pen handy to jot down the number.

Make It Easy For Mobile Web Customers To Call You.

While it may not be well used, there is a protocol for telephone numbers – the tel: protocol. Mark up your businesses telephone numbers like this

Call us on 01234 567890

to make it easy for people viewing your site on a mobile to call you.

It should also of course be possible on a decent desktop browser to associate a VOIP program with this protocol. As VOIP continues to grow marking up your telephone numbers will become ever more useful.

Mobile Web Browsers

In the meantime of course, given the number of sites that do not mark up their telephone numbers in this way it would be helpful if mobile browsers had an option to highlight and make clickable and callable any part of a page that looked like a phone number.

Will Facebook Break The Bank(s’ Authentication Methods)?

Posted on by
Reply
Open Bank Vault

The recent loss of personal details on 25m people by HMRC has certainly raised the public profile of data protection and identity theft. Lots of coverage from the events themselves, to how people were personally affected, to its impact on the government’s proposed identity cards, even the standard of journalism over the issue.

One of the posts on the subject that particularly caught my eye was “Information As Money” on Chuck’s Blog. What did I find interesting?

The potential of severe consequences, understood up-front, has a way of getting people’s attention.

The problem of course is that word “understood”.

While there have been numerous lapses in data protection over the last year from businesses, on the whole I think business does understand the importance of data protection better than the average individual, even when dealing with their own information.

In “Phacebook?” I commented on how compromising the information in social networking sites could be and suggested people should be careful what information they gave to such sites.

Who Was I Kidding!!!
Continue reading →

What Do You Have?

Posted on by
Reply
Welcome Mat For Visitors

No I’m not asking about anything that might be classified as contagious, but what do you call those who for whatever reason make a request of your web server to serve up a page.

On Website To Dos and the blog I have tended to use the generic “visitor” as I have no way of knowing what sort of web site my visitors/readers/whatevers might have. I associate readers more with blogs as other websites may have goals that do not involve repeat visitors (and I was interested to see Problogger also classify a reader as somebody who returns)

If you have a web application/service then you might have users. (And I have to admit that as my day job is as a programmer I frequently have to replace an accidental “user” with “visitor” in my drafts ;) )

Any of the above can, I feel, be subsumed within the term “audience”.

What else can you have?