Hard Drive Failure Update (11/01/14)

Since the drive failure in our Reading based server we have been working hard on trying to rebuild websites for those customers who were affected.

We are currently about 90% of the way through rebuilding those websites. Now that we’ve had some time to focus on repairing what had gone wrong, we thought we should give a little explanation as to what happened to the hard drive, but also what we have done to prevent this happening again.

What happened?

The servers hard drive is partitioned into four sections. One section is for system files, the other is for ‘live’ website files, and two sections are for back ups. One weekly backup, the other a monthly backup.

In theory, and as is normal practice for Web hosts is that if a website was to become infected or vandalised, there would always be two snapshots of the backup on an isolated part of the hard drive. This has happened before as it is inevitable, and the backup is restored.

Unfortunately, with what happened in this scenario is that the physical hard drive failed. It stopped spinning and would not power up, thus making any backup image useless.

Statistically, that’s the first hard drive failure we’ve had in ten years, so it’s almost unheard of that drives fail these days. Remember we’re talking business grade hardware too, not the stuff you buy from a local shop!

However, as much as it is very unlikely to happen again, we have learned from what’s happened put some practice’s in place to prevent it happening again.

What have we done?

1) A full server diagnostic is carried out every morning at 3.30am and a report is emailed directly to our admin team. If one of the drives spins 0.1 seconds slower than the previous day, this will be highlighted.

2) External backup. We have now contracted a backup provider based in Singapore. All data from websites are now securely stored on a separate backup drive. This provider allows for two way data transfer meaning should a website ever be infiltrated, data can be restored backwards from Singapore in less than 5 minutes.

We hope the above explains a little further as the situation we had found ourselves in.