Our transit provider suffered an outage this morning between 11:40 and 12:00, which will have affected ADSL customers and access to servers in our data centre in London. Apologies for any inconvenience this caused.

BT replaced the ATM card and service was restored at 12:38. Apologies once again for the inconvenience.

The BT Central pipe linking all ADSL connections to our network is currently down. We’ve just heard from BT that there is a faulty ATM card in their facility in Ilford and they are working to rectify this ASAP. Apologies for the inconvenience.

Our main transit provider suffered another failure today, resulting in loss of transit in our London facility between 10:30 and 11:40 this morning. There was also a 10 minute ADSL outage from around 11:10. Our apologies for any inconvenience this may have caused. We are looking at ways to improve the reliability of our main transit feed!

At around 14:25 today a core switch failed in our main London transit provider’s network. This resulted in loss of connectivity for all ADSL customers and hosted servers in our London datacentre. The problem was resolved by around 15:50. Our Amsterdam datacentre was unaffected and we were still able to reach our London datacentre via our Amsterdam datacentre, as were customers who have servers in Amsterdam. Our apologies for any inconvenience this caused.

We suffered a DoS attack at 17:27, lasting a few minutes, which affected our ADSL customers’ connectivity. Apologies for any inconvenience this caused.

There was a 10 minute service outage at around 18:07 today affecting our hosting customers (including tunnel bonding FireBrick customers). This was due to a configuration error entirely of my own doing – apologies for the inconvenience.

There was a brief loss of ADSL service between 16:01 and 16:13 this afternoon due once again to a relatively minor DoS attack upsetting a BT router. We are still chasing BT for a long term fix for this issue. Apologies for the inconvenience.

Redbus HEX, where our data centre is located, suffered a complete power outage today, which will have affected all customers. Power went off at around 10:05am, and services started coming back around midday. There was also a power surge which damaged a number of servers.

This is not the first complete power outage Redbus have had recently, and we are asking serious questions as to how they will ensure it doesn’t happen again, and looking at ways of relocating our servers and routers so as to minimise the risk we are exposed to.

For your information, Redbus power supplies are meant to be backed up by battery and generator, and surge protected, so this kind of failure really shouldn’t happen!

Apologies for the delay in posting this status message, and of course apologies for the problems this outage undoubtedly caused you.

Today’s brief loss of ADSL service was due to a DoS attack upsetting one of BT’s routers. The attack was relatively minor and we are encouraging BT to improve the resilience of the router in question. Apologies for any inconvenience.