You may have seen packet loss early this morning, between 01:40 and 07:15. This was due to a faulty switch in our main transit provider’s rack, which has now been reset. Apologies for any inconvenience.

Here’s a more detailed post mortem of this morning’s events. First the power went off at Redbus for 15 minutes from 00:20 – this really should not happen, and we are trying to get answers from Redbus. This resulted in loss of connectivity until 01:25. Then at around 06:54 a switch in our main transit provider’s rack started flooding our network with broadcast packets, causing packet loss until it was reset at around 08:30 – we are not sure whether this was related to the power outage. Then finally BGP routing died between 09:04 and 09:41, as a result of side effects of the earlier power outage. Everything is now back to normal, and we are planning changes to reduce the possibility of this happening again.

We suffered a loss of power in our data centre this morning, which will have affected all customers. All services are now back to normal, and we will be asking Redbus some serious questions. Apologies for any inconvenience.