Here’s a more detailed post mortem of this morning’s events. First the power went off at Redbus for 15 minutes from 00:20 – this really should not happen, and we are trying to get answers from Redbus. This resulted in loss of connectivity until 01:25. Then at around 06:54 a switch in our main transit provider’s rack started flooding our network with broadcast packets, causing packet loss until it was reset at around 08:30 – we are not sure whether this was related to the power outage. Then finally BGP routing died between 09:04 and 09:41, as a result of side effects of the earlier power outage. Everything is now back to normal, and we are planning changes to reduce the possibility of this happening again.