Our wholesale provider for BT lines suffered an LNS reset causing some lines to glitch. All lines are back and working fine.

We apologise for any inconvenience this may have caused.

We have been advised by our wholesale supplier of Be connections that shortly after 14:40 today there was an outage on Be lines due to a possible router failure in O2’s network. All sessions have now recovered.

We apologise for any inconvenience this may have caused.

We are currently suffering low level packet loss on our link to Amsterdam. The packet loss is dependant on the size of the packet, with larger packets worse affected.

We have reported all our test results to the relevant supplier and are awaiting a response.

We apologise for any inconvenience caused and will update as soon as possible.

There appear to have been some routing problems this afternoon which affected many Internet backbone providers. Whilst this has not directly affected out network (which our monitoring indicates is running correctly), it has resulted in many routing difficulties to points on the Internet outside of our network.

At 08:49 this morning, our main circuit to Amsterdam went down. After a few minutes, the backup kicked in automatically, as intended. At approx. 09:00 the circuit came back, but flapped for a while as it did so, resulting in 30seconds or so of routing problems.

It has been stable since then and we are awaiting an explanation from the supplier of the circuit (Datahop). Apologies for the disruption to service.

Explanation of yesterday’s outages

There were 2 unconnected events yesterday which both disrupted service. Firstly, between approx 10:00 and 11:45, one of our interconnectivity providers, Datahop, suffered some failures which resulted in some, but not all traffic between our network and ‘The Internet’ being dropped. Typically, this manifested itself as websites being unavailable and certain outside monitoring services reporting services being down when they were in fact working.

Secondly, at approximately 13:30, our wholesale provider of BT ADSL connectivity, A&A, suffered a fibre outage to BT. Whilst there are backup fibres in place, services apparently did not automatically switch over and instead flapped up and down. Manual intervention saw service restores by approximately 14:00 and the automatic fail-over mechanisms are being reviewed.

As a side effect of the ADSL flapping, a very very large number of SMS and email notifications were generated whilst the outage persisted. These gradually worked their way through various queues, but had the unfortunate effect of continuing to alert users about the outage, for a further 3 hours after it had finished.

We apologise once again for the disruption.

A different problem has arisen with ADSL connections dropping this is an issue outside of our control and we are chasing for a solution. Sorry for the problems today we will post information as soon as we receive it.

The issues we saw earlier this morning have now settled down. We apologise for any issues caused.

The problem customers re seeing is due to an issue with a link provided by Datahop here is what they have advised us. “At 11:04 today we experienced a number of physical hits to the dark fibre connectivity between Telehouse North and GlobalSwitch 2. This caused multiple interface flaps which in turn caused multiple ring topology re-convergence events, that may have caused hits to your connectivity.” We appologise for any inconvenience.

A few customers are reporting, connectivity issues. We are investigating.