British cloud hosting company UKFast suffered an outage at its MANOC5 data center in Manchester yesterday, when workers accidentally cut a mains cable connected to the facility.

The company put out status updates every half an hour informing customers of its progress in tending to the failure.

A Timeline 

At 11:42 GMT, the power was restored, and technicians began efforts to bring the company’s cloud platforms, eCloud and eCloud Flex, back online.

Once this was achieved (between 13:30 and 14:30 GMT), the company tried to bring clients’ physical infrastructure back online, but experienced issues with VMware’s auto deploy service until approximately 17:00.

Finally, at 21:30 GMT, the company published a report detailing the cause of the outage.

“At 10:28 GMT today, our MANOC 5, 6 and 7 facilities were impacted by instability on the incoming mains power as a result of a civil contractor passing a spike through the main feed. This was not work being carried out on behalf of UKFast or within our site but at another location 0.75km away on the path to our onsite transformer.”

“The UPS system supported the load for its designed time and the generators started; however, due to the physical damage to the power cable, service to the site was unstable and intermittent. As a result, the generator sets failed to synchronize and take over service.”

“Our engineers have worked throughout the day to restore individual services and continue to do so for those clients who remain affected by the power issue.”

“Once we have resumed full service, we will be investigating what we can do to prevent this from happening again for both the incoming power issue and also the time taken to restore service for some of our clients.”