US web hoster WebNX suffered a lengthy outage earlier this week when a generator caught fire at its Utah data center, during a power disturbance.

The city of Ogden Utah suffered a power disruption on Easter Sunday afternoon (April 4). WebNX's backup power switched on, but unfortunately, a generator caught fire. The city power was back up by then, but WebNX was unable to switch back to the grid, because the fire services had cut off power to the site as a precaution. Customers suffered a lengthy outage, and WebNX is checking servers for water damage.

Fire department pulled the plug

"Sunday afternoon the city power was disrupted and, as designed, our backup generators automatically switched on. However, during that transition, one of our backup generators that had been recently tested and benchmarked specifically for this situation experienced a catastrophic failure, caught fire, and as a result initiated the fire suppression protocol," said a statement on WebNX's Facebook page late on April 5

The situation was made worse by that fire suppression protocol, according to an earlier statement (shortly after midnight the morning of April 5): "Unfortunately, the fire department opted to cut power to the rest of the building as a precaution even though the power systems were independent."

The Ogden fire chief told The Register that the blaze was controlled by the data center's fire suppression system, and firefighters helped with the cleanup. It's normal procedure to cut off power to a building until an electrical engineer can inspect the facility to be sure it's safe to restore power, he said. According to the fire chief, the fire "originated in a generator in the building and spread to several servers."

WebNX says in its Facebook statements that: "No fire damage was inflicted on customer servers. However, it does say that some customer servers in a main bay were exposed to water and possible damage. "The majority of hardware in the entirety of our data center was spared, but there are machines that need to be inspected for water damage, and possibly rebuilt. As of now [5 April], we are working to restore power, network, and unaffected hardware back up online within the next day or two."

Customers affected by the outage include Ogden City itself, which had some services located at WebNX, and other service providers including Gorilla Servers and Passmore Caviar.

Australian electronic engineering vlogger Dave Jones brightly commented in a YouTube video explaining why his eevblog.com site is not available: "The center of the universe is down. Everyone's getting withdrawal symptoms and they're having to check themselves into clinics and all sorts of things because they can't get their fix of the eevblog forum." Jones hosts his site at Gorilla Servers.

WebNX has said that compensation will be offered in line with customer service level agreements. Some servers will need rebuilding servers, and WebNX hopes to restore customer data.

WebNX has two data centers: 100,000 sq ft (9z300 sq m) in Ogden, and 10,000 sq ft (930 sq m) in downtown Los Angeles. The company mostly offers dedicated server hosting, and its history dates back to running a bulletin board on a Novell NetWare server, according to its LinkedIn page, which says: "We specialize in high end custom dedicated servers. If you can dream it we have likely built it."

The incident came one month after a serious fire destroyed an OVHcloud data center in Strasbourg, France. In that instance, the cause is suspected to be the site's UPS systems.