French cloud provider OVHcloud has today restored some servers to customers hit by the disastrous fire in Strasbourg in March, according to a tweet from its founder, Octave Klaba. Meanwhile, a French technology news site has claimed that the fire may have been made worse by serious errors in the provider's data center construction.

The 10 March fire destroyed the SBG2 data center and four halls of SBG1, leading OVHcloud to mothball both facilities. According to an article on ZDnet.fr, facilities on the Strasbourg site had wooden floors and used circulation of outside air, both of which may have helped spread the fire. The ZDnet.fr article also raises doubts about the site's fire suppression systems.

DCD and ZDnet.fr have invited OVHcloud to respond to the article's questions, but OVHcloud has declined, promising fuller public statements over the weekend.

Meanwhile, the process of restoring customer servers in the other two data centers on the site, and cleaning and shifting, or replacing damaged servers, has been slow. Referring to the damaged and repaired servers, Klaba tweeted: "First servers are given back to [customers]. Sorry for delay. It’s a real nightmare."

Firefighters mention wood and plastics

Initial reports said that the fire appeared to start in uninterruptible power supplies (UPSs), one of which had been serviced on 9 March. Firefighters found UPS7 and UPS8 were ablaze, according to an early tweet by Klaba.

"The fire quickly spread into the first building, reaching a point where firefighters were made to focus on the other building to prevent it from spreading there as well," wrote tech reporter Yves Grandmontagne [Google translation]. "This rapid and uncontrollable spread would have several sources. In a report on StrasTV with the firefighters of the SDIS67 of Bas-Rhin and an interview with DNA (Latest News from Alsace), Commander Damien Harroué, who directs the rescue operations, points to 'the fault of bad luck,' and notes that wood and plastics 'generates significant smoke and flames.'"

Grandmontagne is editor of French data center publication DC-mag, but published his article on ZDNet.fr, a French tech news site (which is unrelated to the English-language ZDnet.com).

OVHcloud, formerly known as OVH, has gone through multiple generations of data center design on its Strasbourg campus, beginning with the use of shipping containers in its first data centers on the site, SBG1, and SBG4.

After an outage in 2017, OVH promised to move all servers out of the container-based buildings and decommission them, and separate the facilities on the site into separate distinct power feeds. Shipping containers were "not suited to the requirements of our business." Newer OVHcloud sites did not use shipping containers, so it promised to migrate SBG1 and SBG4 to SBG3, and to close SBG1 and SBG4.

Grandmontagne believes elements of the shipping container construction remain, telling DCD in an email that the decommissioning was not completed. He believes containers were still in use, and the SBG4 facility was still powered from the other data centers on the site.

According to his article, SBG1 and SBG4 "are composed of maritime containers and equipped with wooden floors (to circulate inside the container and support the weight of the stored goods)." He includes a picture (shared by Klaba) of construction on the site in 2013, with wooden floors in evidence - and refers to the firefighters' mention of wood and plastics spreading the fire.

The OVHcloud data centers combine liquid cooling in the racks with "free cooling" in which outside air circulates to remove heat. The operator has been very proud of its multi-story architecture, where air circulates between floors using convection currents, but this too may have contributed to the disaster, says Grandmontagne: "During the fire, ventilation will fan the fire from the bottom up - creating an aggravating chimney effect - and help it to spread."

Fire prevention lacking?

After the fire, Klaba promised that customers would get free backups in future, and OVHcloud would fund a laboratory to investigate fire prevention systems.

"We decided to create a lab where I want to test and see how the fire is going in the different kinds of rooms and find the best way to extinguish the fire in all the rooms," said Klaba in a video released after the fire. "I want to share the conclusion of this lab with all the industry, because... nobody wants to have this kind of incident in their data centers."

The OVHcloud founder mentioned that fire protection standards varied between countries and even between rooms within a data center - but did not mention what protection was installed in SBG2. The operator has not responded to questions about its fire prevention systems, and Grandmontagne suspects that there may have been limitations, telling DCD that the facilities may have had manual, rather than automatic, fire suppression systems.

"In 2013, OVH installed a fire protection system for the first time in a data center in Canada. What about its French data centers?," he asks in ZDNet.fr, pointing out that, according to Klaba, staff could not approach the fire safely and had to leave the premises and call in firefighters: "This attitude would tend to confirm the absence of an early detection system, an extinguishing system, and a firebreak."

Gradmontagne is critical of OVHcloud's attitude that customers are responsible, and only got the reliability they paid for. "Some customers don’t understand exactly what they bought," said Klaba, soon after the fire, when some customers were shocked that their data had not been safe.

"OVHcloud offers a catalog of services, the level of hosting service depends on the service options purchased by the customer," says Grandmontagne. "OVH's low-cost model is based on segmentation of its offer, with an aggressive base price, and additional billed options. If the customer has not acquired a 'backup' option, his data blown up in flames may be permanently lost without the hosting provider's liability being engaged ... except for subsequent recourse to the courts."

OVHcloud's backup offers may not have helped some customers, even if it had been made before the fire, says Grandmontagne, pointing out that the OVH Private Cloud only began to offer a remote backup service in April 2020, requiring customers to adopt this offer: "If the customers of the offer, before and after this date, have not taken the step to adopt this service, they have no backup. This would be the case for the majority of SBG2 customers. But, then again, its adoption would have had little effect, because as Octave Klaba also confirmed in a tweet, 'Private Cloud's free and paid backups in SBG1 were hosted in another room in the data center. Both are destroyed.'"

OVHcloud is correct in saying that disaster recovery plans are the responsibility of the customer, not the host or cloud service provider, but he asks: "What happens when the said supplier does not respect the practices of the profession?"

It may not just be OVHcloud

While Octave Klaba has been very transparent on the restoration process, OVHcloud has dodged questions about the fire itself, perhaps because it wants to protect its aim of floating on the stock market in an IPO.

Grandmontagne suggests that OVHcloud's low-cost economic model may have increased the risks of such a disaster - but he points out that there are wider implications it could conceivably have happened to other providers: "There is not a data center operator who has not been approached by customers worried about whether they are running the same risk."

Keeping costs low may be having an impact on resilience, and if data centers are cutting things too finely, this may have an impact on their insurance contracts, says Grandmontagne: "How will insurers react? Are the prices of policies likely to increase significantly? Already, insurance companies have started to turn away from new data center projects..."

And he also warns the incident could harm French data center operators and other European players: "In the end, the most pervasive effect of the OVHcloud campus fire will certainly be the loss of confidence in the French cloud giant, and its repercussion on French cloud and data center players, which could well create a pathway for the giant American public cloud providers."