"Never in my career did I dream that the supply chains for cleaning supplies would be absolutely critical to our operations,” Digital Realty’s head of procurement Brent Shinall told us in a July DCD keynote speech about the changes the Covid-19 pandemic has brought about within his company and the data center industry.
How big are those changes? Well, Shinall was speaking to a three day web-conference. As this article hits the web, DCD has had nine global or regional events since April, and they’ve all been online-only.
The Covid-19 pandemic has changed many of our perceptions of value: cleaners and delivery workers are more important than managers; meetings are dangerous; travel is a risk we try to avoid. And digital infrastructure has been part of this.
Enabling online life
Data centers have enabled digital commerce and online meetings, making digital infrastructure more important than ever before. That’s given the data center world a sense of achievement, and kept digital businesses operating and growing.
But lockdowns have tanked economies round the world. Data centers are part of an ecosystem, supporting customers which pay for their services. Some industries - entertainment and hospitality, for instance - have been hit harder than others, and some data centers will suffer an impact.
Some companies have reported they are preparing for customers that may have trouble paying their bills. Even if your tenants are online services booming in the lockdown, there could be long term issues. Zoom is profitable, but others are building their user base and still losing money.
Others tell us that enterprise activity like data center consolidation is on hold. Physically moving servers, closing buildings and opening new space, is massively more complicated in the Covid-19 world.
Mergers and acquisitions are going ahead, along with data center construction, but much of that activity was started before lockdown, and the obvious difficulties with physical activities may slow down due diligence somewhat.
Given these mixed fortunes, it’s no surprise that data centers, and other firms in the sector, have been granted government support to keep their services up and their staff employed (see box below: A helping hand).
Those staff have certainly been dedicated, and operators have stories to tell. In hard-hit New York City, DataGryd CEO Tom Brown says morale is good: “You take a step back and you say, ‘Well, I’m grateful that my health is okay and we’re hopeful that all employees and contractors remain healthy.’”
Stack Infrastructure runs several hyperscale data centers with around a hundred technical staff. Chief data center officer Mike Casey says the staff are essential workers, and “a lot more important than anybody else on the whole Stack team.”
Luckily data centers have been able to adapt their work patterns. DataGryd instituted its own “mini lockdown” (on March 13), reducing its onsite staff, and only letting visitors or staff on site when a task can’t be done remotely: “Technicians are just doing these longer light-load shifts or they’re on more rotations.”
Staff travel also had to alter: “The good news is that we have folks that live in New York City and we are giving strict instructions to not take mass transit,” says Brown. “Take a vehicle. We have parking right next to the building and they have to go in and do what they need to do and then leave. There’s no need to stick around.”
Remote management has helped, he adds: “We were prepared for something like this, we have management systems that allow us to automate a lot of processes. You can check [server] temperatures remotely; you can check if there are any big spikes. Should there be any type of alarm, you can work on your laptop or your phone.”
Working from home was surprisingly easy, says DataBank CISO Mark Houpt: “We were impressed by the effectiveness for us to move online. We were concerned about the ability of systems like VPNs and to sustain the traffic. We did stress testing and so far we’ve seen no glitches.”
Cameron Wynne, DataFoundry’s COO, says: “We had to review all of our work down to the physical nature versus things that we could do remotely. For their health, we had to reduce footfall at our data centers. If people need to be in, they work in different offices and socially distance.”
But some jobs just can’t be done remotely. For those on site, there’s increased hygiene, with staff using sterilizing wipes on any surface that may have been touched. Some companies are investing in ultraviolet lighting to kill bugs. Biometric security is going hands free, with eye scanners replacing fingerprint locks. At the first whiff of Covid-19, the entire facility is sterilized.
Data centers have always had business continuity plans to weather natural disasters, and Stack adapted their plans to the pandemic, says Casey: “We made changes to protect our critical operations staff and our clients. We are taking temperatures as folks come through the door and increasing our janitorial tasks. We also have different rules for shift changeover. We used to overlap shifts, but not anymore, we now don’t have overlaps just to keep the separate shift socially distanced.”
If it all goes wrong and one site has to send all the staff home, Stack has a plan to bus or fly people in from other parts of the US.
Phillip Koblence, COO at colocation provider NYI, reckons it’s down to communications and practical support: “One of the things that I have found, whether it’s Covid-19 or Hurricane Sandy, is that communication is key. So, making sure that the staff is aware that somebody is looking out for them is key. You want to make sure people are aware that they’re able to take things like Uber or drive. Because they’ve got to also look out for their own families as well, you know.”
Despite self-preservation, staff know that important services rely on them. Like police officers or firefighters, they respond accordingly, said Koblence: “During things like hurricanes, when people are hunkering down, in our industry, you’re running towards the data center.”
DataFoundry’s Wynne echoes this: “When a hurricane comes to Houston, We drive into it. While the city is evacuating, we’re taking back roads to go into our facilities to ensure that we keep our customers and our business up.”
Some staff can’t take risks though. DataFoundry set up a special “furlough pot” for employees who felt unsafe, but Wynne says not one employee took a cent: “We gave every single person the ability to, if they couldn’t work from home, to not come in and have no penalty. They could stay home if they felt unsafe or if they felt precautions weren’t met.
“And honestly, all of our folks wanted to come to work. They want to be here. They want to make their living. We all have a neighbor who lost their job or was furloughed and can’t put food on the table. And so, our employees have simply chosen to come to work.”
Part of the reason is team spirit, says Noel O’Grady, director of sales Ireland at business risk company Sungard AS: “Anybody who is part of the essential group tends to not have too many problems. I think it’s because they feel like they’re part of the A-team. In the past, we’ve seen mission-critical workers feeling like they’re at the beating heart of the issue. They feel like they’re special. After this is over maybe they will go back to the office and say ‘yay we did it!’ Whereas everybody else will be saying ‘Well, you didn’t really need me!’”
DataBank’s Mark Houpt agrees: “A number of our staff must work in the data centers.
"We have not seen any of those folks fearful to come to work. We’ve seen people willing to step up and take extra shifts when it was necessary.”
They are still cautious of course: “They wake up in the morning and if they have the mildest of symptoms related to Covid or someone they live with does, then they have to call in and keep everyone notified.”
As the pandemic drags on, it’s clear that some of these measures will be long-lasting, or even permanent. DataBank, Data Foundry, and DataGryd all told us they are looking at moving workloads away from the site, so facilities keep operating optimally with fewer personnel in the data center.
NYI had a head start, says Koblence: “We started embracing [remote working] fairly early on for a number of reasons. New York is a fairly expensive environment for human capital, it’s just expensive to have employees live in the area but now we can have specialists in Ohio supervising those systems.
In Pittsburgh, Data Foundry had to make changes, when local authorities enforced lockdown measures.
Police would give fines to anyone flouting curfew and anyone caught traveling had to make sure they had a good reason to be out.
Seeing the problem coming, Wynne said the company shifted very early to remote working and monitoring: “Before the Governor started locking things down, we went ahead as a company, and decided to do as much as possible to get our folks working from home.
“There are lots of things that can be done remotely, telephone support, ticket support and a lot of customer operations can be done remotely.”
While all this is happening, there’s a fresh surge in already brisk demand for online services, and more capacity is needed. “We’re very busy,” says Stack’s Mike Casey. “We have quite a few enterprise customers and depending on the business they’re in they may have been impacted by the virus. But we’re working with those customers and overall, our business continues to grow. We’ve got construction ongoing during Covid-19.”
Construction projects obviously have to be safe, and those teams are isolated from the operations teams, says Casey: “You don’t want to be taking any risks with the ops guys. If one of them gets ill, the entire team who he or she is working with also has to go in isolation.”
Some are even moving planned projects forward to keep up with demand. DataGryd had to move projects planned for Q4 2020 forward to Q3.
NYI has had some increased demand, but it varies by sector, Koblence says: “We have customers that need to ramp up bandwidth and maybe add a cabinet or two. We have some customers that are part of the entertainment industry whose business has essentially shut down right now and then we have other services that are part of the healthcare industry asking us for an increase in capacity.
“We have empty server boxes piled up near dumpsters because we’re getting in new gear,” says Houpt.
“This is an example of what’s going on at DataBank right now, customers are desperate to increase their capacity. But other than that, it’s just business as usual.”
Technicians on the ground are more aware of the exceptional circumstances. An NYI technician who did not want to be named said: “My family is proud that I’ve been able to work during these uncertain times and provide for them.
"We’ve taken steps as a family to make sure we reduce the possibility of contracting the virus as well as passing it on to others.
"We also check in on relatives and friends to see how everyone is doing. It helps to know people still care enough to check on the wellbeing of others.”
Another admitted they had “a small fear that we could get sick with the virus, but overall pride.”