"There's no reason to use Apple hardware at the bottom of your private cloud, unless you have to.”

Shawn Lankton, chief revenue officer of MacStadium, is honest about why his company exists. It’s not there to compete with standard colocation companies on cost, latency or geographic distribution. Instead, he hopes to target a very specific niche: “We exist because Apple requires their customers to work with Mac infrastructure at the bottom of their stack. And that's not our call, that's just a problem that we endeavor to solve.

“We have come up with technology, infrastructure, expertise, best practices to make that possible, easily and at scale.”

This feature appeared in the July issue of DCD Magazine. Subscribe for free today.

When you need Apple

Targeting companies that need to use the Mac OS, such as Xcode app developers, MacStadium operates server racks full of tens of thousands of Mac computers across five data center locations - Atlanta, Las Vegas, Silicon Valley, Frankfurt and Dublin. Primarily, MacStadium uses private suites in Equinix and Zayo data centers, but has a small footprint in a Keppel facility in Dublin.

"We have plans of getting to Asia in the near future and launching additional points of presence," Lankton said. "But we don't really need to be within 50 miles of a customer for them to have a good experience, because the use case is not super latency dependent.

“If you're sending a build or a test job to one of our data centers, that may be a three minute or a three hour job. And so if it takes an extra few milliseconds to travel to the data center, that doesn't really impact the performance or the user experience."

As a business, the company started “with a bare metal offering of Mac minis and Mac Pros,” Kevin Beebe, MacStadium’s VP of product management & security, told DCD. “And as our product evolved, we got into virtualization and led the initiative to get Mac OS and Apple infrastructure supported within VMware.

“In the last two years, our revenue has exponentially grown in the enterprise market - 70 to 80 percent of our revenue is from that customer base now.”

Core to wooing that enterprise audience is being able to offer the reliability, compliance and security standards of conventional data center equipment. Forced to rely primarily on consumer equipment designed to live under a desk, this can prove challenging.

To deploy the various flavors of Mac systems, MacStadium designs its own racks. Originally, “we had tons of Mac minis deployed in fully custom racks that are 10 feet tall and double wide and very non-standard,” Lankton said. "We still have lots of customers deployed in those racks, but for the 2018 Mac minis and for all the Mac Pros and for anything that comes after, we have designed racks that fit in standard 19” designs. That’s to allow us to deploy in more points of presence, and to deploy more rapidly.”

Pro designs

Currently, the company's most popular device, the one it still deploys in the largest volume today, is the 2013 Mac Pro. Unfortunately, the cylindrical design - referred to disparagingly by some as a 'trash can' - does not immediately lend itself to being stacked in a rack.

"How do you get a round peg in a square hole? We have a custom designed and manufactured sled,” Lankton said. "We add the enterprise upgrades that allow it to go from a consumer device that has a 1GbE port and a single power supply to something that looks like an enterprise server when presented to a hypervisor layer like VMware that has dual Ethernet, dual SAN fabric connection, and dual power.“

When Apple updated the system in 2018, it kept the same shape and size. “People would look at that and say, 'Well, it wasn't a big design change, you shouldn't have to do a whole lot of work,'” Beebe said.

“But the power management and thermals are significantly different, and they updated from 1GbE to 10GbE network interfaces. So we did a whole Cisco Nexus spine leaf architecture for the new racks that we were designing. It ended up being a lot bigger initiative than most people thought it would be.”

Lankton added: “The ports are in a different place, the airflow intakes and outputs are in a different place. The cooling and power requirements are massively different. We had to start from scratch and deploy brand new racks for those in about two months.”

The company faced a similar dilemma with the iMac Pro, an all-in-one machine where the computer is built into the screen. “It's a tragic waste of a 5K screen: nobody's looking at them in the data center,” Lankton said. The company fits 24 of the machines in a standard 19in rack, with two systems packed together, screens facing but disabled.

“It's one of the realities of dealing with Apple hardware,” Lankton said. “We get what we get. And we won't build custom hardware on that front: Apple trusts us to take their hardware and be true to what they have produced.”

Apple will always be in control

Therein lies the greatest potential weakness in MacStadium’s business model: They serve at Apple’s pleasure. "We talked about actually taking the screen apart and trying to just rack mount the internal components," Beebe said of the iMac. "And, that would be frowned upon by Apple and non-supportable going forward."

Perhaps it’s an imbalanced relationship, where power rests with Apple, but MacStadium is confident that it has nothing to fear. Apple is out of the enterprise infrastructure business, and has no need to compete, while the existence of MacStadium makes it easier for businesses to develop apps and test them with Safari. In recognition of this, Apple highlighted MacStadium during its October 2018 event.

"There's always been the question of how much Apple would support or acknowledge us and so that kind of validated that," Beebe said. "I think even a lot of customers have had hesitancy of using MacStadium or any Mac hosting company. Apple has a very strict enterprise licensing agreement that's kind of open to interpretation but, being on stage, validated that they know exactly what we're doing and find that to be acceptable."

For many in MacStadium, the on-stage shout-out was a surprise. “As usual with Apple, we didn't know anything beforehand,” Lankton said. “There were only one or two folks inside the organization that had advanced knowledge, even as a camera crew was coming through our data center to photograph the Mac minis.”

The same level of secrecy shrouded Apple’s latest event, held this June. "We had no advanced tips from the Apple team," Lankton told DCD, days after the conference. There, Apple announced a new Mac Pro, the first major redesign in years.

Packed with Xeon CPUs that go up to 28 cores and 1.5 terabytes of memory, as well as AMD Radeon Pro Vega II GPUs, the new Pro is an expensive rectangular beast.

"There have been estimates that the fully spec'd version might cost upwards of $35,000," Lankton said. "It's obviously a super impressive machine."

It will also come in a 4U rack mountable version, built by Apple. So where does that leave MacStadium? "Anybody can put a Mac Pro in a data center," Lankton said.

"Where we stand out is knowing how to configure those Mac Pros, having a supply chain to deploy them, having Mac expertise on staff, and knowing how to deploy those Macs in a cloud which allows customers to really treat that infrastructure as code rather than just have a Mac in a data center."

Beyond the rack

The company won't know whether it will have to make changes to the rack mountable system until it gets its hands on the Pro - later this year, at the same time as everyone else. "If there are expansion modules to provide enough network connectivity and if we can work with the power modules to connect that to our redundant power systems, then we may not have to do anything proprietary to it," Lankton said. "But there's a big difference between rack mountable and enterprise-ready server.

"Anything that's rectangular you can put in a rack, but you need to have full redundancy and supportability to really call it an enterprise machine. And so we'll make whatever upgrades we need to make."

While the company has received some interest in deploying the system, it also suffers from the same issue impacting all 2018 Mac systems - the T2 Security Chip.

An Apple custom silicon chip, T2 stops Macs from running any operating system that's not signed by Apple - including Linux or VMware ESXi.

"None of those devices are VMware Certified," Beebe said. "We're trying to bring Apple to the table to help solve that, because Apple and VMware haven't [worked] so well directly together in the past. We've had to play middleman in getting that done."

This is why the 2013 Mac Pro remains the company's most popular device, as it shifts further from 'Macs in racks' to providing more cloud services.

"We've done that since the very beginning using VMware as a hypervisor platform," Lankton said, "but more recently, we've supported a new Mac-only hypervisor called Anka, and then Orka is the next iteration of that."

Orka is the company's own in-house virtualization layer for Mac infrastructure based on Docker and Kubernetes technology. "That's really purpose built for the users that need to have infrastructure that looks and feels as generic as possible, while still fitting inside of Apple's requirements that all app builds be done on Xcode," Lankton said.

"What we want to do is help our customers abstract that infrastructure layer as much as possible, so that their iOS and Mac app builds and tests look more and more like the same thing for Android or back-end server code or anything else."

He added: “Our customers don't just want a Mac in a rack, they want a Mac in a cloud.”