Dr. Greg Bruno has developed a theory called the “Pain Curve” (pictured) that visualises his theory that companies using Hadoop for Big Data or OpenStack for cloud are experiencing pain and outages when trying to scale their data centers.

Dr. Greg Bruno, formerly the co-founder of the open-source Rocks Cluster Group at the San Diego Supercomputer Center (SDSC), is the VP of Engineering for the San Diego based startup StackIQ.

The Pain Curve illustrates the fundamental difference in how clusters and traditional data centers scale. One such customer was a tier-one credit card company that managed a 100-node cluster successfully, but a team of 14 engineers failed for 2 months trying to add 363 new nodes to the cluster.

Adding servers to a traditional data center increases the complexity of the overall system by a linear factorial, while adding nodes to a cluster increases the complexity exponentially.

Without increasing the headcount or embracing automation services into your underlying infrastructure, according to Dr Bruno, companies assuredly reach a ‘failure zone’ marked by slow delivery, degraded services and production outages. Whether this is a situation for which the only solution is automation is another matter which DatacenterDynamics FOCUS would be interested in hearing from readers about.