The history of enterprise technology can often be understood as an oscillation between centralization and decentralization as the industry cycles between seemingly opposite positions.

Centralized innovation

Centralization favors the early development of innovative technologies. Large, often well-funded resources are concentrated in one or few places, with competing teams working on a similar issue.

This intense focus leads to an explosion of innovation and an accelerated early evolution. Products and services follow swiftly, and multiple variations of modern technology are collectively ‘birthed’ in rapid succession.

Decentralized application

At this point, the technology must go to market and the process of decentralization begins. Real-world applications differ across verticals and geographies, and multiple feedback loops begin to refine the products and services quickly. These optimizations fuel first-mover advantage, and the return on investment (ROI) begins to skyrocket.

This in turn attracts investment that extends the decentralization as the technology expands into more diverse industries and geographies.

Enter AI

This oscillation is now being seen with AI – specifically the evolution of AI.

It is important to recognize the incredible pace of this evolution. Whilst AI has been part of enterprise technology for years, it has been just twenty short months since ChatGPT thrust AI into the public consciousness and put it at the top of every CIO’s agenda.

But we are already seeing a move from a centralized to a more distributed approach to AI. This is due to the evolution from generative AI and the large language models (LLMs) that train generative AI, to the realm of inference AI.

Whereas ‘generative artificial intelligence’ refers to the use of AI to create new content, such as text, images, music, audio, and videos, inference AI refers to the use of models to produce predictions or conclusions, based on existing data. It is the difference between shaping today and seeing tomorrow.

This intense interest in inference AI is driven by businesses that have learned the lessons of technology investments in the past. They are concerned with how they can actually use AI in the course of their business – and what the benefits are. Not surprisingly, seeing into the future is at the top of that list.

Why the different AI architectures matter

Generative AI favors centralized, hyperscale resources of both data and compute processing. There has already been a wide-ranging exploration of the increased demand for energy and data center capacity that generative AI has created – and the development of generative AI bears all the hallmarks of concentrated, centralized innovation.

But inference AI favors a more decentralized approach. In order to provide accurate insights and predictions, the AI needs to be as close to the data source as possible, so it can deliver real-time analysis. This is especially true if these data sources are particularly rich, as in the case of video footage.

Fast speed mandates low latency which necessitates putting the inference AI needs close to the data source. Hence, it favors a regional, edge architecture.

In this race, nanoseconds matter. Consider an inference AI suggesting or automatically making trades on a foreign currency exchange or stocks and shares. If the data needed to feed the AI has to go across the world, there is an inevitable delay (not to mention a substantial cost). By comparison, if the AI is within the same country or region, that time and cost collapses. And a local AI can act faster than a global one, effectively beating it to the punch.

Consuming different AI

As generative and inference AI begin to coexist, the critical issue will be how these different capabilities are offered to businesses.

Whilst enterprises might have the muscle to create their own, layered AI evolution, small and medium-sized organizations cannot afford to develop their own infrastructure to manage their generative and inference requirements. And no organization wants a range of invoices from different AI providers.

Rather, the expectation of these businesses will be that AI is part of the ongoing upgrades and development of the software they already use. This has already been seen in the addition of AI into Excel, and the clear AI focus on updates to Google Workspace and Salesforce Einstein.

But for those businesses providing this software or AI functionality, they need digital infrastructure partners. These partners must offer not only the rapid and secure transit of vast amounts of data to centralized processing capabilities (or even the cloud) but also dedicated regional capabilities. This takes a portfolio of facilities around the country, connected to each other and then the internet, and clouds beyond.

The evolution of AI has already shown that it will follow the path of centralized innovation to distributed use; the really smart money is now on ensuring that – regardless of generative or inference – businesses will be on the lookout for those use AI cases and applications that can deliver competitive advantage faster.