More and more aspects of business depend on the network, especially as digital transformation drives a “new normal” for virtual and remote operations. This new normal includes ever-increasing 24x7x365 connectivity, bandwidth with very low latency and jitter, and an acceleration of trends around bring-your-own-devices (BYOD), software-as-a-service (SaaS) applications, remote work (especially since the outbreak of Covid-19), and shadow IT. All this means that application or network downtime is more expensive than ever, and ensuring consistent access to mission-critical applications and good end-user experience are essential for customer loyalty and employee productivity. To that end, businesses must implement a strategy to ensure network-aware application performance monitoring.

Yet ensuring this monitoring is increasingly challenging. Data centers continue their consolidation trend, yet data center traffic keeps growing, becoming denser and more complex - thanks to increased virtualization and higher bandwidth links. Meanwhile the ongoing pandemic has accelerated digital transformation. As a result, more enterprises are becoming distributed, with a hybrid infrastructure that spans branch offices or remote sites, data centers and multiple clouds. In fact, there is increasing migration of business applications to the cloud, to be accessed over the public Internet from anywhere. This combination of consolidated data centers and distributed applications and services, hosted in the public cloud and accessed by remote users at home offices and branch offices, is the complex new environment IT needs to manage.

Silos won't cut it

In this new normal, a siloed approach to monitoring simply won’t do – only full network visibility provides the insight and clarity necessary for IT operations. Modern network observability and visibility mechanism such as Network Performance Monitoring (NPM) and Network Packet Brokering (NPB) play a key role in this. Appropriate instrumentation for north-south and east-west traffic visibility may include (but is not limited to) physical and virtual taps, packet brokers, packet capture, flow generation and analytics engines.

One key trend is that data center consolidation and cloud migration are happening side-by-side. Application teams are thus often puzzled by the difficulty faced when trying to lift-and-shift or rearchitect processes of business applications. Often this is because they fail to consider that when applications are moved to the cloud, it is critical to understand the infrastructure and network dependencies so that equivalent performance and security can be provisioned in the cloud. Moreover, normal and peak day baselining is required to benchmark the application and network performance. This baselining becomes the new starting point for the cloud migration – and is not possible without a network-aware application performance monitoring solution.

Achieving full visibility into a hybrid network environment also improves an organization’s security posture. Security tools are only as good as the data that feeds them; if that data isn’t good enough – or has gaps – the tools may miss indicators of malware or network attacks. Getting comprehensive network visibility thus helps to shorten the incident response window and reduces the dwell time of malware. Moreover, many Network Detection and Response (NDR) solutions can be implemented to detect suspicious activity in cloud environments, but only if they have access to high-resolution network data from those cloud environments.

Yet this performance and security monitoring is not easy to accomplish as network complexity increases, because the more complex the network, the more likely there are barriers or boundaries that limit visibility. A truly good network visibility solution must be able to reliably monitor traffic across the entirety of an organization's current and future hybrid network architecture – with physical, virtual and cloud-native elements deployed across the data centers, branch offices and multi-cloud environments with single-pane-of-glass event correlation and analytics.

Unfortunately, the blind spots in the public cloud infrastructure can still confound visibility. In fact, up until mid-2019, every major public cloud platform was a black box in terms of visibility. This made application performance monitoring and security assurance especially difficult. Ways around this lack of visibility do exist, but they all have limitations, and none of these methods provide the high-quality network data required to investigate difficult security, application and user experience issues. Fortunately, in the last year major cloud providers started delivering access to rich network data streams. These features duplicate network traffic to and from the client’s applications and forward it to cloud-native performance and security monitoring tools for assessment. This gives IT much more visibility into public cloud deployments and allows on-prem workflows portability for ease of use.

Big transformations

Many enterprise branch offices are also going through transitions such as virtualization, increased wireless networking, and SD-WAN connectivity to the data center and public cloud. This requires network visibility mechanisms that can integrate with newer technologies. Lightweight probes for remote packet capture and analysis, flow data generation and end-user experience monitoring are some common requirements for these transitions.

IT’s main challenge is to collect data from this distributed hybrid environment, and then correlate, analyze, and convert it into actionable information that can drive business decisions. New automation tools and technologies are emerging to help, including automatic root cause analysis using AI/ML, and prescriptive or predictive recommendations that can feed into overall AIOps (AI for ITOps) framework. But again, this automated analysis requires high-quality network information, or it will produce inaccurate or misleading conclusions that can negatively affect business decisions and the customer experience.

In summary, a cohesive hybrid visibility suite allows organizations to use a consistent mix of tools, workflows, data and insight when managing hybrid environments (the proverbial ‘single pane of glass’). This makes the emerging ability to gather the same deep insights across both private and public infrastructure a game changer for application and network performance monitoring and security. When IT has access to full network traffic, which delivers much richer and deeper situational awareness when compared to log data, they can conduct more effective network and application monitoring or security investigations. Ultimately, better visibility means a better end-user experience and reduced customer churn – all by de-risking IT infrastructure and operations.