Vendor Viewpoints: The Shift to Cloud Observability Involves Cultural as Well as Technical Change

Across all industries, organizations are continuing to accelerate their move toward modern application development built on cloud-native microservices. A cloud-first strategy is rightly viewed as the catalyst for rapid digital transformation and the key enabler for businesses to respond to constantly evolving customer and employee needs.

By reimagining their applications in a hybrid or multi-cloud environment, organizations can embed greater flexibility and freedom in their application development processes and, ultimately, unleash innovation on a scale and speed not seen before.

However, as anybody who has worked in an IT department over the past year or two knows, managing availability and performance across cloud-native applications and technology stacks is a huge challenge for IT teams.

Traditional approaches to availability and performance were often based on long-lived physical or virtualized infrastructures. Going back 10 years, IT departments operated a fixed number of servers and network wires — they were dealing with constants and fixed dashboards for each layer of the IT stack. The introduction of cloud computing added a new level of complexity; organizations found themselves continually scaling up and down their use of IT based on real-time business needs.

While monitoring solutions have adapted to accommodate rising deployments of cloud alongside traditional on-premises environments, the reality is that most were not designed to efficiently handle the dynamic and highly volatile cloud-native environments that we increasingly see today.

It’s a question of scale. These highly distributed systems rely on thousands of containers and spawn a massive volume of metrics, logs, and traces (MLT) telemetry every second. And currently, most technologists simply don’t have a way to cut through this crippling data volume and noise when troubleshooting application availability and performance problems caused by infrastructure-related issues that span across hybrid environments.

The need for cloud-native observability solutions

In response to this spiraling complexity, technologists need visibility across the application level, down into the supporting digital services (such as Kubernetes), and into the underlying infrastructure-as-code (IaC) services (such as compute, server, database, and network) that they’re leveraging from their cloud providers. They also need visibility into user and business impact to prioritize their actions. This is essential for IT teams to truly understand how their applications are performing and where they need to focus their time.

Technologists increasingly recognize the need for full-stack insights and to map relationships and dependencies across siloed domains and teams. This explains why, according to the latest AppDynamics report, The Journey to Observability, more than half of businesses (54%) have now started the transition to full-stack observability, and a further 36% plan to do so during 2022.

IT teams need new cloud-native observability solutions to manage the complexity of cloud-native applications and IT environments. In particular, they need a way to get visibility into applications and underlying infrastructure for large, managed Kubernetes environments running on public clouds.

From a technology perspective, there are several key criteria that IT leaders should be considering when looking at cloud-native observability solutions to ensure they are future-proofed for the next 10 years and beyond. They should be seeking out a new generation solution to observe distributed and dynamic cloud-native applications at scale; a solution that embraces open standards, particularly Open Telemetry; and that leverages AIOps and business intelligence to speed up identification and resolution of issues and enable technologists to prioritize actions based on business outcomes.

Businesses must recognize new cloud-native mindsets

But alongside best-in-class technology, the shift to cloud observability also requires a significant cultural change within the IT department. And it’s vital that business and IT leaders recognize this and act on it to attract and retain the skills and talent they need to achieve their goals.

The move toward a cloud-first strategy has seen the emergence of new teams within the IT department — such as Site Reliability Engineers (SRE), DevOps, and CloudOps. And not only do these technologists have new and highly specialized skill sets, but they also have very different mindsets and ways of working.

Traditionally, ITOps teams have always been focused on minimizing the risks brought about by change. Their mission has been to maximize up-time and unify technology choices, and they tend to take a rigid, centralized approach to digital transformation.

But when it comes to SREs and DevOps teams, it’s a very different story. These new teams value agility over control and are focused on giving each team the freedom to choose the best approach. They accept that there will always be massive complexity with cloud-native applications, but they see that giving up some level of control gives them speed and innovation. They’re able to find peace in the chaos by adopting new solutions that allow them to cut through complexity and data noise and pinpoint what really matters.

Similarly, when considering digital transformation initiatives, these teams aren’t fazed by the scale and complexity involved in these programs. They don’t feel held back by legacy technology or scarred by previous attempts to innovate. They embrace change rather than resisting it and see transformation as an exciting and welcome part of business as usual.

These new cloud-native technologists are unwilling to conform to vendor lock-ins; they believe they can deliver the most value within dynamic technology ecosystems, with all teams having the freedom to select and work with best-in-class solutions for each project.

Finally, cloud-native technologists (for SREs, DevOps, or CloudOps) will evolve to have a very business-focused mindset. They will increasingly strive to view IT performance and availability through a business lens and understand how their actions and decisions can significantly impact the business.

The important thing for business leaders is to recognize the new mindsets and drivers of their cloud-native teams and to empower these technologists with the culture, support, and solutions they need to deliver value. That means developing a strategy that enables these teams to operate in completely new ways while also ensuring their existing teams can continue doing the vital work they’re doing by monitoring large parts of their IT infrastructure.

Most importantly, IT leaders should consider these cultural factors when selecting a cloud-native observability solution. Such an approach ensures their SREs, DevOps, and CloudOps teams have a solution that offers the scalability, flexibility, and business metrics they need to perform to their full potential.

By taking a holistic approach, considering both the technical and cultural needs of their IT teams, organizations can empower their technologists to cut through the complexity of cloud-native environments and deliver on the promise of this exciting new approach to application development.

Abhilash Purushothaman, regional vice president and general manager of Asia at AppDynamics, wrote this article.

The views and opinions expressed in this article are those of the author and do not necessarily reflect those of CDOTrends. Image credit: iStockphoto/metamorworks