Observability can be programmed and automated with observability as code. A maturity model can be used to measure and improve the adoption of observability as code implementation.
Yury Niño Roa, cloud infrastructure engineer at Google, spoke about the observation of programming at InfoQ live in August 2022.
Niño Roa referred to Thoughworks in 2019 recommending treating the ecosystem configurations of observability as code and using infrastructure as code to monitor and alert infrastructure:
They motivated us to choose observability products that support configuration via version-driven code and execution of APIs or commands via infrastructure CD pipelines.
She proposed to think of observability as code as a technique to automate the configuration of the observation tools in a consistent and controlled manner, in particular by using infrastructure as code and configuration as code.
In her lecture Observability is Also Programmed: Observability as Code, Niño Roa presented an Observability as Code Maturity Model that can be used as a way to benchmark and measure its adoption.
The model can be used as a reference to know what level you are at:
The model provides criteria to determine the status of an organization on two axes: refinement and adoption. For sophistication, it uses four stages: basic, simple, sophisticated, and advanced, and for adoption, there are four more levels: shadow, investment, adoption, and cultural expectation.
For example, an organization is at an advanced stage if it has implemented an automation workflow for observability as code and it is in production. The idea is that you identify which stage you are in, look at the criteria of each stage and question yourself about your implementations and performance, Niño Roa said.
InfoQ interviewed Yury Niño Roa about perceptibility as code.
InfoQ: How would you define perceptibility?
Yury Nino Roa: Observability is a broad term whose definition is controversial between industry and academia. Some vendors in the industry maintain that observability has no special meaning, using the term indiscriminately from telemetry or monitoring. I think the proponents of this definition degrade observability when they use it as another generic term to understand how the software works. Monitoring is part of observability because it allows us to anticipate the health of the system based on the data it generates (logs, statistics, traces).
InfoQ: Why should we do observability as code? What benefits can it bring?
Nino Roa: That’s an excellent question, since I don’t think the perks are fully unlocked. Some of them include:
- Less toil needed to set up dashboards for monitoring.
- Repeatable, replicable, and reusable configurations required in the configuration dashboards, alerts, and SLOs.
- Document and generate context using infrastructure as code to configure monitoring platforms.
- In general, the teams store the code for observation in repositories, so that it is easier to check the history of changes.
- Providing security, as perceptibility as code allows us to have tighter controls while using continuous integration and deployment.
InfoQ: What are the challenges of observability as code?
Nino Roa: I think they are aligned with the challenges of other practices, such as Infrastructure as Code and Configuration as Code. specifically,
- Achieving true adoption of engineering teams, leveraging automation where possible to accelerate the delivery of observability across different environments.
- Defining clear KPIs to measure the impact of observability-as-code maturity in the organizations.
- Determine and communicate current status before implementing new observability-as-code capabilities.
- Documentation is a major challenge in any field, as automating documentation generation in an automatic way requires advanced techniques such as machine learning and unstructured text processing.
InfoQ: What is your advice to start with observability as code?
Nino Roa: My first advice is that you should know about Infrastructure as Code, especially Observability as Code. After that, it is very important to get sponsorship for its implementation. I talk about this in the early stages of the Observability as Code Maturity Model. In these early stages, the organizations have decided to implement Observability as Code, so they have started collecting metrics and officially have practitioners who devote resources to the practice.
.
