The adoption of DevOps practices is proceeding apace. Organizations are focused on how to implement DevOps capabilities across teams and how to scale DevOps across the enterprise. But an important aspect to any scaled transformation journey is assessing a team's or organization's capabilities along the way.
The assessment of DevOps capabilities within a team, or a company as a whole, serves many purposes. Perhaps most important, it helps align people with a long-term vision for the company—and avoid making critical mistakes.
An assessment also baselines the current state of DevOps capabilities, which is critical for measuring and identifying the next target state of improvement. It helps individuals and teams devise experiments to move between states, and highlights local discoveries to make global improvements.
Here's what to keep in mind as you're assessing your own DevOps capabilities.
The DevOps journey
Sold on undertaking a DevOps capability assessment? Great! But first remind yourself that a DevOps journey is fundamentally a human initiative designed to modify ways of working. The goal is for an organization to evolve and adapt, to respond to the demands of the digital world more effectively.
The adoption of DevOps ways of working is desirable for organizational health and sustainability, and also has human happiness at its core. DevOps aims for a culture of high trust and psychological safety, where all have autonomy, mastery, and purpose. (Daniel Pink identified this trifecta in his motivation formula for delivering your best work.)
The trigger for this kind of large-scale change is often a sense of urgency and a "survive or die" mentality. The problem with this is that it frequently leads to a big-bang transformational approach to change that's highly disruptive to productivity, tends to fail, and can be off-putting to the people asked to make the change.
A learning expert, Britt Andreatta, explains it in her book, Wired to Resist:
"Several structures in our brain are actually designed to protect us from the potentially harmful effects of change. Humans are wired to resist change and we are working against our biology at every turn. It's well documented that every year 50% to 70% of all change initiatives fail."
A DevOps capability assessment helps tackle the challenges of digital change initiatives by providing clarity and alignment around the long-term vision and by fostering continuous improvement.
The importance of being continuous
The obvious time to assess your organization's DevOps capability is at the start of your journey, but there are other considerations you should take into account:
- Journeys take several years, and the goal is over an ever-receding horizon.
- Many organizations consist of multiple teams with varying capability levels.
- Organizational design changes and mergers and acquisitions are not uncommon.
- A project-based transformational approach to change is inherently flawed.
- Continuous experimentation is essential to creating a dynamic learning environment.
- External pressures (e.g., regulations, competition) continually change.
An assessment initiative can be started at any time in your organization's journey. What's more important is that you approach it as a long-term activity designed to support inspection and adaptation. Think of teams working to their own cadence to discover improvements and then share them across the organization.
Engineering teams, IT Ops leaders, and business stakeholders will want the data that's collected to justify the time, money, and energy expended and to qualify for further investment. As authority is redistributed during the change, leaders evolve into guides rather than managers—but being able to measure progress and describe success stories is still essential.
Teams become masters of their own destiny, balancing the costs of operating and improving their product versus the income generated by delivering value to their customers. They effectively run their own profit and loss.
Leaders can inspect heat maps for each team's capabilities and how they change over time across the organization, allowing them to make decisions about where to direct support and interventions and learn where impediments are slowing the business down.
The importance of being scalable
Few organizations are made up of a single team or value stream, and DevOps initiatives rarely stand alone. Scalability of the assessment approach you take is key. Teams will be busy balancing planned and unplanned work and they will likely be asked for data on a host of other topics.
It's not at all uncommon for organizations to be running similar initiatives in tandem with DevOps for agile, SAFe, systems thinking, or site reliability engineering, for example. There are a lot of cross-overs in these methodologies and frameworks that organizations and change agents/leaders should seek to coordinate or even bring together under a single way of working.
Take into account your teams' cognitive load, since they will have to spend time thinking about their situation and sharing the details, whether it's through an online survey or a conversation with a consultant. It's essential that teams don't feel that these assessments (or any part of a change initiative) are being done to them, but that they are actively participating in the journey and are empowered to choose the things that will work for them.
It's vital to impress upon participants that the assessment is not about beating teams up over their weaknesses. Instead, it's about finding strengths, sharing them, and giving teams the opportunities to optimize their own team's work and the way they deliver value to their customers.
Your teams need to see the value of the assessment process and want to use it for their own groups. Helping them find ways to use the inherent telemetry in their systems to automate the collection of data they need to inspect accelerates the process further. Look to value stream management platforms for support here.
The importance of being reflective
The strength of the assessment is not the data collection, not the active participation in data analysis, not in the workshops to define the next target state and devise the experiments. It's the next step: How did the experiment do? Did the anticipated improvement reveal itself? Is it measurable as intended? What do you do next in light of the answers to these previous questions?
You want your teams to be dynamic learners and you must arm them for that. They need the tools to identify where improvement opportunities lie and where their experiments are working and should be extended, repeated, and shared.
Without a capability assessment you won't be able to measure progress, you'll have no data to drive your conversations, and you'll have no evidence that your efforts are paying off. Continuous assessment must be part of every change initiative and is the only way to assure incremental success.
Want to learn more? The DevOps Institute's Assessment of DevOps Capabilities (ADOC) is designed for individuals, teams, and organizations that want to baseline their current DevOps state and measure and accelerate continuous improvement during their DevOps journey. This crowdsourced, vendor-neutral assessment features a scorecard based on an empirical model that examines five core DevOps dimensions.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
