Why agile transformations fail, and why that is not management’s fault.

By Dennis van der Spoel

​Most agile transformations start out bottom up. Agile coaches and trainers start to explain about VUCA (Volatility, Uncertainty, Complexity, and Ambiguity), and how traditional ways of project management are at odds with the demands of a VUCA environment.

​Of course, some buy-in from management is needed. So, managers get appointed an enabler role, but spend most of their time along the sidelines. And when agile transformations do not go according to expectations of the teams, it’s always the fault of management. And usually middle management gets stuck in a squeeze.

This approach makes the transformation a one-way street:

• Managers should provide a safe space for experimentation and learning
• Managers should be servant leaders and provide the team with everything they need
• Managers should resolve the most challenging of impediments
• Managers must shield the team from the VUCA environment, so the team can focus on their objectives and get into a state of flow
• Managers should enable them to grow individually and as a team
• Managers? Who needs them anyway?

All of these statements, and dozens of statements like these, I have encountered over the years. That seems like an unfair and unbalanced point of view. Because, what does the team provide in return? Here it gets interesting. The premise of agile is value maximization. Managers are told that if they take that leap of faith, do away with their ego, and do all of the above, the team will provide a lot more value. While this in itself is true, the question is: do we agree on the definition of value?

Here things fall apart. For proponents of agile, value means something the end-user is excited about, a.k.a. customer value. Their hypothesis is that, when contesting markets in a VUCA world, the ever-increasing speed of innovation coupled with customer centricity is the only sustainable competitive advantage.

This shows that most product owners, scrum masters, and agile coaches have never had to make (or facilitate) boardroom decisions. Today, the appetite for creative destruction is often displaced by a custodian corporate culture that is defensive about the future and protective of its privileges. It is a culture that incubates habits, strategies, and models that cannot be married with uncertainty or a disposition for radical innovation and life-or-death competition. The more complex the world gets, the more they search for certainty and control. Modern corporate governance is simply not compatible with fast and radical innovation.

Multinationals shifted strategy a long time ago from contesting markets to defending their often oligopolist positions, either through lobbying for favorable legislation or expanding their control across their global value chain. That process also prompted many of them to aim for stronger market control over their end customers. Globalization reinforced the role of corporate bureaucracy and its defensive attitudes toward experimentation. The focus is on raising the costs for others to enter the market, not on creative destruction of their own legacy products, processes, and systems. Nor is it on continuously accelerating innovation.

Uncertainty is also the main problem with complex regulations. They affect innovation because they compel companies to innovate in a way that pleases an increasingly risk-averse and precautionary society. Governments have created a permission-based culture of innovation. This resulted in high penalties for noncompliance, which in turn led to an increase in bureaucracy. When a certain bank got fined several hundreds of millions of dollars for a reasonably minor infraction, this bank at once hired 1,500 additional employees for various compliance related roles. Many of which became direct stakeholders for agile teams.

Another issue is that capital markets have no appetite for innovation. Instead of doing what is right in the long term, management then focuses on the short term. Christensen and his team hinted at this motive when they wrote that when the usable lifetime of an asset is “longer than its competitive lifetime,” managers “may stall in adopting new technology” because they fear the reaction of capital markets. The average manager would prefer to destroy economic value for the sake of keeping numbers predictably. According to a survey among Chief Financial Officers, 75 percent of these executives would “sacrifice some economic value to achieve smooth earnings paths.” Furthermore, approximately 80 percent of the managers in the study would “decrease discretionary spending,” including R&D, to avoid coming in short on a quarterly earnings target. Half of them would prefer value-destroying delays of projects to mitigate the same problem.

What then is value to the average manager? It is risk reduction, plain and simple. So, if you demand that managers risk their reputation, career, fines, and, increasingly, jail time so you can have a safe space to experiment and learn, it is time you offer something of similar value in return.

What most managers need from the team is:

• Objective evidence that the quality of the product and the process is increasing so that industry standards are met, and certification is assured
• Objective evidence that forecasts and estimates become increasingly more reliable, so that the quarterly earnings remain predictable
• Objective evidence of acceptable levels of compliance, so that fines and bad press are avoided
• Objective evidence of increasing user satisfaction and market impact is a bonus

Here’s the thing; you cannot imagine how hard it is to get teams to consistently and willingly gather, process, and share that data in a disciplined, transparent, and open fashion. That is why agile transformations fail. You are asking management to take all the risk but provide them with no reward. You want a safe space? Great! But guess what, so do managers. But the world is not safe, it’s VUCA, and managers are just trying to get a grip on that. So, the best thing you can do is to help them to be a little safer. Then an agile transformation becomes a two-way street.

The same is true for agile frameworks. I often get asked why I love various frameworks so much. I sometimes get sucked into debates, especially when I see ill-informed scrum masters and agile coaches bashing SAFe (Scaled Agile Framework). (Apparently, I have some nags to work on as well.) It is not that I love SAFe but I see its value where others cannot. Besides the fact that it is much more agile that it is given credit for, it serves a crucial purpose in the agile transformation of the multinational.

Scrum, the most popular agile framework, is optimized for working in small teams. And it’s unrivaled in structuring teamwork. But beyond the scope of the team it gets blurry. Anyone not in the team is a stakeholder. But in the IT or innovation division of most larger organizations 60 percent of the individuals are not a member of any agile team. And for reasons of compliance and specialization, they will never be part of an agile team.

Most scaling frameworks build on scrum, such as LeSS, Nexus, S@S, and SoS do not have a desirable solution for actively involving a large number of stakeholders, reducing managerial risk, and increasing compliance. To senior managers, those frameworks feel like letting the inmates run the asylum. That is something they may get fired over, or worse.

Enter SAFe. Even the name feels comfortable to managers. And, for the largest of organizations, SAFe has resolved all those pains that a modern manager has, and even found ways to make the solution as agile as it gets within these constraints. No manager will get fired for implementing SAFe.

Because I read hundreds of blogs and articles on why SAFe is ‘evil’, I can hear the angry screams of agilists claiming that SAFe is not agile. I disagree, but that is not the point. The point is that SAFe invites managers to learn about agile. It creates a safe space for managers to experiment and fail with agile. As a starting point for a transformation within a big organization, it works wonders. Yes, it is easy to just relabel some roles and continue on familiar paths. Yes, it may result in bureaucracy at first. And yes, it doesn’t reduce complexity. But it’s a learning curve. After a few PI’s you can start iterating your customization and tailor the approach, perhaps reduce some of the complexity, as long as you build trust. It beats throwing rocks at managers for not having an agile mindset yet.

The responsibility to act as a secure base, build trust, and create a safe space rests with the one furthest in developing an agile mindset 

What definitely doesn’t work is to start with the ‘reduce complexity’ message. What managers hear is that the organization, its product portfolio, its systems, and its processes must be cut into independent pieces without any form of centrally coordinated direction. What they understand is that there is no alignment, no strategy execution, and an increased risk that, when shit hits the fan, all the blame will fall on them. Apart from some very brave individuals, and we all have some stories to share of entrepreneurs and CEO’s that have been bold enough, most managers just don’t have that much of a risk appetite.

Oh, I know that, according to theory, agile teams will organically align around the organization’s purpose and the manager’s vision and strategic intent and work together as one. However, the current cohort of managers has not been hired for their inspirational qualities. They are hired for their ability to manage risk. Some famous quotes of Mark Rutte, prime minister of the Netherlands, include: “Vision is like the elephant obstructing the view” and “Anyone looking for vision should see an eye doctor.” And as you nor I have the mandate to replace managers, and the current global socio-economic and political landscape will not change overnight, we will have to accommodate this cohort until the paradigm for multinationals changes.

Where does this leave us, the agile community? Well, I strongly suggest deepening your understanding of the ways that managers think and behave in your organization. A good place to start is to get the CEO to invite you to facilitate a session to refine and plan the current strategic objectives or themes. I often use a benefit mapping workshop as a structure for such sessions as this technique will create alignment, surface epics that had not previously been identified, and trigger significant changes to the current set of business models and value streams. This sounds like major changes and big risks, but senior management is on board with that as long as they believe that they are at the helm.

If that doesn’t look feasible, read “The Innovation Illusion” by Fredrik Erixon and Björn Weigel. It’s a real eye-opener. Then read about aspects of agile compliance, agile contracting, lean portfolio management, model-based systems engineering, value-based procurement, value engineering, participatory budgeting, beyond budgeting, agile HR, enterprise agility, agile sales teams, and more. This will help you to connect scrum to other frameworks and practices that help to make an entire organization, not just a development team, more responsive. But most of all, stop selling your idea of agile and start listening to conversations that managers have and try to understand the challenges that they face. Then come up with a workable solution for that, whether it is an acceptable agile practice or not. Let me know how that works out.