Why Do Most Agile Transformations Fail?

October 15, 2024

Don’t be fooled by all the Agile transformation case studies or success stories: most transformations fail.

To make matters even worse, many failed transformations, like organizations that introduced the Spotify Model or Scaled Agile Framework (SAFe), are often presented as fantastic success stories. They brought in an army of consultants, paying them millions, which caused the sunk cost fallacy to kick in and guarantee the fruits of all their labor are presented as a success.

I know that many Dutch Agile success stories using SAFe or the Spotify Model are complete failures. Because when I talk to people who work for these companies, they tell me it’s a nightmare to work there, and nothing fundamentally changed, except in the way they present themselves to the outside world.

Organizations are great at applying copious amounts of Agile make-up to appear more attractive to applicants. I’ve noticed this phenomenon during job interviews. They were always extremely Agile or did Scrum by the book, which was never the case after I joined them and saw how they were working.

As an employee, whenever I changed jobs, I was elated. Changing jobs usually meant a fresh start with a bigger salary, more responsibility, and a better employer to add to your resume. I was also curious to learn from how they worked, by all the excitement they had created during the interviews.

However, all this initial excitement usually quickly subsided. After starting your new job, no matter how awesome the organization presented itself to the outside world, you will begin to notice all the problems:

Leaders who bullied others to get what they wanted.

CTOs who lost touch with what the teams were up to and presented timelines without consulting anyone.

Control-freak managers who were unable to delegate.

Developers who refused to take ownership of their work, usually as a symptom of something else.

Product managers who had never talked to customers were calling all the shots.

In short, from this experience, I learned that every organization has problems. It doesn’t matter how great an organization looks on the outside or how well it performs in the market: there will be issues.

Every system, even if it’s high-performing, is imperfect and ends up producing significant problems.

Of course, I’ll be the first to admit that some companies have more problems than others, but if you examine any organization for some time, you’re guaranteed to find issues and dysfunctions.

Why do organization don’t solve these problems?

Why Do Organizations Don’t Fix Their Problems?

The problem isn’t that these organizations don’t know their problems. I am confident that most organizations know most of their problems, especially if they have a big impact. They may not know how to fix them or the exact root cause, but they are fully aware of what’s happening.

That company with the CEO who refuses to listen to employees: everyone knows about it. Even if they are unable to fix the root cause, they will build systems around the CEO to make the company work as effectively as possible.

Being unaware of the problem isn’t the problem, so what exactly is?

Organizations try to rely on expertise and proven solutions to solve their problems, Dutch banks roll out the Spotify model, or car companies introduce the Scaled Agile Framework (SAFe), fooled by the delusion that it will solve their problems: it won’t.

The problem is that organizations are complex systems that are non-deterministic. This means that the future state of a system isn’t fully predictable from the initial state, which makes fixing a system incredibly hard especially if you try to solve problems by copying others.

What works for one company might not work for you. In fact, what worked for that one company might not work for that same company if they tried to roll it out again.

This sounds pretty abstract, so let’s make it more concrete by looking at ecosystems, which are complex organizations.

What Can We Learn From Nature – Complex Ecosystems

In Yellowstone National Park, they reintroduced Wolves, an apex predator sitting comfortably at the top of the food chain. Scientists were utterly baffled and surprised by what happened after the wolves began roaming the park:

The population of wolves grew and stabilized to around 100

Elk numbers dropped from 17.000 to 4.000

Less elk and them moving around more due to the presence of wolves meant that more trees survived during winter and were able to grow in the riverbank

Due to the presence of more willows and aspen, the beaver population increased

In Ecology, they call this a trophic cascade, which for simplicity’s sake we could visualize as follows:

I want to stress that this may give the impression of a clear causal relationship we could have predicted up front, but this is definitely not the case. If you were to introduce wolves again, it might produce entirely different results. This is exactly what makes changing ecosystems incredibly hard and why simply relying on expertise isn’t enough.

Trying to introducing more wolves is exactly what goes wrong in the realm of Agile and organizational transformations. We see one organization achieve success by introducing wolves (SAFe, Scrum, Spotify model), and then we try to copy the exact same thing to our organizations, and we’re surprised it doesn’t work out in the same way.

Our organization may not have elk, willows, bevers, or the same kind of landscape. But we’re still surprised that introducing wolves to a barren dessert doesn’t have the same impact as in the lush green of Yellowstone Park.

The copy-paste mentality is extremely foolish, but it’s to be expected: we actually made the same mistakes when trying to make changes to ecosystems.

Many countries still have the scar tissues and ecological disasters as a result of our copy-paste mentality to solve problems. The presence of invasive species which completely altered their ecosystems and pushed native species to extinction.

Increasing Sugar Cane Yields: We’re Not as Smart as We Think

Let’s talk about a real-world example: the destruction of sugar cane profits by the sugar cane beetle in Australia.

The sugar cane beetles feasted on the sugar cane plant.

Less sugar cane plants to harvest means lower sugar yields.

Less sugar produced means less money earned.

In short, the sugar cane beetle was eating away their profits, and something had to be done. What do you think they did to protect their sugar cane fields?

In 1932, plant pathologist Arthur Bell attended a conference in Puerto Rico where he learned about the success of introducing the American toad, Bufo marinus, in reducing the population of cane beetles.

In 1935, they introduced 2,400 toads to Australia without performing any studies of the potential impact on the environment or even checking if the toads would eat the beetles. The assumption was that it worked in Puerto Rico, so it should work in Australia too.

What do you think happened? Well, the toad ate anything BUT the sugar cane beetles. Here’s a picture of the toad:

Look at that absolute unit of a toad. The marine toad became such a pest in Australia that it has since been rebranded as the cane toad. The cane toad is a jumping stomach on legs that will eat anything that fits in its mouth.

The cane toad didn’t eat many sugar cane beetles because it preferred to feast on other animals like insects, birds, toads, frogs, lizards, and snakes. To make matters worse, few other animals dare to eat the cane toad because it’s poisonous. Even its tadpole offspring are poisonous.

In fact, the sugar cane toad had so few natural enemies that they started to cannibalize each other. The cannibalization creates an evolutionary pressure for sugar cane toads to become even bigger, to prevent being eaten and being able to eat others. As they increase in size, their mouth becomes bigger, and they will begin to eat even more species.

What can we learn from this story? It’s difficult to predict what exactly will happen when introducing a new species to an ecosystem, just like it’s extremely difficult to predict what happens when introducing a new framework or scaling a framework to an organization.

An excellent example of this inability to predict: who would have thought that sugar cane toads would begin to cannibalize each other?

The moment we believe we can completely predict what’s going to happen and know with absolute certainty a change will have the impact we expect, that’s usually is usually the moment we screw things up completely.

There are countless examples of man-made changes to ecosystems that completely backfired because we assumed we were smarter than we actually were.

This exact same phenomenon is still happening in the world of Agile transformations: systems are more complex than the simple copy-paste solutions we attempt to roll out in hope of solving all of our problems.

Changing Organization, Like Changing Ecosystems, Is Incredibly Difficult

We’re currently in the sugar cane beetle era of Agile, which I’ve dubbed the copy-paste era of Agile. We’re busy copy-pasting solutions that worked in one context across many companies worldwide without much regard for the specifics of their context.

And then we’re surprised it doesn’t work, while we shouldn’t be if we understood complex systems. What works for one organization may not necessarily work for your organization. This doesn’t mean you don’t need expertise—you still do—but what’s more important than expertise is discovering what really works in your context.

If you ask experts about the best exercises to become fitter, they will tell you what works best. But the most important thing about exercise is the following:

The exercise you’ll do and keep up is the best exercise

You must start where organizations are and slowly work from there to discover what works. Once you discover what works, that will influence the next steps. Every step you take helps shape the next steps of your Agile transformation.

I want to stress that expertise still matters, and being aware of patterns that worked in other organizations is important, but the proof of the pudding is in the eating: does your organizational pudding taste great?

When an Agile consultancy company presents a slide deck with all the steps you must perform for the next 12 months, you can be guaranteed one thing: your Agile transformation will fail.

Stop chasing paper victories and keep your boots on the ground to discover what works best in your situation.