Agile development and pluralistic management: why Scrum fails

The Scrum methodology is often introduced because developers want it. On paper, it empowers them—if applied correctly—and developers need decision-making empowerment because development is a creative endeavor.

Of all the different types of people I've known, hackers and painters are among the most alike. What hackers and painters have in common is that they're both makers. Along with composers, architects, and writers, what hackers and painters are trying to do is make good things.
—Paul Graham

On the other hand, there is a shortage of highly skilled developers and companies think claiming to "do agile development" is a low cost option to make themselves more attractive to developers.

So, Scrum is being widely adopted and quickly becoming the de-facto standard methodology for software development.

A Forrester study from 2009 observed that 35% of organizations used agile. Actuation Consulting’s 2013 research shows that 73.68% have adopted agile to develop products. In comparison, only 12.5% of respondents use waterfall.

Unfortunately, the sad truth is that around 70% of agile adoptions fail. This happens because Scrum, and agile, are much more complex to adopt than they appear to be.

Ken Schwaber [...] suggests only 30% [of scrum-adopting companies] will become “excellent development organizations.” [...] Change experts like Harvard Professor John Kotter regularly say 70% of major change efforts fail.

High with the fumes of Dunning-Kruger, companies initially think adoption is easy enough. However, as Scrum is introduced, it becomes clear that the whole company has to embrace the agile principles, including sales and management, for the adoption to succeed in the long run, and many companies find they get more than the bargained for and go in denial. They start to blame Scrum for their own faults, and abandon it or start some weird chimera sometimes called "ScrumBut".

Instead, the correct way forward is to change the company management style and become a pluralistic organization, but it's a big step for most CEOs and management, mostly in terms of ego and self image, as well as fear of letting go and change resilience.

Stack Exchange's org chart

Stop thinking of the management team at the top of the organization. Start thinking of the software developers, the designers, the product managers, and the front line sales people as the top of the organization. The “management team” isn’t the “decision making” team. It’s a support function. You may want to call them administration instead of management, which will keep them from getting too big for their britches.
Joel Spolsky

There appear to be no shortcuts: until high-tech companies become pluralistic workplaces, they will not be good places to work for highly skilled, highly creative professionals such as software developers.

TL;DR:

  1. top-down management generally sucks for software development because building software is a creative act which requires diffuse responsibility to be done effectively;
  2. scrum (or, better, agile principles) are good tools to diffuse responsibility without creating anarchy;
  3. CEOs don't want to give away control and become a support figure to the company, which makes it impossible for their companies to become pluralistic, and so they keep on being ineffective.

Agile and scrum are great tools, but the context is much larger than a methodology.


Do you think it would be cool to work in a pluralistic company with Joel as a CEO? Come work at Stack Exchange, we are hiring full-stack developers anywhere in the world!

discuss this post on Hacker News


Hi, I'm Marco Cecconi. I am the founder of Intelligent Hack, developer, hacker, blogger, conference lecturer. Bio: ex Stack Overflow core team, ex Toptal EM.

Read more

Newest Posts

Can you migrate a company from on-premise to remote-only?

Some lessons learned over the past 8 years of remote work in some of the best remote companies on the planet

Read more
Announcing Intelligent Cache, our caching library

Our newest open source initiative, intelligent cache, is available for use

Read more
Guest Blog: The mythical 10x programmer by Antirez

In this post, Salvatore Sanfilippo puts together a list of qualities that I believe make the most difference in programmers’ productivity.

Read more
Team EMEA

Today I want to introduce our second engineering team: Team EMEA

Read more
Guest Blog: Technology and Storytelling by Orf Quarenghi

LEGO star-ships aren't made to collect dust on a shelf, but to explore strange new worlds

Read more

Gleanings

No, I Don’t Want to Subscribe to Your Newsletter
The Bureau of Programming • Feb 25, 2017

No, I don’t want to subscribe to your newsletter. No, I don’t want to complete a short survey. And no, I don’t want to become a member.

Read more…