Why Scrum doesn't work, and what to do instead

How to manage software projects

2025.07

From a business perspective, the software development process seems chaotic and unreliable. Business owners want to tame this chaos and have some level of control over it, so they look to adopt some process. Many fall into the trap of adopting Scrum, based on its apparent wide adoption by many companies across the industry.

Unfortunately, it does not -and cannot- work, as we'll see.

Overview of Scrum

Scrum is best understood not as a software development process, but as a meta-process, or a framework for structuring the interactions between business and engineering, in a way that gives the business side a (false) sense of control.

Scrum may appear like a proper process because it defines roles and ceremonies: The Scrum Master (SM), The Product Owner (PO), The Backlog, The Daily Scrum, The Retrospective, etc.

But it has nothing to say about how software should be planned or built! How does the engineering team make decisions? How do they divide work amongst themselves? How do they ensure the quality of their output? How to plan the project? How to set reasonable milestones?

These are the actual questions you need to answer if you are suffering from chaotic and unpredictable development process, but Scrum actively avoids these questions. Indeed, it assumes you already have that part figured out.

Scrum Teams are cross-functional, meaning the members have all the skills necessary to create value each Sprint. They are also self-managing, meaning they internally decide who does what, when, and how. -- The Scrum Guide

Instead, the focus is more on the boundaries of communication between business and engineering.

  1. Product Owner decides what should be built
  2. Developers produce Increments in Sprints
  3. The process is reviewed and improved

The structure prevents engineering from collaborating with business, instead, a Product Owner is put in place as a mediator.

  1. A Product Owner orders the work for a complex problem into a Product Backlog. -- The Scrum Guide

Scrum's very first step is already a mistake: it assumes you can have one non-technical person competently break down the project into a series of tickets.

Yes, business side decides what they need to develop next based on business considerations, but engineering considerations should also be taken into account. For example, business might want feature A this quarter, and feature B next quarter, but engineering might suggest that it would be easier to produce feature B first, or that some variation Ax would be much easier to achieve on schedule than A.

This kind of information is valuable for business, and will have some influence over their priorities.

Creating an effective development plan (backlog items) for bringing A into fruition might not be feasible at all without deeply understanding the engineering constraints.

For instance, it often makes sense to develop several pages in tandem, creating a skeleton version first, and only filling in the details after ensuring the basic flow works end-to-end.

However, inexperienced POs tend to structure the Backlog such that each page must be implemented fully before moving to the next one.

In addition, there's often some non-trivial amount of foundation and planning that needs to be done before work any visible "increment" can take place, but the structure of Scrum makes that difficult to implement in a reasonable and effective manner.

Now, Scrum does make room for engineers to give input via "refinement" sessions, but only after the PO has already decided the high-level structure, and developer input is restricted to low-level decomposition work, as suggesting radical changes to the high-level structure would create conflict that most developers would shy away from.

In a healthy organization, there would be a tech lead who would work together with the product owner to discuss these matters in depth before coming up with a development plan that satisfies both business and engineering.

Unfortunately, Scrum makes no place for engineering leadership.

The fundamental unit of Scrum is a small team of people, a Scrum Team. The Scrum Team consists of one Scrum Master, one Product Owner, and Developers. Within a Scrum Team, there are no sub-teams or hierarchies. It is a cohesive unit of professionals focused on one objective at a time, the Product Goal. -- The Scrum Guide

A well functioning team needs hierarchy, with leadership roles clearly defined, responsibilities assigned individually, and incentives aligned.

This is an essential part of the human condition, and applies to any group of people; not just engineers.

Now, it is true that leadership can emerge organically, but this assumes a coherent, well functioning team, and even such a team would benefit from having a formal position with authority to make - and own - the technical decisions.

So not only does Scrum not help tame the chaos, it can even hinder the development process by creating unreasonable structures and boundaries.

The non-agility of the Sprint

Scrum requires programmers to commit to completing a fixed amount of work in a fixed amount of time. This is the heart of Scrum. It appeals to business because it promises consistent progress.

The flip side of fixing both time and scope is this: when you inevitably fall short of the stated goal (even by 10%) it’s considered a failure.

When a Sprint fails, the Retrospective turns into a struggle session, where the PO and the SM remind the programmers that they failed the Sprint, have them "acknowledge" the failure, look for causes, and "repent" by creating "process improvements".

Scrum effectively demands programers to make sharply accurate estimates, then punishes them when the estimates turn out to be inaccurate.

This is where Scrum gives business people the illusion of control: it promises them that they can create an accurate long-term schedule, based on estimating the amount of work that can be completed in each sprint.

This is the very opposite of agile.

Effective agile teams do not fix both time and scope; one (or even both) must remain flexible. What matters is not adherance to a fixed schedule, but steady forward progress.

A predator chasing a prey would not stop just because the chase was a little bit harder than originally estimated; that would be the very opposite of agility.

Misaligned incentives

We've seen several instances where the Scrum methodology creates conflicts of interests and misaligned incentives.

What to do instead

Agile software planning should operate on two levels: long term, and short term.

On the long term level, you have broad topics, such as "authentication flow", "real time chat", "collaborative document editing".

You don't bother creating detailed plans for any of them yet.

On the short term, you are working on one specific topic. You break it down into milestones that can be designed, implemented, and shipped in a reasonable amount of time (e.g. a few weeks). There need not be a fixed time distance between the milestones.

This is not a one-man effort. Arriving at a reasonable plan requires collaboration between technical leadership and business stakeholders.

Before starting to write code, some architectural and design decisions need to be made, even if in broad sketches. If different engineers are to work on different components in tandem, interfaces between these components need to be determined upfront, loosely.

Think about the process of tightening screws: you don't tighten one screw all the way before moving to the next. Instead, you put them all loosely at first, then gradually tighten them in rounds: never tightening one screw too much while the others are loose.

Features should be implemented incrementally: starting with the basic version, and iteratively adding more and more details as we go. This applies to all aspects of development, from visual design, to data flow modelling, to user interactions, etc.

Business side should be kept in the loop, with basic demo sessions given every step along the way. This enables early feedback and course correction, and is cruicial to make the business side feel like they have control over the project's progress.

Finally, here is a rough outline of a possible way to organize engineering roles:

Closing words

You cannot use process to compensate for incompetence. Companies often adopt Scrum as a way to impose structure on a disorganized, incoherent development team, built by hiring for familiarity with the tech stack, rather than for technical excellence and value alignment.

When that is the case, what you need is strong technical leadership, not a rigid framework.

The ideas outlined here are not part of any named methodology that I know of, they are just the obvious way to develop software, and I've seen variations on it practiced informally in different companies.

To summarize the core ideas: