3/5/2020 by jdabulis

Story Points Post-Agile?

Agile Methodology Adoption

More than 70% of organizations have adopted an Agile methodology, though certainly some fraction of these are surely doing Agile the "wrong" way, resulting in processes worse than Waterfall (if that's possible). While there are more ways for Agile to be done wrong than right, in fairness, using traditional methods the correct way is often the wrong thing to do!

Since Agile has been widely adopted, there has been a 50% increase in time to market delivery. But it's too easy to credit all these gains to Agile. You would have to ignore all the improvements in deployment and development tools, better communication between developers, and better QA techniques.

Project management is about planning the total time and expense required to deliver a project of a given scope, and then tracking progress to determine whether goals are at risk. Without specific deliverables, and without the goal of meeting a specific timeline or schedule under definite budget constraints, whatever is being worked cannot really be said to be a project; it's more aptly termed an experiment, or even a boondoggle. Being able to correctly estimate how much a project is going to cost, and how long it will take to complete is what enables project planning and management.

While there are many differences between Agile and traditionally managed projects in terms of doctrine and practice, the key insight of Agile is that it is iterative. It is the practice of having frequent cycles of production, testing, and review that reduces the risk of problems getting out of hand. From that perspective, Agile project management is the science of how to split a large, unwieldy project into a hierarchy of smaller units-- epics, features, stories, and/or tasks.

The infrequent milestones offered in traditional methods serve an analogous function. Missing a milestone provides important information, but many think it merely confirms that the process itself leads to missing milestones.  Using an iterative methodology identifies problems sooner while they are small rather than later when they threaten the success of the project.  The more granular the unit of work, the more accurate the estimate.

Scrum In Particular

The Agile approach in general, and Scrum in particular, also offers a more subtle benefit. As business objectives are organized into epics, feature, etc. as mentioned, in the course of time, product owners learn more as a consequence of having viewed working fail-fast prototypes during Sprint Review meetings, they begin to revise their original vision in ways which can accumulate over time to change the course of a product in positive, market-responsive ways rather than being rigidly locked into a long term plan of ever decreasing value.

The plan can be revised over time-- removing a feature here, inserting one later there-- adjusting priorities and other such course corrections and deviations from the plan originally mapped out on Kick Off day. In short, an Agile method is agile enough to survive changes in scope and requirements. Traditional (i.e. non-iterative) methods tolerate these kinds of changes very poorly. This flexibility cuts both ways, of course; the cutting felt most acutely upon accounting and financial metrics computation.

A traditional (non-Agile) project plan, produces a Gantt chart or some other schedule measured in days and weeks. It has milestones and a date on which the project is supposed to end. This makes possible attempts to compute the total cost of the project in advance. What-if scenarios (regressions) can be run, holding the scope, the schedule, or the cost constant, and analyzing the impact of varying these parameters. In retrospect, at the end of a project total time and monetary investment will be known.  In theory, we should be able to judge whether an idea will make sense from a return-on-investment basis before significant development resources are spent.  

The inability of traditional methods to accomplish this goal, both in theory and in practice, has been a vexing problem for decades, and so the true cost of transforming the culture of an organization to embrace and not merely tolerate Agile methods was an important reason for the conversion of 70% of organizations to these methods.

The Sprint Schedule

When we look at Agile project plans, we see a sequence of fixed-length iterations (Sprints) which more or less correspond to the days, weeks, and months of the ordinary calendar. Contrast this with how a traditional project is divided. These are typically divided into phases, where a phase corresponds to some set of functionality, ending in a milestone. A phase is completed and its milestone deadline is met or it is not.  One reason traditional methods fail is that managers place unwarranted confidence in these target dates which are difficult or impossible to predict, leading to a cascade of subsequent missed targets down the line.

Traditional project plans manage to combine the disastrous practices of planning small tasks to be completed down to the resolution of days (too finely grained), while simultaneously dividing entire the project into functionality-driven phases, each separated from each other by months.

While fine-grained predictions-- those specified to the day or even week-- can often be unrealistic, setting goals to be accomplished after a month or six-week period are too opaque or even useless. A two-week period, has been empirically discovered by many to strike the right balance between these two less effective extremes.

Measuring Effort and Productivity

In traditional project management, estimating and measuring productivity is straightforward, if not very accurate. The number of "man hours" is estimated for each task with or without input from the developers. Sometimes there are hard deadlines based on SLAs or other business commitments. Typically though, deadlines are set for developers assigned tasks based on their own time-to-complete estimates. Future man-hour estimates can be adjusted based on the accuracy of previous results. For example, if a task was budgeted for 16 hours but took 32 hours to complete, the estimate for similar incomplete/future tasks might need to be doubled.

To the Traditionalist, among the strangest tasting of the unnatural ingredients listed in the Agile Happy Meal is Story Points.  When we look at the effort estimates according to Agile practice, we see the labor required to produce a User Story estimated in terms of Points instead of hours. This is called the "size" of the story. Typically only Fibonacci numbers (e.g. 1,2,3,5,8,...) are valid for use as size values. During a "grooming" meeting, a democratic process is used by the team to arrive at a consensus for the size of each story. It is the goal of a Planning meeting to estimate and commit to completing a set of Stories. For the first Sprint, deciding on the total number of Points that can be completed during the Sprint (the "velocity") is guesswork.

However, estimating the amount of work (i.e. the total number of Points) the team can produce during a Sprint is supposed to improve over time.  For example, if a team is accustomed to completing, say, 20 story Points per Sprint, the team should select a combination of stories whose Points estimates sum to about 20.

Points-Hours Equivalence

Since at the end of any project where time is being tracked against project items, anyone can determine the total amount of time that was spent on the project.  Similarly, the total number of Story Points could be added up. Dividing the total number of hours by the total number of points, gives the number of hours, on average, that each Point represented (in hindsight).

As best I can tell, the only difference, and potential advantage featured by Agile here is that points increase according to the Fibonacci series instead of linearly:

Story points integrates the insight that the time it takes to complete a task increases exponentially as its scope and complexity increases. And if it's true that the time needed to complete a set of tasks (i.e. a story) increases exponentially instead of linearly as the complexity increases, it follows that a points estimate will be more accurate than an estimate given in linear time. Of course, there is no reason in principle why project managers and developers who are aware of this phenomenon can't heuristically apply it when making time-based estimates.

Although some Scrum practitioners claim that with experience, it is easier to estimate a Story in terms of Points, there can be some hidden pitfalls as well. Here are a few:

Changing the Inter-sprint Points Scale

This can happen when the team decides that the maximum story point value will be different than that of a previous Sprint. Suppose, for example that last Sprint the most complex story was assigned the value 5. Suppose further that in the current Sprint, the most complex story will be assigned a value of 8 points. This can happen because points, unlike days and hours are relative values. For example, if five different story sizes can be identified this Sprint, then the possible point values are 1,2,3,5 and 8. The simplest story will get 1 point while the most complex will be 8 points. But if the most complex story this Sprint is judged to be about same complexity as the most complex story from last Sprint, how now are we to compare or even establish Sprint velocity over time?

Pointing by Convention and Inflation

It may be tempting to assign a certain point value to all stories of a given conceptual "type" in a perfunctory manner without noticing either that the team has become more efficient over time in handling that type of story, or by contrast, that there is something distinctive about the story which should cause it to have a higher point value than "normal" were the requirements to be more careful considered.  And yes, a team can inflate story points to increase its sad velocity. Inflating points can be less noticeable than inflating hours.

External Dependencies

If a team will be depending on another team or a third-party for the resources it will need to complete a story, then this "complexity" having been identified, the points assigned are intended to capture idle time waiting for others or something beyond the team's control, not really complexity; there is a chance this story will be blocked waiting for some external product or artifact.  There is no way to guarantee that the story can be completed during the Sprint.

Specialized Skills, Knowledge, Expertise

It shouldn't be surprising that some team members will possess special skills, knowledge and expertise that no one else on the team has.  Or they may simply know managers and technical points of contact on other teams or outside vendors with which the team must interface to complete a task.  It seems obvious that this fact should be exploited to benefit the project instead of ignored or diminished.

With Scrum however, it is actually considered most beneficial if all teams members are interchangeable resources such that any team member is capable of completing any task (generalists).  But we know that is never the case because each team member brings with them certain strengths and weaknesses; certain interests and preferences. Not all testers are also developers for example, nor might they want to be.  Not all developers are equally skilled and efficient across all problem domains or with all technologies. The teams themselves are subject to lose or gain various members voluntarily or otherwise. So we find that in practice, if certain tasks need to be accomplished in anything approaching an efficient manner, then these tasks will be performed exclusively by certain individuals known well in advance.

Analyzing the advantages and disadvantages of organizing people into teams, and organizing units of work into user stories instead of traditional systems requirements is probably worthy of its own article.  Much could be written about whether team members should be doing only what love and are good at, instead of forcing them to do a bit of everything as an interchangeable member of an organizational collective.  Something should be mentioned about the value of worker morale to a company and which project management practices elevate or destroy it.  But all this is beyond the scope of this article....

Because the abstract point values assigned to stories without the context of who will be working on them ignores some realities of human nature and common-sense they will not address the problems of task estimation much better than saying time-based estimates. All else equal, the ability to say "this is 5 point story" isn't magically infallible relative to saying "it should take someone 15 hours to do that."

A Post-Agile Approach

There is a way to combine the clearly advantageous and empirical practices of Agile with rational and intuitive elements of traditional methods without fundamental reorganization and disruption, or resorting to revolutionary or contrived metrics.

Some non-negotiable insights of Agile are adopted such as:

  • Iterative cycles of fixed duration

  • Customer collaboration over contract negotiations

  • Frequent releases of working software

These can be made compatible with some concepts or priorities of traditional management such as:

  • Phases with predefined objectives

  • Definite schedule and budget

  • Skills specialization and individualism

The "Lognosys Accelerated Development Process" represents our approach to this middle-way. It is appropriate and applicable in many startup and small business applications and scenarios. The result is a manageable project or set of projects. Each of these small projects is of small, medium, or large size, which predetermines the duration of the first two Phases of the project.  The first phase is the Rapid Prototype Phase which is further divided into either 2, 3, or 4 two-week Kanban pseudo-iterations based on project size. This phase is characterized by frequent customer collaboration, welcoming requirements refinements and changes, and demonstrations of development progress. Development tasks are distributed among team members in a manner similar to a manufacturing production line which achieves superior efficiency through the repetition of specialized tasks using specialized tools and technologies.

The second phase is the Refinement Phase which has the same duration as the first phase. This is also a development and testing phases during which an application is transformed from a "rough" working prototype or proof of concept into a market quality product. All the individual components are thoroughly tested against the most unusual and exacting use cases to ensure everything works together, everything looks good, and everything is configured and released to production environments.

The third and final phase is a Support and Maintenance Phase designed to provide continuous issue free uptime over time, and effect minor low-risk improvements.

© Lognosys LLC. All rights reserved.