Large User Story vs Human Psychology

Much have been written about the problems of oversized user stories. They are commonly indicative of too much uncertainty, usually over business requirements, but sometimes over certain technical aspects. While large stories (or epics) are acceptable during a high-level planning stage, they should be either simplified or broken down by the time they are scheduled within an iteration. Otherwise, they usually end up greatly exceeding their (very inaccurate) estimates, or even becoming blocked.

As a software developer, I've been involved with a few such stories recently. In addition to the above-mentioned problems, I noticed another adverse side-effect. With each passing day on the same story, I became increasingly less motivated to observe good coding practices such as refactoring, cleaning up messy code and proper OO design. As the days dragged on, I became more tolerant of sloppy practices. I eventually finished the story and handed over a piece of spaghetti code that I wasn't proud of. In hindsight, I could have done a lot more to improve code quality.

Thinking back, I realized that this same behaviour manifested itself many times before in the past. Perhaps, I might just assume that I was a lazy bum, if not for the fact that I was practising pair-programming, and the same tendency was also apparent in my pairing partners (some more resistant than others). Maybe it was just the long hours and looming deadlines. However, my experience tells me that doing multiple smaller consecutive stories, while they may add up to the same volume of work as a giant story, does not lead to this behaviour.

Image from: weber.edu

This leads me to speculate that there is a psychological explanation. Research has shown that the human brain has two regions that compete to form a decision when presented with a choice between a small short-term reward and a longer-term goal (larger reward). The more emotionally driven region of the brain seeks the short-term reward while the rational region tells us to hold-out for the overall better outcome. Our brain battles itself to a decision. As professional software developers, we constantly have to make a similar decision :- should we take a short-cut to deliver a piece of work now, or refactor/write more tests/clean up so that the overall code quality improves in the long run?

It is likely that by breaking up a large body of work into many small discrete stories, the completion of each story provides an emotionally satisfying reward in a short period of time, even if extra effort/time has been added to ensure good quality. This extra effort/time is usually at least proportional to the original estimated size of the story, but may cost even more if the story gets so large that it touches many interdependent parts of a complex system.

In other words, a small story usually touches a small area of the code base, hence its parts are easier to visualize in one's head, and thus presents only a small mental barrier to keeping it clean. Conversely, a massive story that encroaches on a myriad of different moving parts becomes hard to visualize. Soon, problem areas stack up beyond our ability to keep track of them in our heads. On top of that, the uncertainties inherent in a large story may reveal themselves to be time-sinks. Before long, the extra effort required to deliver quality becomes a mountain of hurdles. Having spent most of our time and energy just getting to code to work, with no psychological reward from completing anything yet, it is no wonder that we become disinclined to continue.

In conclusion, large user stories work against human psychology for the desired long-term goal of good quality code. Small stories are more satisfying due to frequent emotional rewards, at the same time contributing towards the greater goal.

It is not always possible to foresee how big a story can become at the start, therefore, developers should be given the freedom to split up stories while they are being developed. Personally, I think a story should not take more than 2 to 3 days. If it looks bigger than this, split it. This I endeavor to do more of in the future, with the hope of always delivering code I can be proud of.