Why don’t my Engineers stick to their estimates?

Estimation is a big topic and there are many posts to write on it, but first let’s address one confusion about Software Engineering estimates:

Estimates are not delivery commitments

Maybe that sounds daft or maybe it’s obvious but far too many companies that I talk to are asking their team to estimate in hours, make a deadline at the sum of those hours and then hold the team to that deadline. Guess what – these deadlines get missed. Pretty much always.

noun: estimation; plural noun: estimations

1.
a rough calculation of the value, number, quantity, or extent of something.

Planning

Expectations of delivery date come from a plan, not from an estimate. Plans take in to account availability, team makeup, blockers and dependencies. Estimates do not. Creating a realistic plan can be an art form but it is essential to convert estimates to a timeline that the team and company can believe in. If this process is collaborative or otherwise achieves team buy-in then you have something that people can commit to delivering.

And if your team has committed to a realistic deadline then you have a much better chance that the company can rely on it’s delivery date.


In a future post we will discuss how to get a good estimate quickly and how abstractions like Story Points can lead to better plans.

How can we be agile but still have a plan?

This comes up a lot – usually because a CEO complains “My engineers say it will be ready when it’s ready – they can’t give me a date” or a Project Manager may be concerned that “With time, functionality or quality you can have only 2 – but we won’t flex on quality or functionality and the board need this feature on time”. Clearly these are valid concerns and there is no magic bullet but this is usually as a result of misunderstanding the agile process or being more familiar with a waterfall process of project defiition.

So how can agility be maintained within the scope of a plan? By realising what it is that the team is actually building! As the agile process is about delivering value quickly rather than specific features we can see that a feature list or requirements document paired with a deadline is missing the point of the process.

The outcome of each iteration within an agile project should be a releasable set of functionality that works towards the desired value. Some times the value can be realised in a single iteration (remember to celebrate even the small successes) but most of the time projects will be much larger and contain much uncertainty. When this is the case how can we promise release dates or promise to deliver certain improvements for a deadline? Estimation.

The process of estimation is important for understanding the scope of work as well as when it could be delivered by. A good estimation meeting will involve members of a team from all areas of specialism (development, testing, design etc) and should have people faimiliar with all the areas of the product that could be effected. Whilst estimation is based on completely abstract concepts (story points or t-shirt sizes) these mean something to the team and can be used to understand likely completion dates. A consistent team that is adept at estimating complexity by digging into the details can be relied upon to deliver, on average, a consistent amount of work in each increment. This is the baseline for a realistic agile based plan. The other key element is to continually be reviewing the upcoming work (backlog) to answer questions or explore potential solutions. Bringing these together will create a reliable and sustainable environment for planning.

But I want to plan further out – not just a couple of sprints… Fair enough, there is a business to run after all, and we need a long term plan as well. Here the key is to realise how many unknowns and how much complexity there is in each project as it goes through the process. If you have an innovative environment then the team will never be solving the same problems twice so this can seem like an impossibility. But let’s come back to the concept of delivering value. The plan is to create a new area of functionality for your product by a certain date. Great. Share the high level requirements, let the team explore the approaches and possible features that could deliver this value. With that in mind they can estimate what a complete solution might involve and give an educated estimate (/guess) of delivery date and start the work to incrementally deliver it all.

Of course the overall benefit of working incrementally toward a hypothetical complete solution is that there will be various releases possible along the way. If the team is good and you have articulated the desired value it’s entirely possible that you could either get something that is delivering this value earlier than expected – or that you learn that the envisaged solution was not actually the best way to deliver the value to your customers.

With these possible benefits and the overall improvement in communication that an agile approach requires how can you long for the days of a step by step project plan that becomes out of date at the first unexpected complication?