Strike Teams Part 2

This is a follow up to the original strike team post I did a while back. I’m writing this as the strike team is wrapping up. It’s been an interesting experience. We hit most of our broad goals but did it in a significantly different way than anticipated.

The first big change came early on when we were looking into the proposed changes to an existing piece of software that was going to be a consumer of what we were building. We realized that accommodating it was probably weeks of work by itself. The initial assumption of how much work was needed had been minimal, but apparently that particular solution was untenable to the people who supported the consuming software in the long term. This ended up with them pulling support from the strike team and becoming uninvolved. At the time, the scope and resourcing change was challenging since it threw everything into an uncertain state. I think it ended up being a good thing; we were able to focus on building out the functionality desired without spending a lot of time making changes to one consumer of the service. It does make the follow up more complex since we do need to make sure that work gets done otherwise the whole system will end up in a more complex state.

There was one portion of what we were going to deliver that had a complex deployment step. There were other changes that had to go either before or after this particular piece so we tried to move that deployment as far forward on the project plan as possible. The intention was to deliver it at the end of the first sprint. We encountered some significant scope changes that all unfortunately all had to go before that step. This ended up pushing the actual deployment out until the middle of the third sprint. At this point we had about ten tickets that we did the work for all sitting in this feature branch just waiting for the eventual opportunity to merge the whole thing back to master with nearly 5000 lines of changes. We ended up performing a series of messy merges bringing changes from master back to the feature branch. The final merge from the feature branch back to master was ugly since we wanted to preserve some but not all of the history of that branch. In retrospect we should have seen the merge issues coming and been more proactive about doing more small merges, but lesson learned for later.

Every team member lost some time to dealing with issues brought to them from the team they were on loan from. We had anticipated this during the first sprint but didn’t expect it to continue through the remainder of the strike team. I know I personally got sucked into some production oddities that were occurring and that after the first sprint without me my team missed my presence in code review to the point that they got me back to reviewing basically everything. Both of the other members of the team got pulled back to the their original team for issues that happened. We didn’t fight any of these needs figuring that since they seemed like critical issues for those teams, having their people available was the best overall usage of time, even if it hurt our overall ability to get things done on the strike team.

The team as a whole never really jelled to any significant degree. Everyone kept working the way their team did, which created some minor conflicts. One engineer’s team has a very lax overall structure and he would sort of disappear for days then show up with large amounts of excellent code, but it was frustrating for him to essentially go off the grid, without forewarning. The other came from a team with experience in a different technical stack and vacillated between asking not enough questions and spending too much time stuck and asking too many questions. The three of us were also in three different locations which made it difficult to really all get on the same page and get working together.

Overall the strike team model worked for this, since having representatives from some of the teams that were going to consume this service working on it made sure that we didn’t run into any issues with a misunderstanding of the domain. There were problems setting up a new team, that we should have attacked proactively since setting up any new team needs to initially get organized and form their own norms. The transient nature of the strike team prohibits a lot of identity building which, in my opinion is key to building a good team. Overall based on this experience I think that the strike team model can deal with software projects crossing multiple software team boundaries, but there may be other better ways out there to be found still.

Advertisements

Strike Teams

At work there has been a new practice of starting up strike teams for different projects. The idea is that for projects that require expertise not found on any individual team, you pull in a person or two from multiple different teams to get all of the correct skills on a single team. That’s the pitch of the strike team model, but the hidden downside is that it breaks up the cohesion of the teams that people are pulled from and the created team may not be together long enough to create new cohesion. This post is mostly going to be a chronicle of the issues that I encountered starting up a strike team and what we did to try to resolve the problems.

The first problem was coordinating the various different teams to figure out who was going to be involved. In the case of the strike team I was forming, the two teams contributing resources both wanted to know who the other team was going to contribute before making their own decisions. They also wanted to have a fixed end date to the project before deciding who to contribute. We ended up resolving this issue with a fixed date for whoever they contributed to the team and getting the two of them to discuss the situation amongst themselves.

The second problem was aligning the start date of the strike team. The three teams contributing resources all have different schedules to their individual sprints so it makes coordinating a ‘start’ date for the strike team difficult.  Those who were on teams about to start another sprint wanted the strike team to start now, whereas those with other commitments made weren’t available. We ended up doing a rolling start where as each team finished their sprint they rolled onto the team and the team ramps up as people become available. We did some preparatory work to get everyone up to speed on the goals and challenges of the team so they were aware of what’s going on whenever they were able to join.

The third big problem is more specific to our particular organization, not to the strike team model. As part of setting up the strike team we needed to schedule things like standups and retros for the team. Since the team is split across both coasts, the hours for scheduling these meetings are limited. The conference rooms are also pretty much all booked because every other team beat us to scheduling. We ended up asking IT to rearrange some other non-recurring meetings and managed to get a consistent slot for the standups. The retro slot was more complicated but we managed to get meetings roughly spaced; by not being a stickler for strict week deliminations we managed to find times that worked.

So with all the overhead sorted out we finally get to move on to the real work of the situation which will be a nice change of pace from the administrative aspect.

Book Chat: Pair Programming Illuminated

My team has been doing more pair programming recently so I picked up a copy of Pair Programming Illuminated. I had never done a significant amount of pair programming before and while I felt I understood the basics, I was hoping to ramp up on some of the nuances of the practice.

It covers why you should be pair programming, convincing management that you should be able to pair program, the physical environment for local pairing, and common social constructs around different kinds of pairs. All of this is useful information, to varying degrees. Since the book was written in 2003, some of the specifics of the physical environment section didn’t age well – advising the use of 17” monitors most obviously. Both of the evangelizing sections seemed to cover the same ground, and did not seem to be written in a way to try and convince someone who is not already open to the concept. Neither section seemed to be written to the person who isn’t already in favor of doing pair programming. There were lots of references to studies, and some personal anecdotes, but none of it stuck in a way that felt like it would change someone’s mind.

The social aspects were interesting, however most of the section was stuff that felt obvious. If you have two introverts working together then they need to work differently than if you have two extroverts working together. A lot of the time the tips were common sense, and didn’t seem like it was necessary to write it down in the book. I would have liked to see more discussion of getting someone to vocalize more and clearly what they’re thinking about.

I feel like I’m better equipped to do pair programming because of having read this, but I also feel like a long blog post would have been just as good a resource and much more focused. I don’t know what else I would have wanted to fill out the rest of the book.

The Systems of Software Engineering

This idea came together from two different sources. First, I’ve been reading The Fifth Discipline about the creation of learning organizations. One of the elements of becoming a learning organization is Systems Thinking, which I had heard of before and seems like a great idea. Then I went to a local meetup about applying Systems Thinking to software development. We ran through a series of systems diagrams from Scaling Lean & Agile Development. These diagrams helped us to both understand Systems Thinking by analyzing a domain we were already familiar with in a structured way where we were presented the nodes of the system diagram and asked to draw connections between them.

There were several different groups each looking at the same sets of nodes and drawing different conclusions for some of the relationships. There were a few very adamant people who all felt that an increase of incentives for the developers would increase the velocity of the team, which was interesting to see since they were mostly scrum masters or software development managers with many years of experience doing the management of software projects and should have seen how that does work out. If the impact of an incentive plan on a team is significantly uncertain, it makes me curious about the sorts of teams these other leaders are running.

Through the entire process we had interesting conversations about how various aspects of the software process were related. Everyone agreed that the interaction of a strong mentoring process on the overall system would decrease the number of low skills developers on the team. There was a discussion of whether it would also directly impact the velocity of the team. Some people were adamant that it would lower velocity since the mentoring time was time that people weren’t working on building features. It’s a reasonable consideration, but it doesn’t seem to match with my specific experiences with mentoring. If you were actively spending enough time to lower your velocity significantly on mentoring activities you would be forcing the mentoring relationship instead of letting it happen organically.

The experience made me wonder if we could construct a system diagram that describes some of the other dysfunctions of many organizations. The diagram we built described (1) dysfunctions around hiring large numbers of low skill developers, (2) using certain kinds of financial incentives, and (3) the push for delivering features above all else. It didn’t describe why a lot of organizations end up in siloed configurations, “not invented here” behavior on technical systems, or lots of the other dysfunctions in multiple organizations. I made the below diagram with the intent of describing  the situation about siloed organizations but without any real experience in this sort of analysis I’m not sure if I’m doing it well.

I made another simple diagram about the causes of “not invented here.”

It feels to me like these diagrams describe the dysfunctions that are common in software organizations. Expanding these diagrams to try and find the leverage points in the system might yield some greater insights into the problems. In spending some time thinking about both of the diagrams I’m not sure what other nodes should be there to further describe the problems.

I’m going to definitely do some more reading on Systems Thinking and try to expand on the thoughts behind these diagrams. If you’ve got more experience with System Thinking I’d love to hear some feedback on these charts.

Theories of Technical Debt

There are a couple of different major causes of technical debt even on the best run projects.

  1. Schedule pressure
  2. Changed requirements
  3. Lack of understanding of the domain

You can choose to take on debt strategically to accommodate an aggressive schedule, you can accumulate debt from having things change, or you can collect debt from doing what appeared to be the right thing but turned out not to be once you learned more about the underlying situation. There are plenty of other reasons that technical debt can be acquired, but most of those can be avoided. Changing requirements can’t really be avoided; things can change, that’s the nature of life. Understanding of the domain is a tricky issue, since you can spend more time upfront to better understand the domain but you will still uncover new aspects as you go.

Schedule pressure is the most demanding of the three. You can consciously say that technical debt should be taken on by doing something the expedient way. You can also have implicit schedule pressure that pervades the decision making process. This sort of pervasive pressure causes people to value different things. If leadership discusses the schedule day in, day out, but doesn’t mention quality, it ends up lacking.

Technical debt is fundamentally a lack of quality; not in the defect sense but in the lack of craftsmanship sense. All of those 5,000 line classes got written by other engineers who were doing the best they could within the constraints of the environment at the time. But some engineers look at that and don’t want to touch it afraid of what it is and how hard it is to change. Some other engineers look at it and see a mountain to be climbed, or a wilderness to be civilized. The problem code is something to be taken and broken to your will. Each kind of engineer has a place in the development lifecycle.

If a company needs to hit a product window and is only full of the kind of engineers who see debt as a challenge to be dealt with they might not be able to make that tradeoff. If you only have the engineers who are concerned with maximum velocity but leave behind chaos in their wake you will have trouble as the codebase matures. Every engineer seems to have a range on the spectrum where they are most comfortable. Where in that range they land day to day seems to be controlled by the environment around them.

If you are surrounded by one side or the other you might lean towards that side of the range. The messages management sends out about the quality of software relative to the messages they send out about schedule of the project is another important factor. If they keep hammering home to get more done, people will find corners they think can get cut. What those corners are will differ between people, but they will take on debt in various corners of the codebase. This sort of debt is really insidious since it was never consciously decided on. If you decide that you will defer good error messages, or avoid building out an abstraction now, but you do it explicitly because of the schedule is the right business choice, then since the team discussed and decided to do it, they know as a whole that’s not the best technical solution but is the best business solution, whereas if someone just does it everyone else may not even be aware of the other options.

Fix vs Replace

I was thinking about when to fix a piece of software versus replace it as part of our normal software lifecycle, prompted by a discussion at work of a particular piece of software. The software in question works great, but nobody is confident that they are successful when changes do need to be made. Part of the lack of confidence is because changes are only made once or twice a year, so getting it set up and running locally and testing it is a chore, plus it turns into a complex process with many chances of failure. The other part is it runs on top of another library that nobody is really familiar with; the library isn’t used anywhere else in our stack, so nobody develops any familiarity with it. This particular piece of software is also critical to expanding our business, as it’s the primary way new client data is loaded into the system.

If we wanted to fix it, we could take the existing solid software and enhance it to make the setup and testing easier, or we could clean up the usage of the underlying library so it’s more intuitive. However, the decision was instead made to replace it with something totally new. I wasn’t involved in the decision, but became involved with the original piece of software after the fact to make a change while the replacement is still being developed. It seems like this software could be rehabilitated at first glance, but clearly someone else thought otherwise.

I know in general I’m biased towards fixing existing software. I’ve spent most of my career working on brownfield applications and building oddly shaped pegs to fit back into the oddly shaped holes of those applications. I think that I’ve done this because I enjoy it; building everything from scratch is almost too easy since there are so many fewer constraints involved. I get a different sort of satisfaction from it. I know I’m not the only one who has this particular tendency; the folks at Corgibytes are specializing in this sort of work. I’ve even been nostalgic for a codebase I’ve worked on – not the application but the codebase itself.

I feel like most organizations are biased towards replacing software because it lets you just say the entire thing is bad and try again instead of having to pick a particular thing to do or change. You don’t have to agree on what’s wrong with it, or get into details of what to change. This flexibility of scope leads to the quid pro quo rewrite where a piece of software is replaced with a new version that also contains a major new feature; this concept was introduced to me by Re-Engineering Legacy Software. Re-Engineering Legacy Software describes this as a bargaining tool to enable you to gain acceptance of a plan to do the rewrite, but I’ve always seen the business bring up the rewrite with the idea that they’re unhappy with the team’s ability to change a piece of software and this would clean up the underlying causes of the problems.

That’s the big problem: the current software has problems due to something. And unless you deal with whatever that “something” is, the new software probably is not going to be significantly better than the old. It may not have had the chance to become crufty yet, so it seems better when it’s new, but given a few years goes back to the same sorts of problems you had the last time. You need normal software processes that enable you to create and maintain quality software even as requirements change.

You need feedback into your initial processes of what caused the cruftiness to accumulate. This can be seen as a form of double-loop learning, where the feedback of what happened impacts how you see the world and thereby influences your decision-making process, not just the decisions themselves. If you are accumulating cruft because you put schedule pressure on the initial development resulting in a less modular design, the feedback to the decision-making process would be different than if you are accumulating cruft because the requirements changed radically. To make true long-term improvements, that’s the step you need to take, which sometimes might lead you to fix, and sometimes might lead you to replace.

Modern Agile

I heard some references to Modern Agile and went to investigate further. To me, it’s just a reimagining of values espoused in the Agile Manifesto in a response to how the original manifesto was misused. The four values of the modern agile movement are:

  • Make People Awesome
    • The “people”: here is those making, using, or otherwise impacted by software
  • Make Safety a Prerequisite
    • Mostly psychological safety, but includes physical safety too if that is a concern
  • Experiment and Learn Rapidly
    • Try to find ways to do whatever they do better
  • Deliver Value Continuously
    • This, to me, means the technical practices around continuous delivery. But could mean other things outside of the web application space.
    • This is also breaking work into smaller individually valuable pieces.

At first I wasn’t that enthusiastic about the idea, and it seemed to be splintering the agile community. Then I started thinking about the recent resignations at the Scrum Alliance, and consider that maybe the community was already splintering and this is just people writing down what had already happened. The people for whom orthodox scrum is agile had already disregarded “individuals and interactions over process and tools.” Both LeSS and SAFe give up on “responding to change over following a plan” to varying degrees.  

The “focus on people” is, to me, the only right way to build systems and products, but the diffusion of who you are supposed to be making “awesome” makes this focus less impactful. If you were trying to empower the makers to do awesome you would do things differently than a single-minded focus on the customer, and some of those things are mutually exclusive. Modern agile examples of focus on the customer from Amazon and Apple come with horror stories on the engineering side of the late nights and insane demands – can you really make all the parties named “awesome” at the same time?.

“Safety as a prerequisite” is the opposing force to the focus on the customer and the negatives it can cause. This can be protecting work-life balance and everything else that makes people able to engage with the work. Another aspect of this safety is an openness to discuss problems in a candid way. I don’t know if I’ve ever seen the sort of openness to admitting problems that this is designed to inspire. I’ve been places where small mistakes were admitted to openly, but it’s unclear what would happen with larger mistakes since as far as I know none happened while I was there. I don’t think I’ve ever seen the sort of transparency from above which would make this level of openness either which makes it difficult to reciprocate.

I think experimentation is the key insight of modern agile. The original manifesto implied that if external stimuli did not change, the team did not need to try to change. This implication resulted in a challenge-response feedback loop and sought a steady state where if things around you didn’t change much you didn’t change. Even then there are a lot of organizations where the right thing to do is whatever is specified by the scrum guide and anything else is wrong. This experimentation adds an additional focus on continuously trying to find a way to do better regardless of how well you are already doing.

I see “delivering value continuously” as a continuation of working software over comprehensive documentation. Working software, from the agile manifesto, is the expression of what is most valuable,” so you could say it described this concept in a narrower sense. The focus on “value” rather than “software” is helpful since sometimes software isn’t the right solution – for instance, if you have an internal API that works but nobody uses because they are having trouble with the API, activities other than writing more software would be more valuable.

If we as practitioners wanted to stop and reconsider how we go about doing work, this seems like a good place to start that conversation. I would have liked to have seen it as a more open discussion rather than appearing fully formed from the head of a consultancy. These ideas seem reasonable, but it’s not clear what the next step is in taking up this mantle. I would always try to be on a team that was doing all of these things, but I don’t know if this alone is enough to make a team that I want to work on.

Thinking Back and Looking Forward

I’m writing this after my last day at my previous job. It has me reflecting a lot on the three years I was there. I learned some interesting things and met a lot of great people. There were aspects of the job I really loved, and there were the aspects of the job that I would love to never think about again. I was reflecting back on my accomplishments there and I was surprised how much I had done in some aspects and how little I had done in others.

One of the memories that stuck out to me was the response to some production performance problems we had in July of 2014. I was new to the organization, which made it my first chance to see how it responded to a crisis. I learned a lot about the organization then. It was great to see how disparate teams came together and worked the problem. It also gave me an opportunity to better understand the organizational structure, good and bad. While investigating, I ended up digging into something that wasn’t the cause of  the problem, but demonstrated numerous other little problems that had been festering. There, I managed to make a big impact with different higher-ups once I showed them all of the hidden problems I dug out. It was especially interesting for me to handle from a workplace navigation perspective, since all of the problems were outside of things my team was nominally responsible for.

An area where I tried to make an impact and spent a lot of effort was the build pipeline. The ‘continuous integration’ that was set up by the configuration management team had numerous problems. The basic workflow was that they would – on request or on a timer – take C# code, compile it, and commit the DLLs back to SVN. When I first saw this, I noted that using source control to store the binary artifacts that were created from the repo was odd. The configuration management team didn’t have any interest in changing this, since the entire deployment process was based around deploying DLLs from SVN; it’s understandable they were resistant since changing this piece would require changing everything. This limited a lot of my options on how to change the build pipeline without getting involved in deployment as well. I spent a long time trying to convince the configuration management team to run tests before committing back to SVN, but the server they were building on failed a number of the tests when they tried that. It turned out that the team that owned the code under test didn’t even know those tests existed. This led into an effort to find all of the tests in the whole system and get them passing again. We made progress on these goals but there ended up being a functional area nobody owned, and the rest of my team didn’t feel like we had the capacity to take ownership of it. This left the effort in a sort of stalemate: progress had been made and we could run a lot of tests, but there couldn’t be a clean build unless we started making exceptions. We held in limbo for months, where management continued to debate  who owned that functional area. This eventually led to the effort dying off. That was a massive blow to my morale, since to me it implied a lack of focus on quality; that wasn’t at all their intent, but that doesn’t reduce the frustration after I had gotten so far and made so much progress. I recognize that I may have taken it too personally or read motivation into it on management’s part that wasn’t there, which is an important lesson I’m taking away. I also learned a great deal about how to navigate the corporate environment and how to use soft power to influence change, but most of that was too late to make this change successful; now I know better how to approach this type of major change management from the start.

I don’t want to give the impression that everything was bad – a lot of aspects of the work were great and interesting. I had a great experience working with a system that is much larger than anything I had worked with before. The whole system was hundreds of times larger in terms of computational resources than anything I had worked on before. The codebase itself was approaching ten million lines of code across 10+ different programming languages. Digging into all of this was an amazing adventure to understand how it all went together. Very few people were looking at the system as a whole and I got to come in, learn and understand all of this and help those who were trying to build other pieces of it.

I also had the opportunity while there to mentor some junior engineers and it was quite rewarding. I helped them get up to speed on the code base, which is an easy enough activity. But I also spent a great deal of time working with them on code reviews and helping them understand how to build better software. Helping them to understand the difference between something that works and something that is truly good was really rewarding as they started to see the difference.

My new job is a much different situation than my last job. At the last job I was there to help them clean up the technical mess they knew they had; their issues are common to large organization that have grown through acquisition, and I do appreciate that they wanted to improve. The technical mess wasn’t the whole problem; as you can see from these two anecdotes above there were multiple groups who weren’t in sync and a lack of understanding of each other. The new place is much smaller so it’s unlikely that I’ll have to navigate as many different competing groups, or be assigned such a narrow portfolio. My new company is also much newer so there hasn’t been the time to build up the layers of technical debt. From a technical perspective, at the new job I hope to master a new language and stack and build out some RESTful web services. I’d also like to help build a high performing team and mentor some junior developers, since I found that  really rewarding The person who mentor me to see that difference between “works” and “good” happens to be working at the new place, so that will be a great relationship to renew now that I’ve had an opportunity to play that role. Sometimes the journey is the destination afterall.

Book Chat: Managing Humans

Managing Humans by Michael Lopp, of Rands in Repose, is a humorous take on a management book. It is built as short chapters describing an event or a kind of person and how to deal with or work with them. If you are familiar with his blog there is a lot of overlap between that and the book. However, it fills in the blanks and creates a complete thought about how to deal with common management issues. There is a strong connection to the differences between doing technical work and other kinds of work.

One of the big takeaways I got from the book was the way he described how to give annual reviews. It consists of two parts: separation of the feedback from the outcome (i.e., raises and promotions), and a way to help determine what the employee needs. When I first experienced the separation on the receiving end of the review, I wasn’t a fan, but I can see now that the lack of discussion of outcomes let us focus on a productive discussion. The second part Lopp suggests is to determine what an employee needs using a two-by-two matrix:  if the employee has the skills to do the job, and if they have the desire. High skill low desire individuals need new challenges and responsibilities to re-energize. Low skill high desire individuals need training to get their skills in alignment with their ambitions.

There was another chapter that gave a look at a situation that was similar to something I had experienced recently – a communication breakdown between myself and someone on the team. Lopp describes a way to rebuild communication by going back to a very verbose way of communicating that takes a lot of the implied context out of it, until you get back to a fully trusting arrangement. This was what we ended up doing but it took us a long time to find our way back there. Makes me wish I’d read Lopp’s suggestion a while ago, it would have saved us some time.

Overall, I enjoyed the book. Check out the blog; if you get there and decide you want more of the same check out the book for yourself. He’s also got another book on finding career path as a software engineer that I may check out as well since I enjoyed this one.

Book Chat:Software Estimation

I just finished re-reading Software Estimation: Demystifying the Black Art by Steve McConnell. I was inclined to go back to it after an item at work came in at roughly 500% over our estimate. We thought it would be done two weeks after it started, even with the team only working on it part-time; it wasn’t finished until twelve weeks had passed. A lot has changed in my life and the development world since I read the book the first time, back in 2008. At the time I was doing contracting work, some of which was firm fixed price, so accurate estimates were much more immediately valuable to the company – both in terms of calendar and man-hours. Today I’m doing software product work where estimates aren’t as immediately impactful on the bottom line, but are still necessary to coordinate timing between teams and launch new features.

I wanted to understand how the item had gotten so badly underestimated. With the help of McConnell, I identified a couple of things we did wrong. First, when we broke down the item into its component parts we missed several. Second, we got sidetracked onto other tasks several times while working on the item. Third, even though we had not done similar tasks before we had assumed it would be easy because it was not conceptually difficult. After the fact, each of these points became apparent, but I was looking to understand what we could have done better beforehand.

The three points that caused our delay are interrelated. We missed subtasks during the initial breakdown because we took the conceptual simplicity to mean we fully understood the problem. The longer we spent on the task, the greater the odds that something would come up that needed attention, causing us to spend more calendar time on the task.

While most of the book was on entire project-level estimation, some of the tips and techniques can be applied to smaller tasks. Some of these techniques are immediately applicable to the item we had estimated poorly. The first technique is to understand the estimate’s purpose. In this case, it was so that we could let other teams that depended on us know when to expect the results. The second technique is to understand how the support burden impacts the team’s ability to do work. There was a much higher than normal support burden on our team during this timeframe due to items other teams were working on that would need our input and assistance; we should have anticipated that. The third technique is estimating each individual part separately, which would allow individual errors to cancel each other out. This usage of the Law of Large Numbers would help to remove random error, although it still leaves in systemic error (like our missing some of the composite parts). Since we did one estimate for the item, we had all of the error in one direction. Using these three techniques, we probably would have gotten down to 100 to 200% over the estimate.

100% over the estimate is still bad. The biggest issue in our estimate was that we didn’t stop and fully discuss the task itself, because we thought we had a good enough understanding of the problem. Based on what happened as we went on, it seems like among the team, someone had thought of most of the challenges we encountered, but nobody had thought about all of them, nor had we effectively shared our individual observations. As an example, the person who realized that each test case was going to be extremely time consuming to run was a different person than the one who recognized the need for an additional set of test cases. Therefore, we did not realize the extreme time consumption involved in testing the item until we were already testing. We had not stopped to discuss the assumptions that went into the estimate since we had all agreed closely on what the estimate was. If we had stopped to discuss the assumptions before generating our individual estimates we could have come to a much better final estimate.

McConnell discusses the Wideband Delphi method as a way to avoid this sort of problem. It requires an upfront discussion about the assumptions; if you have an initial agreement on the estimate, someone plays devil’s advocate to ensure that the estimators can defend their estimate against criticism. They take the average of the generated estimates and if the group unanimously agrees on it the average is accepted as the estimate. If not the process continues iteratively until the group converges on an accepted answer. McConnell cites that this method reduces estimation error by 40%. He also suggests using this estimation technique when you are less familiar with whatever it is you are trying to estimate about. He presented other data that showed that 20% of the time the initial range of estimates does not include the final result, but in a third of those cases the process moves outside of its initial range and towards the final result.

Going forward we’re going to make sure to discuss the assumptions going into the more complex estimates. We probably won’t ever do a full Wideband Delphi, but on larger estimates we will definitely be sure to discuss our assumptions even if we are in agreement as to the answer the first time around. That seems like the best balance between time invested and accuracy for us.