It is BDUF, not DUF

iStock_000005541096XSmall

In case the title of this article is confusing to you, BDUF means Big Design Up Front. It essentially means that you will hammer out a complete design up front, before you being to create your solution. The reason that I bring this up is because of Glenn Block’s ALT.NET Criterion post. In his post he mentions “Not doing BDUF” as one of the ALT.NET principles, and I think that making the distinction between doing *Big* Design Up Front and Design Up Front is very important for developers to make.

A lot of developers think that agile development represents doing no design up front. That we will figure out what we want to do and then just jump right into coding it without any concern for what we are going to build. Then we will just come back later, refactor the whole thing, and voila, instant working application. Well, it just doesn’t work that way. I always like to say that if you have one opinion on one extreme, and another on the opposite extreme, then the truth is always somewhere in the middle. Agile development and “not doing BDUF” is all about finding that sweet spot of trying to decide what you are going to build, without getting bogged down in the details that are better left to implementation. I can’t remember who wrote it, but someone said “if you are going to write a spec that is so detailed that the code could be written from it, then you might as well just execute the spec”.

So, why should we be avoiding BDUF? To people from a lot of industries BDUF may make perfect sense. You better believe that BMW engineers are going to design an entire car before they start building it. Now, they may build subsystems independently, but they are going to create the design for the entire car before they start manufacturing the car. I mean, if they create the seats before they design the car, then how do they know that it is going to fit inside the car? In the world of physical parts and manufacturing this seems to be a fairly good rule. You really should have a fairly solid plan about what you are going to build before you start producing any part of it. Well, it is a good thing that programmers don’t operate in the physical world. We may have to deal with the interfaces on two subsystems, but we are able to build subsystems more segregated than these. In fact, we will often aim at our subsystems being black boxes.

So, what do I mean by black boxes? Well, a black box in software development basically means that we have an input and an output, but we know nothing about what goes on inside of the box. Another way of saying this is that these sub-systems are orthogonal. They are completely separate and making changes to one will not affect the other. In the physical world, you could design the interface between the engine and the drive train, then you could design the interface between the drive train and the axles. You could even design the interface between the engine and the frame. So, now you have the entire drive subsystem, but you could still get in a situation where the engine doesn’t fit inside of the body. This is because even though the engine and the body don’t have an explicit interface, the physical dimensions of the body affect the design of the engine.

In software we still have these sorts of indirect constraints, but our medium is quite a bit more malleable than steel and manufacturing lines. If we design two sub-systems and find that there is an unavoidable indirect constraint, then it is much easier for us to change our design. But just because we can change it, should we? I mean, if BDUF works for complex systems like automobiles and airplanes then why can’t it work for our software systems?

One word, “cost”.

The systems that developers are asked to build are often vastly complex, and the resources that are provided are often quite meager. If we had a huge amount of resources, then you might have the ability to do a big up-front design and then to analyze this design to iron out the details that can be the downfall of the waterfall model. This takes a lot of effort though, effort that is often a waste. If a decision can be made further down the road, then it is going to be made with more available information. This is the concept of Last Responsible Moment. This is the key to what BDUF lacks. When designing an application you are making assumptions. You may not even realize all of the assumptions you are making, but you are making them. And when those assumptions end up being wrong, then your design is wrong. Unless you have the resources to analyze and test all of the assumptions you make in your design, then you are likely to end up with a design that has flaws. Flaws that could have been avoided if you put those decisions off until you had to make them.

The important thing to note though is that we still want to do design up front. In fact we always want to be thinking about design. At the start of a project it is desirable to do some design up front. You want to have a clear picture of what business value you are trying to attain, and a more blurry picture of how you want to attain it. As you start to fill in the picture of how you are going to attain your business objectives then things will start to become more and more clear. It is very important that you realize that the value comes from not knowing all of the details. To many business people this may sound terrifying, but how many times have tools been picked at the very beginning of an application lifecycle (along with thousands of dollars spent), only to realize that the tools won’t do what you wanted them to. Or the tools originally did what you wanted them to, but now your needs have changed. Then you are forced to spend resources to force the tools into your particular need, but even then you may end up with a fragile system.

So what does all of this boil down to? Design your application. Being agile does not excuse the need for not planning ahead. Being agile means that you are going to respond to change, and putting off decisions can allow you to more easily make better decisions. Next time you are on a project and during the planning phase someone picks a vendor to fill a specific need, ask them to put the final decision off until that part of the application needs to be built. The vendor will still be there when the software is needed, but if you put off the purchase, then you may find that you need different software or you don’t need it at all.

Be Sociable, Share!

10 comments

  1. Nice post. As I was reading I kept thinking about Lean Software Development. Interesting when you look at lean manufacturing there is plenty of BDUF, yet the actual assembly happens in a lean manner. In this case it makes sense because of mass production. The cost of doing the entire design up-front in order to produce millions of products makes sense. In software, projects are usually one-offs. Investing a huge design effort up front for a one-off project is way too costly and risky. Unlike the car example, we don’t really know what the end goal will look like. There’s too many non-deterministic factors.

  2. @Glenn Thanks! I have not had a lot of experience with Lean Software Development, so I’ll have to take you word for it on your comments. And yes, the cost of up-front design when producing millions of items that *can’t be changed* is certainly worth it. But in software projects that aren’t one offs, is it as important to do a lot of up-front design knowing that we can easily update and patch our product? I’d be interested to hear your comments on that.

  3. [quote]In software we still have these sorts of indirect constraints, but our medium is quite a bit more malleable than steel and manufacturing lines. If we design two sub-systems and find that there is an unavoidable indirect constraint, then it is much easier for us to change our design.[/quote]
    Is it really?
    I don’t think it is. At least not in a lot of cases. I’d like to see more discussion about why people think in software things are ‘easy to change’, as that myth is precisely the reason why clients think they can ask all kinds of changes right before the project is almost done. No-one will ask a builder to relocate a bunch of toilets in a building that’s almost done.

    I’m not saying BDUF is always needed, but what I want to get across is that the reasons a LHC in switserland isn’t build agile and why cars aren’t build agile might be reasons we as software developers can learn from and apply more in our own work.

    I think that Glenn has a good point with the ‘end goal’ being vague and therefore a BDUF doesn’t make any sense, because you don’t know what to design. However, it’s not an excuse. It’s a signal that apparently there are vague goals so one should do more effort to get the goals on the table and put more energy in getting clear what the end product/project result should do.

    Anyone who thinks that starting a project without knowing what will be the end result is OK has no clue what product development is all about. And yes, doing a project for a client IS product development, as the client sees it as a product, albeit a product where there’s just 1 unique instance of, custom made.

  4. @Frans My point was that software is *more* malleable than a manufacturing line. I’m not trying to say that software is easy to change, it is not. Especially as you point out, large sweeping changes toward the end of the product’s construction.

    And I completely agree with "starting a project without knowing what will be the end result is OK has no clue what product development is all about". That was really the whole point of my post. Some people seem to think that agile is all about finding out what you are doing as you go along. You need an excellent idea about what you are going to be building, I just think that trying to decide a ton of the technical details before construction requires a huge amount of resources that software projects often don’t have.

    Your example of the LHC is perfect. They obviously had to design and perfect every piece of it before it was built. They couldn’t build the LHC and then refine their concept, that just isn’t the way that physical objects are built. In software though, we *often* have that luxury.

  5. I agree with the general idea, some design up front is healthy to the project, but can we please stop comparing software development with manufacturing?

    Honestly, how many of us (software developers) have real meaningful manufacturing experience? (My apologies if you do)

  6. @Hector No, I do not have experience in manufacturing. But analogy is one of the best ways for people to convey ideas, and I think that most analogies from programming to any other industry is going to fall apart at some level.

    I am not trying to directly compare building software to manufacturing, in fact I am trying to show that software is inherently different from manufacturing in that we do have the ability to build and refine, build and refine…ad infinitum. This is what I believe makes our industry unique from many others.

    And yes, I agree that comparing software to manufacturing is overused, but it is just so darn convenient. It would be much harder to compare it to something like… dentistry. :-)

  7. If you want to compare building industry to software industry, I would suggest that software industry is at its very early beginning, compare to building industry (just think of the Pyramide, built 4000 years ago) or manufacturing.
    I think there will be many progress in the future in the software industry, to become more and more agile, and easy to change. The best specification and design is the software itself, and BDUF won’t be necessary, as soon as you can build software on demand, and that is where we are going. Lean Software Development, that I have just discovered, is very interesting on this point.
    Thanks.
    (Please, Apologize my poor english)

  8. Great post. Good to see more middle ground thinking. I think the other point to make against BDUF is that you will inevitably waste time on design details that may become unnecessary, as you start to build the system and evolve your understanding of the problem domain and the best way to tackle it. It’s all about knowing when ‘enough’ design has been done at any point throughout the project.

  9. @Tokes Excellent point, I certainly think that waste should play into most decisions. In your case, the waste is apparent in effort that is expended before enough knowledge is accumulated to properly make the decision.

  10. [quote]So what does all of this boil down to? Design your application. Being agile does not excuse the need for not planning ahead.[/quote]

    I agree completely with the article. So many people think that Agile means no design up front. In fact, agile is more about evolutionary design as Fowler describes it: http://www.martinfowler.com/articles/designDead.html

    A part of the design up front that should be considered is outlined in my article: http://jknowledge.wordpress.com/2009/01/03/change-management/

    To summarize the article, common types of changes that should be considered from the beginning of a project are changes in the infrastructure (DBMS, web server, …) as well as changing programming language. Every year, many applications are migrated to new programming languages and we should take the time to plan accordingly from the beginning. The article also summarizes a few ways to cope with these kinds of changes such as using language-independent constructs and language-independent frameworks.

Leave a comment