After listening to an OOPSLA podcast about a workshop on Fred Brooks‘ widely read No Silver Bullets, I was inspired to reread his seminal piece. Though 20 years old, I was struck by just how applicable NSB is today and while there are a few things that place it in time, as I’ve said before, the more things change, the more they remain the same. Heck, I even decided to assign it at dynamic language camp. Much of what Brooks writes about relates to accidental vs. essential complexity, a topic that’s echoed by Neal Ford in this post and Reg Braithwaite here. Stu Halloway touched on this at Code Freeze this year though he rephrased the concept as essence vs. ceremony. More and more, we’re finally heeding the message found in this C. A. R. Hoare quote:
“Programmers are always surrounded by complexity; we cannot avoid it. … If our basic tool, the language in which we design and code our programs, is also complicated, the language itself becomes part of the problem rather than part of its solution.”
Anyway, on to Brooks. In the spirit of a number of Ted Neward’s posts, I’ll take snippets of NSB and inject my thoughts. Let’s start near the top with this gem:
“[Germ theory] told workers that progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today.”
Though perhaps not what he intended, I see this as yet another call for continuous integration as well as fixing broken windows. It isn’t easy, it takes a great deal of work, but when we fail to be diligent, our “patients” get sick. And anyone that’s ever worked on decaying software knows how much fun that is…
This quote should be endlessly fed to those that think programmers are essentially typists:
“I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.”
This work is made up of thought stuff - and anything we do to disrupt flow will ultimately hurt our chances of successfully developing software.
To those that think some tool or modeling language will make software so easy anyone can do, I’d counter with this:
“The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.”
In other words, software is hard…though we often make it harder. Sometimes that’s related to our organizations:
“Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God.”
Further confirming that the problems in software are largely people oriented, one can practically hear Brooks’ echo in the agile manifesto:
“The central question in how to improve the software art centers, as it always has, on people.”
Technologies don’t kill projects, people do and better programmers really are better, a point made by Neal as well as Paul Graham:
“The differences between the great and the average approach an order of magnitude.”
These days, I’m asked often about how we’re going to “scale up” our development teams which is really just management speak for “off shore 80% of the work.” Now, fundamentally, I don’t have any issues with taking advantage of vast labor pools, but I’d much rather have a small team of top notch developers than a large team of, well, less than average ones. I’m not sure if it’s just the fiefdom complex or the overriding dictate of distant management, but big teams are usually problematic. With a few great developers, I can move the world. And let’s never forget garbage in, garbage out.
Back to Brooks - he’s more entertaining than I am. He touches on something near and dear to my heart praising higher level languages:
“Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. Most observers credit that development with at least a factor of five in productivity, and with concomitant gains in reliability, simplicity, and comprehensibility.”
Seems to echo with what people say about Rails, Ruby and a host of other languages these days. Expressiveness matters - a lot.
I’d argue this is largely what Stu was getting at in his Ending Legacy Code talk:
“[Abstract types and hierarchical types] removes yet another accidental difficulty from the process, allowing the designer to express the essence of the design without having to express large amounts of syntactic material that add no information content. For both abstract types and hierarchical types, the result is to remove a higher-order kind of accidental difficulty and allow a higher-order expression of design.”
Getting rid of boiler plate and focusing on the problem at hand is key. I know a lot of developers who defensively shout “but my tool handles all that for me.” Sure. See above. And don’t forget, you or someone coming in behind you will still have to read all those excess symbols.
Though I wish buying new hardware would solve all our woes, Brooks reaffirms what we already know:
“More powerful workstations we surely welcome. Magical enhancements from them we cannot expect.”
Bummer. Guess I’ll need a better reason to get that new MBP.
There’s quite an agile flavor to NSB and a number of Brooks’ comments speak very well of what is becoming a more and more common approach to writing software. I’m not sure about you, but I haven’t seen much success with waterfall…but iteratively developing solutions seems to work, a point he makes quite clearly:
“Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements. For the truth is, the client does not know what he wants. The client usually does not know what questions must be answered, and he has almost never thought of the problem in the detail necessary for specification.”
Despite what some think, you just can’t get it all right up front. This isn’t some fundamental failing, it’s a feature not a bug. Rather than attempt to fight this, just work with it; instead of trying to write it all on the plan before you break ground, iterate. Software types tend to be good abstract thinkers, but our customers often aren’t - thus why getting working products in front of them early is so important:
“I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.”
The value in this approach seems to obvious yet it still isn’t “the norm.” I honestly cannot understand why customers don’t demand this process.
I try not be as wordy as Steve Yegge but Brooks summarizes my post on house building in less than fifty words:
“Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy.”
He goes on to discuss growing software, an analogy that really speaks to me. I remember mentioning something along these lines to a former co-worker only to be rather rudely dismissed:
“Incremental development—grow, don’t build, software.”
When I discuss agile with skeptics, I really try to hammer on this point:
“One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.”
All I can say is, I’m no Fred Brooks. But what does he know right?
Systems that spin off for months (or years) without that iterative review tend to fail - often rather expensively. I remember one project many years ago where the client eventually ran out of money (remember the dot com implosion?) Due to decisions made outside my influence, we really didn’t have anything he could use. Sure, he could “demo” the product, but he certainly couldn’t sell it. We’d spent a great deal of time designing everything in the system - all the screens, all the interactions…too bad we hadn’t spent more time building. Had we worked more iteratively, he would have had *something*. Oh well, lesson learned.
As I mature in this industry, I become more aware of our pioneers; as I find my path I discover how far out they saw. Though recent times have seen amazing advances, we have much to learn from our past.
. OK, I’ve only got three readers so I know it’s all just a big fat coincidence but still! Process theatre really does capture the notion perfectly; pointy haired bosses believe more templates, meetings and blind adherence to methodology are the answer to failing projects. Of course when you remove all decision making from the hands of sentient beings, you get a lost workday instead of someone just hitting the spacebar.