Archive for February, 2008

Emergent Properties

February 17th, 2008 No comments

Like pretty much any office with more than 3 people, we struggle with the ephemeral concept of knowledge management. Now, this takes the guise of everything from cultural lore to more basic issues like where is the latest version of the FooBaz document but the moral of the story is simple: we’re still trying to find the right approach. We have a corporate product that hardly anyone uses because it’s slow, the search is horrid, and it has very rigid ideas around who can post what where.

Recently, a number of teams have started to use a different product, one they are finding to be far more useful then the corporate standard. Though it isn’t officially supported, it’s gaining quite a bit of traction throughout the organization. Imagine my surprise when an IS wide email goes out saying, in essence, that everyone should STOP using the product that’s working and contact the head of the crappy product team immediately so they can “migrate you over” to the “standard.” In other words, cease doing the thing that’s working, not “wow, we’ll fast track the adoption of this new tool since it’s serving such a vital need.” Makes perfect sense.

As I was reading the email, I thought of St. John’s (Go Johnnies!). Anyway, around the time my father was in school, they built the tundra dorms – named as such for the large open space between the bulk of campus and the dorms. Anyway, when they built the tundra dorms, they didn’t put in sidewalks right away, they waited until the students had worn paths and just paved those. Rather then guess what route residents would take, they let the property emerge. Needless to say, this approach worked pretty darn well.

The parallel here is pretty obvious – right or wrong, the “students” are wearing a path towards this new tool. Now I’m sure we can argue that the corporate standard can do “everything the new tool can do and so much more”, but the crowd has spoken. Instead of using scare tactics to keep people from using it, perhaps the bureaucracy would be better served by following the herd.

Categories: Off Topic, Rants, Software Tags:

Silver Bullets

February 10th, 2008 No comments

After listening to an OOPSLA podcast about a workshop on Fred Brooks‘ widely read No Silver Bullets, I was inspired to reread his seminal piece. Though 20 years old, I was struck by just how applicable NSB is today and while there are a few things that place it in time, as I’ve said before, the more things change, the more they remain the same. Heck, I even decided to assign it at dynamic language camp. Much of what Brooks writes about relates to accidental vs. essential complexity, a topic that’s echoed by Neal Ford in this post and Reg Braithwaite here. Stu Halloway touched on this at Code Freeze this year though he rephrased the concept as essence vs. ceremony. More and more, we’re finally heeding the message found in this C. A. R. Hoare quote:

“Programmers are always surrounded by complexity; we cannot avoid it. … If our basic tool, the language in which we design and code our programs, is also complicated, the language itself becomes part of the problem rather than part of its solution.”

Anyway, on to Brooks. In the spirit of a number of Ted Neward‘s posts, I’ll take snippets of NSB and inject my thoughts. Let’s start near the top with this gem:

“[Germ theory] told workers that progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today.”

Though perhaps not what he intended, I see this as yet another call for continuous integration as well as fixing broken windows. It isn’t easy, it takes a great deal of work, but when we fail to be diligent, our “patients” get sick. And anyone that’s ever worked on decaying software knows how much fun that is…

This quote should be endlessly fed to those that think programmers are essentially typists:

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.”

This work is made up of thought stuff – and anything we do to disrupt flow will ultimately hurt our chances of successfully developing software.

To those that think some tool or modeling language will make software so easy anyone can do, I’d counter with this:

“The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.”

In other words, software is hard…though we often make it harder. Sometimes that’s related to our organizations:

“Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God.”

Further confirming that the problems in software are largely people oriented, one can practically hear Brooks’ echo in the agile manifesto:

“The central question in how to improve the software art centers, as it always has, on people.”

Technologies don’t kill projects, people do and better programmers really are better, a point made by Neal as well as Paul Graham:

“The differences between the great and the average approach an order of magnitude.”

These days, I’m asked often about how we’re going to “scale up” our development teams which is really just management speak for “off shore 80% of the work.” Now, fundamentally, I don’t have any issues with taking advantage of vast labor pools, but I’d much rather have a small team of top notch developers than a large team of, well, less than average ones. I’m not sure if it’s just the fiefdom complex or the overriding dictate of distant management, but big teams are usually problematic. With a few great developers, I can move the world. And let’s never forget garbage in, garbage out.

Back to Brooks – he’s more entertaining than I am. He touches on something near and dear to my heart praising higher level languages:

“Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. Most observers credit that development with at least a factor of five in productivity, and with concomitant gains in reliability, simplicity, and comprehensibility.”

Seems to echo with what people say about Rails, Ruby and a host of other languages these days. Expressiveness matters – a lot.

I’d argue this is largely what Stu was getting at in his Ending Legacy Code talk:

“[Abstract types and hierarchical types] removes yet another accidental difficulty from the process, allowing the designer to express the essence of the design without having to express large amounts of syntactic material that add no information content. For both abstract types and hierarchical types, the result is to remove a higher-order kind of accidental difficulty and allow a higher-order expression of design.”

Getting rid of boiler plate and focusing on the problem at hand is key. I know a lot of developers who defensively shout “but my tool handles all that for me.” Sure. See above. And don’t forget, you or someone coming in behind you will still have to read all those excess symbols.

Though I wish buying new hardware would solve all our woes, Brooks reaffirms what we already know:

“More powerful workstations we surely welcome. Magical enhancements from them we cannot expect.”

Bummer. Guess I’ll need a better reason to get that new MBP.

There’s quite an agile flavor to NSB and a number of Brooks’ comments speak very well of what is becoming a more and more common approach to writing software. I’m not sure about you, but I haven’t seen much success with waterfall…but iteratively developing solutions seems to work, a point he makes quite clearly:

“Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements. For the truth is, the client does not know what he wants. The client usually does not know what questions must be answered, and he has almost never thought of the problem in the detail necessary for specification.”

Despite what some think, you just can’t get it all right up front. This isn’t some fundamental failing, it’s a feature not a bug. Rather than attempt to fight this, just work with it; instead of trying to write it all on the plan before you break ground, iterate. Software types tend to be good abstract thinkers, but our customers often aren’t – thus why getting working products in front of them early is so important:

“I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.”

The value in this approach seems to obvious yet it still isn’t “the norm.” I honestly cannot understand why customers don’t demand this process.

I try not be as wordy as Steve Yegge but Brooks summarizes my post on house building in less than fifty words:

“Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy.”

He goes on to discuss growing software, an analogy that really speaks to me. I remember mentioning something along these lines to a former co-worker only to be rather rudely dismissed:

“Incremental development—grow, don’t build, software.”

When I discuss agile with skeptics, I really try to hammer on this point:

“One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.”

All I can say is, I’m no Fred Brooks. But what does he know right?

Systems that spin off for months (or years) without that iterative review tend to fail – often rather expensively. I remember one project many years ago where the client eventually ran out of money (remember the dot com implosion?) Due to decisions made outside my influence, we really didn’t have anything he could use. Sure, he could “demo” the product, but he certainly couldn’t sell it. We’d spent a great deal of time designing everything in the system – all the screens, all the interactions…too bad we hadn’t spent more time building. Had we worked more iteratively, he would have had *something*. Oh well, lesson learned.

As I mature in this industry, I become more aware of our pioneers; as I find my path I discover how far out they saw. Though recent times have seen amazing advances, we have much to learn from our past.

Categories: Agile, Development, Software Tags:

Maybe Software Development IS Like Building a House

February 5th, 2008 7 comments

A few years back, my wife and I built a house. OK, so *we* didn’t actually swing a hammer; if you’ve ever seen me attempt a project around the casa, you know hiring the project out was the sanest course of action. This was my second go round with home construction, a process that many say they’d never repeat. But hey, I’m a glutton for punishment so in we plunged! I’d mostly walled off the entire experience in that place we put painful events like child birth and two-a-day practices, but last week one of my projects got me thinking that maybe, just maybe, there’s a comparison to me be made between house construction and building software.

Before I get started, I have to consider whether I’m on shaky ground here – many people have written pieces questioning the whole construction analogy. For starters, my friends Neal Ford and Glen Vanderburg have their takes with building bridges without engineering and bridges and software. Jack W. Reeves’ classic What Is Software Design? is a must read and Reg Braithwaite has a great post about what he admires about engineers and doctors which has this money quote:

Try this: Employ an Engineer. Ask her to slap together a bridge. The answer will be no. You cannot badger her with talk of how since You Hold The Gold, You Make The Rules. You cannot cajole her with talk of how the department needs to make its numbers, and that means getting construction done by the end of the quarter.

Considering the shared experience behind that impressive collection of wise words, I’m questioning my sanity to even think a construction analogy might fit software. But, even though I’m conflicted about the whole thing, I’m still going to share. Heck, maybe I’ll learn something in the process.

When we built our house, we spent several weeks looking at model houses, poring over floor plans, looking at carpet samples – all sorts of fun stuff. Now, with the last “home project,” during the requirements gathering phase we were just focused on the big issues: do you want a built in here? Would you like a fireplace? How about a skylight here? You know, the stuff you’ve got to get right before the foundation is set and the walls are up. At various stages, I’d get a call from the project manager (yep, that was his title, I’m feeling more confident already!) and he’d setup a time for me to come out to the site and work with one of the trades on issues like outlet placement. With my sample size of one, I expected a similar (iterative) process this go round – alas I was wrong.

You see, this builder had a different approach. They believed heavily in making all (and I mean ALL) the decisions up front (can you say BDUF?) Being a software geek, I think I do a pretty good job of thinking abstractly but needless to say, it can be quite a challenge to figure out where you want your phone jacks when all you have to go on is a 2D model of your future dwelling. I pushed back on the builder and was told they did this for a reason – they felt that if everything was on the plan, I could go on (as they put it) a four month vacation and come back to a completed house *exactly* as I intended it to be.

As much as I wanted to believe the people I was about to give a very large check to, I wasn’t convinced and as you might expect, my wife and I were on site pretty much every other day keeping track of what was going on. It was a good thing we were vigilant customers constantly running our acceptance tests. Nearly every visit revealed something that needed to be fixed, a story to be added to the backlog (or punch list in this case.) Some things were minor – a switch not controlling the proper light or a misunderstanding about what the plumbing code would allow. But others were, well, of the show stopper category. For example, despite a very clear floor plan showing where the washer and dryer were to be, the plumber decided he’d just put the washer where it was in every other house. Thank goodness we caught it early but this whole “we get it on the plan thing” certainly didn’t work in practice.

So what the heck does this have to do with software? Well, one of my projects has a customer group that thinks like my builder – they want to give us all the requirements, we give them an estimate, then they don’t talk to us until the project goes live. Obviously, this doesn’t work too well. I’m not sure why people ever thought this approach worked, heck, if we can’t get it right with houses (something we’ve been building, oh, forever) how can we possibly get close with a discipline as new as software? No, the answer is found in an agile approach where we work closely with our customers. Does that mean we need to see them for eight hours a day everyday? I sure hope not, but if they aren’t willing to commit some time to the project, how important could it actually be?

Needless to say, we’re trying to get the customers to think different, to embrace a more collaborative approach and I hope we succeed. Otherwise, I have a pretty good idea what will happen after that four month vacation.

Categories: Agile, Development, Rants, Software Tags: