Open Discussion Thread: "Test Smarter, Not Harder," by Scott Sehlhorst

Along the lines described by Jeff Staddon, I think an article describing software failures that can happen (especially during deployment) that have nothing or little to do with the quality of the code would be very interesting. Over the years I have observed that a huge percentage of bugs, especially bugs that come up during deployment or shortly after a deployment, are configuration-related bugs, not code related bugs--someone forgot to add an index to a table, or to set a permission, or to update a config file. What usually happens is that the programmer(s) take the blame for the "buggy deployment." One configuration management problem in a critical deployment can negate thousands of line of great code in the minds of managers who do not make the distinction between the different kinds of problems.

I've developed some techniques that alleviate these kinds of issues, including the practice that the developer writes deployment instructions as he or she is coding the solution. This works especially well in an environment with a lot of task switching; it's easy to forget configuration details between the time you write the code and the time you deploy it.

I've been contemplating a blog post on this topic for awhile, but it would be even better with a full article treatment. If someone with the relevant experience is interested in writing the article, I'd love to publish it in developer.*.

Best,
Dan

Interesting comments. Upper level managers nail ya for things that happen beyond the edges.

I found that a List box in the test form for my scanner is unnaturally short versus its size when I developed it. I haven't investigated the problem, but if it is Microsoft's "fault", if the size of the listbox is not truly controllable, part of me wants to say to the user, it is not my "fault".

But this wars with another part of me that has in the past overcome the limitations of crap software, and managers pay programmers to do this.

In this connection, there's a very interesting book by Zygmunt Bauman, Liquid Modernity, in which he contends that because industry as a whole has moved from hardware (the fearsome great whacking machines of my youth, from my Mom's Hoover vacuum cleaner which made baby sister cry, to the steel mills of Gary) to software, in which not only computers but countless industrial machines are more a Visual Basic GUI attached to underlying hardware, completely configurable...to the extent that Chicago factories, today, usually require a college degree for their "hands".

Software isn't fully testable because it's in a "liquid" state, where there are so many particles that it's impossible to control all installation collisions.

Having said this, however, it should be noted that some OS and environments encourage this unwanted flow by providing uncontrolled featuritis and a lack of distinction between levels, Windows being a major offender in this regard.

I like your idea of having an installation plan on day one. How about the daily build, from day one?

Open source does appear to install and run better, and the claim is that this is because there are more people to "test" it. This isn't the only reason. It's also because open source coders don't have to listen to demands for crap features AND can add good features without having to go through layers of review and aren't under pressure to deliver crap solutions in order to get market share.