What can Heidi Klum teach you about software development?

...immediately caught my lecherous, cross-dressing eye, Donna. I am a great admirer, whose admiration becomes Goddess worship, of the female form. Indeed, I am like Garrison Keillor's Earl in Wobegon Falls, who wants, hopelessly in view of simple logic, to be a male lesbian and who is reminded by Dave that, uh, Earl, all us guys are technically "male lesbians".

But, I digress ("Yeah, you digress" - Dan Appleman, private communication, March, 2004).

I would say that in programming there is a necessary and unresolvable tension or aporia between three "stars" in a constellation (in Adorno's sense):

(1) Programming as an unexpected "necessary evil", the unplanned emergence of female writing (Grace Hopper) in what was supposed to be a male domain (John von Neumann)

(2) The Platonic environment of programming, its presentation of simulacra of Platonic Forms

(3) Its artisan status in the scheme of things

I agree that there is overdesign. However, the term "overdesign" has itself been overgeneralized to the extent where it is bandied about and at times refers to necessary exhibitions of professionalism where the reviewer doesn't grok the need...or, in the worst case, doesn't know the technique and is too vain (or too anxious for her position) to admit it.

The traditional artisan knows poor workmanship when he sees it, but in programming, the definition of good workmanship is alienated in large part. Our user-friendly language, Donna, is an American tic: it conceals, in a democracy, brutal ECONOMIC subordination (the out of control arrogance of Trump in The Apprentice, viewed in relation to his limited mental capacities and limited accomplishment in his actual chosen field), by using a New Age language of "the needs of the User".

Santino "overdesigns". Hmm. All this means, I am afraid, that he lacks economic power and control over his work. As it happens, the designer/owner in fashion, the late Versace for example, can dress babes in any contraption he likes because in our society Money Talks.

[As regards swimsuit design, I am a minimalist. You probably guessed that har har.]

The goal of *haute couture* is or should be making women beautiful, gilding the lily in my view (have you seen the 1998 film Pret a Porter?)

Whereas in an organization whose economic needs are in large measure at odds with human needs, I AGREE that programmer illusions as to what the "user needs" can never control.

Just do your best, and ask the users (not the hypostatized User) what they want. Don't expect to be an artist, and if you want to be an Artist, then move to a Garret and Suffer.

Keep in mind, finally, that the contestants in most of these reality shows are humiliated beyond all reason and the winner and still champeen is always Capital, the Lacanian phallus. Sure, in the theme song for The Apprentice, it is sung over and over again (money money money) yet the very repetition blinds the soul to what is going down.

It is to me, based on completing three Outward Bound expeditions in which the personhood of each and every participant was celebrated, OBSCENE what happens to people in Survivor.

Which doesn't prevent me from leching out on the hotties.

As with anything, it's about balance. I've been burned both ways with (1) fixating on the client's wishes and (2) thinking I knew better.

If the former is carried too far, you will ignore the nagging voice that tells you that you should design for extensibility where you feel almost certain it will be needed, even if it isn't now. You will build insecure systems to accommodate client convenience.

Yet, I've also been involved in systems where the developer never visited the client face to face (in an internal IT organization) until he was ready to train the user. Requirements were more or less handed down through hear-say. And what a disaster.

One of my most successful applications was developed for a group which included a handicapped woman who had very limited use of her hands. That was a major design issue and she has often told me what a difference it made in her life.

I agree with you about Survivor and many of the reality TV shows which are scripted, just badly. But of course my point was not entirely to glorify reality TV, but to recognize a common theme. If Santino designs a dress for a woman to wear and it is so laden with baubles that it cannot reasonably be worn (despite his genius), has he accomplished his goal? I don't think so. He should consider a career in textile art, where his products can be featured in a display case hung on the wall.

I understand folks are sensitive about being told not to waste time by recreating the wheel. I am not advocating mediocrity, but reminding us that the user is our patient. If we have become so confident (cocky) in our own abilities that we lose sight of that, then how can we treat them?

... will you read such stuff? Excellent post, Donna. Great parallel drawn and appreciated -- someone being accused of oversdesigning by Heidi Klum.

Mmmmm...... Heidi Klum.....

What were we talking about? Ah yes. Gold-plating. Sorry.
Maybe it's obvious in my stuff, maybe it's not, but I'm a big, BIG user advocate. My thinking is, why do we do what we do if not for them? Simplistic from a scientific standpoint, but hey -- this is business.
So I am very wary of co-worker who always try to shoehorn in a design pattern whether it fits the project or not, or who "HAS to do something with XmlHttpRequest in this app", etc. That's pure egotism and self-centeredness, on a personal level, and 100% bad faith on a professional level. Our job is to do technically what fits the application/project, and to do feature-wise that which is needed and requested. Save the widgets for your personal time.
I tend to work the same way on music -- serve the song, don't make the song serve you. Getting fancy may keep you slightly more interested, but it certainly alienates the listener.
All that said, something Edward mentioned in his comment strikes a chord with me (no pun intended), that there is also a tendency in the inexperienced or uninitiated to view elegant design itself as gold-plating. And that's something I've found I have to resist, and educate people in. Sometimes the flowery bits on the figure-skater's costume can provide lift or aerodynamics, to work in the analogy space.

A nice juxtaposition with these two comments of Andy's:

So I am very wary of co-worker who always try to shoehorn in a design pattern whether it fits the project or not

AND

there is also a tendency in the inexperienced or uninitiated to view elegant design itself as gold-plating. And that's something I've found I have to resist, and educate people in.

Often what happens is that the most elegant design is often the *simplest* design, a concept which may not be compatible with using elaborate solutions and design patterns. I have found that, while sometimes there is an ego component in this tendency to over-design, often the tendency is simply based on a misunderstanding that elaborate, data driven, and generalized to the Nth degree is inherently a good thing--when often it's just overkill.

Dan

We have a major system component here that is exactly what you describe -- data-driven, elaborate, and generalized to the Nth degree. In the space of 6 months it has become understood by none, hated by all. Maybe it's ego, maybe it's job security for the designer...

Your juxtaposition is great -- it's funny though: I was actually thinking more of something that sounds/looks complex but provides so much benefit as to be worth the effort. But your point is 1000% valid. Like David St. Hubbins said, "there's a fine line betweem clever and stupid."

I'd just like to throw another variable into the mix here. Talking about abstraction and so on versus the simple, easy soluton to serve the client -- I'm working right now on a project for a real estate developer who's stated goal is to "track the development of their properties online". That is to say, they want an online facility to help agents in the field update progress relative to houses coming out of the ground. The spec that I've been given is exquistely clear in what it details. However, it's painfully ignorant of the things it knows not that it needs.

So then I can see and understand from my experience things that the client will most likely want once the software is up and running. However I've not been told what the endgame for this product is. I've been tempted many, many, MANY times in designing and developing this product to just take the easy way -- "Ah just put the SQL in the page" or "Who the hell cares about code versus presentation separation?". Or "Why should I give a darn about instrumentation or tracking who did what over how long a period of time?" However I've fought those urges and done ALOT to build an infrastructure that abstracts the problem domain into a series of classes using patterns I'm quite comfortable with. Why? Because I have no idea where this thing is going and, secondarily, because short-cuts taken with the idea of "go-to-market" primacy always, always, ALWAYS live in the code for much longer than intended. Indeed they may become the very crux of the application and are harder to change as time goes on. Abstraction, as you all know, allows distance from implementation and thus flexibility.

I think there's a difference in choosing to use a "gee-whiz" technology just because you want to regardless of the client's need versus doing some heavy lifting to ensure you never have code in two or more places giving the same set of instructions. In both cases though, the devil can sit on your shoulder and second guess you -- only in one case he's right. At my last full-time gig there was a self-styled "cowboy" who was thoroughly steeped in the "git-er-done" school of programming. He had this halo effect around him because he could just turn on a dime and pull something out of his @$$ to co-incinde with some wish from the sales or marketing side of the house. However, it always broke or produced inconsistent, hard to explain results after many thousands of iterations through the logic. I wouldn't say that that serves the client's needs at all.

The last thing I'd like to touch on is that it makes a difference what platform you're working in as to how long it takes to develop a reasonable framework on which to mount your feature set. The above project I mentioned is in PHP 5, and in about 60 or so hours I've managed to create a framework of about 30 classes describing lots, brokers, developments, a user security model and the information related to 8 stages of building progress -- and it works! I absolutely could have done it faster any number of ways, but the end result would have been harder to maintain and extend.

In closing, I guess I heard a little bit in the ensuing discussion that attention to discipline and craftsmanship could be superseded at will by a client's need to go to market. They can, but at a cost -- more like a debt that needs to be paid and all too frequently isn't. Clients aren't an excuse to use the latest, new-fangled, stupid Web 2.0 widget but sometimes they do provide an opportunity.

Trevor writes:

At my last full-time gig there was a self-styled "cowboy" who was thoroughly steeped in the "git-er-done" school of programming. He had this halo effect around him because he could just turn on a dime and pull something out of his @$$ to co-incide with some wish from the sales or marketing side of the house.

I can relate to this story well. I think that consumers of programming services (be they marketing people or middle managers or independent business owners) can become addicted to cowboys, because as Trevor infers, they offer instant gratification. No matter how many times the fruits of these cowboy (or cowgirl) labors come back to bite them in the @$$, they come back for more.

It takes a long, slow process to break this addiction, but the alternative is hop-skipping from one cowboy to the next. I believe the process of breaking the addiction depends on a solid, strong-willed professional (or team of them, as the case may be) to establish sanity within the system, while at the same time partnering with the stakeholder on real business needs and goals. Sometimes that's a tall order.

Thanks for the great comment's, Trevor.

Dan

Many excellent lines of discussion on this thread. I think this is my third comment on it today. Adding to Trevor's thoughts (and pardon me if I get a little long-winded; I'm trying out some material)...

First, I really like the phrasing of this from Trevor's post (paraphrasing):

Abstraction allows distance from implementation and thus flexibility.

I also very much like how you have illustrated the subjectivity inherent in these kinds of questions. I wonder if these three categories are useful:

1. Questionable or clearly bad practices that have high potention of constraining future directions, causing brittleness or maintenance/support difficulties, making the system hard to understand, etc.
2. Proven practices and techniques that make good sense, even if it seems like extra effort (like Trevor's example of keeping the SQL out of the presentation tier code).
3. Speculative generality.

These are all subjective measures to be sure. That last category, though, is where things get interesting. Where is the line between categories 2 and 3? Different case-by-case, I'd say.

I like the term "speculative generality," which comes from Martin Fowler's book Refactoring (Fowler credits the phrase to someone else, as I remember). I think this concept offers a useful test for struggles in this area:

Imagine that here I am generalizing something in my program design, perhaps deciding whether to normalize a table further, or whether to move some settings to a config file, or whether to introduce a polymorphic interface to allow for future variations in input, or whether to implement a Command pattern. Once I have settled the question of whether a given design option will *work*, I have to decide whether it's *appropriate*.

First question: given the requirements (and, importantly, assuming there are one or more stakeholders to whom I am beholden), is this level of generality called for? Is it in line with the requirements (as best as I can interpret them)? Also, and this one's tough, when those questions don't have clear answers, what is the probability that the extra effort (and perhaps loss of clarity in the design) will pay off in the future?

I think considering the real and probable costs of the speculative generality can help in determining whether it is justified. How much extra effort is there? What are the likely or possible configuration management, deployment, maintenance, and administration costs?

Am I going to burden my client (or myself) with anything as a result of this generalization? Do I want to present my client with a 64 piece Ginsu Ultra-Kitchen 3000 food chopper that slices and dices and makes crinkle-cut french fries?

My client might like this super useful machine at first (crinkle-cut fries!), but what about after he realizes that it takes three hours to disassemble, clean, and reassemble after each use, and that it needs to be oiled once a month, and that he really only uses two features that would be just as easy with a good knife?

Another solution I use when there's a tough call (particularly if time and resources are tight) is to is to compromise by taking one step back away from the full generalization I'm considering. Hard code it, so to speak, for now, but leave the door open for later if the generality should prove necessary in the future. An indirection layer is your buddy here (classic Parnas information hiding): hide the thing that might change behind an indirection layer.

Another area of interest: the need to play "proxy" for the "client" (using that term generally) when the client is either not available, or not really in a position to grasp the issue at hand. I need to come up with a good name for this technique. Here I try to pretend I have just spent however many hours or months it would take for me to explain to my client enough that he or she would grasp the complexities and subtleties of the technical/design situation as I see it (after all, it's take me 12 years to achieve my current feeble understanding).

So I imagine my client (or user, as the case may be) sitting next to me in this state of full understanding--what would he or she say? What should I do? Which way should I go? This helps me (to whatever degree possible) to imagine my client's needs and interests on an equal footing with my technical concerns.

And finally, another theme from Trevor's post: in a consulting situation like Trevor describes, weighing your contractual and professional obligations to your client's needs (not to mention your own desire to do a good job and build a durable product) against the need to do exactly the right amount of generalization--providing good value to the client while also not spending so much time that there's no money for the mortgage payment at the end of the month. This is an interesting and tricky line to walk.

For what it's worth, one lesson someone taught me is that when you are in business for yourself, build reusable frameworks, code generation tools, and "engines" to severely chop down the bootstrap time on a new project. In reference to your example, Trevor, I wonder whether the extra time you've had to spend is inherent to PHP or whether after five similar PHP projects it wouldn't be an issue anymore... But here again that tricky speculative generality probability question arises again: how many more project like this one will I be building in the future, and how soon? ;-)

Thanks for reading my long-winded comment. One of the "principles" in my forthcoming book, Principled Programming, is called the Principle of Proportional Effort. This discussion is something I'm been working on for that.

Thanks again, Trevor, for the comments.

Dan

Another solution I use when there's a tough call (particularly if time and resources are tight) is to is to compromise by taking one step back away from the full generalization I'm considering. Hard code it, so to speak, for now, but leave the door open for later if the generality should prove necessary in the future. An indirection layer is your buddy here (classic Parnas information hiding): hide the thing that might change behind an indirection layer.

EXACTLY!!! The scope of my current project is to design an administrative interface for in-house employees to manage property development. I had a demo today to show them the functionality thus far, and they're now talking about getting the buyers of said properties to participate in the process -- uploading required forms and so on. Reading your post made me think of a prime example from my current project where in some cases you can kind of anticipate where the client's going to go but you don't want to make any concrete pre-suppositions. But just in case, what I did was make a conscious decision to structure my pages as classes so that should the time come to put out a public page that might need to leverage the functionality or perhaps a service API, then the conversion from page to class is easy. This is a technique that can work in JSP and .NET as well as PHP since those two environments create their pages as classes anyway.

One should never stop thinking in generalities but, as you point out, it's a fine line where sometimes you have to say, "OK, I know I'm not going to need this logic anywhere else for this stage of the project. But...I might in the future, so how can I write this as tightly as possible so that I can share it if I need to."

Bottom line -- ALL CODE SHOULD BE WRITTEN THAT WAY.

Trevor, I wonder whether the extra time you've had to spend is inherent to PHP or whether after five similar PHP projects it wouldn't be an issue anymore...

Well, I wouldn't say 60 hours to develop a framework of 30 classes is too bad. That's an average of two hours a class. Some classes take 10 minutes (property-bags and query structs) and some take 8-12 hours. I had a pro bono project before this one where I developed a LARGE part of the security model and inheritance scheme for the admin versus public pages I'm using. I also used the pro bono project to cut my teeth on strategies that work well in a PHP environment. Had I not had that previous project and come to PHP a complete newbie, I would have been burning non-billable hours right and left in order to come up to speed. As a consultant developing the same types of frameworks over and over again, you can be sure my goal is to build a common library that I can use in multiple scenarios. Who wants to write their authentication and security model from scratch every time -- or their data access layer? And you'll often find that strategies transcend language. Language after all is nothing but syntax.

Following from that there's a nuance that's much too long for me to go into in this reply -- gauging what techniques must be brought to bear for an established company doing a set amount of volume versus a start-up who's just exploring the waters of internet technology. This particular company is just testing the waters and frankly are excited by just seeing what's possible. However I've set the expectation very clearly from the outset that if this looks like it's going to be valuable to their business we'll need to graduate to a more robust architecture with our own dedicated machines. Its documented in my proposals and everything. I've worked in shops where you're dealing with thousands of transactions a minute and, while it's good to keep those kinds of strategies in mind, they have to be tempered with consideration for the domain your problem exists in.

This is exactly the sort of idea/technique I was glossing when I initially mentioned novices or outsiders interpreting a design as overly complicated. Protected variation is not complicated unless you don't truly understand the full scope of the development effort.
And ANY solution is a development effort that will have a full scope. The sort of mildly complex yet ultimately elegant design manuevers that a savvy coder will make may seem inexplicable to someone who has no insight into the profession. It's like Arthur C. Clarke's maxim about any sufficiently advanced technology appearing as magic.