Domain Specific Knowedge: How much do you REALLY need?

Right or wrong, I've heard executives (CIO types) say that while they can outsource (even offshore) coding, the special talent that in-house developers bring (and the thing that will provide them job security) is domain knowledge. However, that argument can cut both ways. Put all your eggs in a pharmaceutical basket based on domain knowledge, for instance, and then finding another job in another industry post merger/takeover could be more challenging.

Having said this, I don't disagree with your point, Andy. As you say, someone less familiar brings a fresh perspective that is not so entrenched in "the way it has always been done." However, if you stay with a company long enough, after a while you're likely to accumulate some of that offensive or valuable domain knowledge, like it or not. I know that we cringe in MIS when our clients depend on us to tell them what to do (in business situations they should handle) because they know we know more about their business operations than they do.

I will asssert that you need to have domain specific knowledge. But you can pick it up.
I think Software Gurus who go lecturing and write books are doing something really worthwhile. They are teaching the fundamentals. They matter. Chipping flints sucks.
But I don't write generic software. It solves a problem for a customer. In a problem domain. (Defense)
What happens to software developers where I work is:
They get given a task, then they rapidly have to become subject matter experts, or they end up writing software that is useless. They ask real experts questions and try to be the funnel that does the knowledge injection into the computer software. It is helpful to have developers who learn really quickly and have a background in the appropriate sciences, but with less background they just have more to learn quickly. There is a point where that tradeoff fails. We call those "no-hires."
The most experienced developers can talk to external experts in the field and not make fools of themselves.
I care about a developers ability to learn new information, not the domain thay already know because it isn't the one we work in. In four years I have interviewed one developer with domain experience in our domain. Other companies in the domain may recruit differntly; but there are probably less than a thousand people people worldwide with that domain experience, all happily working. YMMV

This post was written directly after writing the comment on Sarah's article (to which I left a hyperlink) and so, in the interest of avoiding repetiton, I left out some key thoughts.
As I mentioned in that comment, I am in no way opposed to the idea that a developer gains or needs to gain domain knowledge as they work through a project. In fact, I state quite clearly there (and in this post, actually) that as a developer/architect works a project, they need to be absorbing that knowledge. If they aren't, they are not doing their job in a professional manner.
That said, I repeat that I wholeheartedly disagree with the notion that a developer is only useful in a given business domain if they have prior knowledge of that domain.
Tim's point is both well-said and well-taken, and succinctly summarizes my view: the developer is the funnel through which domain knowledge must flow into the design and implementation of a system. They must and should be acquiring domain knowledge as they go -- if through no other process than simply by osmosis. Sitting in requirements meetings, establishing specifications, direct contact with end-users, etc should all help to provide that kind of information. And if the developer is resistant to learning it, they're also missing the point.

Having said this, I can directly respond to Tim's comment by saying that I firmly believe that one of the most important "meta-characteristics" of a good developer is that they pick things up VERY quickly. Otherwise, you had BETTER be a domain knowledge expert. Because that's what you're building a career on, not software skills.

And to respond directly to Donna, there is absolutely a value-add when a developer stays in a given domain for an extended period and accretes domain knowledge. But, in my experience, most shops--even in the same line of business--have such different cultures that when that developer swtiches jobs, the domain knowledge is of somewhat limited use. While it is temporarily a benefit (to the current employer), it is not so much so with respect to that developer's long-term goals. Keep in mind, this is EXACTLY the dilemma I was just faced with in my career -- where all I could accumulate was domain knowledge, while I watched my development skills atrophy. Balance is key, yes?

So, to furhter clarify the point I was attempting to make between my comment(s) and post: a good developer is not hallmarked by their specific absorption of domain knowledge, but by their domain-agnostic development skills coupled with by their ability to quickly establish understanding of new domains/problem spaces.

I'm not sure how successful I was in adding to the discussion, but a few days I ago I did some riffing on this thread in a related post to my blog. My basic point is that the debate about "how much domain knowledge is enough," and related debates about technical knowledge, processes, methodologies, are limited unless we consider a larger context, which I label somewhat awkwardly as a "super-domain."

On another note, thank you, Sally Knox, for adding to the discussion (see Sally's comment above). It's great to have a non-programmer perspective. Particularly, I'd like to comment on this passage:

The problem arises when the domain expert is not skillful in communicating those needs or does not know how to determine the needs. I think some of the software failures can be attributed to the fact that the users don't know what they want. But that should be anticipated.

This situation is I think common, and normal, and it has always been my opinion that the burden falls more on the technical professional to bridge the gap, have empathy and understanding for the subject matter expert's point of view, and elicit what's needed for the project. As you say, "it should be anticipated."

Some might say this is unfair, that the burden for bridging the gap should be equally shared. In an ideal (perhaps future) world I might agree. However, in today's imbalanced situation, the technical professional is the only one in the equation who is in a position to understand and appreciate *both* the business and technical points of view.

We're talking about eliciting or "gathering" requirements, of course, and there are some great books on this subject, notably Weinberg and Gause's Exploring Requirements and Karl Wiegers's Software Requirements.

I also liked this comment:

Too often, needs and design elements are determined by some group or committee sitting down with a piece of paper and drawing boxes and arrows. I would contend that that methodology will NEVER result in superior software.

I have seen this happen more than once. In fact, I think it's fair to say that this is one of the Agile perspective's most stinging criticisms of the sequential "Big Design Up Front" methodologies. The Agile solution of working closely with, even co-locating with, the customer/user is a lesson that's easily transferred, though, even to the most giant of waterfalls.

Thanks again, Sally.

Dan

And on Dan's thanks to Sally. Non-technical perspective is always needed.
But the Agile thing -- taking the emphasis off of BDUF is one of the elements I find most appealing about agile methods. It doesn't work, and what's more it gives the false impression that things are further along than they really are.

This has been an interesting discussion, but I think a few key issues have been missed.

The successful communication between the real domain experts and programmers is, in many way, the essence of software development. We've got to get what's in some guy's head (or what should be in his head) into a computer program. But non developers don't know how to think like developers and in order for them to really know what solution they need, they more or less need to be coached to do so.

This is not to say that domain experts need to learn Java (or whatever), but that they need to learn to turn their ideas and business process needs into a software-baed system that makes sense. They must understand the limitations of software. This last one is key. A program, for example, can't take a whole bunch of vastly different and idiosyncratic input and magically make sense out of it. So very many non-developers have so very much trouble with this point.

But that's not what I really want to say here. My main point is this:

To think that every single software developer will become an expert at translating the the business domain into the software domain is fantasy. Clearly this is a specialized job in itself. Even in a small shop, I think you are much better off if you have a requirements gathering expert generally act as a bridge between coder and the client.

This is not to say that at some point communication between the coder and client can't happen. At some point the coder and the client can go one-on-one to go over the finest details and to refine the requirements. But not every developer will be an expert requirements gatherer. I think everyone knows this. The world needs people who can stand all day at the assembly line and twist the same bolt again and again. That's a real job (well a figurative job in my example) and it's not going away.

I also think there are two levels of "the domain" and not one. There is the business domain in the sense of Accounting or Medicine. But then there is the specific domain of a specific solution to a specific business problem.

When you work for several years on the same tool you become an expert on that domain. You may know very well how your own accounting system works and how it integrates into someone's business process. But this requires only the slighetest knowledge of actual accounting.

An this brings up another point. So much emphasis is given in job interviews on computer programming trivia. How do you create an ODBC connection in C#? How do you call a servlet from a JSP? These things are actually quite simple and easily taught and really, developers generally have such a steep learning curve learning the ins and outs of the particular application they are working on that the amount of time it would take to teach them to create an ODBC connection (or point them to the doc) is minimial by comparison.