I just read the latest installment. Many good ruminations on a very hard to pin down subject. How can you tell who makes the cut?
I did take issue with one minor thing in your article, and I thought I'd go ahead and bring it up.
You mention that you had removed the language reference from your audition machine, and I wonder what the point of this was. I'm sure that the results you got from the test were good and that the people you hired were fine candidates. But it seems to me that you went out of your way to make your test unrealistic and to reward rote knowledge over the ability to do research and quickly improvise.
Perhaps the test you gave was so simple that the specific functions needed should be known by everyone. But still, it seems to me that, in the real world, good developers consult references all the time. Why waste hard drive (that is, brain) space with information that is easy to obtain? If you can quickly and easily look it up, it is a waste of time and energy to memorize it.
I'd hate to be given a test like that without access to the Java api or code completion. I can't say that I would automatically be able to come up with the exact correct name of every Java object and method signature that I would need. And really, memorizing all this information would serve no purpose.
I say, turn people loose with all the information they could need if you are going to give such a test. The ones who complete the task well will do a good job for you, whether or not they had to look anything up.
Just my grumpy two cents worth.
Daniel Read Responds:
Rob, you bring up an excellent point, and I really should have been more clear about this in the column. I certainly agree with you in principle regarding the rewarding of "rote knowledge over the ability to do research and quickly improvise," as you put it so well. I am generally against that sort of thing, and it's a big problem with many language certification tests I have taken. That said, I do think it is reasonable to expect an experienced developer to have memorized a great deal of basic stuff about the language and platform.
I would like to clarify my decistion to not have the documentation installed on the audition machine.
I hope this clarifies matters, Rob. You are exactly right, of course, that valuing memorization over improvisation is a mistake. I would have been better off leaving that sentence out of the essay if I was not prepared to explain the above specifics.