developer.*

The Independent Magazine for
Software Development Professionals

Hey Dan,

I just read the latest installment. Many good ruminations on a very hard to pin down subject. How can you tell who makes the cut?

I did take issue with one minor thing in your article, and I thought I'd go ahead and bring it up.

You mention that you had removed the language reference from your audition machine, and I wonder what the point of this was. I'm sure that the results you got from the test were good and that the people you hired were fine candidates. But it seems to me that you went out of your way to make your test unrealistic and to reward rote knowledge over the ability to do research and quickly improvise.

Perhaps the test you gave was so simple that the specific functions needed should be known by everyone. But still, it seems to me that, in the real world, good developers consult references all the time. Why waste hard drive (that is, brain) space with information that is easy to obtain? If you can quickly and easily look it up, it is a waste of time and energy to memorize it.

I'd hate to be given a test like that without access to the Java api or code completion. I can't say that I would automatically be able to come up with the exact correct name of every Java object and method signature that I would need. And really, memorizing all this information would serve no purpose.

I say, turn people loose with all the information they could need if you are going to give such a test. The ones who complete the task well will do a good job for you, whether or not they had to look anything up.

Just my grumpy two cents worth.

Rob

Daniel Read Responds:

Rob, you bring up an excellent point, and I really should have been more clear about this in the column. I certainly agree with you in principle regarding the rewarding of "rote knowledge over the ability to do research and quickly improvise," as you put it so well. I am generally against that sort of thing, and it's a big problem with many language certification tests I have taken. That said, I do think it is reasonable to expect an experienced developer to have memorized a great deal of basic stuff about the language and platform.

I would like to clarify my decistion to not have the documentation installed on the audition machine.

• First, the language being used in this test was Visual Basic. The Visual Basic IDE has two features that should keep a developer from needing a language reference: automatic code completion as you type, and an object browser. The entire Visual Basic language--including function names, parameters, return types, constants, etc.--is available right there in the IDE.
• Second, the test itself was indeed very simple, and each candidate only had an hour to complete it. The test involved building a calculator engine using an abstract class to define an interface with concrete classes implementing that interface: one concrete class to handle whole numbers only, and another concrete class to handle single precision floating point numbers. Not a terribly realistic example, but useful nonetheless. Very few actual language functions were needed for this since the program only needs to add, subtract, divide, and multiply numbers. When I mentioned in the article that certain candidates could not remember basic language functions, I was talking about the CLng() and CSng() functions, which cast any numeric value into a Long or Single, respectively. I don't think it's unreasonable to expect a developer to remember basic type casting functions like this--and like I said, if a developer knows where to look, the names of the functions are right there in the IDE--no help file required.
• Third, in the case of Visual Basic 6, the installed help file is not just a help file--it's the entire VB-specific MSDN library. There's no way I know of to install the language reference without installing the entire VB MSDN library. The VB sections of the MSDN library contain exact instructions and examples on how to use abstract classes to define interfaces and how to create concrete classes to implement that interface. I did not want this information available to the candidate since this test was a design test as much as a coding test. If the candidate could not define an abstract class and a couple concrete classes without reading the MSDN library, then I would not be interested in hiring him.

I hope this clarifies matters, Rob. You are exactly right, of course, that valuing memorization over improvisation is a mistake. I would have been better off leaving that sentence out of the essay if I was not prepared to explain the above specifics.

Thanks,
Dan