In Making The Cut, you mentioned that phone screening was part of your hiring process. This is something which I've wanted to include in my hiring process for awhile, but am not quite sure how to approach it, like which questions should be asked then over the phone, and which ones best asked in person. Can you offer some help on this topic?
How do you determine the design expertise of a developer? I throw in a couple of snippets of poorly designed code and ask him to refactor it into a better design. How do you approach it?
Daniel Read Responds:
I can appreciate your predicament about how to best go about phone screening. I don't necessarily have a one-size-fits-all formula, but I can tell you a little about my approach. I keep each call to about 25 minutes. What I do is block out a bunch of 30 minute time slots throughout the day and have the recruiters I'm working with schedule appointments in those slots based on the resumes I liked.
My basic technique is this: I start the person out by asking the candidate to describe her most recent positions: what was she working on, what was her role, what was the architecture and design of the system, etc. I listen closely to what she says to get a sense for the sophistication of her understanding. I probe the candidate with specific questions about the systems she is describing. I also look at the resume in front of me and ask specific questions about the technologies the candidate listed with each position. Many times people list technologies for a position that the team used, but that they never personally touched.
After about ten or fifteen minutes of this, I switch to asking the candidate some standard questions that I ask everybody. My favorite litmus test is "Can you tell me what coupling and cohesion are?" Most of my questions, though, fall in the realm of technology-specific questions. I want to try to determine the level of this persons mastery of the tools used on the project I am hiring for. Some of the questions are pass/fail type questions: if you miss it, you are off the list. However, not every question is a do-or-die question. For example, depending on the level of developer I am trying to hire, I might not necessarily disqualify someone if they can't tell me what coupling and cohesion are. It all depends on what type and level of developer you are hiring.
As for determining design experience and knowledge, that's a little tougher to do over the phone. I tend to look at this more closely in the in-person interview, especially with the "audition" that I wrote about in the column. The audition specification I used in this most recent candidate search is as much a design test as it is a coding test. On the phone, though, when I ask the candidate to talk about her most recent positions and the systems worked on at those positions, I am largely probing for design and architecture knowledge. I ask the candidate to describe how the system was laid out: how the tiers were designed, how state was passed back and forth between the tiers, how errors and transactions were handled, what criteria was used to decide what kind of code would go into a stored procedure as opposed to the middle tier--that sort of thing. If the candidate can answer these kinds of questions in detail, then I can get a pretty good sense of her knowledge and experience.
Another good way to determine design knowledge over the phone is to ask about design patterns. Does that candidate know what they are? Has she ever used them in a system she has designed? Which ones were used? That sort of thing.
Another good area to ask about is database design. I ask, "Have you ever designed a database from scratch?" If yes, then I ask how big it was, how normalized it was, was it mostly transactional, mostly for reporting, or a hybrid. Stuff like that. I also ask how many databases the candidate has designed.
Yet another good area to probe for design knowledge is object-relational mapping. Does the candidate even know what that is? In cases where the database exists before the objects, what general criteria would one use when mapping objects to a relational database? Would the candidate generally prefer to design the database first or the objects first? (This last question does not necessarily have a "correct" answer, in my opinion. I just like to see if the candidate has an opinion on the subject.)
Your idea about refactoring is a good one, but I would think that would work best in person as an "audition" exercise.
I hope this helps you to introduce phone screening into your interview process, Kelvin. I have definitely found it to be a time-saving technique since you don't bring someone in for an in-person interview until you are pretty confident about his abilities.
Further Response From Kelvin Tan:
After reading Daniel's response, Kelvin had the following additional comments. Kelvin's comments are dead-on, especially regarding the complexity trade-off with design patterns, so we wanted to share them.
The question about OR mapping is something which I intended to throw into my next interview even before you mentioned it, and I'm really glad you did. It's really quite amazing how many ways there are NOT to do OO-programming with an OO-language (Java, C++, etc). From my experience, a really healthy number of programmers actually perform what I call database-oriented programming instead, where the "objects" are really database tables. Inheritance is really a familiar but little-used concept to these people.
One comment about design patterns I have is this: following the growing popularity of design patterns (as a tool for design as well as a recruiter question), many programmers have taken to studying GoF's bible, and may be familiar with them from an academic level. What many don't realize is that design patterns are introduced primarily to provide flexibility and additional layers of indirection at the cost of greater complexity. A design composed of multiple patterns may be more elegant, but it's usually more difficult to understand. It's kind of like a trade-off, which I want the interviewee to be aware of. One thing I like to do is throw in a question about the Singleton pattern, without telling them that it's a design pattern question. And even if the programmer isn't familiar with design patterns, he should have encountered the need to ensure an object is only instantiated at most x number of times in a system during his journeys.