After my session at JavaDay I had a conversation with a colleague from Sun on technical interviews. We started with the question whether a Java candidate should be in the details of Generics (e.g. know the term "erasure" and what it means). I'll touch this in a moment but let's begin in an ordered manner.
What do you need to achieve in an interview?
The famous answer comes from Joel Spolky's "Smart & Gets Things Done" known article (it has an updated version, but I like better the original one. And it got out by now also as a book carrying the same title!) Sorry for the massive PR for Joel (he is NOT my cousin), but this is really a great source to begin with.
Following Joel, let's just mark out that you need to achieve two things in an interview:
- Attract the candidate: sell yourself, your company and the offered position. Make sure that if you want this person you will get him.
-- This is highly important and sometimes neglected (e.g. making the candidate wait for his interview for 30 minutes), but we will keep this for another post. - Make a Hire / No Hire decision (is the candidate Smart and Gets things done?)
__________________
What do you need to know about your candidate?
- Communication skills - can he explain himself well. Ask him about his previous job or big project at college, see if he can well explain what he did.
- Does he see the big picture? - continuing with his big project, make him explain what other did in the same project, ask about other alternatives that were there to implement the same thing and why the one taken was selected.
- Technologies - ask about the technologies used in his big project and what can he tell about each of them.
- Design - you can learn about the candidate's design abilities by talking about the design of his big project, but it would be better here to tackle him with something he haven't thought about or at least didn't talk about in his previous interviews. It can be a pretty simple design, like some kind of LRU Cache, or Copy on Write mechanism, or anything else you have in mind. But make sure to listen - ask shortly, give enough time to think and then listen. This is a good opportunity to see how the guy thinks.
- Implementation - if the candidate is going to write code, let him do it in the interview. You will be amazed how many fail at this stage. There is a debate whether the code phase should be on paper, on a PC but only with a Text Editor (and Help?), or with a full-blown IDE. Each of the alternatives may shed light on a different interesting angle. The super-candidate should excel in all of the environments, he may be using all of them in his day-to-day life simultaneously. What would we say about a candidate that gets the IDE but when lag-behind gives the excuse that he is not used to this IDE? I believe paper exam for short implementation is good, as long as you tell the candidate that it's OK if he makes small syntax mistakes. The code should be considered as pseud-code, but all important aspects should be included (e.g. resource management and exceptions). Then of course he has to explain what he did.
- Reading other people code - many developers are really good in writing their own new code but do miserably when it comes to working with other people's code. During the implementation phase you should check this with your candidate, give him some strange API that he has to use, suggest your implementation to one of the methods and ask for his opinion (it might be a good thing to throw into your implementation some obvious bug).
- Effectiveness, Tidiness, Laziness - do not ignore the tidiness you see during the interview. Usually this is what you will get later. Pressure is not an excuse for a mess. Watch the way the code is placed, either on the paper or in the IDE. Laziness is OK as long as it does not effect the essence. In fact laziness is something you want to find in your candidate. You want to see that he gets things done and for that he should not finish some of the assignment you gave him, but it should be the trivial part. A candidate that cannot live with an unfinished task in a way that prevents him from even doing the parts that he could have finish, is someone that does not get things done. Effectiveness means starting with the tricky stuff leaving the trivial for the end. Identifying what is tricky and what is trivial is being smart.
- How does he fit the position? - this is the place, after presenting the position and the assignments he would have to handle, to ask the candidate how does he see himself in this position, what he likes and what he doesn't, what he is strong at and what he still has to learn. This is the place to see that the candidate is attached to reality. You saw him already, you got some feelings and impression about him. Does your impression fits what he says about himself?
----------------------------------
Addition, 12/6/2008:
----------------------------------
Asking for the candidate's grades, maybe even his SAT score (Psychometric exam called in some countries) - can also be a smart thing, assisting you to pinpoint the smart guys. It doesn't necessarily help, this is why you are having the interview, but it may shed some more light. And in my experience, even if there isn't a complete correlation with performance, high grades do say something and poor grades also have their say. Usually, not a surprise, a guy with high grades would do better than his fellow with poor grades. But do not let the grades confuse you, I saw some promising candidates with marvelous grades who may barely complete an if-else.
----------------------------------
How much time should it all take?
Including the first part of attracting the candidate (speaking about the position, the organization, and yourself) - not less than 90 minutes. 2 hours may be reasonable.
__________________
Is it a true / false question?
Joel says it is. There is only Hire / No Hire, nothing in between.
I agree, if we have the budget and attraction to get all the MacGyvers that we need, then yes. I do not mean that all employees should be superheros. They should, however, be superheros in their field. If we need someone to handle XML configuration files we need someone who is a master in this domain, whistles XPaths in his sleep and grains XMLs with XSLs daily. He will not find his duty boring because he will deal with making it more efficient by inventing new utilities. Same thing would be with a secretary, we will want someone who knows how to create table of content in documents and use sophisticated attributes of the spreadsheet.
But what if we cannot attract superheros in the domain? Suppose the budget we have for developers doesn't bring us the most brilliant ones and we have to compromise.
Building your compromises
- The candidate should have a positive marginal output. Writing bugs gives negative output, writing bugs half of the time may also. When saying positive marginal output, it means that you have to take into account the effort and investment to be made in this candidate if recruited.
- Marginal output is related to the current mix of your team. If you have "good thinkers" but lack "coders", head for those who can code fast, even if not considered design superheros.
- Remember that there are qualities that can be gained (e.g. experience), while some other qualities are inherent. Prefer smart over experienced and capable over well trained. A smart novice can quickly become smart and experienced.
Now back to the question we begun with. Suppose we recruit a Java developer. Does the candidate need to know what is "earsure" for Java Generics?
Answer is simple: if you already have in your team someone who read the Generices literature and can serve as the team knowledge base, it's not the end of the world if the candidate does not know it. The team will have the knowledge. BUT - if you are now seeking for this "curious developer", make sure he proves his curiosity!
Summary
Hiring decisions are of the most important decisions made by managers. The damage that can be made by an unfit employee are huge. Investing time in an organized recruitment process and in each of the relevant candidates is a must. At the end, the employee selected must have a positive marginal output that justifies his salary. Marginal output is related to the current mix of your team, so recruitment decisions should be made with the set of people that you have in mind, and with understanding of the qualities that you miss in your team and want to focus on.
----------------------------------
Addition, 12/6/2008:
----------------------------------
Is Smart and Gets things done the all thing? What about motivated, doesn't get bored too quickly, having a good temper?
You may want to read these comments on Joel as well.
----------------------------------
2 comments:
Amir,
Thanks for your writeup, it was a great follow-up to our conversation.
I appreciate your including an assessment of the amount of time an interview should last. Unfortunately I don't think most people spend this much time with a candidate. Here we try to have a number of people talk to each candidate, each one taking about 45 minutes. There is a trade-off between respecting the candidate's time and trying to really have an informative meeting.
I would question the order in which you presented the topics to discuss. Talking about the previous job can very easily get out of hand and most of the interview time can pass without you knowing anything about the candidate's hard skills. I think this part of the conversation should come later in the interview, after you've established that the candidate has coding skills and the other basic qualifications for the job.
Having said that, I really liked your suggestion to have the candidate explain what OTHER people did, what were the alternatives, and why was the selected one chosen. I can definitely see how this would lead to interesting and relevant conversation.
When I was a job candidate, at all of my toughest interviews for the jobs I was most interested in I was asked to write code on a white board. I know first hand how hard this can be and am not sure why so many people feel it's important to make the candidate write code on the board. I always offer the option to write on the board or on paper, wherever the candidate feels more comfortable. Anything that makes him or her more nervous is just interference - what interests me is the code. I keep the exercise short enough so that it is reasonable to do it without an editor. I think it is important that someone remember the syntax for a for loop without his editor's code completion (yes, I have interviewed people who couldn't write a for loop without the editor).
As for reading other people's code, what I have done is given an assignment and then an implementation of the assignment that doesn't work 100%, and asked the candidate to find the problem(s) and suggest an improvement. This lets me assess his or her ability to understand written requirements, form reasonable assumptions about where the edge cases are and where bugs are likely, understand someone else's code, and suggest reasonable improvements other than "rewrite it from scratch."
I very much like your comment "Effectiveness means starting with the tricky stuff [and] leaving the trivial for the end." Solid advice. At the same time, it is reasonable advice to a candidate that (s)he should always put *something* down, and that simple code that works is better than complicated code that is broken. The trick for the interviewer is to find a simple but interesting challenge so that there is something non-trivial to figure out, but not so complicated that implementing it will take most of the interview time.
As for "how does he fit the position," I think that much of this question should be cleared up-front, perhaps even in a pre-interview phone call. Too many people come on interviews because a placement agency sent them, but they are looking for a team lead or managerial position while we are looking for a developer. Many candidates never see the job description, the connection is made by a placement agent. Before inviting the candidate to come in the company should make clear (again) what the requirements are of the open position, and what the interview will involve. If the candidate is uncomfortable writing code then (s)he can say so up front and save everyone some time.
I would add to your compromises section the suggestion that there is always an element of how fast do you need this person to become productive? If you have the leisure of hiring well in advance, then you can afford some training and ramp up and should look for the brightest person you can find -- (s)he'll learn the specific technologies on the job. But if you need someone to immediately be a productive contributor to your project, you can't afford the ramp-up and need to interview for specific skills.
Finally, I would suggest to all candidates out there that it is important to prepare for your interviews. It's not hard to find out what the company does and what skills are of interest to them, even if this requires a phone call to the HR department or hiring manager before the interview. Brush up on your CS basics. Serious companies are going to ask you about data structures and algorithms, and "it's been so long since school" is not an acceptable answer. Showing the interviewer that you've done some preparation and have refreshed your memory of the skills *listed on your resume* will go a long way to establishing you as a credible and serious candidate.
Moshe, thank you for the conversation and the comment.
I must emphasize that my post is focused on the face-to-face interview. In order to be efficient I recommend on a phone screening in which the candidate history and fitness to the position can be checked, by some conversation and even a small and easy question (e.g. some simple algorithm question). So CV and phone interview should be the first screening (I usually try not to screen too much based on CV, it worth the 20 minutes over the phone to make sure you don't miss someone good who doesn't know how to write CV. I met a few). After screening some of the candidates according to CV and phone interview, there should be enough time left to invest in long face-to-face with the rest. And during the time the candidate is writing a solution in the face-to-face interview I can go back to my desk for some short assignments.
I agree that respecting the candidate's time is important, I plan to write on "respecting your candidate" sometime, but a long and serious interview (in which the candidate can also learn new things!) can be built in a way that wouldn't be considered as a waste of time.
Thanks again Moshe!
Post a Comment