Are Brain Teasers Good Hiring Criteria? 672
theodp writes "Your brain teaser prowess may win you a job at Google, but the folks at 37signals don't hire programmers based on puzzles, API quizzes, math riddles, or other parlor tricks. 'The only reliable gauge I've found for future programmer success,' explains 37signals' David Heinemeier Hansson, 'is looking at real code they've written, talking through bigger picture issues, and, if all that is swell, trying them out for size.'"
Those of you who have hired employees: have you seen correlation between interview puzzle success and job competency? How should an interviewee best handle these questions?
Bring puzzles as an applicant (Score:5, Interesting)
I had a hedge fund ask me physics problems (Score:5, Interesting)
Needless to say unless you spend time puzzling over this specific type of problem you don't have the skills to answer them.
The impression I had was they were going through a dog and pony show of "trying to find a candidate" for their position. I am not sure what they were up to. Whatever it was, they weren't looking for a candidate for the advertised position.
There was an absolute reek of duplicity, insincerity and dishonesty about every single employee I met on that interview, starting with the prostitute-cum-receptionist who greeted me to the project manager who wouldn't look me in the eye to the interviewer who looked over my resume (which had only a distant physics class) and said "we're not going to ask you about programming, I can see you've got that down, we want you to solve some puzzles" and sprang on me some physics puzzles I could only solve if I were a physics major.
I couldn't wait to get out of there.
I saw that ad for a few more months online. I always wondered what they were up to.
My thoughts and reply (Score:5, Interesting)
In short, I disagree. I find brain teasers invaluable. But not in determining skill, but in determining personality and how a candidate behaves when they are faced with a challenge that they aren't familiar with...
Re:Puzzles aren't to test programming skills (Score:5, Interesting)
I ask candidates puzzles (Score:5, Interesting)
I also look at samples of previous work, and we make all candidates carry out real world tasks along side us.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
Yet when the people hiring are other programmers or (as in my case) artists, they still feel the need to ask these questions because that's how they THINK they should be interviewing people.
How do you think I got my job? I BSed my way through their stupid questions and when they finally asked pertinent programming questions I answered intelligently. From what they told me afterwards most people managed to BS the first set and failed on the second.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
God I always hate those fucking questions. "Why did you chose to apply with us?" Because I need a fucking job! Why else do people apply for a job? Why is that not enough? "Where do you see yourself in five years?" Uh, gainfully employed? Do my life goals really matter to whether or not I can fill this position? What if I saw myself working at the fucking circus in five years, would that have a bearing on whether or not I was hired? Why? "What are your goals?" To make enough money to pay my bills with a little left over for fun once in a while? Is that too mundane?
Man, I despise interviews. I fantasize about going all Peter Gibbons in Office Space [youtube.com] every time someone asks me one of these stupid, irrelevant questions, but my sense of self-preservation reigns in those crazy ideas.
Personally, when someone asks me why I chose to apply with them, I've got a very good reason. When someone asks me "Where do you see yourself in 5 years", I tell them the truth, and then I ask them what they will do to help me get there. If they answer wrong, I walk. And when they ask me to engage in meaningless work so they can judge me, I tell them they're welcome to judge my portfolio, but if they want me to start problem solving, the meaningless of the task is irrelevant... they're still going to have to pay for it.
You can weed out most bad employers in this way. Not all of them, but most. It helps if you have 3 months salary in reserve for emergencies like you should so you don't end up entering a bad situation out of desperation.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
There's a technical ladder? Anywhere I've ever worked, it's more like a stool - start a decent distance off the floor, then go nowhere.
Sort of (Score:4, Interesting)
If you're using brain teasers as pass-fail criteria, then you're stupid. And if you're using them in an interview process that lasts less than an hour or two, you're even more stupid. They can be good for understanding a person's thought process during problem solving, but that's it. It's not the answer, but how they come up with it. That being said, the "how many gas stations in such-and-such city" where you have to pull estimates out of your ass are not good for choosing programmers, so don't even go there.
I used to work for a big company that you've probably heard of. When we interviewed people, our group had a brain teaser that we liked to use, probably because the answer (and there was a fixed, definite answer) was not the obvious one. And we got to draw pictures of it on the white board while asking it. But it was the programming test that really mattered to our group.
First of all, we had them do it on a white board in front of the group. (This was after all the individual interviews were done, and we had warmed up the group part of the interview with a brain teaser or two.) We weren't looking for getting API parameters in the right order, just that you could do the algorithm on the fly, and in a less than quiet environment. (typical cubicle farm level noise from us chatting to each other during this) We didn't even care what language they wanted to write in, the point was getting the algorithm right. And if they got something wrong, we would tell them how the output would be wrong, and let them fix it. Again, the goal was to see how they write code, and show us how, not that they could spit out the right thing from memory.
First was to implement strcpy. Any C programmer (our stuff was mostly C++) should at least understand how strings and pointers work to build something around *p++ = *s++ with a loop. So you probably got an off-by-one error, so what, we point it out, you fix it, but you at least got the basic idea right if you got even that far. Second was to write code to copy a file, since you should also be able to understand how to get data in and out of files. Then we would ask how to make the file copy faster, since most people would try the one-byte-at-a-time approach, and you ought to know about buffering, too. Finally, reverse a singly-linked list. This is something that any CS student should learn in their second year Data Structures class. Not to memorize it (because it's kind of pointless to memorize such a function), but to figure out how to do it from scratch. If your degree says "Computer Science" on it, you should be familiar enough with how to walk down linked lists to at least make a decent start on this one.
Well, guess what. The fresh out of college CS grads generally failed horribly, especially the ones that had been weaned on Java, where you don't have to deal with pointers like you do with C and C++. It was really stunning and even sad to see people fail at this. (The EE grads did much better, FWIW.)
Wrong question: is a hammer "good"? (Score:4, Interesting)
Ive been on both sides of the table plenty and have both faced and given brain teasers. To say they are inherently good or bad hiring criteria is thinking of it the wrong way. Its just one tool in the toolbox. A hammer isnt inherently good or bad, you use it when its appropriate and not otherwise.
I personally put little stock in them, except that I love to get a wiseass answer because it shows personality. For example, I got hit with the Google-ish "how many golfballs fit in a schoolbus?" question once. My answer, almost immediate, was: "Come on, thats just silly, everyone knows golfballs do not ride the bus when they go to school... they don't need to, they're balls, they just roll!" The interviewer absolutely loved that answer.
To me, there's far better ways to evaluate a candidate. For a programming job its actually easy: give them a real-world task typical of the position, tell them they have as much time as they need, set them up at a workstation, show them where the bathroom and snack machine is and give them some space. See what they produce in that situation.
Re:Well, they're a good indicator of intelligence (Score:3, Interesting)
Re:Well, they're a good indicator of intelligence (Score:3, Interesting)
As a person who is involved in hiring, we just want to get a sense of whether the position is right for you or whether you'll just flake out because you rather be doing something else. For example maybe in 5 years you expect to be a manager or a team leader but we don't expect openings. Or the opposite, maybe we do expect an opening and we are looking for someone with aspirations to become management so we can groom them for that position. Personally I like to see candidates achieve their objectives in our company. So if we can hire them on and both our goals align then both parties can potentially be satisfied or even happy.
It may seem like a stupid question but it's actually quite relevant. I guess relevance does depend on the job though.
Re:Well, they're a good indicator of intelligence (Score:4, Interesting)
If you did what you describe when applying to the company I work for, you would not be considered for employment. And even though I may disagree with some of our candidate selection processes, we tend to hire great people and the work environment / compensation / benefits are awesome.
Re:My thoughts and reply (Score:4, Interesting)
I find brain teasers invaluable. But not in determining skill, but in determining personality and how a candidate behaves when they are faced with a challenge that they aren't familiar with...
Exactly. At my last job when interviewing candidates with coworker (software team would hire other team members, a practice I thought worked well) we would use:
All the questions had value.
On a team personality match is a huge aspect. Questions should bring that out (such as "shooting the shit" during an interview to put the person at ease, e.g. after a teaser). Of course that means the interview must be performed by at least one member of the team. Generally we'd have a pair interview and if the candidate went further they'd meet the rest of the team (rather than a panel grilling them). It doesn't matter if the candidate is super coder who can code at 80wpm if they can't communicate and are at odds with the rest of the team. Those type of resources can be useful if you can apply them correctly, however a good candidate that can work well with the team (in my experience) is a lot more valuable than an excellent developer that needs to work solo.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
If you don't have a specific reason for going for *that* company, you wont get the job. You *MUST* show that you have spend at least some time in researching what they do and why you think it would suit you to work there. There are perhaps a hundred people applying for that job, you need to make you look better than the others.
What I want in 5 years? What I would really like, a boat in the Med, an Audi R8 and enough money to live like a spoiled brat - in fact I'd love to live like that right now. On a more realistic outlook I'd like for x,y,z etc. E.g. manager position, advanced field stuff, travelling whatever.
But if you can't answer it and show you've thought about it they will almost always pass.
To be frank, these questions are "designed" to vet people like you. My gut feeling just from reading your post is you wont fit in - and that is your biggest problem; it's not about being the worlds best at whatever you do, if you can't sell yourself you might be the next Bohr, but still not get a chance at proving yourself.
Oh and for all the puzzle/whiteboard programming haters out there. I was once tasked with doing a hashmap on the whiteboard as part of an interview. Instant thought was "Fuck!? who would ever remember those fuckers?"; but I went to the blackboard outlined most of the map and when it came to the actual hashing algorithm I told them that I had no idea how to do that, but knew what book to look in. Afterwards they offered me the job and specifially complemented me on how I handled the whiteboard task.
If you get asked to do a puzzle or whiteboard test, do it, get it over with - yes you might not have access to your favorite cheat sheet, but thats life. Sometime you might end up in a room full of clients and act like you are on top of the problem even though you have absolutely no clue what just went boom. It's all about selling yourself and showing you can handle pressure.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
Actually, many large companies have parallel engineering and management ladders. The goal of this is to allow your best technical people to advance as individual contributors without moving into management (which they may not be their strength anyway) For example, at my company, Architect and Director have identical HR classifications. Likewise for Fellow and Vice President. If you are not advancing as a technical employee, you should look at yourself, just like if you were not advancing as a manager.
climbing up the technical stool (Score:5, Interesting)
There's a technical ladder? Anywhere I've ever worked, it's more like a stool - start a decent distance off the floor, then go nowhere.
That's a function of the size of the company and industry you are in. In general, the technical ladder (or stool) becomes steep very quickly. And as you climb it up, you start to see that you do more management than actual hands-on thingie-building. But do not delude yourself into thinking that this type of management is of a non-technical nature.
Software Architect. Enterprise Architect. Technical Lead. Principal Engineer. Technical Director. Chief Scientist. Let's call these upper-stool technical positions.
These types of positions require you to do less hands-on stuff, but the management you will have to do must be technical-oriented. How you assign technical tasks to people and teams will depend on whether tasks are technical feasible, on identifying the technical capabilities of your team, on understanding the resources required to complete technical tasks.
Granted that a lot of people who get into these positions let go of themselves, gradually detaching themselves from the technical realities on the ground, where the pedal hits the metal. And as a result, their decisions are no longer technical, with technical consequences that is beyond their grasp. But those are examples of doing a bad job in their positions. And that exists at all levels, from the uber-chief of technical reality down to the lowest code monkey.
These are the fabled paper tigers.
That is, being detached of technical realities is not an inevitability of working so high up the ladder/stool. Good technical people remain strategically and tactically technical always, regardless of their pecking order. A good above-the-clouds architect can drop back to code with only a few days to clear the mental cobwebs. A good technical foot soldier can extrapolate the reasons behind good high-level technical decisions, even if he/she does not have the management experience (which naturally they don't at their entry level of their careers.)
My suggestion to people who find themselves staring at the technical stool: put another stool over it, secure it with nails, crazy glue or some other good shit, and then climb it. That is, like a good engineer, you need to engineer and build your technical ladder.
This can only be done without realizing first that to climb it, you will have to gradually move away from hands-on work without losing your technical wits. You cannot allow yourself to become a paper-tiger.
This will also means that when you find yourself at a company where there is nowhere else to go but down (because the stool cannot go any higher for whatever reasons), then it is time to go somewhere else where there is a chance to nail/glue another stool over the one you have built so far.
Re:Test (Score:5, Interesting)
Interviewer: You're in a desert, walking along in the sand, when all of a sudden you look down...
Applicant: What one?
Interviewer: What?
Applicant: What desert?
Interviewer: It doesn't make any difference what desert, it's completely hypothetical.
Applicant: But, how come I'd be there?
Interviewer: Maybe you're fed up. Maybe you want to be by yourself. Who knows? You look down and see a tortoise. It's crawling toward you...
Applicant: Tortoise? What's that?
Interviewer: You know what a turtle is?
Applicant: Of course!
Interviewer: Same thing.
Applicant: I've never seen a turtle... But I understand what you mean.
Interviewer: You reach down and you flip the tortoise over on its back.
Applicant: Do you make up these questions? Or do they write 'em down for you?
Interviewer: The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can't. Not without your help. But you're not helping.
Applicant: What do you mean, I'm not helping?
Interviewer: I mean: you're not helping! Why is that?
Interviewer: They're just questions, Leon. In answer to your query, they're written down for me. It's a test, designed to provoke an emotional response... Shall we continue?
Re:programming is not what programmers thing it is (Score:4, Interesting)
Anyone who can clearly explain a topic in English probably won't write readable code either.
Is this a clever dig at native English speaking programmers?
(which 7 variables do you need to track in your mind at any one time)
After 15 years of business programming, a smattering of embedded and OS programming, and a lot of algorithm programming competitions, I can't imagine a kind of programming where "which 7 variables do you need to track in your mind" is of any relevance. If this normally comes up for you, you're doing something very wrong.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
current salary?
try this: companies are now asking (expecting!) your WHOLE CAREER SALARY TRACK!
I laugh and refuse. I don't even give current salary, I give a range of what I expect for THIS job.
one guy even asked if the # I gave for my salary was verifyable. I asked him 'this is a trick question or test question isn't it? there's no legal way you can 'verify' my salary, so who are you trying to kid, here??'
of course, some actually want COPIES of your pay stub to prove it.
I walk away from such places. those are big red flags that there will be mgmt trouble later on, at that kind of place.
look, if you want to hire me, stop playing games. but if you want to start things off poorly, start DEMANDING I tell you private things, like how much I made last year. that's none of your fucking business, mr corporate asshole hiring mgr. talk about rude questions - and the fact that so many people just give in and answer them! astonishing.
then again, its a horrible time for job seekers right now. they have us by the shorties, as there is NO bargaining and NO unions to help us keep the big co's in check and in their place.
Funny anecdote (Score:5, Interesting)
I once applied as a programmer to work on a server infrastructure for a next-generation search engine. They were looking in particular for people with great expertise in the C++ language and in the Boost libraries, areas in which I was a very good candidate.
They asked me to perform a task and send the result by email before meeting me in person.
The task was to write a program that would take an integer n, and display the nth integer that satisfies a particular condition involving primes (I have forgotten what the exact condition was). I was told I would be judged on the performance on my program.
It was obvious that what they wanted was for me to know the mathematics about primes so that I would know the right formulae to compute the nth value quickly. As I didn't know them, it was irrelevant to the job I was applying for, and I didn't want to spend time researching it on the Internet, I chose to fit their requirements differently.
I computed all the values beforehand, and simply made the program return the nth value of a table. Technically, it fitted the specifications they had given me exactly, and was the fastest solution possible.
Yet they chose not to make me go to the next stage.
Looks like brain teasers don't like being beaten at their own game...
(Another funny thing about this event is that I sent the code to the person as a tarball, and he was unable to open it and asked me to send him a zip instead.)
Re:Well, they're a good indicator of intelligence (Score:4, Interesting)
What they actually determine is whether the interviewee can correctly guess what the interviewer wants to hear. But even if they worked, would they really help you? After all, the goals of both people and organizations change, and long-term employment under the same employer is pretty rare nowadays.
Re:Well, they're a good indicator of intelligence (Score:4, Interesting)
There's a degree of interview technique required (on both sides). Avoiding an 'over qualified' label is often merely a case of expressing a career direction you'd like to take, and identifying some of the skills you'd like to learn that will help take you there.
In one swoop you've shown self-awareness, a reasonable level of ambition and a willingness to learn. As an interview I also now get to discuss how you might gain those skills within the role.
I've been a candidate before where the interviewer has flatly stated, "I'm not sure we'd be able to offer you " in relation to a specific soft skill - but then offered me a job anyway. It came down to them knowing that they aren't my ideal job, and me deciding whether they ticked enough boxes or not - the job certainly was appropriate in many regards, so just how much did I value that one skill (and could I find other ways of gaining it, outside of work). I don't go into the wrong job and they don't hire someone that's going to walk straight back out again.
Re:Well, they're a good indicator of intelligence (Score:5, Interesting)
"The point of these questions is that everyone has career aspirations."
No, they don't. *I* don't, for a start.
It used to be perfectly common to pick a job - i.e. a function that would be useful to society - and keep doing it more or less indefinitely. I can't, for the life of me, see what the hell's wrong with that. But it's remarkably difficult to convince HR droids that I don't actually want to quit my job (which I rather enjoy) and start a new one every two years (in effect) just because the new one is allegedly 'ranked higher'.