Follow Slashdot stories on Twitter


Forgot your password?
Businesses Facebook Microsoft Programming

Tough Tests Flunk Good Programming Job Candidates 743

snydeq writes "Fatal Exception's Neil McAllister discusses the use of quizzes and brain-teasers in evaluating potential software development hires, a practice that seems to be on the rise. 'The company best known for this is Google. Past applicants tell tales of a head-spinning battery of coding problems, riddles, and brain teasers, many of which seem only tangential to the task of software development. Other large companies have similar practices — Facebook and Microsoft being two examples,' McAllister writes. 'You'll need to assess an applicant's skill in one way or another, but it's also possible to take the whole interview-testing concept too far. Here are a few thoughts to keep in mind when crafting your test questions, to avoid slamming the door on candidates unnecessarily.'"
This discussion has been archived. No new comments can be posted.

Tough Tests Flunk Good Programming Job Candidates

Comments Filter:
  • by eparker05 ( 1738842 ) on Thursday November 03, 2011 @06:22PM (#37941316)

    Tests can be a good measurement of quality when the test is material that can be studied for. In school you have a test at the end of a class. For certifications, tests are meant to measure knowledge gained during training. In graduate school, qualifying exams are done to second year students who have time to prepare and hone their skills.

    Testing somebody from a cold start, on subjects they have no practical way to prepare for seems like a good way to hire a trivia expert, but the productivity of an employee should be evaluated by his resume and portfolio.

  • Mix it up (Score:4, Insightful)

    by Tablizer ( 95088 ) on Thursday November 03, 2011 @06:25PM (#37941362) Journal

    Teams with diverse thinkers are often the most effective. The one who is not good at math puzzles may instead be good at understanding the customer's needs or the intuitiveness of user interface designs in the eyes of non-techies, and vice verse. They each can focus on their specialty, or at least help each other out in their weak spots.

  • by stanlyb ( 1839382 ) on Thursday November 03, 2011 @06:26PM (#37941368)
    Tell me wrong, but if google is full with EXCEPTIONAL developers, why there is not even a single product made entirely by them? Please, even one is enough (and don't say: Google) Anyway, my point is that these mehods are good enough for finding anything, but good developers, which is maybe their aim in the first place.
  • by JonySuede ( 1908576 ) on Thursday November 03, 2011 @06:31PM (#37941428) Journal

    google is not full with EXCEPTIONAL developers, it is filled with EXCEPTIONALLY clever people; you get what you select for. As D&D should have taught you, INT is not equals to WIS !

  • Re:the way to go (Score:5, Insightful)

    by anomaly256 ( 1243020 ) on Thursday November 03, 2011 @06:33PM (#37941448)
    This would be preferable to what one company I applied for a job with did recently. Gave me a fairly straight forward maths problem involving modulus, gave me about *5* seconds to solve it using real code and not just pseudocode. Sure, that was fine. Then they added the caveat 'What if % is an expensive operation? how would you work around it?'. Turns out it was a trick question. They were expecting you to statically store the result explicitly instead of finding different maths that achieved the same result dynamically but more efficiently. less than 2 seconds later the interviewer interjected with the answer before I had a chance to even say or do anything right *or* wrong.

    I don't see how *this* particular kind of quizzing *can* weed good candidates from bad, it's stacked against everyone equally. It's hostile. I'm pretty sure they didn't ever find the 'right' candidate.

    I'm all for puzzles and quizzes to test someone's experience and ability and problem solving skills during job applications, but they MUST a) be unambiguous otherwise you're just being a jerk, and b) must be given a reasonable amount of time to actually perform them otherwise, again, you're just being a jerk.
  • by Anonymous Coward on Thursday November 03, 2011 @06:36PM (#37941476)

    The offshoring of software development over the past 15 years hasn't just trashed the quality of the software that many American businesses use, it has also trashed the ability of software developers to become managers.

    The best software development managers were formerly software developers themselves. They know that experience is what counts. They know that bullshit HR tests don't work. But these kind of managers are now retiring or getting promoted to executive positions outside of software development. There's nothing but a huge void following them, since there have been very, very few software developers in America over the past 15 years.

    The people we have following them often have no software development experience. Many of them are MBAs who don't even know of any programming languages beyond JavaScript, and they only know of JavaScript because they read about it once in some article that was hyping it. The worst are the "professional project managers" who don't even have any relevant college-level training in any useful field (yes, that's right, sociology majors don't know how to be software development managers).

    We don't find good managers in the places where the software development was offshored to, either. Skilled management was never a factor there to begin with, and thus the void has always been present over there.

    Offshoring software development has been one of the biggest economic mistakes that any civilized nation has ever made.

  • Re:the way to go (Score:5, Insightful)

    by anomaly256 ( 1243020 ) on Thursday November 03, 2011 @06:36PM (#37941484)
    In hindsight though, I'm really glad I didn't get that position. I'd hate to work for someone so near-sighted and irrational. :)
  • by acidradio ( 659704 ) on Thursday November 03, 2011 @06:37PM (#37941494)

    It seems like every job posting now has around 50-100 people who apply. To weed out this many people en masse they will make you do just about anything - tests that have little application to the job that you are applying for, bark like a dog, sing the interviewer's favorite Barbra Streisand song, paint a painting of a nice wilderness scene, tune the carburetor on the interviewer's old Triumph motorcycle... Many of the people are well-qualified and even over-qualified! To weed them out on that alone would go nowhere.

    If I had to tell you how many times I've been asked something stupid and cliche like "Tell me about a time when you experienced change" or "Tell me about a time when you faced challenge" I might go postal. It's almost like HR people invent these questions to pad their interviews because they don't really understand what or who they are interviewing for. I long for the days when a hiring manager or, god forbid, the company owner/proprietor calls and asks you "So, tell me what you are about and tell me why you think I should hire you."

    They can treat applicants like total bastards and get away with it. With this kind of market what is really to stop them?

  • Amen (Score:4, Insightful)

    by bradgoodman ( 964302 ) on Thursday November 03, 2011 @06:38PM (#37941512) Homepage
    I've sat through interviews before where I got hit with some of these "puzzle" questions. The thing that really struck me was that the questions were the kind you'd need to be a Nobel Laureate to solve (not really), but were completely obvious if you *knew* the answer. So, as these were "novel" or "neat" "brain-teasers" - they really assessed only if you were some mathematical savant - and had nothing to do with your coding, or even problem-solving abilities. I'm 100% sure that the interviewer wouldn't have been able to figure out the answers, if he hadn't already known them.

    (In the end I admittedly had absolutely no idea how how to solve the problems, and didn't even attempt to. I got the job anyway.)

    When I interview people - I feel it is my job to "extract" the best out of the candidates, and to find out what "their best" actually is. If I come away from an interview and don't have a strong feeling for a candidates abilities - good and bad - I feel as though I didn't do my job as an interviewer. I've seen too many people "freeze up", or just be shy in interviews. These people maybe were VERY qualified - I feel it is always my job to understand that. My creedo is this: Get the people talking. Get them talking about what they do, and what they love. If you can do this - they'll go into the depths and bowels of their technical knowledge, working style, experience, etc.

  • Re:the way to go (Score:4, Insightful)

    by acak ( 2362174 ) on Thursday November 03, 2011 @06:40PM (#37941534)
    Too often the interviewer's questions are loaded in two different ways - there is no one right answer and not understanding this or dismissing anything outside the narrow set of answers he/she is looking for shows a lack of maturity on the interviewers side of the table.
  • by frosty_tsm ( 933163 ) on Thursday November 03, 2011 @06:41PM (#37941548)
    Or their objective isn't specifically verifying the individual in front of them.

    Picture this: a hiring manager at some medium company reads an article about the brain teasers asked by Microsoft and Google interviewers. He wants his application process to seem more like theirs so they can say they are doing MS / Google style interviews (which sounds good to other managers and executives and theoretically impresses applicants).
  • by Asic Eng ( 193332 ) on Thursday November 03, 2011 @06:43PM (#37941560)

    Looks like you just flunked the "sample size" test. Don't worry about it, brain teasers aren't for everyone.

  • by Tawnos ( 1030370 ) on Thursday November 03, 2011 @06:55PM (#37941678)

    Which is bogus, because at least here at Microsoft, those brain teaser questions haven't been used in many years, after it was determined they were ineffective and dumb.

  • Re:the way to go (Score:5, Insightful)

    by beelsebob ( 529313 ) on Thursday November 03, 2011 @07:02PM (#37941742)

    Do they? Why? You're going to be working in a development environment. What good is a guy who can solve trivial problems on a white board when you need a guy who can solve complex problems in a dev environment with a bunch of reference material.

  • Re:the way to go (Score:5, Insightful)

    by mhotchin ( 791085 ) <> on Thursday November 03, 2011 @07:09PM (#37941824)

    The saddest thing is - the 'no right answer" questions are (in my mind) one of the *best* ways of evaluating a prospective hire if the interviewer then follows up with "Why did you do it that way?".

    Rather than finding out if the hire can find the answer, I would want some insight into *how* the answer was achieved. Was it something they memorized? Did they evaluate trade-offs? Did the even *see* the trade-offs? Can they evaluate their own answer if (as happens in the real world!) new constraints are placed on the problem?

    An 'inefficient' answer is *just fine* if the code wouldn't be used much, and better if it's more maintainable, for example.

  • Re:the way to go (Score:2, Insightful)

    by interval1066 ( 668936 ) on Thursday November 03, 2011 @07:17PM (#37941954) Homepage Journal
    Yeah, having hires solve a direct problem seems good to me. During the interview process (if I am involved) I try to get one simple question in like "name an algorithm, any one." If they stammer and haw, which is surprisingly often, that's a mark against. If they can at least name Nagle's, or binary tree, or something that makes sense, they're in as far as I'm concerened. But giving them a damn term paper, that's just a waste of time.
  • Re:the way to go (Score:5, Insightful)

    by Ethanol-fueled ( 1125189 ) on Thursday November 03, 2011 @07:34PM (#37942124) Homepage Journal

    On a side note i'm not that great with people skills so while i write the tests i no longer do the interviews - last too applicants ended up crying during the tests.

    Huh, you might work for my company. An entire software department in a company I may know about has jumped ship wholesale, with one person literally getting up and walking out with no notice, and chatter of the remainder leaving, because its run by an insecure touretty-aspie asshole.

    The funny thing is, he (you?) asks all new hires how they can "think outside the box" and make the department better. When people actually bring up useful suggestions, he gets all butthurt and snappy and puts them on his shit-list for making him feel stupid, harassing them at every opportunity. He's certainly one of you, raised in an affluent household and given everything he wanted while mommy made excuses for his rotten behavior. Working for the company is a double-edged sword - it is impossible to be fired for anything short of murder. It's hell when you get stuck with the wrong boss.

    But Ethanol, isn't harassment illegal?

    In corporate Amerika, harassment is not illegal unless the harassment falls under a protected category like race or gender. That means your boss yelling at you daily for inadvertently making him look stupid is not really harassment.

    But Ethanol, why doesn't anybody complain?

    Because the economy sucks and some people have families to feed. The ones who don't would not dare jeopardize their good references within the company, because nobody likes a complainer. You know those 3-page exit surveys you get? HR laughs at them and tosses them in the trash anyway.

  • Re:the way to go (Score:3, Insightful)

    by heironymous ( 197988 ) on Thursday November 03, 2011 @09:27PM (#37943106)

    When i do it i make sure to use an environment that has the same capabilities but they are not familiar with.

    If that really were a good idea, then do more of it. For example, give them only Dvorak keyboards if they are fluent in qwerty. See how silly that sounds? Introducing unnecessary cognitive overhead isn't testing much of anything except whether the applicant is willing to accept ridiculous constraints just because they are directed to do so.

    And actually I suspect that obedience is the real property your company needs to test for anyway.

  • by JoeMerchant ( 803320 ) on Thursday November 03, 2011 @09:28PM (#37943112)

    Offshoring software development has been one of the biggest economic mistakes that any civilized nation has ever made.

    I disagree. It wasn't a mistake. It was a deliberate act by short-sighted executive management who profited obscenely from the "bold, visionary paradigm shift." The people who were responsible for the decision have all moved on with their millions in bonuses, stock options, etc. Their boards of directors are mostly made up of people who expect to be dead within 15 years or less.

    It was no mistake.

All seems condemned in the long run to approximate a state akin to Gaussian noise. -- James Martin