Forgot your password?
typodupeerror
Businesses Facebook Microsoft Programming

Tough Tests Flunk Good Programming Job Candidates 743

Posted by samzenpus
from the lowering-the-bar dept.
snydeq writes "Fatal Exception's Neil McAllister discusses the use of quizzes and brain-teasers in evaluating potential software development hires, a practice that seems to be on the rise. 'The company best known for this is Google. Past applicants tell tales of a head-spinning battery of coding problems, riddles, and brain teasers, many of which seem only tangential to the task of software development. Other large companies have similar practices — Facebook and Microsoft being two examples,' McAllister writes. 'You'll need to assess an applicant's skill in one way or another, but it's also possible to take the whole interview-testing concept too far. Here are a few thoughts to keep in mind when crafting your test questions, to avoid slamming the door on candidates unnecessarily.'"
This discussion has been archived. No new comments can be posted.

Tough Tests Flunk Good Programming Job Candidates

Comments Filter:
  • the way to go (Score:5, Interesting)

    by lecoupdejarnac (1742408) on Thursday November 03, 2011 @06:20PM (#37941304)
    Radical idea: have them write code for a few hours to solve a given problem - then see how their solution looks. This goes a long way towards judging their fit for the job. You can even give them a couple of data structures and algorithms references - on the job we use references all the time, and being able to implement something from a reference and apply it to a problem is a real skill.
    • While a programming test can be a valuable part of the interview process, it would be a mistake not to do some in person "tests" as well.

      Why? Because the entire point of those tests isn't to see if they get the right answer, it's to see if the candidate can work with the people in your office.

      • Because the entire point of those tests isn't to see if they get the right answer, it's to see if the candidate can work with the people in your office.

        The trouble with brain-teaser puzzles and trick questions it that the entire point is usually to make the interviewer feel that they are smarter than the candidate.

        If they make hiring decisions based on that kind of test, this is probably not the case. :-)

    • Re:the way to go (Score:5, Insightful)

      by anomaly256 (1243020) on Thursday November 03, 2011 @06:33PM (#37941448)
      This would be preferable to what one company I applied for a job with did recently. Gave me a fairly straight forward maths problem involving modulus, gave me about *5* seconds to solve it using real code and not just pseudocode. Sure, that was fine. Then they added the caveat 'What if % is an expensive operation? how would you work around it?'. Turns out it was a trick question. They were expecting you to statically store the result explicitly instead of finding different maths that achieved the same result dynamically but more efficiently. less than 2 seconds later the interviewer interjected with the answer before I had a chance to even say or do anything right *or* wrong.

      I don't see how *this* particular kind of quizzing *can* weed good candidates from bad, it's stacked against everyone equally. It's hostile. I'm pretty sure they didn't ever find the 'right' candidate.

      I'm all for puzzles and quizzes to test someone's experience and ability and problem solving skills during job applications, but they MUST a) be unambiguous otherwise you're just being a jerk, and b) must be given a reasonable amount of time to actually perform them otherwise, again, you're just being a jerk.
      • Re:the way to go (Score:5, Insightful)

        by anomaly256 (1243020) on Thursday November 03, 2011 @06:36PM (#37941484)
        In hindsight though, I'm really glad I didn't get that position. I'd hate to work for someone so near-sighted and irrational. :)
      • Re:the way to go (Score:4, Insightful)

        by acak (2362174) on Thursday November 03, 2011 @06:40PM (#37941534)
        Too often the interviewer's questions are loaded in two different ways - there is no one right answer and not understanding this or dismissing anything outside the narrow set of answers he/she is looking for shows a lack of maturity on the interviewers side of the table.
        • by sapgau (413511)

          Agreed +1

        • Re:the way to go (Score:5, Insightful)

          by mhotchin (791085) <[slashdot] [at] [hotchin.net]> on Thursday November 03, 2011 @07:09PM (#37941824)

          The saddest thing is - the 'no right answer" questions are (in my mind) one of the *best* ways of evaluating a prospective hire if the interviewer then follows up with "Why did you do it that way?".

          Rather than finding out if the hire can find the answer, I would want some insight into *how* the answer was achieved. Was it something they memorized? Did they evaluate trade-offs? Did the even *see* the trade-offs? Can they evaluate their own answer if (as happens in the real world!) new constraints are placed on the problem?

          An 'inefficient' answer is *just fine* if the code wouldn't be used much, and better if it's more maintainable, for example.

        • by Splab (574204)

          When I hired for the backend department which worked mainly with databases for a small telecoms I used to hand them some questions.

          Some of them was really basic database stuff, but others had no right answer, they did however, have downright wrong answers:

          E.g. a client reports performance problems with a certain function, you trace it down to the following query:
          "SELECT * FROM PICTURES"
          What do you do?

          This question is not really a trick question, but it weeds out those who have worked with databases but does

        • by syousef (465911)

          Too often the interviewer's questions are loaded in two different ways - there is no one right answer and not understanding this or dismissing anything outside the narrow set of answers he/she is looking for shows a lack of maturity on the interviewers side of the table.

          Agreed. The interview is often a very good indicator of what the job is like. It's just as much of a way for the interviewee to evaluate their prospective employer as it is for the employer evaluating the employee. Even if HR is administering the test or it has been outsourced, that tells you something about how you'll be treated. I remember being rejected based on one of these idiotic interviews for one particular job. It was crushing at the time but in hindsight boy am I glad I didn't get that job! It cer

          • The interview is often a very good indicator of what the job is like. It's just as much of a way for the interviewee to evaluate their prospective employer as it is for the employer evaluating the employee.

            Amen.

            I once interviewed for a programming position at a very young and small company. Most of the interview was done by the owner, a business/marketing guy with some technical knowledge of the industry they develop applications for but with no real knowledge of software development (other than it takes longer than expected). For the programming test I was handed over to his lead developer. The test was crap but I probably did well on it. When I was back with the owner he asked how the test went. I decide

      • by ackthpt (218170)

        This would be preferable to what one company I applied for a job with did recently. Gave me a fairly straight forward maths problem involving modulus, gave me about *5* seconds to solve it using real code and not just pseudocode. Sure, that was fine. Then they added the caveat 'What if % is an expensive operation? how would you work around it?'. Turns out it was a trick question. They were expecting you to statically store the result explicitly instead of finding different maths that achieved the same result dynamically but more efficiently. less than 2 seconds later the interviewer interjected with the answer before I had a chance to even say or do anything right *or* wrong.

        I don't see how *this* particular kind of quizzing *can* weed good candidates from bad, it's stacked against everyone equally. It's hostile. I'm pretty sure they didn't ever find the 'right' candidate.

        I'm all for puzzles and quizzes to test someone's experience and ability and problem solving skills during job applications, but they MUST a) be unambiguous otherwise you're just being a jerk, and b) must be given a reasonable amount of time to actually perform them otherwise, again, you're just being a jerk.

        Interview panels I've been on, we generally throw out a few simple problems - we're not so concerned with whether the candidate arrive at a correct answer as we want to observe their thinking - "Explain how you would do X, in general terms" More telling than someone who has the IQ to pass the test, but then proves to be a problem in how they go about work.

        That aside:

        'The company best known for this is Google. Past applicants tell tales of a head-spinning battery of coding problems, riddles, and brain teasers, many of which seem only tangential to the task of software development.

        This seems to explain some of the idiotic updates Google has been rolling out to Chrome and other products. Brilliant, but lacking wisdom.

      • Hello,

          I think they just didn't want to hire you, so they "failed" you on those two questions. They could then give half an hour on each to the person they actually wanted. Maybe their preferred candidate was the interviewer's brother in law or something.

        Then they justify choosing their BiL by saying "he scored the highest on our test!"

        --PM

      • by Machtyn (759119)
        Perhaps these guys were so proud of their questions, they couldn't wait to tell you the answers to show how clever they were.
      • by Animats (122034)

        Then they added the caveat 'What if % is an expensive operation? how would you work around it?'

        Do they program really low end microcontrollers, or what?

        That's a reasonable question for people hired to program very low end microcontrollers. Something in the ATTiny11L class, price about $0.35 each, might need that. If, for example, you're writing code for a transponder key for a car, you might have to cram a crypto algorithm into a device that only has 8-bit add, subtract, and shift. This sort of thing is Not Fun. Especially since you usually have to program the thing in assembler.

        Around $5 per C

      • by Ash Vince (602485) *

        This would be preferable to what one company I applied for a job with did recently. Gave me a fairly straight forward maths problem involving modulus, gave me about *5* seconds to solve it using real code and not just pseudocode. Sure, that was fine. Then they added the caveat 'What if % is an expensive operation? how would you work around it?'. Turns out it was a trick question. They were expecting you to statically store the result explicitly instead of finding different maths that achieved the same result dynamically but more efficiently. less than 2 seconds later the interviewer interjected with the answer before I had a chance to even say or do anything right *or* wrong.

        Quite often in interviews people will ask you this sort of bs question. Some parts of and interview are about testing your technical ability but some parts are going to be about seeing how you cope with an unsolvable problem or impossible situation. Quite often they are just checking you can keep your cool and not react to crap being dumped on you without getting irate. Calmly responding that you need more information to solve a problem or need more time never did anyone any harm in this situation.

        For the b

    • by Anonymous Coward on Thursday November 03, 2011 @06:36PM (#37941476)

      The offshoring of software development over the past 15 years hasn't just trashed the quality of the software that many American businesses use, it has also trashed the ability of software developers to become managers.

      The best software development managers were formerly software developers themselves. They know that experience is what counts. They know that bullshit HR tests don't work. But these kind of managers are now retiring or getting promoted to executive positions outside of software development. There's nothing but a huge void following them, since there have been very, very few software developers in America over the past 15 years.

      The people we have following them often have no software development experience. Many of them are MBAs who don't even know of any programming languages beyond JavaScript, and they only know of JavaScript because they read about it once in some article that was hyping it. The worst are the "professional project managers" who don't even have any relevant college-level training in any useful field (yes, that's right, sociology majors don't know how to be software development managers).

      We don't find good managers in the places where the software development was offshored to, either. Skilled management was never a factor there to begin with, and thus the void has always been present over there.

      Offshoring software development has been one of the biggest economic mistakes that any civilized nation has ever made.

      • by JoeMerchant (803320) on Thursday November 03, 2011 @09:28PM (#37943112) Homepage

        Offshoring software development has been one of the biggest economic mistakes that any civilized nation has ever made.

        I disagree. It wasn't a mistake. It was a deliberate act by short-sighted executive management who profited obscenely from the "bold, visionary paradigm shift." The people who were responsible for the decision have all moved on with their millions in bonuses, stock options, etc. Their boards of directors are mostly made up of people who expect to be dead within 15 years or less.

        It was no mistake.

    • Re:the way to go (Score:5, Interesting)

      by CannonballHead (842625) on Thursday November 03, 2011 @06:45PM (#37941584)

      I interviewed at a company, some sort of advertising thing. Relatively small - I was fresh out of college and talked to some heads of something-or-other. Anyways, there were two of us that happened to interview at the same time; they had two test stations setup. There were a few multiple choice sorts of questions, and then an actual programming test dealing with extracting ads out of an XML page, as I recall, in PHP. Well, I'd never dealt with XML in PHP at all, so it was totally new to me. I found various things online (XML parser, etc). After a couple hours and making some progress, but not completing it - and having the other guy already finish roughly an hour or so earlier and whatever - I said I wasn't going to be able to complete it very quickly and asked if I should leave or not.

      The head hiring people asked if they could talk to me, so I said sure. Asked if I thought I could complete it, given time - I said sure, it wasn't that difficult, it's just that I had never worked with XML parsing before and was having to learn it as I went. Then they pretty much startled me. They said, basically, the following: the other guy interviewing had all the skills, he was able to churn out the problem, and his test scores were good (as were mine, incidentally). But they didn't like his attitude (somewhat arrogant, loudly proclaiming "oh yeah, it was easy!" and that sort of thing). They said that even though I couldn't complete it within an hour or two, they liked my attitude, they liked that I stayed at it for about two hours (without being asked really), they liked that I was honest to say I would not be able to complete it by the end of the business day (my interview was at like 2:30 and it was now 4:30). They basically said they'd offer me a Jr. Dev position based on the work ethic, honesty, and generally good attitude/personality (along with knowledge of general programming stuff based on test scores, resume, etc).

      I didn't end up accepting as I got a much better offer elsewhere - also, primarily, based on work ethic, personality, general knowledge, etc.... it was a "we'll train you to do what we want you to do, because we can see you're versatile enough and willing to learn" - but the experience was pretty eye-opening.

      • by ari_j (90255)

        You're the kind of cheat who would rig the Kabayashi Maru...oh, wait.

        There really is value in recognizing not only that you can't solve a problem in the time allotted but also why you can't. There is also value in recognizing that value in others, and in working for people who recognize it in yourself. I would much rather have someone interrupt me at 9 a.m. to say they need some direction to get a task done by 5 than come in at 5:15 p.m. to tell me it didn't get done. I would also much rather work for so

        • In my particular case, I came in around 2:30pm (this was about 3 1/2 years ago, so I'm a bit foggy on the exact timing) and I asked, after about an hour, if I should keep trying. They said yes, please do.... so I kept trying until I thought it'd be close to when they'd want to leave (and I had a previous engagement that evening).

          But yes, I agree; asking for help on the job is a good idea. Utilizing the skills of your team members, being willing to say "hey, any ideas on how I might be able to do this?" or

    • Actually that'll be a good test of their individual programming and development style. Have them work with another employee who has no idea what the problem is and see how your candidates work in a group.

    • Yup. The programming assignments work.

      I even go so far as to give them a couple of days to think it through, and send me code that compiles, along with written test cases / data.

       

    • by shentino (1139071)

      Or maybe they just have you write their code for free under the guise of evaluating.

      Poaching an evaluation for free consulting isn't exactly new.

  • by eparker05 (1738842) on Thursday November 03, 2011 @06:22PM (#37941316)

    Tests can be a good measurement of quality when the test is material that can be studied for. In school you have a test at the end of a class. For certifications, tests are meant to measure knowledge gained during training. In graduate school, qualifying exams are done to second year students who have time to prepare and hone their skills.

    Testing somebody from a cold start, on subjects they have no practical way to prepare for seems like a good way to hire a trivia expert, but the productivity of an employee should be evaluated by his resume and portfolio.

  • Whatever happened (Score:4, Informative)

    by ignavus (213578) on Thursday November 03, 2011 @06:25PM (#37941360)

    Whatever happened to tests like drinking the interview panel under the table?

    Now that is a skill needed on the job.

    • Whatever happened to tests like drinking the interview panel under the table?

      Now that is a skill needed on the job.

      Recreate late-night thinking by preceding the test with 3-4 shots?

  • Mix it up (Score:4, Insightful)

    by Tablizer (95088) on Thursday November 03, 2011 @06:25PM (#37941362) Homepage Journal

    Teams with diverse thinkers are often the most effective. The one who is not good at math puzzles may instead be good at understanding the customer's needs or the intuitiveness of user interface designs in the eyes of non-techies, and vice verse. They each can focus on their specialty, or at least help each other out in their weak spots.

  • by acidradio (659704) on Thursday November 03, 2011 @06:37PM (#37941494)

    It seems like every job posting now has around 50-100 people who apply. To weed out this many people en masse they will make you do just about anything - tests that have little application to the job that you are applying for, bark like a dog, sing the interviewer's favorite Barbra Streisand song, paint a painting of a nice wilderness scene, tune the carburetor on the interviewer's old Triumph motorcycle... Many of the people are well-qualified and even over-qualified! To weed them out on that alone would go nowhere.

    If I had to tell you how many times I've been asked something stupid and cliche like "Tell me about a time when you experienced change" or "Tell me about a time when you faced challenge" I might go postal. It's almost like HR people invent these questions to pad their interviews because they don't really understand what or who they are interviewing for. I long for the days when a hiring manager or, god forbid, the company owner/proprietor calls and asks you "So, tell me what you are about and tell me why you think I should hire you."

    They can treat applicants like total bastards and get away with it. With this kind of market what is really to stop them?

    • It seems like every job posting now has around 50-100 people who apply. To weed out this many people en masse they will make you do just about anything - tests that have little application to the job that you are applying for, bark like a dog, sing the interviewer's favorite Barbra Streisand song, paint a painting of a nice wilderness scene, tune the carburetor on the interviewer's old Triumph motorcycle... Many of the people are well-qualified and even over-qualified! To weed them out on that alone would go nowhere.

      Another trend we never used to see are the online tests for minimum wage/low paying jobs. Had a buddy of mine who can pass these with no problem tell me you have to lie and don't list your education/training anywhere on the application/resume. Never thought in my life I would see a test battery being used to be able to clean bathrooms or flip burgers. WTF has this country become when slavery to a corporate master is becoming the way of life in the US.

      We want a job to be able to have a place to stay and h

    • by owlstead (636356)

      Go to NL, we've got a (rising) shortage of IT personel. Of course, your pay level and housing cost may be affected. We are a friendly people that like to drink real beer, talk English and use US keyboards :)

  • Amen (Score:4, Insightful)

    by bradgoodman (964302) on Thursday November 03, 2011 @06:38PM (#37941512) Homepage
    I've sat through interviews before where I got hit with some of these "puzzle" questions. The thing that really struck me was that the questions were the kind you'd need to be a Nobel Laureate to solve (not really), but were completely obvious if you *knew* the answer. So, as these were "novel" or "neat" "brain-teasers" - they really assessed only if you were some mathematical savant - and had nothing to do with your coding, or even problem-solving abilities. I'm 100% sure that the interviewer wouldn't have been able to figure out the answers, if he hadn't already known them.

    (In the end I admittedly had absolutely no idea how how to solve the problems, and didn't even attempt to. I got the job anyway.)

    When I interview people - I feel it is my job to "extract" the best out of the candidates, and to find out what "their best" actually is. If I come away from an interview and don't have a strong feeling for a candidates abilities - good and bad - I feel as though I didn't do my job as an interviewer. I've seen too many people "freeze up", or just be shy in interviews. These people maybe were VERY qualified - I feel it is always my job to understand that. My creedo is this: Get the people talking. Get them talking about what they do, and what they love. If you can do this - they'll go into the depths and bowels of their technical knowledge, working style, experience, etc.

    • I've sat through interviews before where I got hit with some of these "puzzle" questions. The thing that really struck me was that the questions were the kind you'd need to be a Nobel Laureate to solve (not really), but were completely obvious if you *knew* the answer. So, as these were "novel" or "neat" "brain-teasers" - they really assessed only if you were some mathematical savant - and had nothing to do with your coding, or even problem-solving abilities. I'm 100% sure that the interviewer wouldn't have

      • I don't even consider them to be IQ tests. Some of the questions I've seen rely on algorithms that have taken decades of computer science research to create. If you're able to write down an implementation it's not going to be because you worked it out. It's going to be because you rote learnt it.

        Which is crazy really, because you don't want programmers who rote learn implementations of algorithms. That goes against the principles of not recreating the wheel (and all the bugs). You'd be much better off with

  • I interviewed over the phone a couple times there.. They try to "ease" the difficulty by providing hints and asking you to "talk through" your process, etc. The problem is that these questions are often poorly defined (and I suspect made up on the fly), and one interviewer's "hints" were very distracting. In both cases I figured out the answers 5 minutes after hanging up, when I could finally be alone with my thoughts.
    • I had a similar experience in the interview process for my current job I but was very blunt about it. I told the interviewer:

      I can give you a good answer to your problem but I have to stay silent about 5 minutes while I solve it and only after that I can explain my solution. However, if you really wish so, I can do a bad job while talking about it but be warned that you will eventually hear me talk about how I think when I think about "talking through" my process while I crapply solve you trick problem. I value quality and correctness, but as I said, if you really wish so, I can botch it for you. The choice is up to you.

      The interviewer evidently selected quality (who would choose crap?) and about 3 minutes later, I had an elegant solution based on graphs theory.

  • Google invited me to interview for a Java programming job. They started the interview by informing me that I would be "the oldest person in the group" (I was 39 at the time). Then, I was invited to code a linked list in C on the white board while they watched. I can do this, I suppose, having done it 20 years ago while getting my computer science degree. And never done it since. I questioned the relevance of the problem pointing out that this was surely not required for programming in Java. It kinda went do
    • Re: (Score:2, Troll)

      by mandelbr0t (1015855)
      I haven't coded a linked list in C in 10 years either, but I could still do it in my sleep. Sounds like you know less than you think you do.
      • by DeadCatX2 (950953)

        I haven't coded a linked list in about 8 years. And I shouldn't have to. There are standard libraries for that, and if you're writing your own linked list, then you're a failure of a programmer in the first place. We learned how to make linked lists in college only so that we knew their strengths and weaknesses [ O(n) traversal, O(1) adding/removing element, etc ], not so we could go on to find jobs writing linked lists.

        • Which, of course, is why I haven't written my own in over 10 years. The fact that you have forgotten shows that you didn't find it that important in the first place. C is an excellent language for understanding how a computer works. Managed languages are great because they remove elements that are likely to lead to human error. The tradeoff is that your program isn't as fast. These days, there are ways (CORBA, for example) to communicate performance-intensive code with the easier-to-maintain managed code.
    • by joggle (594025)

      At least they didn't ask to code a linked list in Java. In C it's trivial with pointers. In Java, I would be stuck at "Why?" since there's already a LinkedList container. I'm only partially kidding.

      It's not really that unfair of a question because the answer is so simple and fundamental. It basically shows that you understand the concept of pointers and memory allocation, which even in Java is relevant due to the similarity to references.

    • by TheSync (5291)

      "They started the interview by informing me that I would be "the oldest person in the group" (I was 39 at the time)."

      Too bad you weren't over 40, you could have then said: "Of course I'm sure that won't be held against me due to the Age Discrimination in Employment Act [eeoc.gov]..."

      When you are hiring, you never should mention anything regarding age, marriage status, race, veteran status, etc.

       

  • Less False Positives (Score:4, Interesting)

    by Tsarnon (4195) on Thursday November 03, 2011 @06:42PM (#37941556)

    Sure this technique has tons of false negatives, but I think it has fewer false positives than many other interview techniques. False positives are a much bigger problem then false negatives when hiring.

  • by Nom du Keyboard (633989) on Thursday November 03, 2011 @06:46PM (#37941594)
    Considering that these questions can be asked once, maximum 3 times, before someone posts it to the Internet, now that I've memorized all of the right answers there they go changing the rules. I'll never get that job now.
  • Has anyone considered pitting them against each other in mortal human combat? That would surely make the cream rise to the top!
  • I almost want to apply for a job just to play the puzzles.
  • ...but it's also possible to take the whole interview-testing concept too far.

    No kidding. I was a software engineer for almost a decade and got some bizarre interview questions with no connection to any relevant problem I had to solve in nearly 10 years of writing apps. What was bizarre is, most of the time, the problems had no connection to anything the company did, either. Nothing about logistics, data import, connecting to a web service, or setting up batch jobs, none of the skills you need every da

  • by TheGratefulNet (143330) on Thursday November 03, 2011 @07:06PM (#37941790)

    partly - perhaps even mostly - due to this new strict 'screening' bullshit that everyone in the silicon valley seems to be doing, now. I've been out of work for more than a year now and the interviews I have been on have been markedly different than they were 20 years ago when I first came to the bay area. lately, the interviews are confrontational. they are assuming you are a liar, incompetant and many other bad things - and its up to you to disprove that. they do not seem to want you, they are there because their bosses told them to interview some new people.

    this is quite different from the case where they really are looking to hire and are excited to have found a match (at least on paper) and just want to verify that you are who you say you are. even as little as 5 years ago, the interviews were not so negative. it was more of a verification that you met your paperwork. now, they don't bother to read your resume; or they assume its all lies and you have to start from ground-0 and prove an entire background to them. *over and over*, too; with each new guy that steps in for his 45min slot. show me linked lists; show me trees. show me O(n) stuff.

    the problem for me is that I am starting to not care anymore about this level of detail. I'm 50 and have been doing C since I was mid 20's. I have done my time, to be sure; but I just don't really get into having to prove it over and over again, like my resume is all a pack of lies. it gets tiring and its making me question whether I really want to re-enter this field with these kinds of people 'running things'.

    I do think that people like me are worth retaining in the software development field. true that I'm getting tired of the lower level details and things that are reference (like precise steps in deleting a linked list node) are of no interest to me when I can quickly get the correct sequence in a few minutes of search. there is too much to keep track of and older things do age out, its true. but older folks do have a lot of other things to offer. its a shame that we get passed over due to how the 'tests' are structured these days. I once was able to jump thru some hoops, but now, I'm just not so motivated to play their bullshit 'test me' games anymore. its a shame since I'm not alone in this and losing experienced guys like me is a real loss to the industry.

  • My last interview was for a "C" programming position (Point-of-Sale system for a large retailer) which was a very good fit for my experience and skills (30 years C programming, almost a decade on another large P.O.S. system).

    Unfortunately, the only technical interviewer who talked to me was a Java programmer. He thought he was also a C/C++ programmer, but he was wrong. After 45 minutes of questions about details of Java class usage, it was clear that he couldn't tell a good C programmer from a bad one.

    I d

  • I recently got a new job and have definitely seen a lot of this. It ranges from the esoterically retarded ("Can you call a destructor in C++ directly?" (The answer is yes, but it's a terrible idea -- so is there any difference in practice between never doing something and thinking you can't? Why not quiz me on goto syntax -- that's at least useful sometimes)) to one company I really liked that sat me down for 3 hours with a Linux box and said "Solve these 3 problems as efficiently as you can" -- all of wh

  • I've made it a point to ask ahead to confirm that there will be no "brain teasers" and if there may be, to kindly ask them to rescind the interview so that they may focus on more qualified candidates who share their core values (being a self-congratulating asshole who gets a boner doing brain teasers)

    Frankly, the very fact they ask brain teasers tells me a lot more about them than my answer will ever tell them about me. The fact that I put it bluntly, but kindly to opt out in such an event should show my "f

  • Having had co-workers who were apparently pretty good at bluffing in the interview, and having had to clean up for several others, I can appreciate the desire to do whatever it takes to not have those people working in your company.

    A lot of companies are also just very bad at interviewing. A lot of times they'll just have a couple folks from the team you'll be working on talk to you for a while and you get in or not based pretty much on how much they like you more than any technical skills you may or may

  • by SoftwareArtist (1472499) on Thursday November 03, 2011 @07:21PM (#37941986)
    I've interviewed lots of people using puzzles of this sort, and I find they work really really well for picking out the better programmers. You need to understand how to do it correctly, though. You're not looking for whether they "get the right answer". You're looking to see how they approach the problem and what sorts of solutions they try (even if they end up not working). When you interview a bunch of people this way, you find they split into a few groups, and the differences between groups are really obvious. For example, some people will just have no idea how to even approach the problem. Others will struggle to figure out an O(n^2) solution. And others will instantly take it for granted that of course there's a trivial O(n^2) solution, but you're obviously looking for something better than that. The differences aren't subtle.
    • by swillden (191260) <shawn-ds@willden.org> on Friday November 04, 2011 @12:29AM (#37944262) Homepage Journal

      I've interviewed lots of people using puzzles of this sort, and I find they work really really well for picking out the better programmers. You need to understand how to do it correctly, though. You're not looking for whether they "get the right answer". You're looking to see how they approach the problem and what sorts of solutions they try (even if they end up not working).

      Mod parent up.

      I work for Google, and interview engineering candidates at Google (which doesn't make me special -- all Google engineers are expected to participate in interviewing), and the above describes what these puzzle questions are really intended to do.

      Evaluating candidates is a really, really hard thing to do. Honestly, it's darned near impossible to do accurately unless you can bring someone in for a few weeks, put them to work on real problems and see how they really perform. Even then you're just scratching the surface. Trying to get a realistic read on someone's capabilities in just a single day? Yeah, right. But hiring decisions must be made, and Google gets hundreds of thousands of applicants per year.

      So, the question becomes not how to truly, accurately evaluate each candidate, but how to fill the positions and ensure that no egregiously bad hires are made. It's sad to reject someone who is really good... but it causes tremendous grief to hire someone who just can't do the job, because firing people is really painful for everyone involved, and quite expensive to the employer.

      With that in mind, here are my perspectives on the article's points, based not only on what Google does, but also on the dozens (hundreds?) of job interviews I did at previous employers.

      1. Recognize that tests are artificial scenarios

      This is inarguably true. The interview is a completely artificial environment which bears no relationship to the actual job. But there's no way to avoid that, so the question become what is the best way to extract as much useful information about a candidate's abilities and characteristics as possible in the brief time allowed. And the time has to be very brief, because getting any kind of an accurate reading depends on getting multiple points of view, and fitting four or five interviews into a day without killing the candidate means that each interviewer really only has about 45-50 minutes.

      The article comments that the problems are hypothetical, without context, etc., but that's because there's no way to present real-world problems in the timeframe given. And, actually, underspecifying problems is deliberate, because it's a good way to see how the candidate reacts to inadequate requirements specifications, which is perhaps the one part of the interview which accurately models the real world at every place I've worked in 20+ years.

      So, the goal is to construct an artificial environment which tests the things that can be realistically tested. Google is perhaps a little different than most companies in that Google doesn't really care what the candidate knows. The infrastructure is so different from everywhere else that it's really not important; there will be so much to learn anyway. Well, basic CS knowledge is crucial, but specific tools and techniques? Nah. What really matters is how quickly the candidate can understand problems, how effectively they can reason their way through the solution space, how well they can communicate their solutions -- verbally and in code -- and how well they can analyze their solutions and explore alternatives. It's all about problem solving and the only way to understand someone's problem solving abilities is to watch them solve a problem they haven't seen before.

      The "they haven't seen before" is crucial... and it's also pretty easy to verify. Here's a word of advice: If you go into an interview and are presented with a problem you already know the solution to, tell the interviewer up front that you've seen it before. Odds are the interviewer will figure that

    • by gweihir (88907)

      Well, but you seem to be investing real effort into this and the interpretation of the results. Most of these tests are just used in form of a standardized, cheap-to-administer form and that does not work at all.

  • by kc8jhs (746030) on Thursday November 03, 2011 @10:41PM (#37943708)
    Alex at The Daily WTF wrote about this problem back in 2007: http://thedailywtf.com/Articles/Riddle-Me-An-Interview.aspx [thedailywtf.com]
  • by kiehlster (844523) on Friday November 04, 2011 @11:55AM (#37948528) Homepage

    Like some of my PhD friends have told me, putting a technical quiz in front of well educated and experienced job candidate is a great way to insult them, and is deserving of a good punch in the face.

    What you get from a quiz is a candidate who is intelligent enough to write a program that is plain to the interviewer. That is, it is neither a wise answer nor a clever one. It is simply an explainable one, and it is usually the explainable ones that show up in "For Dummies" books and have no practical value. You could be interviewing Einstein who would give you an answer that breaks ground in uncharted territory, but you wouldn't hire him because your mind couldn't comprehend his explanations. You could be interviewing Jesus, but his wise answers would be so over your head that you'd not hire him because you couldn't grasp how many risks were calculated in giving you those answers.

Wherever you go...There you are. - Buckaroo Banzai

Working...