Programming Interview Questions Are Too Hard and Too Short (triplebyte.com) 463
Programming interview questions can feel unnecessarily difficult. Sometimes they actually are, a new study has found. And this isn't just because they make interviews excessively stressful. The study shows that harder programming questions actually do a worse job of predicting final outcomes than easier ones. From the study: Programming under time pressure is difficult. This is especially true during interviews. A coding exercise that would seem simple under normal circumstances somehow becomes a formidable challenge under the bright lights of an interview room. Stress hormones cloud your thinking during interviews (even though, sadly, neither fight nor flight is an effective response to a menacing programming problem). And it can almost feel like the questions are designed to be perversely difficult. I actually think this is more than just a feeling.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
sing for your supper (Score:5, Funny)
Re:sing for your supper (Score:4, Interesting)
"Do this arbitrary coding problem" is usually a good sign that you don't want that job anyway. Such things have little relevant to the job or to producing good software most of the time.
The only exception is on embedded where you actually do need to know the ins and outs of the compiler and stuff like that. But for Javascript or C# devs they should use String.Reverse() because whatever they code on their own will be worse and a waste of time. And even for embedded I would only ask by way of some questions about some sample code the candidate was showing me.
Re: (Score:2)
Re: (Score:2)
Correctly reversing an UTF-8 string with combining diacritical marks and embedded RTL substrings and other features could be quite fun, though.
Indeed, and also a great example of why you should use String.Reverse() instead of trying to handle it yourself.
Re: (Score:2)
Correctly reversing an UTF-8 string with combining diacritical marks and embedded RTL substrings and other features could be quite fun, though.
Indeed, and also a great example of why you should use String.Reverse() instead of trying to handle it yourself.
What's actually more important to me is knowing that you have to deal with all those Unicode bits when doing the reverse... because someday you'll run into a situation where String.Reverse() (or equivalent) doesn't actually handle them correctly.
However, actually implementing this robustly is far too difficult and time consuming to fit within the confines of an interview question.
Re:sing for your supper (Score:4, Interesting)
It shouldn't be about whether or not a person can program a string reverse, it should be about the entire team watching the new interviewee's process of writing the code on a white board (after they have interviewed him personally two or three at a time) to get a feel for his bullshit to brilliant ratio. (or his brillant to brilliant ratio if you're a TDWTF fan).
Unfortunately when I was at a company that did this, it was during the dot-com crash of the early 2Ks, so we only had one hiring round while I was there. Our two programs were reverse a singly-linked list and copy a file. Watching recent CS grads who were weaned on a diet of Java trying to do file copy (it's important in C/C++ to know how to sling data around in buffers) was such a train wreck. In my case, I knew the basic file copy inside and out, including the performance implications of buffering. The CS grads were having trouble with the basic idiom of read a byte, while not EOF, do something with the byte. It was the EE grads that actually knew what the fuck they were doing.
Re: (Score:2)
A whiteboard would be a problem for me. I have some arthritis so extensive use of a whiteboard will get painful fast. Typing is fine though.
Even if that wasn't the case, a whiteboard isn't a great way to see if someone is good at coding. Aside from the poor editing capabilities it favours people who count on their fingers over people who just think things through and then write down the answer. It reminds me of the driving test where they advise students to make exaggerated head movements when looking in th
Re: (Score:2)
White boards have excellent editing capabilities.
You should simply practice with them.
Doubt you have arthritis, more likely gout, seek a good doctor, it is easy to treat in our days.
Re: (Score:2)
It's arthritis, and not the kind you get from old age. Thanks for trying to diagnose me over the internet though, obviously I never bothered to see an actual doctor about it.
Re: (Score:2)
Whiteboards are fine. You just don't measure people on things that every IDE provides these days, you measure them on problem solving and code fluency (not library trivial pursuits, but if the candidate chooses the language he should know the common tools very well).
Any modern professional interview will start with the presentation of the problem, followed by "talk me through your design ideas before you start writing code". After all, the process of reasoning through the design is half the signal your tr
Re: (Score:2)
The CS grads were having trouble with the basic idiom of read a byte, while not EOF, do something with the byte. ... obviously they have trouble to some bullshit like that when they never got taught how to do it.
But you are aware that time has changed?
I mean
Why the funk would an University teach completely obsolet skills to freshmen?
Why are you testing for such obsolet skills when you should have noticed decades ago, they are not taught anymore?
It was the EE grads that actually knew what the fuck they were
Re: (Score:2)
The CS grads were having trouble with the basic idiom of read a byte, while not EOF, do something with the byte.
But you are aware that time has changed? ... obviously they have trouble to some bullshit like that when they never got taught how to do it.
I mean
If you can't solve a problem that's new to you, what good are you? Very simple problems like that are good especially because they'd be new to someone fresh out of school (where recursive descent problems are useless for new grads). Now, obviously you don't want to play "library trivial pursuit" and want to explain any bytewise I/O library calls they'd need, but given the primitives it's a fine question.
I like to ask new grads to design an arbitrary precision integer class that supports addition. In the
Re: (Score:3)
However there will always be a time when someone somewhere needs to know how to do arithmetic at a low level on a computer. These aren't things that are done once and then never needed again for all eternity. Maybe not many people will need to know this, but until we have AI that can do this we'll need people. Every time you have a new processor designed you will need to know how to do this, or when you have a new compiler, or when the third party libraries don't have what you need, etc. Even if you don't
Re: (Score:3)
Our two programs were reverse a singly-linked list and copy a file. Watching recent CS grads who were weaned on a diet of Java trying to do file copy (it's important in C/C++ to know how to sling data around in buffers) was such a train wreck. In my case, I knew the basic file copy inside and out, including the performance implications of buffering.
Or skip the buffering. I wrote an example using mmap() [wikipedia.org] for Ultrix / 4.3BSD on a VAX-11/785 back in 1991 to demonstrate Unix memory mapping to a colleague at NASA LaRC who was coming from CDC NOS and NOS/VE [ my opinion is that NOS stands for Not an Operating System :-) ]. The program cats files to stdout, but could be changed to copy to a destination. The benefit here is that the OS does *all* the paging/buffering for you and (literally almost) all the run-time is accounted for as system time. The 50 lin
Re: (Score:3)
Embedded coding is actually much more easy.
You simply have your ints and chars and do your & and |.
No idea why people try to push embedded programming as a pillar of high magic.
A modern business "framework" regardless if it is Qt on a desktop or Spring, Hibernate in the Java ecosystem or what ever the .Net equivalent is: requires 100 times more knowledge than simple stupid C programming (or even assembler) on a 1960 invented chip.
The /. idea that only assembly programmer (we tolerate C programmers) on a
Re: (Score:2)
Nooo, you shouldn't be using ints and chars for embedded! Always use intX_t that states the width explicitly.
Embedded is hard because to do anything decent you have to really understand the hardware. Not just the CPU, the whole system. You don't get any memory protection, exceptions crash the whole thing, and you have to handle your own interrupts. Oh, and your code needs to run perfectly for years without stopping or running out of memory, and in some cases needs to survive external problems like supply fl
Re: (Score:3)
I'm sure there's difficult parts to embedded programming, just like any other technical discipline, however, you should not dismiss higher level work as easier, or even less paid. Now I don't know how much you make, but there are software engineers working at Facebook who make $400k in total comp. And yes, they are software engineers, not managers.
Difficulty for them comes from 3 directions: performance, multi-threading, and customers.
Performance is a given. A 5% reduction in CPU usage directly translates t
Re: (Score:2)
It seems difficult because people are unfamiliar with it. When everyone is graduating from a Java school, bitwise operations are high magic.
I find bit bashing to be the most fun thing to do in code, but I'm at a loss for a real world problem to apply it to. Man, if I ever found a reason to count the "1" bits in a word, I could do that so fast ...
Re: (Score:2)
take this string, now reverse it, now convert it to pig latin, now reverse that.. How many internal threads are used to perform writes in a ConcurrentHashMap, now bark like a seal..
It's all about the specific questions. Asking someone to design a ConcurrentHashMap from scratch is a great question, at least for someone senior. Lots of interesting design choices, and still relevant to some parts of the industry.
It used to be popular to ask someone to write a function to find a string inside another string. This topic is so deep that I've read an actual book of nothing but different algorithms. That one has faded from relevancy these days, and doesn't seem appropriate any more.
Loaded Interview (Score:5, Interesting)
I shortlisted for an interview and got ambushed on my arrival. There were 19 other candidates, and we're all ushered into a small room with 20 desktop spots on a table that went around the entire room. We were handed one single sheet of paper with a coding problem we were to find a solution for. All 20 workstations were Mac Minis. None of us were told there was a programming exercise, none of us were told it was a Mac shop.
I walked out of the "interview" in disgust. Eleven others went with me. By the looks on the interviewers' face, this has happened before.
I don't mind being put to the test, but I don't like being ambushed.
Re: (Score:3)
Place sounds like a dumpster fire.
Does the place have a name? Sounds like in your round alone, there were 12 people who walked out. Probably dozens, if not hundreds, have walked out over time.
Re:Loaded Interview (Score:4, Insightful)
None of us were told there was a programming exercise
Do you really need to be told this? Were you expecting to be hired as a programmer without demonstrating that you can program?
Re: (Score:2, Interesting)
Demonstrating I can program means showing them examples of my work and discussing issues with them, not sitting down and completing some arbitrary task under pressure with a bunch of rival candidates.
That kind of testing just screams code-monkey sweatshop.
Re: (Score:2)
No, because I don't know that you actually wrote the examples of you work. I can't even count the number of times on freelancing sites I was asked to create them (and a few times outside of that- graduating college I was asked by a lot of people to do it for them). If you don't do it in front of me, I can't trust that you can actually do it.
Now a roomfull of people at once- yeah that's a dumpseter fire get the fuck out. This type of thing should be done one on one.
Re: (Score:2)
That's why you discuss the code. You ask them why they wanted to show you that bit, why it is interesting or impressive, and why they designed it that way.
Re: (Score:3)
That's when you talk to them. Pretty easy to determine if they wrote the code or not.
Re:Loaded Interview (Score:5, Insightful)
None of us were told there was a programming exercise
Do you really need to be told this? Were you expecting to be hired as a programmer without demonstrating that you can program?
I imagine it wasn't so much the test, but the bulk 'we don't want to waste our time, but are happy to waste yours' nature of the interview process. It can be a lot more effort for a candidate to come to an interview (time off current job, dry cleaning, transport costs) then for the company to allocate a few staff to the interview room for an hour, and it is a pretty good indicator that the company is happy to mess you about if they expect you to put the effort in to turn up, but won't even give you a one on one interview.
I would always expect a one on one interview unless they told me before hand as a matter of courtesy.
Re: (Score:2)
Re: (Score:2)
I had two interviews in the last 30 years with programming excrises. Oh, actually 3.
The third own wanted me to finish the programming exercises before the interview.
The other two announced there will be programming exercises, one only was making a kind of UML diagram, the other one canceled it.
So: yes I expect to be informed before hand if there is an programing exercise and what language they use, so I can prepare a bit.
After all, as far as I know NULL in C is not the same NULL as in C++. Your millage may
Re:Loaded Interview (Score:5, Interesting)
Re: (Score:2)
Re: (Score:2)
Exactly! ...
Obviously they where window users ... I doubt a Linux user had skipped a mac challenge.
I mean, we are old fashioned and use man pages, but that can hardly cause a grudge against us, can it?
Re: (Score:2)
Ask them what all the HTTP status codes mean
And yet both people failed the embedded chip portion of the interview.
Re: (Score:2)
True but 9 stayed (Score:2)
Heck, the current administration just added another 20,000 PHD candidates to the H1-B cap (they sold it as a benefit to workers because they'd require more PHDs ignoring that those PHDs are in addition to the existing 65k cap. Got away with it too...).
I'm really disappointed. I didn't ex
Re: (Score:2)
Re: (Score:2)
It's probably a good test for them. They're looking for people who will put up with being treated like shit. Imagine actually working there.
Re: (Score:2)
There were 19 other candidates, and we're all ushered into a small room with 20 desktop spots on a table that went around the entire room.
Could have been worse. There could have been 20 candidates and only 19 spots at the table. :-)
Re: (Score:2)
1 guy actually accepted a job there and he quit within 18 months as it was a properly shit place to work.
It was a mac shop. You have been warned!
Re: (Score:2)
It was a mac shop. You have been warned!
Idiot very much?
It was a programming job. There is no better OS/computer for programming than a Mac. Linux is miles behind and MS Windows lightyears. Only some obscure Unixes are somewhere clustered around Linux, like HP-UX and AIX ... and Solaris.
But I guess in your blinded mind you tried to make a lame joke ... you failed.
Re: (Score:3)
So there were 8 people left who had the required skills too complete the requested task?
1: If they were looking for Mac users they should have screened for them in the first place
2: If being familiar with using a Mac is the main qualification then it worked (but see #1). Otherwise they screwed themselves by rejecting good candidates who could have become familiar with a Mac in a week or two. Very stupid and inefficient.
Missed the boat... (Score:2, Insightful)
Small hint: sometimes the answer is not what they are looking for...stress handling, bullshit-o-meter, implementation ideas, probable likability, etc...
Only once I had an interview where I didn't get the job (as a software dev - back when I was a junior, was totally not prepared). The key is to be prepared and actually tell them you don't know and/or you don't want to bullshit them if you don't know the answer.
Interview questions? (Score:3)
It's pointless to ask anything other than a basic programming question during an interview. IF you insist on verifying that a candidate knows the language they claim, do that OUTSIDE the interview. You can present them with a "how do you fix this error" or give them a bit of code and ask them what it does, but any detailed questions like "Solve this problem using recursion" really doesn't tell you anything.
Personally, I ask what seems to be a programming question, but really isn't. I ask them how they'd go about writing some algorithm and then start throwing system integration tests that are failing at them, their coworkers not being available to fix something and a deadline they don't think they can meet. The point is to find out how open they are to ask for help, how readily they will report problems meeting schedule and how they choose to interact with uncooperative coworkers. To me THAT's the stuff that is more important than being able to implement a swap sort on a linked list using recursion.
Re: (Score:2)
To me THAT's the stuff that is more important than being able to implement a swap sort on a linked list using recursion.
Reverting a linked list is trivial (regardless of recursion or not - actually I don't know why would anyone use recursion for this).
But I assume "swap sort" is a term you expect me to know, sorry, I don't know it.
Re: (Score:2)
Rapid fire (Score:2)
Re: (Score:3)
Re: (Score:2)
For example: âoeWhat problem in your career did you most enjoy solving?â
None, the problems I solve are fucks up someone/a team made a decade ago.
A Favorite T-Shirt Slogan nails it... (Score:3)
This might not work for all developers in all situations, but one of the best ways to establish credibility as a developer is to find an Open Source project you like and contribute.
When it comes to the technical aspects of an interview, any technical interviewer worth their salt should be able to go to Github and verify that you are the person who committed the code you claim to own. Take along hard-copy, annotated if necessary, and be prepared to talk to it, to explain why it is something special, elegant, efficient, and flexible.
If you can get testimonials from project leads or other contributors, to demonstrate that you're a good team player, so much the better.
There are several advantages to this approach:-
1. The body of work that you build up stays yours and can be taken from job to job
2. You can share this well in advance of an interview, giving your interviewer time to review it and thus lead to a good discussion
Yes, someone is going to argue that this approach could result in cheating and could allow you to claim ownership of code that isn't yours. This is not a new problem for the interview process to factor - and a good technical interviewer should be able to ask piercing questions able to determine if you really did write what you claim is yours.
It's a completely [utterly] different example of credibility, but I am for some reason reminded of that scene in the original, Black-and-White movie version of the "Dam Busters" raid from the Second World War. Barnes Wallace, the inventor of the bouncing bomb, needed the use of an RAF aircraft from which to test drop his prototypes. It quickly became apparent that he needed an actual Avro Lancaster bomber, but of course, in the middle of a war, these were in short supply. His response?
"Do you suppose that if we explained to your man in the Ministry that I designed the Lancaster that he might be willing to lend me one?"
The whole reason we hold interviews in the recruitment process is to try and establish whether the candidate has the chops to actually do the job. Nothing says that better than hard, real-world evidence.
Re: (Score:2)
Re:A Favorite T-Shirt Slogan nails it... (Score:4, Interesting)
My biggest problem with your approach is I don't have any code in github. A lot of professional programmers don't. I program 50-60 hours a week for a living, the last thing I want to do when I get home is program. And the company owns all that I code, and they don't put it on github. Any programming I do at home is typically related to learning new languages and libraries, and to put it mildly, none of that is worth publishing anywhere.
+1 Insightful (Score:2)
An obvious parallel (Score:3)
Stop interviewing, start bidding contracts (Score:3, Insightful)
I was a lot happier when I bid contracts for work and stopped interviewing for jobs.
Being an employee sucks. The tax system is set up to screw over the working schlob. If you're smart enough to program well, you're smart enough to hack the tax system too. Programming is the only profession that has anything like trivial tests in interviews. The flip side of this is without professional credentials, that's where things go.
Otherwise.. stand up and do tricks.
Unionize already. Doctors are unionized. Lawyers are unionized. They call it something else, but that's what it is..
Subjective. (Score:5, Interesting)
Re: (Score:2)
'which trick did the person who designed this test have in mind?' or even worse 'which niche within the language is the person who wrote this obsessed about?'.
Exactly.
But then again I had a phone interview, and as the real software was in C/C++ and the CI environment was in Java, they asked me: what are the byte sizes of char, short, int, long in C and in Java.
So I answered. Got the job. And later I asked one present in the interview: what was that lame question about word sizes of primitive types? He answ
Reminds me of the phone book joke (Score:3)
A lecturer asks students to learn the phone book by heart.
The physics students ask: "Why?"
The medicine students ask: "Until when?"
There's a longer version, but you get the point. Apparently some of these "hard interviews" are just testing how fast you can cram irrelevant knowledge into your head every sane person would look up when they need it. That someone knows how to write down a quick sort algorithm in c with every semicolon in the right place doesn't mean they know when best to use it.
And now for something completely different:
https://www.youtube.com/watch?... [youtube.com]
Er isn't stress testing part of the test?? (Score:2)
I would have thought that most employers might be interested in your ability to perform under stress as well or at least your ability to handle it? If you can't program at all when under stress, that's not going make you a very effective programmer in the real world.
Re: (Score:3)
WTF?! In the real world, when are you going to need to write a program to escape a burning building? If the place is such a dumpster fire that they are trying to push out critical, last minute fixes with the customer standing in the waiting room, then good God, get me away from there!!
Comment removed (Score:5, Interesting)
Re:I've conducted many interviews (Score:5, Insightful)
Indeed. I find that within a few minutes of talking to a candidate, I usually have them nailed down pretty well.
A good developer simply talks differently than a bad one. They talk about what matters rather than whatever rote thing they learned in school, they are excited to talk about cool projects they did, enjoy getting into the mud answering deeper questions, have opinions about things they've been using for a long time, and don't bullshit claiming they know things they don't.
The ones I do give a longer coding exercise to are juniors -- these guys don't usually know how to chat just yet, and don't have any interesting projects under their belt. Enthusiasm can only go so far, so I like to send them home with something to spend an hour on, letting them send it back the next morning.
Re: (Score:2)
Re: (Score:2)
So far out of the 8 people I've hired/recommended, only one didn't work out.
I think that's an unacceptably high false positive rate, though it correlates pretty well with what I saw when I used your approach.
The problem with the approach is that there is a non-trivial percentage of people who can talk the talk, but can't actually do the job. They're good at understanding the successes of their peers in enough depth to be able to successfully claim them as their own, and even fool a competent interviewer in spite of not being able to do the work themselves.
Coding interviews are
Re: (Score:2)
Re: (Score:2)
Re:I've conducted many interviews (Score:4, Insightful)
The problem with the approach is that there is a non-trivial percentage of people who can talk the talk, but can't actually do the job. They're good at understanding the successes of their peers in enough depth to be able to successfully claim them as their own, and even fool a competent interviewer in spite of not being able to do the work themselves.
No. The problem with this approach is that it requires an interviewer that has been in the trenches long enough to have worked with enough of the moochers. Once you've worked with a few, they're easy enough to spot. Unfortunately, competence in interviewers closely tracks competence in the general population.
Re:I've conducted many interviews (Score:5, Insightful)
This is another problem with the modern interview process - you only get to see half the results. Your success rate among hires is 87.5%. But you have nothing to compare that against because you don't know what the success rate would've been for the people you didn't hire. Who knows, maybe 90% of the people your interview process filtered out would've made good employees too, and your interview process is actually picking worse candidates than random chance?
The only method I've been able to think of to test this is for management to hire "perfect applicants" to apply for job listings at their company. People with a made-up history, skill set, and knowledge which makes them perfect for a job. See how far they get through the HR filtering and interview process. If HR or an interviewer rejects them, then that indicates a problem with HR or the interviewer.
Re: (Score:2)
Programming test during the interview? (Score:3, Interesting)
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
Re:Programming test during the interview? (Score:5, Insightful)
I'm not surprised HR raised holy hell over that they were 100% right to do so. Only people in fairly narrow niches have time to do take finye coding tests. It skews heavily towards the young and especially those without dependents. Older candidates tend to have more in their lives outside work which preclude doing such tests.
And older, more experienced candidates know you don't do work for free.
They don't know you or your company. You're spending zero effort to ask them for a lot. They have no indication you'll put anything like a commensurate amount of effort into evaluating their work. We've all seen jobs where the winner was selected even before the advertisement. Who on earth but the naive or desperate wants to risk their work on that.
There's no guarantee that you'll get a fair shake if you have an on site, but you know that the company is spending about as much time as you are, so there's a good chance they're not simply wasting it.
Programming Test During the Interview? (Score:5, Interesting)
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
I just rebooted and forgot to login to /. so this is repeated AC post.
Here in the Valley.... (Score:2)
I've talked with several programmers here in Silicon Valley, who seem to state that HR and management wants you to code a whole project for nothing.
Mixed bag, wait until later (Score:2)
In the interview process just determine if someone can talk to talk. Save the proof for the first 30-90 days so people can work under normal circumstances. Some people suck at interviews anyway. For myself, I always bomb the easy questions because they are usually nonsense the compiler will catch, minor syntax, minor implementation semantics.
Unless you are hiring entry level these things are all minor noise and what separates talent from drones is higher level thinking. You are going to end up with someone
not testing for the right thing (Score:3)
Problem solving is important but having a good work ethic is at least as important and most interviews do little to identify that.
I give simple coding problems. (Score:2)
I check to see if they spend enough time to understand the problem, do they ask questions to make sure the specs are, I like it if they ask, "do you want a fast algo or minimum memory algo?". I ask them think out aloud, and guide them towards the solution. Then the fun st
Interviews should be a two way street (Score:3)
Best option: question with variable difficulty (Score:2)
My preferred approach is to use a question that has many different variations, and to start with the easiest variations and then ramp up the difficulty/complexity based the candidate's performance. I only have the candidate write code for one variation, usually the second-hardest one that we talked about, and then ask them how they would go about testing what they implemented. This approach has many advantages.
First, it lets me see how mentally agile they are. Weaker candidates have a hard time adapting
Easy questions (Score:2)
Definitely agree about the easy questions. When candidates can't answer easy questions correctly, that provides a strong indication that the candidate is a bad hire.
When candidates can't answer hard questions, what does that tell you? They need extra time to solve difficult problems? Who doesn’t?
Re: (Score:2)
No wonder so many programmers suck ... (Score:2)
We Give Live Programming in Interviews (Score:3)
What we do is provide a list of about 6 (depends upon skill set we're hiring) problems that equate to first or second year CS studies. They are all straightforward and require no fancy manipulations. The candidate picks one. We set them up with an environment, they can access Google (keep search history intact, no Googling the actual problem).
They are given an hour to do the problem; anyone on the team can do it in fifteen minutes or less. If they're having trouble, we will give them another hour. There is a proctor present to help with non-programming issues (for example, for C++ folks we'll help them compile if they're rusty with command-line gcc).
These types of simpler exercises provide insight into some of the fundamentals: can you write code? What methods do you use to solve problems? Can someone else easily understand your code?
The number of C++ 'programmers' who can't compare two lists based upon criteria, or web programmers who (no joke) can't code a page that dynamically retrieves data using a (provided) web service, or database folks who can't join two tables is staggering. The exercise weeds out people who have spent years - decades - copy/paste/trial-error coding. Sorry kids, we have real work to do.
Good Article but Need Thought Question (Score:2)
Re: Harder? (Score:4, Insightful)
Answers are never that important to me as an interviewer unless I am verifying they possess a certain skill. I am much more interested in how conscientious a candidate is in looking for the right answer. It is predictive of how they will solve real problems when they are unsupervised later after being hired.
Re: Harder? (Score:5, Interesting)
Answers are never that important to me as an interviewer unless I am verifying they possess a certain skill. I am much more interested in how conscientious a candidate is in looking for the right answer. It is predictive of how they will solve real problems when they are unsupervised later after being hired.
Quoted in full for emphasis.
It's not always important to have a problem answered fully or correctly in the allotted time. It is far more important to see *how* the candidate tackles the issue, what steps they take, what mental tools they bring to bear, and most important of all, knowing when to ask for more information and/or assistance (and then paying attention to what they're asking for, and the reasons they state as to why.)
I used to do this back when I dealt with just sysadmins. One of the fun bits was to chattr +i a config file, but then have them alter it as part of the testing (but not give them the permissions to fully do that).
The point wasn't to torture anyone, but to see how they handle non-ordinary problems. To watch how they tackle it, how they determine what's going on, then know whether or not to ask for help - and to see if that request for help is specific ("you'll have to change the attributes back before I can update it"), or if it was vague.
That told me more about the skills of the candidate than whether or not they successfully knocked out a problem.
Re: (Score:3, Insightful)
Re: (Score:3)
Maybe you could talk to them, instead of trying to deceive them?
You don't have to deceive them entirely - just emulate a problem or two to see how they handle it.
Re: (Score:3, Informative)
Part of a sysadmin job is to fix the mess of others, get lied to and deal with incompetents.
Sometimes, you'll have to fix a problem, and no one will be there to help you. You are the expert, you are supposed to fix problems, not the other way around. When asked, people will invariably tell you they did nothing. If there is some problem you can't fix (ex: that "read only" attribute), you'll probably have to tell the person who can fix it exactly what to do. Either because that person doesn't know (remember,
Re: (Score:2)
A good talker can use up the interview time on subjects they know well. Then you don't know their weaknesses. Skilled interviewers can deal with that situation, but how many people were hired for their skills interviewing job applicants? It's a complete crap shoot whether anyone conducting an interview is a skilled interviewer. Different types of interviewers choose different approaches based on what works for them.
Re: Harder? (Score:4, Interesting)
I have found it useful to give them questions that they may not know.
I had a test that seemed to be rather good at judging ones ability. And it took people an average of 4 hours to complete.
Question 1. I have an HTML file with a Picture of two boxes overlapping each other, and a box with two empty Div's asking them to make the other box look like the picture. (Non Web people will need to google to learn how to use style sheets, and position properitys)
Question 2. A simple Application form, that asks for the address, validates that the format is correct, and just pops up a text box giving the errors or showing the address in a proper US mailing address format. (The zip code with leading 0's gets them every time. )
Question 3. A SQL Stored procedure, that returns a table where the rows become the column. The code works correctly, however there is a null in the data, preventing it form working correctly. (So they either will need to do something with the null, or exclude it, extra bonus points if they state how insecure that code was.)
This seemed to have allowed the company to get some good employees, but still it isn't fool proof, a few people slipped threw the cracks, mostly because they actually had experienced those particular issues before, so they knew how to handle them, but turned out they would struggle on new problems.
Re: (Score:3)
I have found it useful to give them questions that they may not know. I had a test that seemed to be rather good at judging ones ability. And it took people an average of 4 hours to complete.
Question 1. I have an HTML file with a Picture of two boxes overlapping each other, and a box with two empty Div's asking them to make the other box look like the picture. (Non Web people will need to google to learn how to use style sheets, and position properitys)
Question 2. A simple Application form, that asks for the address, validates that the format is correct, and just pops up a text box giving the errors or showing the address in a proper US mailing address format. (The zip code with leading 0's gets them every time. )
Question 3. A SQL Stored procedure, that returns a table where the rows become the column. The code works correctly, however there is a null in the data, preventing it form working correctly. (So they either will need to do something with the null, or exclude it, extra bonus points if they state how insecure that code was.)
This seemed to have allowed the company to get some good employees, but still it isn't fool proof, a few people slipped threw the cracks, mostly because they actually had experienced those particular issues before, so they knew how to handle them, but turned out they would struggle on new problems.
This is the correct outlook.
Unfortunately, there are enough "interviewers" out there looking for canned code monkeys, that it really makes an already stressful activity (job hunting) into an even worse hassle.
Re: Harder? (Score:2)
My first reaction is "Why?" to these types of questions. I work for a small company now: they have things that need to be done. Coding in large enterprises tends to be "coding in the small", with little creativity required and everything spelled out for you.
Re: (Score:3)
The company I work for has multiple divisions. We primarily build technology. In some divisions, the people who write code are, indeed, just laborers who build exactly what is provided them.
But in a couple - magical - divisions, the people writing the code understand the intent, the business, and the objectives. It's much harder to find, hire, and develop these people. It's especially bad when someone is good at one facet but n
Re: (Score:2)
What came first? the egg or the chicken?
The egg.
The life cycle of a chicken begins with the egg, results in procreation to produce more eggs, and eventually the death of the organism. Ergo, a "chicken" to this argument is viewed from the point of the egg.
Applying evolution, one must define a strict categorization of what a chicken is. Once that is established, there is no "slippery slope" logical fallacy, and you can determine, in binary, what came first on the life cycle, as a chicken.
An "almost chicken" laid a mutated "almost chicken" egg, w
Re: (Score:2)
An "almost chicken" laid a mutated "almost chicken" egg, which is now classed as a chicken. Ergo, the mutated "almost chicken" egg is the world's first chicken egg, which becomes the first chicken.
q.e.d.
Not really.
Speciation occurs within a population which is mixing its DNA, not within one individual. It's really impossible to be so genetically different from your parents that you couldn't mate with them, and successfully mating is how we categorize a species. Within the population of a species you'll find a wide variance of DNA, and as time goes on, because they're mixing that constantly, the whole species is changing. Over time, that species may change so much that it's not the species it used to be, bu
Re: (Score:2)
I went to an interview and was handed a programming test to implement a web based vending machine. The test also required implementing a service that mocked a credit card processor.
Pfft, no problem. Mine even handles multiple card processors.
Re: (Score:2)
Re: (Score:2)
the quiz is not the interview. the actual interview process is a more involved two-hour live call. source: i do remote interviewing for them as a side thing.
the actual interview does have live coding -- but note that it's coding using whatever tools the candidate is comfortable with, not a whiteboard. also more involved question/answer things, and more. it gets a lot of additional signal beyond what the initial quiz does.
Re: Don't take them. (Score:2)
I have worked for decades, for companies tiny and huge. I have a github account. I write articles about cutting edge techs. You give me some abstract problem to solve that has no bearing on what I do every day. You don't need me, I don't need you.
Re: (Score:2)
Re: (Score:2)
I have a proof, but it's too long to fit int he margin of this post.