Hiring Programmers and The High Cost of Low Quality

An anonymous reader writes "Why is it so hard to find good programmers? And why should companies favor hiring fewer more senior developers rather than many junior ones? Frank Wiles discusses his thoughts in his article A Guide to Hiring Programmers: The High Cost of Low Quality"

Sigh.

FTA: ...Experience is key, but not necessarily in ways you might imagine. Time in the saddle, with a particular language is not as important as diversity of experience. Someone who has worked in several disparate industries, a generalist, is often a much better developer than one who has spent years in the same industry. There are exceptions to this, but in general I have found this to be the case. Bonus points if your developer was a systems administrator in a former life.

Some of the best developers I know were originally trained as journalists, mathmaticians, linguists, and other professions not normally associated with software development...

As a generalist programmer, originally trained in cognitive science, who has formerly worked in several disparate industries, was a systems administrator, programs in half a dozen languages (including perl), etc, etc...Apparently I'm supposed to be making twice my salary. Goddamnit!

*stomps off in search of his boss*

These days, being a programmer generalist (even worse, one with admin experience) just increases the types of shit that get dumped on you...Where they might have had to hire a person to do the front end GUI code, a person to do the database work, a person to set up the server, and a person to code all the services that need to constantly run in the back end, instead, since they've got you, you can do it all, while the specialists sit around drinking coffee and making catty comments about how much better they are at what they do than you are.

My advice is specialize in something to the point where when you do any work on it, it's immediately out of the comprehension of a generalist or a less accomplished programmer...Sure, everyone will hate you, but they'll have to deal with you, and you'll be in a position to dictate terms. What's a generalist got? They're great employees. Big deal. Being a great employee is like being a great dog; at the end of the day, they'll still euthanize your ass when you're no longer of use.

//Not bitter or anything.

Re:Sigh.

But the advantage of being a talented generalist is you have a N+1 higher chance of remaining employed then someone that can only do one thing, no matter how well.

Re:Sigh.

Tell me again? Just how is it you've managed to get this far in life having never fallen victim to office politics?

Three possible methods... may be used in combination:

1) Small companies/organizations
2) Being completely oblivious to politics and not getting involved
3) Consulting/contract work

Note, I'm not the original person you were asking. I just thought I'd chime in.

Re:Sigh.

Well, the question wasn't how can anyone do it, the question was how did *I* do it. The first point, avoiding large corporations, is probably the most effective way that I have avoided office politics deciding the fate of my job. You can form more personal bonds in a small company and it is much more difficult for someone else to hide their incompetence.

Another good strategy is simply to be good at what you do and don't give anyone any reason to doubt your sincerity or integrity. Always be upfront, frank, and honest. Never be afraid to say "I don't know" if you don't know something. If/when someone approaches your boss to complain about you (presumably for no good reason), your boss will take your side by default and you can therefore remain oblivious to the politics.

Of course, if you're in management, too bad. Politics is pretty much your jobs then. ;-)

Re:Sigh.

The first point, avoiding large corporations, is probably the most effective way that I have avoided office politics deciding the fate of my job. You can form more personal bonds in a small company and it is much more difficult for someone else to hide their incompetence.

My personal experience (clearly being a statistically significant single datapoint . . . ) was in two small business (one around 20 employees, the other around 40 employees), and both were extremely cut throat. In particular, the second one had the employees forming close, almost family-like ties, but management was completely insane. That's where I discovered that being the guy that is considered absolutely invaluable doesn't actually insure job security, but rather makes you a target by people who consider you a threat to their own position.

Now I work under a consulting company under a Fortune 500, where I'm almost completely insulated from the normal office politics. Whenever I have a bad day, I watch "Office Space" and remember why I'm so lucky.

Re:Sigh.

Avoiding being a victim to office politics is doable. It is about making yourself look good as well as the rest of the department. Politics come in when you are trying to make yourself look good either by focusing completely on yourself or at the expense of others. If there are other people trying trying to make themselves look you you help make them look good, if they are trying to make you look bad you make sure you still look good without making the other guy look bad. I work in a small company but I am one of those evil contractors who do work for bigger companies and there are always people want to see me fail but normally after I help them succeed then they are normally more welcoming to me.

Re:Sigh.

"But the advantage of being a talented generalist is you have a N+1 higher chance of remaining employed then someone that can only do one thing, no matter how well."

Not so true: say you are quite expert on A, B and C. On a mature or stressed market that only will mean that you won't get work neither on A, B nor C because those job positions will go for "real niche expert on A", "real niche expert on B" and "real niche expert on C", repectively.

Re:Sigh.

Says you. I'm a "generalist", but damned if I don't solve tons of problems around the office that stymie specialists because they don't know the full system, or how to think of it from any other angle than what they've been trained. As in, the Java interface programmer who keeps asking me how to use LDAP, which he hasn't had experience with, or why the 64bit version isn't working. Generalism still has a lot of place in a business.

Re:Sigh.

I think we're talking about two different things.

By "specialist" I wasn't talking about idiots who specialize in one part of a particular programming language. I was talking about the people who specialize in writing software for one part of a particular industry and know that part of their industry very, very well.

For example, people who specialize in industrial robotic systems, telecom systems, automotive computer systems, avionics systems, etc.

To put it another way: Do you want the avionics in the plane your flying in to be written by 10 specialists with 200 years combined experience writing avionics systems, or by 50 general programmers with 5 years combined experience writing avionics systems?

Re:Sigh.

I would say an expert can at times generate up to 3 times a much output, but 10 times is ludicrous.

Heh, quantity is not quality. Do you have any metrics? DeMarco and Lister [amazon.com] do, and their data seems to show 10x. As in, not 'more code' but 'better code, fewer bugs, faster execution'

Re:Sigh.

My advice is specialize in something to the point where when you do any work on it, it's immediately out of the comprehension of a generalist or a less accomplished programmer

Perl and Batch files it is!

Re:Sigh.

Not specific enough. Just batch files!

Re:Sigh.

Every five years someone rediscovers, The Mythical Man Month [amazon.com] and thinks they've had a great insight. People should be handed a copy of this when they start their tech jobs. Managers should have it inserted forcefully into appropriate orifices. Hardback copies for senior management.

Basically, some people are just better coders, and adding sub-standard assistance just ensures late, sub-standard software. Adding people to late projects makes them later.

Re:Sigh.

^^ No kidding, it's like this guy just read about the Code team as surgical group. I'm a baby programmer (young, i don't actually program babies) and even i think this is old news.

How do you tell the difference???

While many people have an intuitive feeling as to what constitues a Good Programmer from a Bad Programmer, there are very few quantitative measures. Bad software does not look vastly different from Good Software.

By some estimates, Good Programmers can be a factor of ten or more productive than Bad Programmers, yet they are seldom paid more than a few tens of % higher. It would be far better for most companies to pay double the going salary to attract only the best, but unfortunately business thinking does not seem to be structured that way.

Most organisations base their planning on some convenient notions like programmer-months etc, using some standardised measure for programmer capability. These measures are great because they make the spreadsheets look neat and tidy. They also make all the outsourcing logic work: "I can get programmers in country xxx for $10 per hour". Untimately they are flawed because you get what you measure. If you don't pay a premium for good programmers you won't get them. You end up spending mucch more on crappy programmers.

Re:How do you tell the difference???

Yeah, being able to tell which programmers are the good/great/uber is HARD. It's much easier for companies to go on metrics as above instead of attempting to filter through for the excellent people, or even the most relevant person for the position.
Compounding that, it's rare that a coder will admit to being subpar. Chances are even if you're dailywtf material they think they are great programmers! I've been doing code in one form or another for over a fifteen years and consider myself pretty good, great sometimes. I've worked with one uber programmer in my entire career (John Kichury of SGI), maybe 2 or 3 others that came close, but have met many that act and talk like they are. Following on their projects always has a common thread of being overly clever, loosely documented and hard to maintain.

Re:How do you tell the difference???

"It would be far better for most companies to pay double the going salary to attract only the best"

1) Everybody knows that some horses run faster than anothers. The problem, my friend, is telling appart *which* one will run fastest this evening's race.

2) Do you really think that by paying double bad programmers will be repeled and won't try to apply for your job offer?

find the enthusiasts

I think the single best way to find good programmers is to find enthusiasts.

The reason is that it's easier to determine how enthusiastic someone is than how good a product they develop:
    - enthusiasts usually have side projects
    - enthusiasts often create libraries of code that can be reused
    - enthusiasts will have a variety of favorite tools - and can explain why they like them
    - enthusiasts will likewise have a variety of favorite methods - and can explain why they like them
    - enthusiasts read widely in their field
    - enthusiasts know the names of those who have made impacts on their field
    - enthusiasts often find themselves putting in too many hours - because they *enjoy* the work
    - enthusiasts gravitate together - put them in a room together and you'll have a lively conversation

And there are technologies, methods and tools that attract enthusiasts. For example, I've found that even if python and ruby aren't the most marketable languages out there - they are great ways to find the enthusiasts.

Of course, this won't help a manager that lacks enthusiasts on his team. But a technical manager who is himself an enthusiast, and builds such a team should be able to easy find more. At least in my humble opinion. :-)

Easy answer.

As a generalist programmer, originally trained in cognitive science, who has formerly worked in several disparate industries, was a systems administrator, programs in half a dozen languages (including perl), etc, etc...Apparently I'm supposed to be making twice my salary. Goddamnit!

See The Market for Lemons [wikipedia.org]. The existence of tons of bad programmers, and the inability of employers to tell them apart from good ones, drives the salaries of all programmers towards that of the average programmer.

Re:Sigh.

Ha! Trust me on this, COBOL programmers will take their goddamn jobs to the grave with them. Those S.O.Bs get called out of retirement and paid consultant money to fix things they arsed up decades before.

Anything that sits on a financial system will be changed as seldom as possible, and COBOL is alive and well with people who maintain those damn ancient not-to-be-upgraded systems.

Re:Ant farm engineer

Your wish is google's command:

http://dribibu.xs4all.nl/dilbert19950813.html [xs4all.nl]
http://dribibu.xs4all.nl/dilbert19951230.html [xs4all.nl]

mythical man month

that is all.

They exist, but they don't know it.

Here's what Paul Graham had to say about Great Hackers:

Because you can't tell a great hacker except by working with him, hackers themselves can't tell how good they are. This is true to a degree in most fields. I've found that people who are great at something are not so much convinced of their own greatness as mystified at why everyone else seems so incompetent.

http://www.paulgraham.com/gh.html [paulgraham.com]

More to it than skillz

There are other issues besides technical skills. The higher you rise in the food chain, the more the "soft skills" matter. Organizational skills, people skills, communication skills. All the elegant code in the world doesn't make up for a prima donna who won't show up for a critical meeting or who openly disrespects "lesser" members of the team. The last thing in the world most people want is to hire the developer equivalent of Terrel Owens... because, just like Owens, they will leave damaged teams in their wake. Morale counts. The reason that leads get paid more than individual contributors is not just because of technical skills. It's because they can herd cats. It's because they can recognize that business reality sometimes has to trump "ideal" elegance or philosophy-of-the-week. It's because they can convince Dev to talk to QA to talk to Product Management to talk to Sales. It's because they can somehow get a clear functional spec from the marketing guy. It's because they can get by with existing equipment instead of demanding an Intel Core 31337 for their desktop. It's because they don't have to have an HR apologist in tow smoothing ruffled feathers everywhere they go. "Senior" implies so much more than "technical guru".

Re:Internal Inconsistency in his Argument

I worked for a company that got bought by a bigger company.
We had an über programmer. He left because rather than exceptional pay, he wanted good enough pay and a small company style work life.

We then got in a pickle where some kernel mode drivers for NT4 needed to be revised and SoftICD'd in. Even though he doc'd everything and gave training to our programming staff about gotchas and pitfalls as well as maintenance, it was something that only about 100 people in the world could really do. All we could get done is a widely variating series of BSODs. We hired him back at $12K/day + travel for 5 days of work. He did work his ass off, further document everything, and provide additional spot training to our two brightest. The job was done and he had a check for $60K. I suspect that the training and docs took the vast majority of his time. I asked him why so much (and why MegaCorp would pay that) and he said it was simple. They were a big company not interested in paying him either in lifestyle changes or money so he didn't stay, but for a short job he charged what it was worth.

The free market did work. (considering his solution was cheaper than a contract with MS for the same work by $40K).
-nB

Re:Internal Inconsistency in his Argument

But I think that if the über-programmer really does exist, then eventually the free market will figure that out, and compensate him accordingly.

It has, and then some. These "über-programmers" are what you and I know as "wildly successful startup founders." Part of the reason it's so hard to hire them is because they are mostly already independently wealthy and / or personally invested in a project of love that no offer of cash and benefits will draw them away from. Most likely, if the former is not true, the latter will eventually cause it to be true. The best and most common way of hiring an über-programmer is to buy the company they currently work for.

Re:Internal Inconsistency in his Argument

Not really. The uber programmers rarely have the skill set needed to found a buisness. Those tend to require high people skills and financial skills, which are almost completely disjoint from the programming ones. Plus it would mean spending time doing all that bullshit, instead of programming.

The real answer is that most uber programmers work for that small premium. They get to do what they want, get a few other perks like their choice of assignments, and are generally happy as is. Money isn't really the key motivator for them- if it was, they'd be in another field that pays more.

Re:Internal Inconsistency in his Argument

There's no one programmer who does the work of ten other programmers. One uber-programmer does just as much work as one ordinary programmer. It's just that the results solve ten times as many problems. Programming is fundamentally a design problem. A great bridge designer doesn't do the work of ten lousy bridge designers; the great one designs one great bridge in the time it takes the ten lousy ones to design ten lousy bridges.

The best approximation is that each problem has a certain complexity and a certain size. The size determines how long it will take, and it doesn't matter how good the developers are. The complexity determines how good a developer is needed to make progress at all. If you've got only easy problems, an uber-programmer doesn't help you much (unless the programmer can find a smaller, harder problem that replaces the big easy one). If you've got a hard problem, ten average programmers will work on it forever without getting any results.

And there's one last thing specific to computers: the computer can solve easy problems for you, but making it do so is a hard problem. But solving that one hard problem (plus some processor time) resolves a lot of easy problems. Another type of hard problem is writing a magic library function that makes a range of moderately hard problems easy enough for average programmers to solve.

If you've got ten people essentially doing data entry, an uber-programmer may be able to eliminate the need for them to do that at all. If you've got ten developers working on some problem, an uber-programmer may be able to double their productivity. In either of these cases, the uber-programmer directly produces something that isn't part of the actual project, but the benefit to the project is on the order of ten average programmers' work. And, if the uber-programmer reduces the complexity of the problem to put it in reach of the rest of the team, no amount of ordinary programmers' work would benefit the project as much as the uber-programmer's contribution. Of course, if you require an uber-programmer to literally do the work of average programmers, there's no benefit at all.

ueber programmers do exist!

>There's no one programmer who does the work of ten other programmers

Oh there is, I am one. I've had several occasions where I solved problems in a few days (or 4 hours once) that others were struggling at for months; some of these others were plain incompetent, others were pretty good.

I find that I'm pretty unique in fitting the right tool to the problem at hand, as well as in general overview. I've never met anyone as productive as myself.

I'm posting this anonymously, because I have to work with others, and one of the things you cannot do is alienate everyone around you; one sure way of doing that is being more skilled than them in all job related aspects.

Best damn article in a while

While I am not a dev, Sysadmin here, this is probably the best article I have read on the subject in a long time. This idea of lets get someone in and train them up is assinine. Of course not every company can afford 120K a year but what about the lower end, midwest people get hit up with 45K a year jobs all the time, if the company would jump to 60-70K they would get 2X the dev and also get a much better product. I am currently with a company that made the mistake of hiring a below par employee to dev a site. Now they lucked out and got someone for the same price who doesnt care about salary but it a hell of a PHP developer, probably the best I have ever worked with. He spends 90% of his time fixing mistakes of the last dev and does things in minutes that took his predecessor days.

Same concept goes with my job field, I spend a considerable amount of time consulting, fixing poorly configured networks and servers. You cant just grab a joe off the street and expect him to be a professional or put out professional work without having learned his/her lessons, they will make mistakes learning, do you want it to be on your buck and your network?

Re:Best damn article in a while

This idea of lets get someone in and train them up is assinine.

Dumb question, but if nobody trains new developers, then where the heck are those more experienced developers supposed to come from? And of course the related question, where did the few that we now have come from?

T

Re:Best damn article in a while

Some people are self-taught or learn on the job. Not everybody needs to learn from another person.

Re:Best damn article in a while

welcome to the working world. The specialized knowledge you seek doesnt come from your first job. It comes from taking a thousand crap jobs and compiling information as you go. If you seek to be specialized in mainframe development then you have a long path in front of you, as there are many more experienced mainframe developers with a good 15 years of experience ahead of you.

However if you are one of the aforementioned mainframe experts then you need to take a crap job or some side jobs as a web developer, starting off working for free for non-profits and the like, then as your skills progress you can start charging then work your way up from there. Then you can start looking for the mainframe web devs that you are aiming for. However if you are like most people you become a specialist because that is where the job field led you. You may see that there is a good paycheck in the area or it may interest you but that is exactly why those jobs pay well, because they are a rare person and unless a trade school offers a program you are SOL.

The purpose of the article is to explain that a good dev can make up for some serious problems as a beginner programmer, or if your company is big enough you can hire the underlings to do the crap coding while the expert does all the engineering, planning and actual Development. Those entry level people pay their dues coding and recoding the same scripts over and over for the guru and after a while they move up, taking that knowledge with them. However if a company cannot afford a whole host of devs, they had better hire the good ones rather than the entry level employees.

Re:Languages

The hardest part of finding a new job is getting past the recruiters, who are generally not capable of anything more than keyword matching against your experience. Use your contacts if you can to get in front of the technical managers who will understand that your domain knowledge and overall experience is more valuable than which languages you have been using for the past few years.

Re:close your browser now boss

I'm new to it all also. I've worked at this company a few months and have designed two separate applications, while maintaining/improving a third. All I can say is it seems that I'm a better developer than the people that came before.. The project I've had to maintain is garbage spaghetti code that was said to have been developed over a few years, and would take me 6 months tops to write from scratch. I'm not saying I'm good at what I do but I am saying whoever came before was terrible at what they do. This boggles my mind considering that it was written by 'Senior' developers, who probably make double what I do.

I obviously haven't seen the code you're talking about and hove no opinion of it. However, I've heard this crap from new programmers for years. So let me through some ideas your way.

1. it isn't garbage and spaghetti because you have difficulty following the technique, there are a thousand ways to do anything. It's garbage and spaghetti when it doesn't work well, and when it's under documented.
2. Could you really write it in six months without referencing the code you are talking about? It's one thing to write code when the problems have been solved, quite another to solve the problems.
3. the scope of a job changes over time. For a new programmer, start looking for scope creep, it's a friend and an enemy.
4. code changes over time because languages change over time. Look for those situations in the spaghetti code where those guys that you are better than wrote functions that were not available in the language. You may get a surprise that they did something really elegant to overcome something missing in the language originally that exists now.
5. lets say that there were five programmers on the code before you, one after another. The first four might have been gods gift to programming, it only takes one pretentious newbie in the chain to really screw up pretty code.

certainly none of these apply to your project, but it's something to think about as you are exposed to projects in your career.

Sales is looking at demand and driving supply

One aspect not discussed: Programmers are in short supply because demand for code and new features is limitless.

My company right now has huge demands for new features and new software. While development is desperately trying to fight the urge to pump out more and more features, they fail miserably each cycle. This is coupled with the fact that we have tons of work to do cleaning up bugs. No one can stop and catch their breath, the work keeps piling on.

This cycle will continue until a customer realizes they can't get something on time, or the quality is so bad the software won't sell any more. Customers think the software just materializes because they see it on the shelf. It took years to get it that way.

Salesreps do and say anything to get the contract signed, and the details get ironed out later. As long as the cash rolls in, large companies aren't going to change this.

The most interesting link in the whole thing

Was to the 400 disc CD Changer.

Seriously, we knew ALL of this a long time ago. HR just has yet to catch up- they'd rather hire 100 slightly-less-than-competent people who have the right keywords on their resume than a single lazy generalist who will figure out the right way to code it the first time regardless of how new they are to the language. And it's the second one you want. The real bottleneck isn't finding expert programmers- it's finding HR people who understand this industry.

We all have to start somewhere...

As a recently graduated CS student, I find this type of thinking to be incredibly infuriating at times. Companies only want to hire people with experience. Yet to gain this experience, I need a job. The circular logic goes round and round until you have a brain aneurism.

My college never stressed learning any one language well. Rather, it taught us the tools and techniques we would need to survive in the ever-changing world of software development. Yet none of this seems to count for anything. No past experience with a company? Goodbye. The fact of the matter is, I need to start somewhere. Right now I'm sitting at a job that I feel doesn't tap my abilities, yet I put up with it for the "experience." The number of opportunities for fresh graduates are few and far between, and you have to take what you can get.

Re:We all have to start somewhere...

Your tools and techniques are probably bad, especially if you learned them in school. Do you know how to:

* use source control?
* analyze someone else's code (multiple people's code), figure out what it's doing, and map that to what it's supposed to be doing?
* can you understand the bug at all (what is is supposed to be doing)?
* can you figure out how to verify that your fix actually worked?
* do you understand how to configure and use the product you're working on?
* explain what you're about to do, and justify why it should be done like that?
* be focused enough to fix one (1) bug, and not go off and rewrite a whole lot of stuff that looks like cr*p?
* not break the build?

In real life, doing architecture and writing stuff from scratch rarely happens...but that's all they teach you in school. In real life, you're working on some big pile of code that you're stuck with, can't change, and don't understand. You can fix #3, but usually #1 and #2 are immutable...until the magical day when they need a new feature (hey, we need to redo a whole chunk of that thing to get the new feature to work).

Do you need experience? Write something. Nothing sells your coding skills like code. The downside is that people will be able to see how your code is. If the programmers in your target company are good, they'll be able tell the difference between someone who's new and someone who just sucks. If they aren't so great, then your code is still a plus, because they won't know how bad it is.

Re:We all have to start somewhere...

As a recently graduated CS student, I find this type of thinking to be incredibly infuriating at times. Companies only want to hire people with experience. Yet to gain this experience, I need a job. The circular logic goes round and round until you have a brain aneurism.

As someone with experience I'd just like to suggest that maybe they aren't stuck in a logical loop. Maybe you aren't the person they want, they want someone with experience.

Here's an idea: As a computer programmer, you are in a unique position to make your own experience. I got started in "the business" by developing a totally crappy (seriously, I'm ashamed) graphing calculator for the HPC. I have an HPC, I needed a graphing calculator, and I'm cheap, so I really wrote it for myself. Then I put it up on the web and people liked it, so that became the first point on what is now a long programming resume.

My point is, you don't have to get hired by anyone to get experience. If you don't have an idea or an itch to scratch of your own, pitch in on some open-source projects.

When I am hiring junior programmers, the guy who is fresh out of school I'm going to overlook. The guy who's fresh out of school AND has some projects of his own he's worked on is EXACTLY the guy I want, though -- not only do I know he's knowledgable and capable, I also know he's a self-starter who's not going to wait around and whine and bitch that no one is giving him an opportunity. The opportunities are out there, you can't wait for someone to give them to you... you have to go take them.

Seriously, go *do* something. It'll take your mind off being out of a job and if you do it right it will be the thing that gets you a job. It's win-win.

Isn't this called...

...the mythical man-month?

>why should companies favor hiring fewer more senior developers rather than many junior ones?

*swish*

Uber Programmers Don't Exist

Domain knowledge is the primary difference between a 1 day LOE and a 1 week LOE, not programming "skill".

There is no class of general "uber" programmer that can be brought on to an arbitrary company's internal development project and hit the ground running at a pace 10 or even 2 times that of the standard-fare developers already on the project. This is a complete myth.

However, the domain knowledge gap can in most cases be narrowed very cost effectively through knowledge transfer, training, and tools.

If you skimp on resourcing and experience anywhere in your development organization, it should be on programmers. Inexperienced and unskilled programmers can be compensated for effectively through targeted specification, management, and quality assurance processes. The key is to have processes in place to identify and rectify programmer failure early and often.

Computer programming isn't rocket science, it's bridge building. You have planners and you have builders. Builders pour cement and put rivets in place, and there are processes in place to identify, rectify, and robustly handle individual builder error. Bridges do not arbitrarily drop cars off into the river below due to individual builder error, and neither should software programs crash due to individual programmer error.

Re:Uber Programmers Don't Exist

Computer programming isn't rocket science, it's bridge building. You have planners and you have builders. Builders pour cement and put rivets in place, and there are processes in place to identify, rectify, and robustly handle individual builder error. Bridges do not arbitrarily drop cars off into the river below due to individual builder error, and neither should software programs crash due to individual programmer error.

When you separate the planners from the builders like that, you get the Twin Cities bridge that was a load of under-engineered shit because planners built it to look good on paper, and the builders made it work, but overall it was a disaster waiting to happen.

I mean honestly, you shouldn't have overall architects who haven't actually written code before. They will absolutely fudge the software requirements because they really don't know what they need to get the job done. Likewise you can't get an infinite number of stupid programmers to implement a perfect specification because it takes too long and is too error prone.

Ultimately the question is where do you get your perfect software architects, and why can't you just get programmers from the same place? Without good programmers, how do you know that your software architects aren't full of shit?

Re:Uber Programmers Don't Exist

There is no class of general "uber" programmer that can be brought on to an arbitrary company's internal development project and hit the ground running at a pace 10 or even 2 times that of the standard-fare developers already on the project.

Possibly not on day one. But definitely a few weeks afterwards (ie: domain knowledge). Experienced programmers tend to see subtle things that most people gloss over. Even stupid little things. Those things make one developer more productive than a whole department team combined (especially if those other programmers were primarily hired by HR).

Most projects at most corps have 1 (or maybe 2) developers doing about 90% of the work, and a dozen or so folks on the payroll for the project.

Yeah, right

I'm going to take advice on hiring programmers from a Perl cool-aid drinker. Sure, just the very minute I get my brain replaced with a cauliflower. Perl is an horrifically bad language. It's called "write-only" for a reason. It makes great programmers produce merely adequate code, makes good programmers produce bad code, and makes bad programmers think they're great. Feh. A properly trained, incentivized and provisioned Java team can run rings around a Perl team in terms of working code produced, as well as (more importantly) cost to develop and cost to maintain.

Re:Yeah, right

A properly trained, incentivized and provisioned Java team can run rings around a Perl team in terms of working code produced,

If measured in terms of number of lines of code written, absolutely ;)

Re:Yeah, right

A properly trained, incentivized and provisioned Java team can run rings around a Perl team in terms of working code produced, as well as (more importantly) cost to develop and cost to maintain. and a python team can do it in half the time, and half the code

HR ...

... is still looking for a senior programmer with 15 years of .NET experience.

Mastering a whole language ecosystem takes months

You don't need to hire an expert in language X, you can and should look for expert programmers that are willing to learn language X. An expert can easily cross over from being a novice in any language in a matter of a few weeks.

I read this a lot, but it's misleading, and raises wrong expectations. While learning a new syntax and grammar can be done in hours, it doesn't buy you much. To get to that 10x productivity level on a real-world project, you have to master the whole ecosystem surrounding a language - standard libraries, open-source libraries, tools, idiomatic use, patterns, conventions, best practices, common architectures etc.

As an example, coming from Java, if I switch to Ruby, how long before my code truly follows "the ruby way"? How long before I know the ins and outs of Ruby on Rails and the standard libraries including their gotchas? How long before I can architect a serious ruby application that makes good use of its meta programming facilities, instead of one that looks as if it was ported from Java?

Or, as another example, if you switch from Ruby to Java, let's say on a web project: How long before you can make a informed choice which web framework to pick? How long before you know the architecture implications of picking Hibernate, and when iBatis would be a better match? To know what Spring can do for you, and what you are giving up by not using it? Until you know even a standard set of tools like Eclipse plus which plugins to use, FindBugs, Ant, Cruise Control, Emma, ..., plus another dozen or more libraries typically used even on a small Java web project.

Of course, you can be productive even when you don't yet know all these things, and are still learning - but you won't be productive on the expert / 10x level we are talking about. By all means, become an expert in as many languages as you can - but don't plan on getting there in 24 hours, days or even weeks.

(Disclaimer: Switching between fairly similar environments, e.g. Java C#, is of course much easier).

Read Joel's book

Smart and Gets Things Done.

I'm a little dismissive of the mystique around the required "super hackers" that never need to look for work but there is a ton of great advice on just hiring people.

I'm an engineer. Been there and done a lot of that. I'm not going to say I'm one of the greatest but not too shabby, I've built some stuff and made some good money, you, know left a few marks.. As a more senior guy I've been taking on more and more leadership and to be completely honest, I like to think there are things I know to look for and catch, but I suck at hiring and team building. At the end of the day it's about building products, selling them and making money and the balance between people you think are a good personality match vs. the people that are technically good enough vs. people that are actually motivated and want to work and be successful is hard. We've hired folks we though were good personality matches for the team and turned out to be terrible technically and completely unmotivated, as much as you might want to like them, you simply won't when they are trying to play big business "CYA" games and not actually contributing to the team.

I'm kind of dealing with a situation now, we're a small team, 4 or 5 developers and 2 testers. We hired the 5th developer based largely upon a recommendation from one of the testers. He's a marginal tester to be honest, good guy, just not super motivated, why we hired his recommendation is looking more and more stupid by the day, we value recommendations. After reading Joel's book I've found like 6 or 7 indicators that probably would have flagged this guy that we simply didn't think about. We were in a hurry, we thought the req would go away, etc.. Honestly, I'd rather have one fewer people and better morale that this guy, seriously in 6-8 months of having him, I cannot point to a single substantial contribution. Now we get to go through the process of firing him which sucks for every one involved also.

Basically, you always want smart people, you want motivated people, people that do a good job, people that have some passion, good communicators, strong team people that know what it is to be on a team, you want all of that stuff, all of the time and it's hard to find. We pretend that parts don't matter or don't matter as much. Having shitty people on your team just flat out sucks, doesn't matter how good everything else is.

Unfortunate reality

On the flip side, why is there such an excess of not-so-great programmers out there? The answer is simple: The higher education system is turning out not-so-great graduates. In an ideal world they would not, but we live in a world where there are "CS Graduates" who have never seen anything more than pseudo-code and java. There are some great programs and great graduates to be found for sure, but I think the writing on the wall is apparent -- the average graduate is a below-average programmer. There needs to be more hands-on exposure to real, complex code, or better yet, production code.

In the interim, unfortunately, we realistically need to take in some of the graduates that we have and finish the education they apparantly never received in full. If we don't -- if we let a bubble form between now and when the educational system corrects itself, then we will effectively lose much of the "tribal knowledge" that is passed down through the generations of the workforce.

You cannot sustain a class of experts in any endeavor simply by surrounding them with other experts. At some point they must mentor or pass down their knowledge to the next generation -- but the best way to ensure the next generation is to make sure that they're at least on-par as a developer.

I say all this as a relatively young developer who graduated in Computer Science in 2002.

Education is only a foundation. You must love it.

I've been a software engineering manager for a long time. I don't blame the higher education system. Nearly all CS programs cover enough of the basics to form a foundation for lifetime learning in CS. The rest is up to the student and their own innate passion. If they have a passion for the technology and chose CS as a labor of love, then they'll do fine. There have been many graduates of CS programs who declared CS majors when counseled to 'get into computers' by a high school guidance counselor. I always look for the passion players when I hire people. I avoid the people who chose CS as a 'sensible career in computers'. I've seen some passionate lovers of CS that come from tiny state universities run circles around graduates of Stanford, MIT, and Berkely.

Design by Management

I think the biggest stumbling block for above-average developers is the attitude of managers who indeed think of programmers as 'cogs in the machine'. And a sausage-machine at that. I have seen too many projects without a serious design phase at all. Typically they start with a team of managers and tutti quanti dreaming up a specification (usually to vague to be useful, but still specific enough to be unworkable) and then expecting that the software will simply be written.

When you try to explain to them that actually, there is nobody in the IT department who is actually capable of designing and building such a system of the required complexity, you meet blank stares and incredulity. How is that possible, when that place is filled with programmers? Trying to explain the difference between routine GUI programming, systems administration, database administration, and designing a company-wide system of interlinked applications is no use. Managers would apparently trust any competent car mechanic to design a Formula-1 car. Well, they must, because they are not willing to pay for the skilled engineers that it takes to do the real job, or to invest in serious training for the people that they have.

Of course, if they don't have the people, managers are willing to contract them. So they pay to bring in programmers from consultancy firms, and expect them to hit the ground running, seamlessly integrating in teams with people they have never met before, and writing business logic for a business they don't understand. You would need to be really lucky to find all the right people right away, more often you need to get rid of about half the consultants you hired at first, because they are no good or because they experience is too far removed from the needs of the projects.

So far, I've met just one or two really highly skilled and creative developers. Flexible thinkers who quickly grasp problems, readily adopt to whatever technology they need, plan realistically, and deliver quality work. They are a delight to work with, and you can achieve as much with them in one hour as you would else do in whole month of meetings and deliberations. But they are usually self-employed or running their own little firms, and they can afford to pick and choose the projects they are interested in. If they don't want to do it, or don't have the time, they won't. Management regards often them as 'difficult' and expensive, considering that other programmers are a dime a dozen... So the pleasure is rare.

The normal condition is to have a mix of more or less skilled people who will learn on the job and might be quite competent at the end of the project, and inevitably a few idiots you have to entrust parts of the project to with appropriate feelings of resignation and foreboding. (When in IT waters, I live on my wits. I have no authority there.)

Testing....

I work for a small game developer. We recently announced we are developing a pretty big title, and some fanboys of the game said that we'd make a lousy product because our Programmer Job description on our job page reads like this :

If programming games is your dream, we're the place for you. Please send in examples of your work, no matter how trivial. Websites are acceptable as well. You can't always work in one language long enough to know the syntax by heart. But the concepts are reusable, that is what we look for. C/C++ knowledge is an advantage but not necessary.

The comment went along the lines of, how can a company make any good games if they hire just anyone off the street? Well, we do. But everyone has to pass a test. We basically hand them a copy of the engine and give them the instruction of. "Complete it" It's their job to read the code, figure out what it does, and add their own code to extend functionality. The test basically tests their ability to grok it. To get it to compile. Then to extend the code while following proper standards and naming conventions( As long as you follow the style of the rest of the code, we're happy ). Finally, their creativity. We don't tell them how to complete it. So some people do the bare minimum which shows competence, and some go all out. But usually you can tell what they like. Some are AI heavy, some do a lot with player mechanics, and some start extending the engine when they want to do more (Warning : Not Invented Here Syndrome Probably Present).

So we've got some older guys from the VIC-20 days, some young college grads and some non-traditionally trained programmers. I'm a tools and build process guy myself. Engine is another set of people that hate muddling with gameplay because you can only spec so much, the rest is up to...testing and feel, and they hate repetitive work like that. The gameplay guys that love to noodle and pull magic numbers out of the air and test until it feels "just right".

We also do products in staggerred teams. Several experienced developers, several inexperienced. Let the experienced funnel the knowledge down. Rotating R&D cycles, so everyones fingers is in the engine at least a little bit. Really experienced outside developers have their issues sometimes as well. They are very set in their ways, and it's hard to mold them to your system. Which is why Microsoft prefers hiring physics and math majors. They haven't learned any bad habits yet. Sure when it comes to crunch time, we do neglect the juniors, but that is when the smart one's start to shine.

Oh god, they never get this right.

For 100's of years you have the senior crafts man and his apprentices.
You can't get good quality with just Senior crafts man, or just junior apprentice types.

The Junior ones just don't know how to make the trade offs or the how and why things are done a certain way and end up painting then selves into a corner or making a mess for the next guy to deal with. But they are young and full of energy.

The Senior ones, just don't have the patients or excitement about over all of the stupid little details. They just can't get excited about doing the same thing over and over. But they think ahead for the next guy, they know how to avoid problems and usually know how to fix problem when they arise.
Also they have usually have a long list of other senior developers they can call on for help and advice.
Often on a really difficult problem the phone and E-mail are your best tools.

As my former partner the infamous Jesus Monroy used to say, on a boat you need rower and captains. Too many of either doesn't work.

As a senior developer, I find I am best at working the really difficult problems, but lack the patients for the more mundane bulk coding.
Also like doing architecture work.
But thing work best when there a Junior programmer that will get stuck on a problem and usually hide in the cube for weeks trying to solve it. Where when I am around I usually can take one look and tell then exactly how to fix it. As a result they tend to get 10x or more work done when I am around.

Also junior programmer usually just start writing code when giving a project with little consideration on design. The end result tends to be large, slow and almost impossible to debug.
As someone experienced, I find that laying out the design, the foundation, if you will that everything else is to be built of from is critical.
Once designed correctly, the code is much smaller, simpler, is easy to work on and debug. It also less code means it runs faster, loads faster and uses less resources.
Also less code, means it faster to implement. So I'll spend more then 1/2 of the development time on research and testing, and design, before I ever type the first line of code in. But in the end, I get done faster and almost never have any logical bugs, memory leaks and have never needed a debugger, just type-o's as mostly, of only I could spell...

I hate nothing worse the Bloated code.

Re:Oh god, they never get this right.

This is *exactly* what I think is best. Let the experienced hands create the design and let them solve the difficult stuff. Then have the juniors "fill in the blank" and help them along the way. Occasionally have them participate in design tasks to give them some experience and a little more of the "why" part about the design - this will reduce the number of annoying questions tremendously afterwards.

As you can probably tell, this is what I do - design and difficult stuff. I really enjoy that. Others do GUI, build process, configuration, and what-have-you. Some of them like their work because they get to go home and not think about their work until next morning. They will stay "junior" forever, but are perfectly happy with that. They don't have a passion for their work, so they should never ever be allowed to do the design or solve the difficult problems, and most of the time they don't even want to.

Now management, that's another issue. They simply fail to understand that there can easily be a factor of 5 or 10 difference in the total time it takes for any two of us to complete a certain task with any reasonable measure of quality. This works both ways - I end up spending more time doing GUI work than those who like that work, simply because it is a kind of work that does not motivate me. Likewise, solving that OS and transport-independent highspeed failover service protocol was my work, because that is what I do 10 times faster and better.

the real question

Who's hiring these recruiters? I may have all the qualities of an expert developer, but I'll never in a million years get even a sniff if I don't have all the checkboxes in order. Recruiters don't care if your resume screams out that you can be idiomatic in a new language or system within a few weeks. They'd far prefer you have a ten year history of making the same pinhead mistakes over and over. The attitudes of recruiters reflect the desires of the company, whether they're implicit or explicit. Companies that have trouble finding expert programmers are just lazy.

It's the hiring process

I think the real problem is that people don't know how to interview to find talented programmers. The best predictor of future performance is past behavior. 90% of an interview should be about past projects or academic work. Instead many people seem to have this weird notion that asking how many socks you need to pull from a drawer to get a matching pair gives insight into an engineer's talent.

Re:My own experience...

Instead of only hiring cs people with degrees in engineering and cs, broaden your horizons. You may be pleasantly surprised at what someone who has learned programming from a different background can bring to the table.