If you haven't read Jeff Atwood's Why Can't Programmers.. Program? go ahead, then come back.
Now, are you scared enough? Don't be, the problem there is with the hiring process.
Yes, there are lots of people who show up for programming positions and can't program. That's not unusual!
Suppose you are a company that tries to hire in the top 1% of programmers, and have an open position.
You get 100 applicants. Of those, 99 can't program. 1 can. You hire him.
Then the company next door needs to do the same thing. They may get 100 applicant. 99 can't program ... and probably 80 of them are the same the previous company rejected before!
So no, hiring the best 1 out of 100 is not a way to get a programmer in the top 1% at all, that's just statistics intuition getting the worse of you.
You don't want to hire in the top 1% of applicants, you want to hire in the top 1% of programmers. Different universes.
These two things are the two sides of the same coin. 99% of applicants are useless, that's why they are applicants, because they can't get a job and they can't get a job because they are useless as programmers.
So, judging programmers by the standard of the applicants you get is like judging quality of a restaurant by licking its dumpster.
But now, having taken care of this, how do you find a programmer that can actually program?
Easy! Find one that has programs he can show you!
I would never hire a programmer that can't show me code. There must be something wrong with him, because programmers write programs.
That's just what we do. If we didn't what kind of programmers would we be?
Let's see some obvious objections to my argument:
He wrote code for his previous employer and can't show it.
So, he did. What else has he written? Some open source code? Maybe snippets in a blog? Answers in stackoverflow?
Nothing? He has written nothing he was not paid to write? He is not who I want. He only programs for money, he lacks passion for programming, he doesn't enjoy it. He is probably not very good at it.
He is just finishing college, he has not written much code yet!
Why? What stopped him? He has been learning to program for years, what has he done with the knowledge he has been receiving? Saving it for his 25th brthday party? He has not practiced his craft? Not the programmer I need.
But having him show you code is not enough, of course. It also has to be good code, if you are serious about hiring excellent programmers.
So here's some bonus criteria:
Check the languages he uses. If he codes COBOL for pleasure, he may or may not be what you want.
Open source == bonus points: it means he is not ashamed of his code, plus it makes his credentials trivial to verify.
If he leads a project with multiple contributors and does a good job he is half way to becoming a programmer/manager, so huge bonus points.
Projects with long commit histories show responsability and a level head.
Development mailing lists let you gauge his personality. Is he abrasive? Is he thin-skinned? Is he annoying?
Then there's the obvious stuff, references from previous employers, interviews, exercises, an such. But those are the least important filters, the most important thing is that he must be able to code. And showing you his code is the way to do it.