Last week one of my friends applied for a job with the NJ Transit Authority. Long story short, he didn’t get the job.

Before I continue – let me tell you a little bit about my friend (I’ll call him Greg; not his real name). Greg is probably one of the most charismatic people I’ve ever met. His energy and enthusiasm are infectious and everyone around him is uplifted by his larger than life personality. He would be the perfect candidate for the transit authority because he would make everyone around him happier. Having him work on a bus or train would meaningfully improve the commute for thousands of people every day who would get to spend time with him. When he originally told me he was applying to work at the transit authority – I thought: that is the absolute perfect job for Greg.

There was only one thing standing between him and the job: the entrance exam. I believe the exam is a written test that covers many parts of the transit job. Here’s the thing – Greg blanked on one of the questions on the test about brake pressure. He’d never driven a train before, so the muscle memory just wasn’t there. When he looked at the words on the page about how to maintain proper brake pressure, he just blanked. Greg had experience driving big rig trucks and knew about brakes, he just didn’t know the answer to the specific question they were asking.

The result: he failed the test and now has to wait six months to re-apply for the job.

Sure, brake pressure is important, but how important is it that he know, hypothetically, how to manage brake pressure before even driving a train?

That’s like teaching someone to drive a car with a manual transmission and upfront, before they even get into the car, requiring them to memorize the exact RPM’s required to shift into each gear. Not only is this information difficult to remember, it’s actually largely irrelevant to performing the task. When you’re behind the wheel and on the road, it doesn’t matter if you have the RPM’s memorized, it’s far more important to feel how the car is responding to your controls and develop the coordination and muscle memory for completing each gear shift.

So now, because Greg got this question wrong, the job at the transit authority is probably going to go to someone who is far less engaging and charismatic. Thousands of people will be robbed of his magnetic personality because the organization relies on an entrance exam – a terrible way to measure true qualification.

Just think about all of the people out there who didn’t get their dream job, even though they would be excellent at it, just because they aren’t any good at taking written tests. There has to be a better way.

Qualification Tests: A Terrible Way to Measure Qualification
Tagged on:         
  • What’s the comment? – The goal of [interviews / qualification test / hiring evaluations] is not to hire the best person. It’s to hire (a) someone who can do the job, and (b) avoid hiring someone bad (who will cost the company: management time, failure to perform, piss of customers.

    I’ll happily agree with you that a lot of tests are *bad* and don’t really filter people out. It’s why I’m against putting requirements (years experience, technologies, etc) in job descriptions, because it introduces a selection bias which may be _unrelated_ to how good the candidate actually would be.

    But really, using tests is to weed out people who may do poorly, not to identify people who will do well. Perhaps most people who miss the same question have worse safety records – of course, we’ll never know. But if missing that question means you’re 20% more likely to damage a train – still a very low probability, i.e. from 5% to 6% – should the business be able to use that information, based on their cost estimate of average damages?