Years ago I was getting to know an attorney who was working for my boss. His list of accomplishments was striking. He had a long history of helping transact remarkable deals and negotiating resolutions to delicate situations. “I was the main attorney for the Sun acquisition of MySQL,” he remarked at one point. Truly impressive. That deal, a $1B acquisition of an open-source software company, is still the largest such in history as far as I know, and is sensational in several ways.
Later I needed an attorney for my own company and a friend introduced me to someone who had the particular qualifications I needed. Because this was a referral through a similar network, this person had also spent time in the MySQL business community. At some point we were discussing the circles we had in common, and I mentioned the first attorney and his key role in the Sun/MySQL transaction. She laughed dryly. “I was at the table for every single meeting and discussion involved in that deal,” she told me. “That lawyer was certainly not central to the deal.” She went on to name a team of several people she considered to have really been responsible for it.
Since then I’ve met at least a couple other people who believe they were the main or most important attorneys involved in the Sun/MySQL transaction.
Bias. Conscious or unconscious, deliberate or accidental, all of these people are responding to their own perceptions of themselves and others. These biases are very common and most people aren’t going to notice and moderate themselves. That’s why it’s up to you, and there’s nowhere this is more important than in interviewing a job candidate.
I’ve interviewed a lot of candidates who claim to have done something notable in their past roles. Often these accomplishments are well-known to the public, like the Sun/MySQL, so the candidates emphasize those achievements above others during the interview process. However, on deeper examination, I have often concluded that the candidate didn’t deserve as much credit as they thought, due to these biases.
It’s important to ferret out the truth, because past performance is one of the best predictors of future success. I’ve found that most of the time the achievement they’re claiming was due less to them as an individual, and more to other factors:
- The team around them. Other people were involved and enabled the success in ways they’re either deliberately minimizing, or forgetting, or failing to understand.
- The environment. Presidents always take credit for improvements in the economy, and always blame recessions on factors outside their control. In many cases the advances are just as serendipitous as the declines. Many job candidates likewise fail to realize or mention that they were just riding the coattails of a favorable situation.
- And finally, sometimes there just wasn’t as much of a success as they claim, or the achievement is more nuanced. “I grew the sales team’s lead flow by 300%,” they say. Talk to the sales team and you might hear that this was true, but the leads were utter garbage, which is worse than no leads because the sales team wasted their time on the bad leads.
Your job is to get to the truth. However, it’s also important to do it in a way that respects the individual, because they may not have ill intent in taking credit for these achievements. Your role as an interviewer is not to judge the person, but to build an accurate picture of their abilities and determine whether they are likely to do well in the role for which you’re interviewing.
Here’s the trick: to do this you have to walk a delicate line indeed. It doesn’t matter whether the candidate is mistaken or lying; what matters is what they really achieved. You have to find this out without implying ill motives, offending, or waking sleeping dogs. (No good can come of revealing misconduct to a past employer or letting a candidate know that you’ve uncovered their lies; everyone has the right to a fresh start if that’s what they’ve chosen.)
How do you do this? I’ve found the best way is to try to ask the minimal set of questions that would indicate, directly or directly, one of the following:
- Direct contradiction to the claim. For example, if I were interviewing an attorney who claimed to be the sole driving force in the Sun/MySQL deal, I might ask a question about a particular detail that I think another attorney might have been responsible for instead. If I get a vague or contradictory answer I might have just knocked down the claim.
- Lack of supporting evidence to the claim. For a software developer who claims to have implemented a particular algorithm, for example, I know or can quickly find a number of important details they must have had to deal with. A lot of Computer Science students will tell you they built and programmed robots to play soccer, for example, when they were actually given robots and 99% complete software, and all they did was write an event loop. A couple of quick questions about trigonometry can reveal whether they had to actually create steering systems or just used existing ones.
- Indication of nuances that might have been glossed over. An example question might be, “what was the lead-to-opportunity conversion ratio?” If the candidate claims to have increased lead flow 300% and doesn’t know or won’t say the conversion ratio, or how it changed, there might be more to the story.
Not everything has to be examined during the interview, and not everything can be. Reference checks serve a vital purpose. Many claims will hold water in the interview but won’t be verified by references.
It’s also possible that the candidate really did achieve the results they claim. In this case you still have an additional question to answer: was it conscious and under the candidate’s control, and therefore reproducible? Or was it an act of unconscious expertise or luck? I’ll write more about this in a future post.
Candidates will likely make dozens of performance claims. You don’t have to, and won’t have time to, investigate every single one. You need to investigate the most important ones that get to the heart of the matter and have the most predictive power.
Originally published at Evaluating Candidates’ Performance Claims, Part 1: Who Achieved What?.