Hiring in product is broken, and those hypothetical product questions you get asked in interviews are just the tip of the iceberg.
“Describe a coffee cup.”
No, you’re not in the middle of a therapy session, you’re in a product manager interview, and a bad one at that.
If you can, you should get up and walk out, because your interviewer’s bias is about to count against you.
Yes, bias.
These types of questions are great if you look, sound, and have “the resume” the interviewer dreams about when they put up the job posting (none of these things are explicit. It would get them sued).
But because they can’t be explicit about what they want (anti-discrimination legislation and all that) here you are, describing a coffee cup, and unintentionally running directly into a process that is full of bias.
Sorry to break it to you, but if you don’t look like you fit on the company’s “about us” page, you’re in a world of trouble.
How did we get here?
Enter the product hypothetical – for a long time, companies have been using interview questions like “how many matches could you fit inside of an airplane” or “how many gas stations are in the country” as a way to understand how someone thinks.
But do these questions really help you to understand how product people think? I posit that they don’t. Instead, they land us in a world where bias wins and rational thinking loses.
What do I mean by that? Let’s ask the world of behavioral science.
Behavioral Science isn’t new
Behavioral science teaches us about our mind’s proclivity to lean towards bias, specifically cognitive bias.
In fact, we’re cognitive bias machines. I learned this while reading about behavioral science from two of its foremost experts, Daniel Kahneman and Amos Tversky. The pair spent their lives trying to understand the WHY behind our behavior, and for their efforts, they became two of the world’s most cited academics.
Kahneman’s seminal work, Thinking Fast and Slow, talks about our system 1 – intuition, and system 2 – rational thought. Most importantly the book shows us that our intuition is a hotbed of cognitive bias, one that can be controlled by almost anything – from how much we’ve eaten in the morning to the last argument we had with our spouse. Do you want your choice of breakfast to determine the next person on your team? Relying on intuition means it could play an important role.
Back to bias
The Kahneman and Tversky research points to system 1 as being susceptible to bias.
Most product hypotheticals have no guideline, no rubric, and rely on the “intuition” of the person giving the hypothetical to determine what happens next.
Ask yourself, when you’ve seen a product hypothetical with clear criteria? And by this, I mean a situation in which the criteria have been clear for both the interviewer and the person being interviewed.
While the person on the other side might be trying their hardest to work out a complex system, the interviewer is just listening, usually passively. With no rubric, no grading structure – they end up at the whim of their system 1 thinking.
What happens as a result?
Unfortunately, in this situation, the interviewer is more likely to play heuristic bingo (a term I’ve created that describes when we let our bias control our decisions).
Heuristic bingo is a good thing when deciding where you want to eat for lunch, but when it comes to hiring, it leads to disastrous consequences. The interview process becomes less about skills, experience and know-how and more about what we expect people to look and sound like.
For folks like me, who aren’t white from privilege, it means you end up a casualty of the process, no matter what your skills are. This plays itself out during the hypothetical section of interviews where the interviewer relies more on whether things ‘feel’ correct rather than if they are interesting or match criteria.
If that person says something novel, it is discounted. This is particularly insidious in product management because most of the work is contextual, and a best practice at Facebook may (actually, probably) mean absolutely nothing at your company.
How do we avoid bias?
Let’s shift to system 2, our rational thought, which is less susceptible to cognitive bias.
One of the easy ways to get into system 2 thinking is to have a long conversation about someone’s life. The good news is that interviews already have a vehicle for such things – it’s our experience.
People have no problem telling stories about how they did something. The important thing to remember is that you’ll need to really dig in, as an interviewer, to know what “something” you are looking for. Your questions are important.
This is no different from trying to solve a problem for a product you own. This is the time to ask questions and get context around how this person’s experience helps your company to solve that problem.
It’s time to talk to people more
Our brains are built for stories, and in hiring, we need to take full advantage of that. So instead of asking people to create, just ask them to recall.
Talking to people and asking them to recall will help you to uncover their stories, through which you can learn about what they’ve done.
Their stories will be filled with details and these details are enough to see if someone is being honest. They also give you room to ask follow-up questions to get to the bottom of what you need. Write them down so you can check them against your criteria and understand what you’re looking for.
For extra points, recording and rewatching the conversation (especially if someone is further along in the hiring process) can give you, the interviewer, more insight into this person’s experiences. It also gives your ‘system 2’ enough time to think about the conversation.
Hiring is broken
Let’s be clear — hiring in product is broken. In fact, many tech companies are failing outright when it comes to hiring equitably, despite the statements many have made in response to recent police brutality. These statements have been highlighted in a database published by The Plug, in response to which, The Plug founder, Sherrell Dorsey, has reported that at the majority of companies in the database so far, less than 4% of employees are black.
Talking about product hypotheticals is, therefore, only the tip of the iceberg, as there are systemic injustices that lead to inequality everywhere, as recent events clearly illustrate. If we’re to make a better world, we need to work for it.
With that said, getting rid of hypotheticals now can give you the space to focus on a more equitable hiring system by listening more to candidates and asking them about the problems that they are dealing with.
That is how we solve problems.