An interview with Adam Thomas. Adam is a passionate product leader & product coach who wants to help you drive organisational alignment. By day he’s Lead Product Manager for a recruiting platform, and by night he’s the hero that Gotham needs with product consultancy Approaching One.
We speak about a lot, including:
His work with Approaching One and how he’s trying to help product managers & product leaders get better
How he started out as a mainframe programmer, and ended up falling into product management when a mentor realised how unhappy he was
The story of his two startups, whether they succeeded or failed, and some of the lessons he learned from the experience
His journey from individual contributor to leadership, the resources he used and how he mixed mentorship with repeated mistakes to get good
The importance of driving organisational alignment, the types of negative & positive feedback you can get due to misalignment
Why alignment is the product manager‘s job, how you should never assume anything, and have to do the work
Some of the warning signs of misalignment, techniques you can use to get back on track and why you should trust but verify
The importance of having a compelling story around your product that you can align your team around
Survival Metrics – what they are & how you can use them to decide whether to pivot, double down or give up on an initiative
And much more!
More on Survival Metrics
Why not visit the website to find out more about Survival Metrics?
How does the level of trust after a decision is made impact product teams (and other teams in your organization too)? How much waste might be happening in your organization because of cognitive biases and what are practical ways to avoid falling into the traps of these biases when it comes to decision making?
We explored these questions and many more with Adam Thomas, a product person and technologist based in Harlem, NY who is focused on strategy, team organization, and product management. Adam is currently the Lead Product Manager over at SmartRecruiters, a columnist at BuiltIn, and also shares weekly product management tips and other fun resources in his newsletter.
Adam believes that product management isn’t just about getting done what’s on the roadmap; it’s fundamentally about people. So he looks at things in a way that leverages behavioral science, psychology, philosophy, and more to build extraordinary teams that thoughtfully bake culture into what they are developing.
In this episode of the In Trust podcast, we talk to Adam about how he helps product teams get better results by putting people first. His practical perspectives and astute observations apply far beyond product management. Whether you’re a product person or a leader in a different domain, you’ll want to give this podcast a listen and take notes.
Overview of Episode 39: Helping Product Teams Get Better Results By Putting People First with Adam Thomas
Adam’s posture and philosophy as a technologist and product person
Why Adam rejects the box-checking, order-placing view of product management
Adam explains something he calls product calculus
Unpacking how trust comes into play specifically after decisions are made
Why there has to be a trust in failure
Why leaders can better serve their entire teams and their work by looking at work and culture through a lens of cultural anthropology
Adam’s practical tips for leaders to study the culture that they are immersed in
How sometimes one little push can change your entire organization
The high tax that bias plays in the workplace and how it is a silent killer of teams, processes, and development
The importance of conducting post-mortems on decision-making to understand how bias might have come into play
How regular pre-mortems can help minimize the bias tax
The importance of having a high cadence for company communications and how this can help bake in accountability into a workplace
The importance of talking about both successes and failures
How to reduce your chances of getting into trouble at work to nil
Starting small and building the muscle to take bigger and bigger risks
The basic thing you can do to have better meetings, more focused conversations, build trust, and other benefits
Where to learn more about Adam and read, watch, and listen to his work
Quotables
“I’d like to get completely away from that, let’s just call it determined roadmap. That preordained idea that we have some understanding as to what the future looks like, and instead move towards a place where we heighten our communication and we heighten our understanding of, not just the infrastructure of the company or sales requests, but also the people inside of that company, in order to live in that ambiguity – live in the space of just the unknown that exists inside of building things and making things. Be honest with ourselves around the concept of what I like to call product calculus, the fact that we’re never reaching something. It’s never a determined place that we’re reaching. We’re always approaching something.” – Adam Thomas
“I’ve seen companies create these huge plans, the leadership team is aligned, they have a big party, everybody goes, ‘This is awesome. We’re gonna really kill it over the next year.’ And then afterwards, six months or a year later, all the competitive advantages they had are gone and they have a product that no one uses no one cares about. A lot of that falls squarely on the product team because they didn’t have the trust inside of the organization post these decisions in order for other folks to trust them enough to carry them out. So I think decision trust is a huge, huge, huge piece of what product folks do.” – Adam Thomas
“There has to be a trust in failure, because I have never seen someone make something right the first time. No matter how much information I’ve sent back and gathered, I’ve never done it right. I could sit back for a year and gather information and talk to customers and do this and do that and I’ve never been right just saying, ‘this is what we’ve made. Ta-dah!’ So there has to be a level of comfort in the failure of what you do because the failure is coming. How folks trust your ability to deal with that and then to take that as a learning lesson, as opposed to playing the shame and blame game, really affects your ability to build stuff. Not just once, but consistently throughout your tenure as a product person.” – Adam Thomas
“As a leader, you should constantly be studying the culture that you’re in.” – Adam Thomas
“One thing to change the world: sometimes it’s just one little push and you can change your entire organization.” – Adam Thomas
“If I were to sit back and look at your operating expenses as a business, and I would suggest shadow your teams for three to six months, I could probably put a percentage on how much waste is happening based on bias. My bet for a lot of companies is that waste is upwards of 20 to 30%…With cognitive bias, since it’s so subtle, it’s such a silent killer of teams, processes, and development. It’s something that goes underneath the radar.” – Adam Thomas
“If you are communicating clearly, and you are being thoughtful, and I can tell that you’re showing your work, the chances of you getting in trouble are practically nil. You would have to do something extremely diabolical and in bad faith. I think most places work that way, especially once you understand what is the level of work that you need to show and who you need to communicate with.” – Adam Thomas
There’s a lot of uncertainty about the future, but one thing we are sure about is that The Future Is Trust. Which also happens to be the title of our forthcoming book.
The Future is Trust: Embracing the Era of Trust-Centered Leadership officially goes on sale on June 15, 2021
We are so excited to bring this reimagination of what a leadership book can be.
I’m going to share a harsh truth here. No, you don’t.
You need an architect, a project manager, or any number of jobs that directly speak to the problem you are facing. Your issue is that you haven’t defined that problem, and are more than likely listening to outside voices that “think” they know what product management is.
In my 13 years in tech, which I’ve spent building startups, consulting, and growing product teams, I’ve never, ever seen the use of a technical PM. My hope after you read this is that you’ll understand why, and as you build your product teams, avoid the trap this represents.
So, let’s start there — what does a technical PM represent?
“You know what we need? A technical PM.”
This is a prime example of what I like to call “conference babble:” listening to talks and workshops at a conference and mixing them all together with prior experience to create the world you want. “Technical PM” as a job description is conference babble of the highest order.
As product has taken another leap forward in consciousness as a discipline, plenty of folks have taken the stage and talked about their trials and tribulations. It’s part of the process. As product people, we are working out the nuances of what we do.
Experienced folks understand the nuance, while non-experienced folks may end up in conference babble land. And that is where the rise of “technical PMs” comes from. Why not take advantage of this increased nuance with “technical” skills, and we’ll have some kind of super engineer. Right?
Who doesn’t need that? With that thought in tow, folks come back and start typing job descriptions. “We’ll have a little bit of this, and a little bit of that, and they will be able to fill all these gaps. We’re unsure what we are asking for here, so let’s ask for everything.”
Another unfortunate second-order effect of product talking about what they have been able to accomplish is people getting the wrong idea of what product managers do.
In the eyes of some, they’ve become magic wands. If I wave a PM in that direction, then everything is fixed. They can “CEO” that function. So, why not throw “CTOing” that function in there as well?
That reminds me of what famed business author Seth Godin often speaks about, and what I’m calling the Cheesecake Factory dilemma.
Cheesecake Factory has a menu as thick as your doorknob. The food is mediocre because they cook everything — there is no focus or specialty.
There are no two ways about it; engineering and product are different disciplines. Both take years of experience to do well. By asking for someone to do both, it starts them down the path to mediocrity.
“Alright, we’ve found our technical PM. Now they can go solve the problem.”
Another tough truth: no, they can’t.
Let me hit the fast forward button on this since I’ve seen it a lot while consulting. Your engineering and product teams will have a cold war over this person. As a result, one of two things will probably happen.
They will turn into a project manager/architect by proxy with nagging commitments
They end up doing “PM” work anyway, title be damned.
No one hires senior technical PMs. There are no directors of technical product. This battle will play itself out, and that third-order culture effect could be devastating. If you are a person with hiring authority, part of your job is to minimize any cold war that can happen in an organization. Internal stressors are a company’s silent killer.
So let’s hit the rewind button.
“What is our problem?”
Hiring is a puzzle, no different from finding product-market fit, alpha, a bug, or whatever paradigm you subscribe to.
Eating at the Cheesecake Factory may mean you leave full, but the culinary experience won’t blow you away. Ask yourself: what are you really looking for?
If it’s someone to get projects running on time, go get a project manager. If it’s someone who understands the innards of a technical system, an architect will do well (plus, they are super technical).
Ask yourself what this person is for. Remember that being sincere about what you need can help your business grow.
“Wait, how will I know this what I need?”
Talk. Ask more questions.
Bring in outside counsel if you have to. There are plenty of product coaches that will help you settle what problem you are trying to solve and how doing so will improve your business.
The wrong hire won’t solve your problems — they will just exacerbate them. Worst-case scenario, you are starting a cold war between your product development teams that will slow down rather than speed up progress.
As product management becomes more popular as a discipline, the more trends — hiring and otherwise — will come and go. Remember that you don’t have to follow every one of them. Stay true to your organization’s needs and your product’s differentiators and you’ll avoid the “shiny new object” pitfall.
In this episode of the Product Science Podcast, we talk with Adam Thomas does to create a disciplined and clear approach to product research.
Adam Thomas is a product manager with 10 years of experience as a “wartime” product person. As Director of Product at Informed, he built a product research and development practice from the ground up, teaching his team solid research methodologies as well as getting buy-in from stakeholders to build things the right way.
In this episode of the Product Science Podcast, we talk about what Adam does to create a disciplined and clear approach to product research.
What was Adam’s experience like founding a startup at the age of 19? How did he transition to FiTech as a mainframe programmer? What were the differences moving into a mainframe architect role, and how did that prepare him for a future in product? How did Adam develop the EdTech startup the Arcade School? What did an advisor say to him that led him to kill the project? What lessons carried over into his role as a product leader today?
What is the difference between working in a product management role in large enterprises versus the traditional Silicon Valley model? Why is it important to understand the politics behind shipping something at these kinds of organizations in order to be effective? How did those experiences influence Adam’s work at Informed building product from the ground up? What were his priorities when he had the ability to set them himself?
How can a Product Requirements Document be useful even if you don’t show them to anybody? Why did Adam make younger product people go through the exercise of writing a PRD? How can you learn to ask the right questions in product? Why is writing out a list of assumptions at the end of the document so important as a starting place for product research? Why does Adam push product leaders to always identify their stakeholders and why they care?
What did product research look like at Informed and what challenge did Adam face in building out a research discipline and practice there? How do you get buy-in for your research? How do you get buy-in for making decisions based on your research? What is a falsifiable hypothesis? Why is it so important to be able to prove your expectations wrong with research?
How do you train people new to product in experimentation? How do you teach people to identify bad research, and understand why it was bad? Why did Adam do a study as soon as he started at Informed, and how did he use it to teach a lesson to his team? How does he think about the political side of research?
Quotes From This Episode
Folks tend to dismiss, in telling the story of whatever they’re trying to build, the point of view of the stakeholders.
I feel like a lot of times when somebody has tried to do research they did it badly. And now their mental model of what research looks like is just a waste of time.
For a usability study…You need to have a falsifiable hypothesis. You need to have clear questions that don’t lead the person. You need to know what you’re ultimately trying to get out of this both politically and also through the person. You need to be very clear with your script.
Transcription
Holly Hester-Reilly:
Hi, and welcome to the Product Science Podcast where we’re helping start-up founders and product leaders build high-growth products, teams, and companies through real conversations with people who have tried it and aren’t afraid to share lessons learned from their failures along the way. I’m your host, Holly Hester-Reilly, founder and CEO of H2R Product Science.
Okay, so this week on the Product Science Podcast, I’m super excited to have a conversation with Adam Thomas. So, Adam, welcome.
Adam Thomas:
Hi, Holly. How are you?
Holly Hester-Reilly:
Well. For anybody who’s listening to this later, we are recording in the middle of the COVID quarantine phase, which we don’t exactly know when this will end. I’m doing as okay as anybody can be doing, kind of. How are you?
Adam Thomas:
I’m in the same place. With this pandemic happening, it’s a blessing to be able to talk to someone else as we’re kind of stuck in our apartments and places. I’m sure the folks listening are in kind of the same place of this is still going on when you hear this. Yeah, thankful to be healthy and I’m hoping that everyone else here listening to this is in the same place.
Holly Hester-Reilly:
Cool. Well, yeah, let’s try to distract everybody from all of that and talk about product and software and building good stuff.
Adam Thomas:
Let’s do it.
Holly Hester-Reilly:
Why don’t we kind of go back a little bit? If you could tell me sort of how did you get started in working in tech? Did you start in product? Did you start in something else? How did you make your way over to product management?
Adam Thomas:
I made the mistake that a lot of young people do and started with my own company. When I was 19, I started building this company called JRLI Media. The product that I produced was called The Gamer Studio. It was a media start-up in around 2007, 2008, and we focused on just everything news kind of at the tail end of when you could make money doing ads in any type of way. That’s kind of where I got started in tech.
Then after learning all types of bad habits after running that for about four years, I ended up in, I like to call, old-school fintech at a company called ETCC where I was an engineer. Mainframe programmer, actually, which I’m not sure how many folks listening to this have had any contact in the world of mainframes. But I spent six years there.
Holly Hester-Reilly:
Whoa, whoa, whoa. I’m not going to let you keep going there. You programmed mainframes? I don’t think I’ve ever talked to somebody who does that or did that. How did you end up doing that?
Adam Thomas:
So I went to an HBCU, historically black college university, and a lot of the financial companies and a lot of big data companies were targeting HBCUs around 2009, 2010, to help hire for their mainframe teams. They were paying well. Somebody proposed me. I came up to New York and viola, that’s what got me here to New York. Then also that’s what got me into mainframes. Best job available.
Holly Hester-Reilly:
Why is it the best job available?
Adam Thomas:
At the time, we’re coming out of a recession. I’m talking to folks that are making fractions of what they were offering just coming out of school. People were taking the first job they could get. It had a good salary. It had a good company. It felt like the culture was good when I went up there. It was something new and exciting, at least for me. Mainframes aren’t necessarily known as new and exciting.
As somebody that was programming, I built my first start-up content management system from scratch. I built that with PHP. It was pretty in-depth with modern technology at the time or what is seen as modern technology. Mainframes is different, it’s new, it’s weird. Something to learn.
Holly Hester-Reilly:
Yeah. Cool. So you were an engineer there and then what happened after that?
Adam Thomas:
Then, after three years of being a mainframe programmer, I ended up switching over to being a mainframe architect, which is pretty much like product, essentially, on the mainframe side. I was managing products. I was doing research. I was building interfaces. The whole software fulfillment lifecycle was up to me. It took me a couple of years of working in product on this side of the fence of distributed computing. There’s mainframe computing and distributive computing. Distributive computing is generally what people are listening to this on, probably, or using their laptops. Coming over to this side of computing to know this is essentially the same job but on mainframe side. That is where I started product.
Holly Hester-Reilly:
Yeah. And were there any people who were called product managers when you did that or was it just mainframe architects?
Adam Thomas:
They would just call it mainframe architects. When I would go to conferences and things like now with the background I have in product management, it’s like, “Oh, this is basically product.” But then, in that world, you’re just a mainframe architect.
Holly Hester-Reilly:
Yeah, yeah. I think there are a lot of people who come up and realize at some point that they’ve actually been doing product but they didn’t know that anybody called it that; weren’t familiar with the term and what not. I mean, we still come across people all the time who are like, “What does that mean? What do you do? What exactly?” It’s a classic question. Where did you go from there?
Adam Thomas:
So from there, I ended up getting founder fever again. I started another company with an old rival, actually, by the name of Anthony Fraser. We started a company called Arcade School which was an ed-tech platform designed to help college students in the world of video game and design move towards full-time jobs and the corporate atmosphere in a more successful way. They had a high burn-out rate, high drop rate from folks that are graduating even from the best schools like Carnegie Mellon. There was a whole lot of money being sunk into the new folks that just weren’t making it so we were trying to bridge that gap to increase the rate of success.
Holly Hester-Reilly:
How did you stumble upon that problem as the problem to solve?
Adam Thomas:
We looked at teachable at first. I think we started with this idea that, “Hey, we’re just going to teach people how to build video games.” Then, after a little bit more research and talking to recruiters and talking to folks that work in the industry, we started to stumble on to what this problem was. There were just a whole bunch of students that were very frustrated. A whole bunch of universities that just weren’t quite sure what to do with the students that matriculated through them. And then, also, with corporations that were just lost in terms of their human capital problem. It’s the first time I ran into the term human capital.
Holly Hester-Reilly:
So how did that go as a business, as a product?
Adam Thomas:
Failed. After about six months bootstrapping, got a few folks on board. Going to fundraise, just didn’t like what I was hearing. I had breakfast with an advisor one day and he was just like eh. He’d worked in the ed-tech space for a while. Was on the board of a pretty popular school here in New York. He was telling me like, “It’s ugly out here right now. We’re losing money and we’re pretty popular.” I don’t want to say the name of this place. But he was saying, “Yeah, we were losing money. We’re pretty popular. I like your vision. I like what you’re trying to do but the financiers aren’t really digging this right now. BCs aren’t really digging what you’re saying. They’re going to try to make you turn into this other thing that’s not really working.” So if you keep hearing, “Save your time and your energy and get out.”
Holly Hester-Reilly:
Must have been hard.
Adam Thomas:
Yeah, yeah. No. I had to have the conversation I think a lot of founders… If you’re in this long enough, you have the conversation with folks that believed in your vision and what you were trying to do. I had to tell them, “I’m sorry. It just didn’t work.” Yeah, it was quite tough. Kind of threw me into a bit of a depression there for a little bit.
Holly Hester-Reilly:
You are not the only founder that I know who has been through that cycle. It’s hard when you get to the point where you didn’t achieve take off. It’s hard to not go through a depression as you’re coming out of that.
Adam Thomas:
Most definitely. It’s funny. If I knew then what I know now, I guess, there would definitely be a lot of things I would do differently. But I think it was a critical part of my life. It shapes the product leader that I am today. I really appreciate it and none of the folks hate me. We still keep in contact with everybody that went with me on that journey. If they don’t think I’m a flaming piece of garbage then how mad can I be with myself, right?
Holly Hester-Reilly:
I think that’s a great way to look at it. So interestingly to me, your story, the timing was very similar. I also got into product by way of being part of co-founding a company and doing that first. It was founded in 2007 so same time frame. I remember well what it was like being in the New York start-up scene in that time. I was in New York already. That didn’t survive, either. We went through the same cycle of, “Okay, what do we do with this?” I don’t know about you but to this day, I have friends of mine that were friends at that time that I feel like they still think that it was a bad decision for me to have founded a start-up instead of going and getting a job. I’m like, “But I wouldn’t be able to do what I do now.” This is a big part of how I became the product person that I am because I’ve lived through the things you can do right and the things that go wrong. They’re not joking when they tell the statistics on failure.
Adam Thomas:
Yeah. Very much so. The likelihood that all the people you confide in, all the founders that you talk to, there’s a high rate of failure. You’ll all look back now, a couple years later, and go, “All of our start-ups are gone.” It was just a thing we did but it’s a thing that binds us, people that have gone through that failure.
Holly Hester-Reilly:
Yeah. So after this second round of the founding bug, what happened next?
Adam Thomas:
I ended up getting hired at a company called Philosophy, which, Chris Butler who’s also been a guest on this podcast was the person that hired me there as a product strategist. From there, I spent a lot of time working on a whole bunch of different projects. The great thing about consulting at Philosophy was just the diversity of things you got to work on. It was the first time I had dealt with things like AR and VR. It was the first time I dealt with mobile apps. I had a little bit of time at DTC working on AI so I came to there with a little bit of knowledge around AI and the process of actually turning something that is using AI into something that’s product-ready. But this was the first time doing it on this side of computing. So very cool to work on chatbots. Very cool to work on AI design systems for these big companies that I would have never guessed to work for.
Holly Hester-Reilly:
Tell me a little bit about when you’re working on that kind of project, you’re often doing product in a different environment than the Silicon Valley product manager because you’re working within enterprises and big old companies. What was that like?
Adam Thomas:
Coming from someone that started their own thing and kind of worked, it was frustrating. I did have experience at DTC and some other huge enterprise companies, so my experience there helped me navigate it a bit more and understand the politics that are behind getting things out the door of those places. Some of the softer skills. Some of the things you probably don’t have to do thinking about Silicon Valley-based start-ups where the org chart is very convoluted and it’s a lot of dinners, a lot of coffees, a lot of conversations that don’t seem like you have to have but you do in order to get something moving.
I think on the flip side though, it teaches you about patience in terms of getting things out the door. It also teaches you politics which, I know this is something I think we’re going to touch on later, but I think the politics of getting products moving, they’re everywhere. Learning how to do it starting in finance and then also working in these big companies that I worked at when I was working with Philosophy was almost a master-class in learning how to navigate things to get things done.
Holly Hester-Reilly:
Yeah, for sure. I think you’re right. We probably will touch on that a bit later. I know that you’ve got another step to your journey after Philosophy, right?
Adam Thomas:
Yes.
Holly Hester-Reilly:
So tell me what else is in your portfolio of experiences that we might use?
Adam Thomas:
Gotcha. So I have a very short start-up that shall not be named. Then the last major step in my career was being director of product at Informed. Informed, where I was at most recently, the last few years, is a company that helps third-party sellers on marketplaces such as Amazon, eBay, and Walmart, price their items to get better revenue through search results.
Holly Hester-Reilly:
Cool. And so there you got to be presumably in a smaller team than those big enterprise projects?
Adam Thomas:
Yes, yes. A lot smaller team. Essentially, I built product from the ground up there. Basically, I got to use everything that’s I’ve used in the past in order to craft the culture and get the product out of this mature state it was in and into something that’s new.
Holly Hester-Reilly:
Cool. How old was the company when you started?
Adam Thomas:
They would say seven years. Let’s just say seven years.
Holly Hester-Reilly:
Yeah. I know. Sometimes founders count funny.
Adam Thomas:
Yes.
Holly Hester-Reilly:
That’s one of the things that I wanted to talk with you about too because I know that while you were there, you really got to take the things you learned throughout this career of early-stage start-ups and products within the enterprise and working with people like Chris Butler to a company where you could really apply it and help build out their practice. What were some of the things that you got to do there that maybe had been really hard to do in some of the previous jobs?
Adam Thomas:
I think one of the things that was really cool when I got there was the ability to just pick things to do; pick processes, pick tools, pick anything. Since it was a small company but it was profitable, I had a pretty sizable budget compared to sometimes when you’re consulting you might be working with a company with billions or super billions of dollars. But they’ll be like, “Hey, you got $200 to figure this out.” Here, I had a pretty sizable budget to work things out so it was nice to be able to build a research practice.
Also in terms of running the product process as a whole was really cool to be able to take some of the ideas that I’d been learning and figuring out in theory and then apply them with folks that haven’t been… Like at Philosophy, I was probably the least experienced product person there. Here I was the most experienced product person. So all of a sudden I have all these young, associate-level folks and I got a chance to apply some of the theory that I have.
Holly Hester-Reilly:
Yeah, so tell me some more about that. What are some of the theories that you applied and how did you apply them?
Adam Thomas:
One of the things that I got to do, which is something that I’m super proud of, is I got a chance to manipulate the PRD structure and change it into something that’s a little bit more useful to folks outside of the product team. I feel like that’s what everybody does. Everybody has their own version of PRD that they like to play around with. I don’t know if this is weird to folks but I also love, on regular PRD, I think it’s a great document which probably has some contention with folks. It’s not something to show anybody.
Holly Hester-Reilly:
Picking up on that, if you see a PRD as a useful document but not to show anybody, what is it for in your experience or your usage?
Adam Thomas:
I think a PRD is a great document to get your ideas down. It asks a lot of questions that if you’re not trying to be rigorous you can dip, dodge, and duck around to get through. But forcing especially younger product people to go through that process. I often got a lot of, “Why are we doing this? We never had to do this before. Why are we writing down these metrics? Why are we doing?” But having them have better questions to ask the teams around them as they were building out products and then also having them more confident in the work and the research that they were doing combined with the idea of what’s on the ground, I know it’s because of forcing them to write these things.
Holly Hester-Reilly:
Mm-hmm (affirmative) Yeah. So making them think critically about why they’re building this product and what metrics it’s supposed to achieve. What to measure to know if it’s successful.
Adam Thomas:
Right.
Holly Hester-Reilly:
Is that what I’m hearing?
Adam Thomas:
That is what you’re hearing.
Holly Hester-Reilly:
So one of the things that I think is interesting is that I feel like the word PRD, the acronym. The word is product requirements document. The acronym is PRD. Fix my own language here. But I think the phrase, the acronym, polarizes people because similar to how there are other phrases in product that polarize people like, “Is the product manager the mini-CEO?” Some people will be like, “Yes,” and other people will be like, “No.” I think the idea of the PRD is one of these things. And in many cases, I think that comes from each of us having our own assumptions about what a PRD is, which come from our own backgrounds instead of experiences of what it was like in whatever companies we were in that used it. Or if we’ve ever been in a company that’s used it.
The more people you work with who are younger, the more likely you are to come across people who have never come across one. So I think it’s really important to share what you mean by that. And I’m curious if you could tell us, since you mentioned sort of reimagining it, what are some of the things that you had people think through and answer as part of the process of thinking through what the product requirements document would include that was helpful for these young products people that you were guiding?
Adam Thomas:
It’s kind of touching because I was just thinking about a young researcher that just built her portfolio and she was like, “Adam, I did your version of the PRD and I put it in my portfolio because it really helped me.” It came right to mind. What are some of the things that I think are required there that was a little different? I’m really big on story so having those folks having to create somewhat of an abstract on the top of the PRD. Have a way to tell or talk about it because, knowing that no one is going to read the whole thing, but having something that’s shorthand that you’re able to talk about that kind of summates everything that’s in that document is something that I really pushed at Informed.
Another thing that I really pushed at Informed were their assumptions that were being made. I made it a point at the end of every document just to write a list of assumptions that I saw that the product person was making or asked them to write what they saw as they were writing through the document. I would tell them that, “Okay, this is the start of your product research, right? We have to get to the bottom of these assumptions. How confident can you be in them?” I think the team found it very helpful to have to sit there and think about just how often we tend to just assume things and let it go. It functioned as a pretty big check.
I don’t think there was anything else that was relatively different other than that for those internal documents that I had them write. Same metric success. Who are the stakeholders? I guess this is different. Who are stakeholders and why do they care?
Holly Hester-Reilly:
Tell me more about that.
Adam Thomas:
I know politics is being a topic or theme that we’re kind of touching on. I think one of the things that folks tend to dismiss in discussing or telling the story of whatever they’re trying to build is what is the point of view that other folks that are stakeholders are coming from. It’s super important as you’re trying to get resources or you’re trying to get people to believe or moral to go up, you have an understanding of why they care.
Holly Hester-Reilly:
Yeah, or if they care. Why should they?
Adam Thomas:
Or if they care. Yeah.
Holly Hester-Reilly:
A lot of time people just come at it assuming that these other people are going to care. I’m going to give them documents to read and they’re going to read it because they care. But maybe they don’t care and you need to understand why.
Adam Thomas:
Most definitely. Sometimes not caring is fine. It saves you time. If you know this engineering manager doesn’t care or believes you or whatever, it’s not really affecting their day, then you know you don’t have to give them updates a million times a day. Hey, I can use this understanding to send an email out at the end of the week and say, “This is what your engineers are working on.”
Holly Hester-Reilly:
Yeah.
Adam Thomas:
Or the marketer, they really see that this feature has the ability to really, I don’t know, increase referrals. You know how to keep them up to date and keep them abreast and give them stuff that they can send out to the folks that are anticipating this so that they can… You have an understanding to send that stuff out to help boost folk’s excitement about what you’re building, right?
Holly Hester-Reilly:
Yeah.
Adam Thomas:
All these things become possible just by thinking about, very early in the process, why do they care? What is their point of view? What do they want out of this?
Holly Hester-Reilly:
Yeah. You mentioned as well, and let’s say, sort of this process that you’ve built there. They start with this document and it makes them think critically about what they’re doing and why, who cares about it, how do they measure it, and what is this initiative, right? You said it’s the beginning of their research so tell me a little more about what did product research look like? And what kind of challenges did you face in building out a research discipline and practice there?
Adam Thomas:
I’ll tell you, Holly, I’m sure this is something that you’ve faced a lot doing the work that you do. There’s barely any discipline. Barely any consistency. A lot of gut reactions.
Holly Hester-Reilly:
What? You don’t say? People like to just go with their gut?
Adam Thomas:
No documentation, no anything that speaks to a very mature research practice. When I got there, I did this with clients a lot. I would say, “Okay, give me everything you’ve done prior.” I know with clients, it would give me an understanding of where were they? How do I need to talk to them regarding research? Are they at the very beginnings? Is it good research? Is it bad research? If they have a lot of it? Let’s kind of break it down here so I have a game plan going in.
At Informed, it was more the former. It wasn’t much of any research to speak of, even though before I got there they swore they had product folks and they swore they had other things happening. At a certain point, if there are folks doing the work there should be something that shows the work being done. They just… Yeah, yeah.
Holly Hester-Reilly:
Yeah, so you faced them having not much of a practice there. One of the things that I come across a lot of times when I do work with people where whether it’s the person I’m working with directly and their team or it’s the people around then one step removed, is disbelief that it’s worth it. Whether they think that it’s not needed or they think that it’s not possible to get useful learnings from, it’s not uncommon to hear, “Why would we invest in that?” I’m curious, did you come across that? Was there a pushback on it? This maybe is a place where it comes back to sort of these politics of like, “Well, it’s not just what does it mean to do good research, it’s also how do you get buy-in for that research? How do you get buy-in for making decisions based on that research?” What does that look like for you?
Adam Thomas:
Right. I feel like a lot of times, I don’t have any data on this, but somebody has tried to do research. They did it badly and now they’re mental model of what research looks like is just a waste of time, full of these quotes that no one is going to use and full of this bad data that doesn’t really tell us anything. When assessing and when getting to Informed, looking at some of the research I saw some of that. I saw quant data surveys that had maybe six or seven responses and folks making decisions based on that. I saw all research that were hypotheses that weren’t falsifiable. I saw…
Holly Hester-Reilly:
Let’s pause. Let’s pick on that for a second because I don’t know that I’ve covered that topic before. What do you mean a hypothesis that is not falsifiable?
Adam Thomas:
Well, Carl Popper says… No, but not to get into it too much, a good hypothesis needs to have the ability to be false or else we’re just going to believe whatever we think. There has to be a very clear bright line reason why this won’t work or this will work.
Holly Hester-Reilly:
Yeah.
Adam Thomas:
And the sharper it is, the better it is for the experiment. When I say the hypotheses weren’t falsifiable, they’ll say, “We’re doing this research study to see if someone likes this.” I can’t qualify like. I don’t know what that means. You can create what like is in your head at any given time.
Holly Hester-Reilly:
Yeah.
Adam Thomas:
As opposed to saying, “I think that out of three folks, two people will do X and that tells me if this is why, that’s far more… ” I hate three but just using that as the first number that came to my head. That tells me what a hypothesis is and I have something very clear to really speak to what we’re trying to understand.
Holly Hester-Reilly:
And there’s a way for it to be wrong. If your hypothesis is two out of three people will do X and then one out of three people does X, you’re like, “Whoops, I didn’t get that quite right. There’s something there.” Yeah, absolutely. If you were working with younger product people who were newer to experimentation in the world of product, how do you train them?
Adam Thomas:
I think there are three things that you have to really nail to build a really solid research practice. The first thing is you have to train that discipline. They have to know why the research in the past was bad. For quant, you just got to give them, I think, at least two things. One, you have to give them an understanding of statistics. Seven is not going to be a sample size that’s going to tell you anything why. You have to tell them about confidence. You have to tell them how to get sample sizes. You have to tell them about margins of error. You have to tell them all types of things, these things that will help them understand why they need X amount of people to answer or else the survey is nonsense.
You also have to tell them how to build good survey answers, or questions. I’m sorry. Questions and answers, honestly.
Holly Hester-Reilly:
Yeah. I was just going to say, “And answers.” Yeah.
Adam Thomas:
Both. Both matter. You have to teach them how to screen correctly when looking at surveys. You got to have gotchas that help them understand this is useless data, let’s get this out of here. Giving them the understanding of how to do that stuff on the quant side is super important. On the qual side, it’s creating research outlines, real research outlines that take work.
I remember when I first got there, I had a designer tell me for a usability study, he was just like, “I really don’t understand why I’m spending this time. I just want to get them in front to see if they do the thing.” After walking him through the reason why, which is essentially you need to have something clear. You need to have a falsifiable hypothesis. You need to have clear questions that don’t lead the person. You need to know what you’re ultimately trying to get out of this both politically and also through the person. You need to be very clear with your script. When he did it afterwards, he was like, “Oh, this data seems a lot clearer.” And it’s like, “Yes.” You’re starting to get things that are useful for you to take and give it to someone and say, “This is why I’m making this decision.” As opposed to just people feeling like it’s from your gut and that you don’t have a lot of data to really back that up very cleanly.
There’s also recruitment discipline. I mentioned screeners in the past with the quant stuff that I brought up. There’s something, I’m sure you’ve heard this before, I don’t want to bother the customer. I’m scared to bother them. I don’t know if they want to hear from me.
Holly Hester-Reilly:
Yeah.
Adam Thomas:
Meanwhile, they don’t know their customers are biting at the chomp. They just can’t wait to tell you stuff.
Holly Hester-Reilly:
Right.
Adam Thomas:
I remember the first time, and this goes into being a model which is the second part, being a model for them I remember I got there immediately and did a beta study, just immediately I think the second week I was there. I think most of the participants were like, “I don’t need a… ” They didn’t even want compensation from me. They just were like, “I can’t believe someone’s finally reached out to me to talk about X. I’m super happy to tell you what’s going on. You want more? What’s your email? I’ll send you all my thoughts.”
Holly Hester-Reilly:
Yeah.
Adam Thomas:
Customers are super excited to hear from you. All it takes is an email, for the most part. If you have tools like Intercom or Mixpanel or Pendo or whatever, this makes this super easy to do. Really, a simple email just saying, “Hi. I want to hear from you,” will get you a whole bunch of stuff. You tighten up their discipline around your quant, your qual, and also your recruitment.
The next step is modeling that behavior. As I said, one of the first things I did was just do a study just immediately and also critique the study they did. Showing them, one, this is how I got the data. Presenting this over to leadership. Showing them how I did that and having leadership listen in a way that they haven’t before. I think for them was a good way for them to see, “Hey, people will listen to research.” There’s a way that people do listen to this stuff.
Holly Hester-Reilly:
And what is that way? What, for you, has made it different so that the leadership was listening?
Adam Thomas:
I think it goes back to understanding what their motivations are. It’s about getting the data that they care about and also presenting them with the work that you did. Showing them that you do show your work. Show them that you do understand what they need. Giving it to them in the most efficient way possible. I did the big show once a quarter, which was all the research all bundled up, all summarized all beautiful. It was really a one sheet at the beginning of the presentation that I sat and had them read. I’m an Edward Tufte fan. Of course, I’m going to tell them to read silently. Showing them that, yeah, we do the work but the truth is they all knew that by the time we got there.
Before the big show, it’s a constant feedback cycle of me showing them what they want to know. Sometimes they want video. Sometimes they want face to face presentations. Sometimes they want text. I had an engineering manager that loved to read the transcripts. He was just like, “Send me the transcripts. I just want to read all of them.” Happy to do that. Showing them that, eventually, my team is able to give you all this information how you want it so you can make better decisions on your own. Opens everything up.
Holly Hester-Reilly:
Yeah. Cool. Are there any other tips you have on the politics-side of making that research useful, maybe outside of leadership amongst the engineers or any other teams that you’ve kind of worked through that on?
Adam Thomas:
I think the politics matter internally too, to the product team, research team. As the leader, I think it’s important that you model this stuff working. You model a way to disagree. You model a way to, I guess I want to say, get your will across the organization. That gets the product closer to the books they read or the blogs they keep up with or the podcasts like this that they listen to. I think there are probably a lot of product folks that will listen to this like, “My organization doesn’t do any of this.”
There’s an opportunity for product leaders to kind of jump in and start to model that for the folks internally, politically, so that they get more confidence as they go out on their own. You can’t be there for every interaction. You have to give them the tools to be able to do so in a way that helps them not just for this job, and this is something that Butler told me a lot, “I’m not here to help you for this job. I’m here to help you for the next one.” Giving them the confidence and the skills to move forward with their careers is ultimately, I hope, a legacy for that team and the folks there.
Holly Hester-Reilly:
Yeah. Absolutely. I think we’re going to need to wrap it up because we’ve been talking for a while. I’m wondering where people can find you if they’d like to follow you and hear more of what you have to share?
Adam Thomas:
Sure. So just finished my website. Done, the remodel.
Holly Hester-Reilly:
Congratulations.
Adam Thomas:
Thank you. It’s funny because it’s one of those types of projects where you go, “I’m the boss of this. They can just wrap it up.” Then you just spend weeks and weeks and weeks figuring it out. So you can find that at theadamthomas.com and then also you can find me on Twitter, same craziness, at thehonorableAT on Twitter.
Holly Hester-Reilly:
Wonderful. Awesome. Well, thank you so much for your time, Adam. I really appreciate you sharing your experiences with our listeners and getting to talk about product with you.
Adam Thomas:
Thank you, Holly, and thank you for inviting me. This was a wonderful, wonderful conversation.
Holly Hester-Reilly:
Product Science Podcast is brought to you by H2R Product Science. We teach start-up founders and product leaders how to use the product science method to discover the strongest product opportunities and lay the foundations for high-growth products, teams, and businesses. Learn more at h2rproductscience.com. Enjoying this episode? Don’t forget to subscribe so you don’t miss next week’s episode. I also encourage you to visit us at productsciencepodcast.com to sign up for more information and resources from me and our guests. If you love the show, a rating and review would be greatly appreciated. Thank you.
