The Dan Olsen Hypothesis: You Can't Just Take What You Learn In A Big Company And Apply It To A Startup

Dan Olsen is a product management trainer, consultant, and speaker. Dan wrote the bestseller The Lean Product Playbook. Through his interactive training workshops, Dan helps companies build great products and strong product teams. He is also the founder of the 11,000-member Lean Product Meetup community.

In this episode of the Product Science Podcast, we cover Dan’s journey through product at both enterprises and startups, how that experience became the Lean Product movement, and how to validate a user's responses and prevent false negatives.

Subscribe for the full episode on Apple, Google Play, Spotify, Stitcher, and more. Love what you hear? Leave us a review, it means a lot.

Resources

• Follow Dan on Twitter
• Follow Dan on LinkedIn
• Follow Dan on Youtube
• Dan’s Training Workshops
• The Lean Product Meetup
• The Lean Product Playbook
• Follow Holly on Twitter
• Follow Holly on LinkedIn

Questions We Explore in This Episode

How did Dan get started in his career in product? How did an interest in technology play a role in Dan’s early career? How did the experience Dan gained at Intuit apply to the rest of his career? How did Dan transition into working at startups? How did Dan develop the Lean Product Playbook and Meetup from his time as an interim vp of product for multiple companies? What was it like for Dan to run his first startup? How does a product manager’s education into product differ when learning in a startup environment? Why is it common to see product managers transition between enterprises & startups repeatedly throughout their careers?

What was it like to win the people’s choice award at TechCrunch50? How was Yourversion able to turn adversity into their advantage to help gain traction at TechCrunch50? Before his time in startups, what was it like for Dan at Intuit in 1998? What was Quicken’s presence like in the market in 1998? What was the benefit of how Intuit scheduled their product and feature releases? How did a clear schedule dictate the process for the product team at Intuit? How do feature teams differ from one another? What are qualities of a good manager? How does a large company like Intuit naturally help promote career development? How did Intuit invest in their employee’s professional development? How did Intuit find the difference between their products creating customer value and capturing customer value?

How did Dan transition from Intuit into working at startups? How is working on a legacy product different from working on a V1.0 product? Why was working on a 1.0 product at a large company like Intuit a great testing ground for what working at a startup is like? What are your options if you feel you’re no longer growing in your career? How do product managers build relationships at large companies? How can setting stretch goals help to empower teams and affect cultural change at a company? How does a startup environment differ from a large company’s? What was Dan’s experience in boot strapping a business like? Why is it difficult to keep teams aligned on a vision? How often should you check in with teams to re-align on vision and strategy? What is user research like in early startups? How did Dan transition from startups to being a consultant? How was Dan able to apply what he was learning in school to his work? How was Dan able to create low cost prototypes to test ideas?

How did the Lean Product Playbook get written? How did Dan get involved in public speaking? How did speaking at 1 event lead to further speaking engagements for Dan? How did consultancy help Dan gather information for his book? How did the Lean Product Meetup get started? How can product teams distribute experienced and junior product managers? Why is having experienced product managers in a product org essential? How was the Lean Product Meetup able to adapt to the pandemic? How has the Lean Product Meetup grown internationally?

What is experimentation like when you’re a consultant? How can experimentation be used to optimize your processes? How is experimentation different before and after launch? How does Dan structure an experiment from start to execution? How can prototypes help to speed up experimentation? How can prototypes help you know if you’re solving the user’s problem? How can experimentation help to validate your strategy? Why should you have multiple prototypes to test with? How do you validate the results from multiple prototypes? How can personas help narrow your target customer? How do guarantee getting authentic responses from user testing? How do you get users to invest in a product they are testing? How do you avoid false positive responses from users? How do you get users to have some “Skin in the game?” What are other strategies to test a product that are not focused on cost of the product?

For product managers looking to advance into product leaders, what is that journey like? Why is being a product manager a siloed experience? Why is it difficult to feel like a cohesive product team? What are some strategies to help teams feel more connected and engaged? How do product managers fill multiple roles in their company? What changes from being a product manager to being a leader? Why is being a product leader a very hands off role? Why are good product leaders always invested in the development of their teams?

Quotes from this episode

That's the beauty of creating mock-ups, it's a lot quicker than creating code. I'm a huge believer in using interactive prototypes to do the experimentation and test different variants.

PM can be a very lonely job where we're all so busy, we're dividing and conquering. You're on a PM team, but it may not really feel like a team. As the leader, try to do what you can to make it feel like a team.

You can't just take what you learn in a big company and apply it as is to a startup. You have to mold it and tailor it.

Transcription

Holly:
This week on the Product Science podcast, I'm excited to share a conversation with Dan Olsen. Dan is a product management trainer, consultant and speaker. Dan wrote the best seller, The Lean Product Playbook. Through his interactive training workshops, Dan helps companies build great products and strong product teams. He is also the founder of the 11,000 member lean product meetup community. Welcome, Dan.

Dan Olsen:
Hi, Holly. Nice to be here.

Holly:
I'm excited to have you. I know you've had a long career in product. I'd love to hear a bit about how that got started.

Dan Olsen:
Yeah, definitely. Probably started when my parents got me a computer when I was a kid. I'm old enough that like personal computers were kind of exploding, people getting Apple IIs and Commodore 64s and all that. Luckily they got me a computer, so I was comfortable with coding as a kid. I still had electrical engineering in school where I also kind of worked on computer design, which was fun.
And then my first job out of college actually was in the navy doing submarine design. And my title was technically like engineer. We were doing all this mechanical engineering stuff. But after I got into product management in hindsight, it really was just kind of very technical product management, very cross functional work, a lot of complexity.
Anyway, and then after that I had decided I wanted get more on the business side of things so I wanted to go to business school since I had only worked in the navy to kind of diversify my experience. And that's what brought me out here to Silicon Valley to attend Stanford. And while I was at business school trying to figure out what I wanted to do when I grew up, I learned about this emerging career called product management. The more I learned about, the more exciting it sounded.
And because I'd never done it, I asked people, "Hey, where's the best place to learn product management?" And at the time most people said Intuit was a really good place to learn. I was fortunate to get a job at Intuit, I worked on the Quicken team, so consumer software, millions of users, which was great, annual release cycle and I learned a ton there. I mean, most of what I learned in product management, still the foundations all come from Intuit.
And then I worked on the Quicken team for a while, worked on a web product there. Then after five years at Intuit, decided to move on and take what I learned to apply at startups. I joined the bootstrap startup with a friend and then I actually joined, it was a head of product at Friendster way back in the day at first social network. And then I was CEO and co-founder of my own startup. I was at Intuit, then I was at startups and it was actually while I was doing that that I basically said, "I know how to code, I just don't know how to code all this web stuff."
I'd worked on several web products. And so I made it a point to go and study HTML, CSS, JavaScript, all that. And I got a job offer to be the head of product for a startup. And I really liked the startup and what they were doing, but I was literally taking 20 hours a week of classes just to kind of get it done. And so I said, "Hey guys, I can't be your full-time employee. Can we work out some kind of consulting deal?" And so that's how I kind of stumbled into being like this interim VP of product for startups. And it was a lot of fun because I got to work with one startup for six to 18 months and then I'd work for another one.
I really learned a lot about what different product teams were doing right and what challenges they were facing. And so it really accelerated my product learning. I then kind of got to the point where I was doing more than one at a time, two or three at a time, so it was a lot of fun. And then that's kind of what led into the speaking and the book and all that stuff, so yeah.

Holly:
Yeah. That sounds like a great journey. I want to back you up a little bit and hear more about your startup. What was your startup about?

Dan Olsen:
Oh, my startup was a personalized new startup. The only kind of product left in that category is Flipboard. People familiar Flipboard, it basically was like that. It was actually 2007, 2008. So it was like when the Sequoia RIP nuclear winner debt came out, it was really hard to get funding for startups and things like that. I actually bootstrapped it out of the back of my apartment, where there was kind of a two bedroom connected to a studio. So the studio became a startup, we launched at TechCrunch, we won the People's Choice Award. It was a lot of fun, but it's a really tough category. It's really hard to make money in that space, nobody really pays for news. Actually a couple other people in that category had acquisitions early on. We had an acquisition offer, but I decided not to take that, but it was a lot of fun, it was great.

Holly:
Yeah, that's awesome. I actually really love that because it's around the same time that I was part of a startup that I co-founded with a couple people as well. And so I'm like, oh yeah, I remember that period really well and what it was like being a startup.

Dan Olsen:
It was exciting.

Holly:
Yeah. And we also bootstrapped. But for me, it was the beginning of my tech journey actually. So you brought to it a lot more experience and knowledge by that point in time.

Dan Olsen:
Yeah. And that's what's interesting, some people are like, "There's pros and cons to starting your product career in a startup versus a bigger company."

Holly:
Yeah. Tell me more about that. What do you see as the pros and cons?

Dan Olsen:
Well, I mean, I think that, especially if it's an early stage startup and you're a brand new product person, there's usually not a lot of support and infrastructure and mentorship that happens, so you're kind of learning on the fly, throw into the fire, have to figure it out. It can be very tough and frustrating.
The pro is you can kind of go from ID to execution very quickly. You can move fast, you can be nimble, you can pivot. At my startup, someone would just comment on Twitter about something that a bug or something they didn't like, they wouldn't even like really mention us, but we had the alerts going. We'd see it, we'd fix it within a day or so. And then we'd tweet them back and they'd be like, "No way." You could do that in a fast startup, right? In a bigger company might take longer.
But in a bigger company, depending on not all the companies, but if you go to a big company where they've got a really good product management team and you've got mentors and their processes are down and got a track record, you can really learn best practices so that when you do go to startup, you can bring those. Now the last thing I'll say is you can't just take what you learn in a big company, apply it as is to a startup. You have to mold it and tailor it.
I often see people in their prior career oscillate between the two, the pendulum swings, they'll start in a large company like, "Okay, I'm tired of this. I want to try something different." And then you go to a startup for a while and then you're like, "Oh my gosh, this is chaotic and crazy. I'm going to go back." They go back and forth over their career, that's pretty common.

Holly:
Yeah, absolutely. Cool. Well, tell me a little more about ... I mean, what was it like winning the People's Choice Award at TechCrunch? That's pretty cool.

Dan Olsen:
Oh, I mean, it was awesome. It's so funny because I love Silicon Valley HBO with Pied Piper and they launched at TechCrunch and so they're on stage, so I had a chance to live that experience. It was cool. It was the last year they did tech ... then they'd pivot it to calling it TechCrunch disrupt. And basically they would pre-select a certain number, used to be TechCrunch 30 then 40 then 50.
Pre-select a certain number of companies that if you got selected, then you just kind of got to go on stage automatically. We got to the final round and didn't get picked. And my team was so bummed, but then I was like, "Hey guys, we'll just get a booth in this demo pit. And if we win the People's Choice awards, we still get on stage. It's pretty much the same thing." We kind of set that as our goal to do that.
And I had marketing interns and so one of them came up with the idea for slap bracelets that had our branding on it. And those were one of the top swag items, so there was like viral, like people, "Where'd you get that?" Oh, you got to go to the year version booth. And then my co-founder was really good at demoing and very enthusiastic in rallying people, so I think that those kind of things helped out.
But it was great, I got to present on stage. My [inaudible 00:07:24] was like, "Okay, you guys won People's Choice, you got to be ready to get on stage in five minutes." I'm like, "Okay." And then Marissa Meyer was on my panel and actually after my demo, she complimented the UI, which I was proud about because we had done a lot of user testing and stuff. And then there were like Paul Graham from Y Combinator was on there. It was just kind of funny who was on the board and stuff. It was great. It was fun. It was a lot of fun.

Holly:
Yeah, that sounds pretty awesome. A lot of legends there.

Dan Olsen:
Yeah.

Holly:
That's cool. Let's go back further. Tell me more about what it was like working at Intuit early in your career.

Dan Olsen:
It was awesome. It's so funny, Holly. Let's talk about these pivotal times, 2007, 2008 was like the nuclear winner where all the startups were trying to get through. When I accepted my job in Intuit was like 1998. And so it was like the first dot-com was starting to get really hot.
And Quicken, which had been around for a long time and was a successful product with 80% market share had launched quicken.com couple years ago or a year or so ago. It was like the hot new dot-com portal. And so when you interviewed into it, usually interview with two groups and I got an offer in both groups and they were like, "Okay, which one do you want?" When I chose Quicken, they couldn't believe it. They're like, "What?" Because Quicken.com's ranks were so hot and all this jazz.
And it kind of goes back to what I was saying about the startup versus established company. I was like, "Well, I want to learn from people that have been around and done it." The Quicken team has a very strong track record. I joined that team and it was like walking into this machine. They had been cranking out successful versions of Quicken every year for many, many years. And so what's interesting is each year the people change somewhat. There's always new people coming in, people moving up, people moving to other jobs. But the kind of institutional knowledge is there.
And so you just kind of walk in it just like, "Okay. Now it's the time when we do consumer research. Okay, now's the time when we write our MRD. Okay, now's the time where we do the kickoff." It was nice because you would bring your skills and individuality and creativity to the equation, but what had to get done by when was pretty clear. And so you'd work with your feature teams. It was great. I mean, that's where I learned as a PM you would get put on multiple feature teams, three or four feature teams.
And that's where I learned the early lesson that no two feature teams are alike. Every feature team has its own mix of people and pros and cons and challenges and things like that. We had great cross-functional partnership with UX and dev and we would have betas and we would do usability testing. It's all the best practices that companies still try to aspire to today of customer centric development we would do basically, right. So it was great.
And another thing I often mention is, the founder of Intuit Scott Cook came from Procter & Gamble. And so Procter & Gamble does package goods companies. They have to convince you to buy their soap versus someone else's soap or their toilet paper versus someone else's toilet paper or their bleach versus someone else's. So they're really, really good at customer research and kind of differentiation and understanding customer benefits and messaging, things like that.
As a result, that's what I attribute to why we had a PhD in market research on our Quicken marketing team. And so I learned tons from her about both qualitative and quantitative research, which has really helped me a lot in my career. And I'm excited to see in the last few years, more and more excitement and interest in kind of research, and discovery and qualitative and quantitative, so it's pretty interesting.
It was great. I learned a lot and then it's a one year product cycle, so it's kind of cool because it's long enough that you spend enough time in each phase. You're not rushing like crazy, but then you move on and then you get to do it again, so I did three product cycles there. Once as an individual PM contributing to it, the second time as leading the whole version. And then the third time is a second level manager managing teams that were doing the version, so it was a really good progression. And my GM there was the best boss I've ever had.

Holly:
Wow. What made your GM the best boss?

Dan Olsen:
He was very good about ... And Intuit in general is good about setting ... OKRs are popular these days, but setting objectives, like what are our business objectives for the company. And then what does that mean for the Quicken division? Or what does that mean for you? What I call cascading OKRs, they were really good at that. And they would use the term line of sight. Usually had a pretty good line of sight into what you were doing, how it fit in and contributed to the bigger picture goals.
Plus it's kind of like Intuit culture was good at that. Intuit culture was also good at one-on-ones and growth and development of people. So you had your day job, but they were always oftentimes talking about what did you want to do next? And things like that. And that's the advantage of a big company is if you get tired with a certain area or want to do new things, you can move around basically within the company. But he was really good at all that, good at one on ones, good at growth and development.
One thing he specifically did because we had a one year product cycle, people would often change jobs and/or get promoted during the summer, right after the release, it was like musical chairs. Okay, we just launched. Okay, who's going to stay, who's going to move on, who's going to get promoted? And one thing that he did is when I started there, there were two levels of management. It still was relatively flat, but there were two levels of management between me and him. And after the first cycle, both managers left and then I got promoted, but he deliberately didn't backfill the level of management between the two of us because he saw a lot of potential. And so that was great.
I got to sit on his staff and attend all those meetings. And he also invested a lot in training. Every month we would have training from external experts come in and things like that. And I still remember to this day, there's one Columbia professor person who talked a lot about customer benefits and things like that. Anyway, we would do fine events too. Once we do ship trips and when we would ship and fun events. And so he was really good about that kind of the soft skills of hiring great people, creating a great environment, focusing on their growth and development.

Holly:
Yeah. That sounds like a great leader. How did you decide to leave that?

Dan Olsen:
Well, basically I mentioned, so I worked on Quicken for three years. I was like, "Okay, I've been there, done that on Quicken." And the web was taking off, so like my growth and development goals would be like, "Hey, it'd be great to work on a pure web product." I mean, Quicken was kind of a hybrid where it would download stuff and increasingly it was kind of a mix of web and desktop. But I was like, "You know what, it'd be great to work on a web product and it'd be great to work on a 1.0 product instead of an upgrade of a product that's like version 30 or whatever it is."
That was my growth and development goals. And when there was an opportunity that came up, they let me do that. I was the head of product. We decided to launch an online brokerage because we had this amazing website, quicken.com where people would come, investors would come, do all the research. They had all these award winning innovative tools and then they would go to Schwab or E-Trade and place their trades and they would make the money and we would only make money off advertising.
And so when advertising was good, it was cool. But when the crash happened and advertising go through, we had this highlighted difference between creating customer value and capturing customer value. There was no doubt this quicken.com investing website was creating so much customer value. Millions of investors were coming and using it every day. It wasn't my baby, I didn't work on it. It was definitely the best financial website back in the day.
But then we weren't capturing the value because we didn't have a good business model or good revenue model. I imagine like a Ferrari sports car that's got this 500 horsepower engine that's generating all this value, but the transmission's broken. So none of it's getting translated to the wheels basically.
And Intuit always likes to be neutral and have a Switzerland strategy when it comes to financial institutions because we need to work with them all, so they're very reluctant to actually become one themselves, a bank or anything. But we said, you know what, let's partner and let's do a brokerage, so I did that and it was great. I'm glad you asked me about that because that was the transition from working on a mature established product like Quicken, so then going to these startups was working on a V1.0 within Intuit, which was amazing because we basically were working on a 1.0 product but we had all the resources of Intuit. It's kind of like a startup within Intuit.
It was really cool because we had great teams, we could leverage the assets, that product did okay. But then we had to partner with someone and we ran into some partnership issues and I knew that, and basically we kind of nailed the release. We nailed what we needed to do for version 1.0. We did a couple of dot releases in the next few months, but basically it was a fully functioning working brokerage. And so I kind of knew when I took that job that like a startup there's risk that your job may go away or whatever.
I basically said, "Hey, you don't need my level person running this anymore. We can kind of just fold it back into the Quicken team and maintain it at this point." And so then I was kind of left without a job, which I knew could happen. And then as I looked around other places and into it for the next role, I was like, "You know what, I've been here five years, I could kind of feel the learning kind of start to flatten out." It was a really steep learning curve. I was like a sponge for the first three, four years.
And in that fifth year I just felt like, yeah, I could go to QuickBooks or something, but I've kind of learned most of what I think I'm going to learn here into it. I liked working on a 1.0 product, so I was like, let me go try startups. So again, it's the pendulum swinging after five years of being an established company, I'm like, "Let's go see what startups are about."

Holly:
Yeah. I like that part of your journey that you got to move from the established product to the 1.0 product within an established company, it can be so much fun to work on a 1.0 project.

Dan Olsen:
Yeah. And I didn't think about it this way, but a lot of PMs in bigger companies these days after two, three, four, five years working on the established company, you're kind of well positioned, right? So when a 1.0 company, who are they going to toss the ball to? They're not going to give it to a brand new PM. They're not going to give it to some new external hire who doesn't know the company. You're well positioned if that's what you're interested in, I think you're well positioned to take those things, which can be super exciting.

Holly:
Yeah. What about the downsides? What were the downsides of being part of a 1.0 in an established company?

Dan Olsen:
Well, I mean there's risk, right? If you care about, hey, I've got this nice cushy job as ahead of Quicken, I can just keep launching new versions of Quicken. If you just want to rest invest, then it's fine. But if you want new challenges and things, then the main risk is that, hey, this may not succeed and your job may go away, that's the main thing.
Also it wasn't a risk per se, but there was definitely a cultural difference too. It's interesting because they had a broken revenue model that team before we decided to do the brokerage with them, they would ship features. And it's really hard to understand what the revenue impact was going to be of those features. You know what I mean? If you slipped, they would slip, they would slip a week, slip a month, it's like okay.
There was no real consequence. Because of that decoupling, the lack of the coupling of clear business results with what you're building, I think that's what I attributed to. They had kind of gotten this culture of, ah, it's okay if we slip. So then when we came in with Quicken brokerage, we had to set an aggressive timeline. There was a bit of a culture shift there, where I had to partner with this quicken.com engineering team that had been used to being like, "Yeah, dates don't really matter." And now the senior VP is like, "We got to launch this thing by this day."
And so I had to be the voice of reality and be like, "Hey, if we want to hit these goals," not everyone was like that, some people got and were excited. One of the lead architects I got to work with, it was awesome to work with this guy. I'd worked with his brother on the Quicken team and I ended up kind of doing a patent on some of the trading functionality that we had done. It was just so fun.
He got it and then he was rallying, the director of engineering got it, she was rallying everybody. So it was kind of fun. I don't like arbitrary deadlines, but deadlines can be helpful to rally a team. In this case, it was helpful to have an aggressive goal, a stretch goal, but not impossible. And then kind of hit it. That kind of feels really good when your team's able to do that.

Holly:
Yeah, that does. All right. Let's fast forward a little bit then, so you do that 1.0 within Intuit and then you start getting involved in startups, and what surprised you when you first went into startups from there?

Dan Olsen:
Imagine I just got done telling you how awesome Intuit was and how structured it was. And not over stiflingly structured, but well organized, you know what I mean? And then just startups are just chaotic, which I don't mind, I can deal with that. But some people that are used to structured environments, they have a tough time, some people that leave some of the bigger companies and go to a startup. It's like be careful what you wish for.
The first startup I did actually was I kind of was co-founded a startup in Spain with a buddy that was just out of his apartment. It was me and him and a few developers. And so there was a complete lack of structure, but I actually loved it, it was great. We didn't have a designer, so it's like who's going to do the [inaudible 00:19:27] the flows and wire frame, so I stepped up and did that.
And I remember at the time googling like, "Okay, what's the best kind of medium fidelity prototyping tool?" And at the time there was a white paper on why PowerPoint was the most medium fidelity. Now we've got all these amazing tools. I literally was doing all these flows and wire frames and PowerPoint copying and pasting symbols, but it was great in Spanish, but it was great, that was kind of a unique bootstrap situation.
And then the next thing was Friendster which was a 50 person startup when I joined post series A, funded by Kleiner Perkins. It was just amazing just the kind of chaos and just a lot of our job, I do a lot of trading workshops and every once in a while, play the video. The super bowl commercial from EDS about the cat hurdles. It's like instead of herding cattle, they're all riding horses and herding cats. I feel like you have to curb cats.
And the more chaos there is, the more time, energy you as a PM need to put into that. If everybody has line of sight into what we're trying to accomplish and they understand how they work together, you can spend less time on that. There's always some amount of time you have to do herding cats. And even at Intuit I remember, get everyone in a meeting, in a room, conference room, get alignment. And I could tell the second we all left that conference room, the entropy was starting to seep and the cats were starting to walk in different directions. So a week or two later, you got to get everybody back synced up again. But at startups just to pace, the chaos is much higher, so that's probably the biggest thing.

Holly:
Yeah. And what was customer research like when you moved to the startup?

Dan Olsen:
Oh, well, that's interesting. I mean, so a lot of times in 1.0 startups, there's a lot of focus on just figuring out what you're trying to build and trying to get it built and that you have so limited resources that you're just focused on that. We did go out and the startup and the bootstrap are we did go out and talk to some people. And in talking to some people recognized that my friend, the co-founder was focused on a specific target market segment, just a general consumer target market segment.
And I realized, gosh, well, there's this high volume user that instead of doing like buying one of these a year or using this once a year, they're going to be using them multiple times, so that was great insight that we got from talking to users. We went out and talked to people, qualitative users, and then also getting feedback on the early alpha version of the product doing usability testing, things like that.

Holly:
Yeah. Awesome. And then you go through several startups and then you get to the place where you started consulting. How did that come about?

Dan Olsen:
Well, like I said, basically I got an offer to be head of product, but I decided to take all these classes and web programming languages. And so it would've been just like the next startup I would've been the head of product for, but instead it was on a consulting basis because I was taking all these classes.
And so I basically was their VP of product, just not working 100%, maybe working 70% or 75%. And then getting paid hourly instead of getting paid like a salary kind of a thing. That was great. This startup had just signed a major deal with Sony and they're like, "Ah, now they just got real and we need a product manager and we need to get this stuff built." And so that was a lot of fun.
That was a lot of fun because the team was super sharp. They were doing music discovery. So things you take for granted now on Spotify and things like that, you pick a song and how it finds similar songs, they were an early pioneer in that kind of how do we find similar songs based on non obvious things like just genre or something. And so that was pretty cool.
And so in that one actually, one thing I didn't mention is before I came out of here, I actually got a master's in industrial engineering at Virginia Tech. And I remember I took an MIS class, management information systems class. And you had to propose a product for that. And because I was this kind of early on, my dad kind of got me started with an IRA right out of college, just to kind of teach me about investing and stuff.
And so I kind of was into that and kind of went all out and I was like, "Let me prototype this thing." I actually prototyped it in visual basic. So it was like an interactive prototype in visual basic. The only reason I mentioned that is this first startup with the music discovery. I actually ended up prototyping stuff in visual basic for them and then getting usability testing on it, which was a lot of fun. That was before all the new interactive tools now like Balsamiq and InVision and Figma and all that, so that was fun.
But then it ended after five or six months because the way things went with Sony. And so then I was like, "Okay, cool." And then I just did it for YouSendIt. So the pattern was pretty much like a post series, a startup that had enough of a team and product success to raise the A round, but knew that they didn't have the product management chops in house that they wanted. And so I would come in and help them. And then I did it for Box, One Medical Group group as well. I did it for a lot of companies back in the day, it was a lot of fun.

Holly:
Yeah, absolutely. And so where did the book come into play?

Dan Olsen:
Oh yes. So on the side, I basically, again consulting for all these different companies. Instead of just working in one company, I saw all these multiple data points. And so I started speaking basically, speaking was kind of the start of it probably where in 2006, in 2007, I had two different people at Stanford were teaching classes and they asked me to come in and talk. And so that was the first time I ever put any slideware together.
I'd done internal training at Intuit for my team and so some slides there. I created some slides, started speaking and got positive feedback and then people would ask questions. And so then I'd create additional slides based on the questions that they asked and over time just kind of made new talks and things and started speaking at bigger and bigger venues. I used to speak at the Web 2.0 Expo that O'Reilly put on, which was an awesome event. It was in Moscone, I could still picture it like thousands of people getting together. It was a lot of fun and I kept giving talks and they would get posted online.
I gave a talk at Google that got posted online and then my editor at Wiley saw that talk and then reached out to me and said, "Hey, have you thought about writing a book?" And I actually had thought about writing a book, but it's a lot of work to write a book, so I never actually buckled down and did it. And so things worked out to do the book with Wiley and I buckled down and did it. So it kind of emerged from the speaking and consulting.

Holly:
Yeah. And what year did the book come out?

Dan Olsen:
2015. It's coming up on seven years.

Holly:
Yeah. Nice. The book is The Lean Product Playbook.

Dan Olsen:
That's right. Yeah.

Holly:
And definitely a popular book. Where in there did you start the meetup?

Dan Olsen:
Oh, so I started the meetup actually before the book came out in 2014, so eight years ago. And actually this month is the eight year anniversary of the meetup. I was actually consulting for a client Medallia, one of the other unicorns that I consulted for, so they've IPOed and basically they had a great team. I mean they had an amazing team. They were in Palo Alto at the AOL building.
They had this amazing space, amazing team. And it's funny because they brought me into like advise them and help them with their product team. And when I came in, they had a great culture. Medallia has a great culture and their culture in general was hire young, smart people, they'll figure out what to do and we'll train them. That was kind of their general thing. They'd hire people from top schools that were smart and things like that.
And so when I came in there, they had bootstrap for the longest time, they had raised around from Sequoia. They finally let a venture capitalists invest in them Sequoia. And so they were starting to grow a lot and they needed to grow their product team. So they wanted to hire a bunch of PMs who wanted to go from like six PMs to 21 PMs. And I looked at their team and they're like, "Okay, cool. We're just going to hire another young up and coming person who's smart, they'll figure out product management."
And I looked at the team I'm like, "That's 90% of your team are these people that have never done product management before they came here." That's fine, but let's not make our whole team like that. Let's start to bring in some experience to the team. If you think of it like a pyramid, our pyramid was very flat, just a bunch of junior people and one senior person. I was like, "Let's add some middle labs, some senior PMs. Let's add some directors in there." You know what I mean? And so I helped them do that.
What happened is I was talking with the VP of product. I always wanted to do a meetup, but it takes a lot of time and resources. And so they had a great space there and they had a great HR and recruiting team. And so we basically partnered and said, "All right, I'll take care of building the community and getting the speakers. You guys provide the space and deal with logistics." And we did it, so that's how we started basically. We started, it was in person, March, 2014 at AOL building at Medallia. And then it just grew and grew and grew.
And then Medallia moved out of that building that kept growing. And then we moved to actually Intuit, which was great. So we were holding them Intuit and then COVID hit. I remember sitting on my couch and my wife going, "I guess I have to take this online now." And so we took it online and actually it's been awesome because now you don't have to live within 40 miles of Mountain View, California. We have people from all over the states, South America, central America.
Actually Australia the time zone works out pretty well. New Zealand and Australia people dial in and then even people that stay up late in Europe and Africa. That's how it came out just from good opportunistic timing of, they wanted to basically do an event to help recruit product people to grow for their team and get a better presence. I always wanted to kind of start a community and start an event series like that. So it worked out really well.

Holly:
That sounds amazing. One of the things that I always love to talk about is experimentation. And I'd love to hear a little more about the later years of the consulting years and the time speaking and running meetups and everything. Tell me more about what you advise people when it comes to experimentation.

Dan Olsen:
Yeah. And I think about experimentation narrowly, it's usually when your product is live, when you're like, "Okay. Let's tweak this and see what happens. Let's tweak this, see what happens, right." And it's funny because it reminds me in that industrial engineering program that I took, there was an optimization class and a linear programming and things like that.
I always think about that experimentation and kind of an optimization mindset. It's like, okay, we've got what we've got, let's change one or more variables and see if we can do better right [inaudible 00:28:58]. That's kind of the purest form of experimentation. But I think even before that, before you launch, you still have hypothesis you need to test with experiments too. I kind of divide it into pre-launch, which is largely qualitative testing. And then post-launch, which can be more quantitative.
And then basically I was typing up notes for our call. It's basically in my book, like the pre-launch, that's the Lean Product Process. That's how I advise you to articulate your hypotheses and test your hypotheses, right? You should go out and do discovery interviews with who you think your target customers are, clarify who your target customer is. Clarify what you think their problems are, the benefits that you can provide, the needs that you can meet for them.
Then specifically I like to apply importance versus satisfaction on those needs to figure out which needs are well served versus underserved so we can target the most underserved needs. And then basically the next step is doing a value prop grid against your competition. How are you going to be better than what's already out there. And these are all hypotheses that you're basically forming. And then you figure out what you think your MVP feature set should be, and then you prototype it. And the whole point, that prototype, that's where I would test back into my consulting days.
A lot of times I would get hired to do product validation testing. It was all about that interactive prototype. All the thought that led up to the interactive prototype, but then that prototype is what we did the experimentation with to get feedback. Even that startup I mentioned with Visual Basic, this is a very sharp team that I work with when you were to get to match the songs with other songs, you would have to get data from users. And so it was really important that it'd be streamlined and the minimal number of steps, the minimal number of clicks, high focus on usability which I love.
And so I remember we had one debate of like, I can do the Little Wizard in Visual Basic in two screens or in three screens. And it's like this comes up in UX a lot. It's like, okay, we've got this reg process, it's got 20 questions or 15 questions. Should we do it all in one screen of 15? Should we do it on three and five? That whole thing, right? That was the experiment. I actually built both prototypes, the two page and the three page version, screen version in Visual Basic and tested them both.
Sometimes we do that. Step one is just make sure you have a prototype to test. But a lot of times, even recently an e-commerce client of mine, it was like a browser thing where you know how sometimes the little panel pops out on the side and you see the functionality on the inside the browser window, it was like one of those. And again, there was a spectrum of, oh, we can go really detailed and have a lot of detailed information or we can go lighter with less information.
I just work with the designer to mock up actually like five different variations. And one would focus on this benefit. A couple things going on, which benefit are we going to focus on? Are we going to focus on saving your money when you shop or helping you find the size that you need when you shop or better selection or finding related clothes.
There were three different benefits. So we kind of said, "Okay. Let's do a concept for each main benefit." And then within each of those, it was like, "Do you go detailed or do you go light." We had five or six different variations that we mocked up. And that's the beauty of creating mock-ups, it's a lot quicker than creating code, working code. I'm a huge believer in using interactive prototypes to do the experimentation and test different variants. And so that's what I like to do. And sometimes what you can do is then take the best of the different ideas and then combine them into like the [inaudible 00:32:12] idea and then test that guy out.

Holly:
Love it. Tell me a little more about the actual act of testing those prototypes when you have two or three or four different prototypes, how are you testing them?

Dan Olsen:
Well, I mean, in general, so in the book, after I discuss Lean Product Process, I go through a very specific, like the best ... I'm glad I saved all the documents. I saved all the prototypes. So I could tell the story afterwards at marketingreport.com where we basically did all that thinking, came up with a prototype and then tested it and then iterate it.
In general, just one of the key things is the foundation of the pyramid, the product market pyramid in my book is target customers. So you need to make sure you're clear on your target market segmentation, your personas, and then recruit to those. And one of the things I like to say all is like, if we think that our target customers, whatever definition, and we go and we talk to 10 of them and six of them say, "Oh, this is great." Before them go this is horrible. Then what that noise in our data tells me, we haven't gotten sharp enough or crisp enough in our target market segmentation. That's one of the top things for me is tying it back to the target market customer.
Say you have a B2B product for e-commerce people and you go and you test it in 10 people, six say, "I love it." And four say, "I don't." You may learn that it's because the four that didn't like it have complex inventory needs you didn't anticipate. So your product doesn't handle inventory, but the other six don't have inventory. So then you go, "Aha, our target market." We have two choices here. We could say, "We're going to go for the target market that doesn't have these complex inventory needs." Or we could say, "Let's adapt our product to go after it." That's a big thing.
Usually it's just testing one concept, but when you do experimentation with multiple, then I will pick the right order to show them to people and then show them multiple to the same people. And this is just good anyway, when you're showing people a bunch of stuff, it's at the end, they go, "Okay, out of all the stuff we showed you, what did you like best?" It kind of forces them to do kind of relative prioritization. And you see what they remember too. If they go through the whole test and like, ah, I don't really remember anything that was that great, then sorry, you're still quite awake in the bullseye at that point.
And this is where also you got to watch out for false positives. People are nice. Oh, this is a lovely prototype. Oh, this is great. Oh yeah. My favorite line Holly is when you ask them, "Would you use it?" Oh, I wouldn't use it, but I could totally see how other people would use it. It's like, well, we're doing the user test with you, so it's kind of funny when people they're super nice. That's why at the end if you go ... we call it what? The skin in the game. This is probably another important concept to think about. But that's the beauty of asking at the end, hey, what did you like the most? It kind forces a relative prioritization.
And you can also say, "What did you like the least, or what was least valuable to you?" It's easy for people to kind of pick the best and the worst. You can try to get through the rank order a bunch of stuff, but it might be too complicated. So at least ask them to pick the best and the worst is great. But back to this false positive issue where people are going to tell you they like it, the best technique there is to introduce what's called skin in the game, some skin in the game so that if they say yes, they have some skin in the game.
And my friend Steven Cohen, who was the CEO and co-founder most succinctly kind of taught me this. It's like you can ask people for time. Wow, Holly, sounds like you really like this prototype, we're going to have a new version in three weeks. Can we schedule time on your calendar right now? If they really are so excited about it and as a good fit, they're going to be like, "Yeah, let's book it right now." If they were just giving a false positive and going to be like, "Well, I don't know what's going on. Check with my admin. I'll have to get back to you, just email me, you'll figure out a date." That's skin in the game.
The other one is money, especially for B2B. You can be like, "Wow, it sounds like you really like this prototype. We're going to be ready with our private beta in a month. We're taking deposits right now, 5K, 10K, whatever. Just some token amount. Would you be willing to put down a deposit? And again, that just makes the false positives go away.
And the last one is kind of their reputation or social capital. Wow, Holly, it sounds like you really like our prototype. Do you know three other CMOs who would also value looking at this prototype that you could connect us with? Again, and it's funny because with that exact e-commerce client I was helping, right before when the mock-ups were ready and we're about to do the test, I'm like, "Okay, let's talk about what we're going to use for skin in the game."
And I was thinking it was just a free app on the App Store. It's a free thing. I was like, "Well, maybe we should ask them, if asked to charge like 1.99 for the app and see if they would pay for that or not." And the founders were like, "I think that's a little too heavy. What if we just say there's going to be a private beta and we need you to devote five hours a week to test it out and give us feedback during the private beta." I was a little like, "I don't know if that's going to be enough skin in the game to really weed out the false positives or not."
But it turns out they were totally right. We'd be doing this test. Oh, this is amazing. This is great. Oh yeah, [inaudible 00:36:52]. Oh great, Joe, sounds like you really like it. We're going to be private beta, all we're asking you, you can have early access and start shopping with this thing right away. All we're asking is five hours a week. Like, oh, no, no, no, I don't want to, I'm too busy. Sorry. I can't. It actually worked out that five hours a week for a private beta was adequate skin in the game for that kind of consumer e-commerce app.

Holly:
Yeah. Those are great examples of skin in the game techniques. I love it. Well, I think we're about running out of time. I always like to ask people for their advice. What advice would you give to aspiring products leaders?

Dan Olsen:
Aspiring product leaders, well, the transition, I was just thinking about this actually, the kind of the career ladders paths for product people. It's a big transition when you go from being an individual contributor PM to being a product leader. Product managers most of the time ends up learning to be very self-sufficient. And as I like to say, good PMs fill holes in their team, they have horror vacuum.
If there's no one around to do QA, who's going to do the QA? The product manager. If there's no one around to do wire frames or prototypes, who's going to do that? And so you get, you become very much like this solo superhero doing all the things and then you get promoted to becoming a leader and it can be hard to kind of let go, because you've been so trained, conditioned to do that.
I think the number one thing I guess is if you're making that transition, it's just like what we do is good product people like why are we doing this from how are we doing it? And again, back to my amazing boss, my amazing GM at Intuit, he would always set objectives. He would say, "Here's the objectives that we have. And it's up to you to figure out how we're going to do it." He wouldn't micromanage. He wouldn't jump in and say, "Here's how you're going to do it." Or that kind of thing.
Of course I would come back to him and explain, here's how I recommend we go after it and get feedback from him and things like that. So that's it, I just think be mindful of that. It can be tough, especially sometimes you're a player coach, you didn't divest all your individual contributor staff. You're probably still like the PM, the biggest team or the most complicated feature. And then that's 30% to 60% of your job. The remainder is managing the rest of the team.
I think that's one piece of advice. The other thing too is PM can be a very lonely job where we're also busy, we're dividing and conquering. So you're on a PM team, but it may not really feel like a team. I would say as the leader, try to do what you can to make it feel like a team like the things I was mentioning about my manager. Doing educational events once a month or every two months, doing fun events every couple of months.
One of the recent leaders that I coached at a startup, he was great at this. He was really good. Always thinking like quarterly planning, doing quarterly planning events with his team, doing offsites with his team, thinking about their training, thinking about their growth and development, planning out where they're going to move next, those kind of things. It's a whole new frontier of you're just an ICPM, none of that stuff is on your plate or on your radar, so just things to think about.

Holly:
Yeah, that's awesome. And where can people find you if they want to follow you, Dan?

Dan Olsen:
The main place is just my website, dan-olsen.com, D-A-N hyphen O-L-S-E-N.com. There are links there to my talks and the book and the meetup and things like that. The meetup is meetup.com/lean-product. I also have a YouTube channel, we just crossed 9,000 subscribers. So it's just youtube.com/danolsen. My talks are there, but most of the talks there are of all the speakers that I host at lean product meetup, so all the top speakers. Those are some, and LinkedIn and Twitter.

Holly:
Fantastic. Awesome. Well, thank you so much. It's been such a pleasure to talk to you, Dan.

Dan Olsen:
Thanks, Holly. Great talking with you.

Holly:
The Product Science podcast is brought to you by H2R Product Science. We teach startup founders and product leaders how to use the product science method to discover the strongest product opportunities and lay the foundations for high growth products, teams and businesses. Learn more at h2rproductscience.com. Enjoying this episode, don't forget to subscribe so you don't miss next week's episode. I also encourage you to visit us at productsciencepodcast.com to sign up for more information and resources from me and our guests. If you like the show, a rating and review would be greatly appreciated. Thank you.

‍