The Amy Jo Kim Hypothesis: Drive Deep Product Engagement by Optimizing the Core Loop with Game Thinking

Named by Fortune as one of the top 10 influential women in games, Amy Jo Kim is a game designer, community architect, and innovation coach. Her design credits include Rock Band, The Sims, eBay, Netflix, nytimes.com, Ultima Online, Covet Fashion, and Happify.

Amy Jo has helped thousands of entrepreneurs & innovators bring their ideas to life through her coaching programs at gamethinking.io. She pioneered the practice of applying game design to digital services and is well-known for her books Community Building on the Web (2000) and Game Thinking (2018).

In addition to her coaching practice, Amy Jo teaches Game Thinking at Stanford University and the USC School of Cinematic Arts, where she co-founded the game design program. She holds a PhD in Behavioral Neuroscience from the University of Washington and a BA in Experimental Psychology from UCSD.

This week on the Product Science Podcast, H2R Product Science Founder and CEO Holly Hester-Reilly sits down with Amy Jo to get into how ideas from game design can translate into product development, and what separates the successful projects she’s worked on from those that didn’t get off the ground.

Subscribe for the full episode on Apple, Google Play, Spotify, Stitcher, and more. Love what you hear? Leave us a review, it means a lot.

Resources

• Game Thinking
• Gschool
• Become a certified Game Thinking coach
• Community Building on the Web by Amy Jo Kim
• Will Wright teaches game design and theory
• Connect with Amy Jo on LinkedIn
• Follow Amy on Twitter
• Follow Holly on Twitter
• Follow Holly on LinkedIn

Questions We Explore in This Episode

How did Amy Jo put herself through graduate school with coding? Why did she move into tech with a degree in behavioral neuroscience? How did she transition from programming into game design? What kind of projects did she work on at Paramount and Viacom? What did Amy Jo do with her specialty in multiplayer social systems? What kind of work did she do on games like the Sims, Rock Band, and Covet?

How was the New York Times’s first paywall a multiplayer social experience? What is Game Thinking Methodology? What is the difference between a gamelike experience and gamification? What kind of customer research did they conduct to design the NYT paywall? How do you track users’ social experiences and reward prosocial behavior? Why didn’t gamification work for this project, and how did Amy Jo convince the stakeholders of that? How does she walk key decision makers through research at large companies? Why are videos of customers saying your conclusions so convincing?

What does discovery look like in the game industry? What is the most common way for games to get greenlit? What’s an example of a game that was held up in this process, and how did it finally get made? What is the Superfan Funnel and how do you use it to identify leading edge customers? What is a concept scenario and how do you use it to validate your ideas with customers? What does Amy Jo mean when she calls the core loop “finding the fun?”

What is a common mistake teams make with onboarding and discovery? Why do you need a core learning loop that engages users and changes over time? How does this create sustained engagement? What is the Mastery Path Model and what are its four stages? What is discovery? What is onboarding? What is habit building and how is it different than onboarding? What is mastery and what does it look like? How does League of Legends manage toxic behavior, and how does it illustrate these stages?

What was it like working with Will Wright on the Sims? What does he do at the earliest stages of a project? How does he use throwaway prototypes? What work did she do when the project got to beta? How did she use the Sims website to make that single player game into a multiplayer experience? What was it like being the interview for Will Wright’s Masterclass course? What is Amy Jo’s biggest piece of advice for product designers and product managers early on in their career? What is Gschool and how can it help you?

Quotes From This Episode

"There is nothing as convincing as short video clips of real customers saying the key messages that you're trying to communicate to your stakeholders."

"Don't separate testing and design. The best designers are aggressive about finding out what's wrong and what's right with their idea as early as possible."

"Start with the core loop, find the fun, we call it in gaming. Which is, what is that core activity that people are going to do over and over and over again? You need to make sure that works, you need to make sure that's fun."

Transcript

Holly Hester-Reilly:
Hi and welcome to The Product Science Podcast, where we're helping startup founders and product leaders build high-growth products, teams, and companies through real conversations with people who have tried it and aren't afraid to show lessons learned from their failures along the way. I'm your host, I'm your host Holly Hester-Reilly, founder and CEO of H2R Product Science.
| --- ---
This week on The Product Science Podcast, I'm excited to share conversation with Amy Jo Kim. Amy Jo Kim was named by Fortune as one of the top 10 influential women in games. She's a game designer, community architect, and innovation coach. Her design credits include Rock Band, The Sims, eBay, Netflix, thenewyorktimes.com, Ultima Online, Covet Fashion, and Happify. She's helped thousands of entrepreneurs and innovators. She's been an author more than once. She's got a book, Community building on the Web and Game Thinking. And she also teaches game thinking at Stanford University and the USC School of Cinematic Arts, where she co-founded the game design program. And not only all of that, but she also holds a PhD in Behavioral Neuroscience. So welcome.

Amy Jo Kim:
It's a pleasure to be here.

Holly Hester-Reilly:
I'm so excited. I have a pretty conversational style. I love to just kind of hear from people a bit about how they got started. You've been doing this a long time, I'm sure yours is a long story. But how did you first move from behavioral neuroscience into game side?

Amy Jo Kim:
So I put myself through graduate school by programming. I was initially a self-taught programmer, but then I took a number of graduate level courses. When I got out of graduate school, I got a job at Sun Microsystems when they were getting started. So it was really the dawn of the internet or at least the internet that most people were using. I started working in tech as a programmer, then as a designer, then as a producer and creative director. I kind of worked my way up. After Sun, I worked for Paramount and Viacom, in their advanced product design lab. That's really when I got into game design.
I was working with top brands like Nickelodeon and MTV and Star Trek and Entertainment Tonight on taking their brand and translating it into the internet, which was still pretty new then. Building websites, building multiplayer experiences. And I started going to the Game Developers Conference when I was working at Paramount because I was the person that knew the most about the internet since I had worked at Sun. When I went to the Game Developers Conference initially just to kind of check it and see if Paramount wanted to get involved, I really felt like I had found my people. People were really weird. Some of them were a little greasy and didn't smell too good, but they were so creative and imaginative, while also being intensely analytical and smart. And it was that combination that just drew me and made me feel like, “Ah, these are my people.”
So I did some game work for Paramount, particularly I did some prototyping for MTV. And then when they moved our lab to New York from California, I elected to stay in California where I live, I started my own business. And I did community design, game design, product design, all different kinds of things. But my specialty was always multiplayer social systems. That's what I worked on at Paramount, that's actually what I worked on at Sun. And so my career in gaming is really largely working on multiplayer games and building out the social systems as well as having some creative direction on the overall game.
The move is really a move into tech. With my behavioral neuroscience background, I decided I wanted to have more freedom and flexibility than I would if I got a postdoc and worked at a university.

Holly Hester-Reilly:
Yup.

Amy Jo Kim:
But I've also had ties to university. I co-founded the game design program at USC with my friends Tracy Fullerton and Scott Fisher. I co-taught the first game design classes there with Tracy, and then I've also taught game thinking at Stanford. But I am largely a practitioner versus an academic, but I love to teach as well as to do. So that's been really where my career has developed.
I was a contributor and system designer on the games that you mentioned earlier; The Sims and Rock Band, and Covet. I helped the New York Times build their first pay wall and the experience beyond their pay wall when they were turning their newspapers into a subscription, which again was a multiplayer social experience for them, especially in how we dealt with comments.

Holly Hester-Reilly:
I want to hear about all these things, but let's dive into that one actually because I feel like that's one that's particularly interesting because it's not a game itself. Tell me more about that.

Amy Jo Kim:
When the New York Times was first creating their subscription service, they worked with experts such as myself to figure out what the service should be and how they should run it. Now, it seems like the olden days because so many subscription services exist, particularly for news media. But it was one of the first time that a major newspaper was doing that.
We did a combination of things. The game thinking methodology, which you can read about in the Game Thinking book is absolutely for non-games as well as games. It's for any creator who wants to create a game-like experience. Not gamified, but game-like. And we interviewed a lot of their best customers. We talked to people that have considered a subscription, we talked to people that had other subscriptions to really understand the landscape and understand the consumer mindset. We modeled out how we would reward people for their subscription. If they were a subscriber for one month or six months, what would that look like? What would be different if you were a subscriber for six months than one month? Those sorts of questions. But you're pretty basic to designing long-term engagement.
We looked at some gamified solutions where we had levels and points. We put those aside. We didn't want to do that. We looked at what powers and impact people could have. And one of the things we worked the most on, and again this was many years ago at this point, was... One of the things we worked a lot on was comments and if people could earn the right to get their comments published or more likely to be published. We toyed around with introducing sort of earned badges next to people's name. So if they had been a commoner who had gotten a lot of good feedback from others, they would get more cloud and visibility. That was the project that actually went forward was integrating the subscriber experience into the social experience of commenting and giving other people feedback.
And that is a long-standing tradition in the internet communities of all kinds is you create some sort of social engagement, whether it be message boards or chat or channels in an online game like World of Warcraft or whatever. You create social experiences and then track to make sure that people are behaving well. And if people are being pro-social in some way, you give them more power, privileges, and visibility. If people are being antisocial, you curb them and give them less powers and less visibility. So that dynamic is common, anybody who designs multiplayer social systems whether it's a game or not. And we absolutely brought that to the New York Times. But a lot of the work as well was considering and really analyzing all the ideas that people had.
Gamification was very big then and some people really wanted to do something that looked very gamified, so we analyzed it and we tested it and we really looked at it, decided not to. A lot of work that goes into products that ship falls on the cutting room floor and you consider things and you go, “Oh, I see, that actually isn't that good an idea.” Which is a great thing to find out before something ships than after.

Holly Hester-Reilly:
Absolutely. One of the things that we talked about sometimes here is how we convince people of that, because sometimes people fall in love with their ideas and there may be a high-level executive who supports a particular perspective such as gamifying something and you have to convince them that maybe that's not the right approach. How did you go about doing that? Were there challenges in convincing people not to go to gamification route?

Amy Jo Kim:
There were absolutely challenges. On that project, the way we did it was in layers. We started by carving out a squad of people internally, which included some higher-level, director-level people to work through all the ideas. So we worked through those ideas. That wasn't always easy, but we worked them through. And then we collected our analysis and our findings, including feedback from customers into a brief, a product brief, which is something that I do a lot on projects. And we have a very specific way of doing a product brief, which includes short interview clips from the interviews and play test that we do with customers. And so that product brief, we presented to the leadership at The New York Times.
There's a group of people, I think they called it the G9 or the G7 as a reference to the G8. But the leaders of The New York Times, including the very head, have to approve all major initiative. So at the end of that project, we went up to the top floor in this very tall building in Manhattan and presented our product brief and our findings to these people, and they did not crack a smile. It was so nerve-wracking. But it worked out pretty well. We got started with one of the initiatives that we presented because you got to start somewhere. And that's often how it goes, especially with larger companies, more established companies.
We do work with a smaller group. We make sure there's a stakeholder involved in the smaller group at least for weekly check-ins so that somebody is championing us and understands what we're doing. And then there's a presentation at the end, which where we summarize everything we learned, how we learned it, how we collected the data, how many people we talked to, and what we found. And what I've discovered is that, that's all great, that works really well. But there is nothing as convincing as short video clips of real customers saying the key messages that you're trying to communicate to your stakeholders. You can't just have that, but if you have a very well-organized presentation that let's people get inside and understand your process, how you got the data, and then you show them their customers giving feedback on these ideas, that's very convincing.
I have worked with many game companies that got games that have been struggling for months to get greenlit, greenlit meaning they get budget to be developed by using this methodology. And it isn't just a convincing, it isn't just it's a nice convincing presentation, it's that we've actually validated ideas with real people and found out what's going to work and what's going to be a dud before building anything. And it's that validation process and then the expression of it that I have found so compelling for working with stakeholders.

Holly Hester-Reilly:
Yeah. Absolutely. Yeah, we talked to a lot of people about validation in product discovery and moving things forward, but I'm curious because I'm not as familiar with the game industry, what does validation typically look like in the game industry? I mean, I've certainly heard about play testing, but tell us more about what's normal there.

Amy Jo Kim:
When you say what's normal in the game industry, first of all, it's all over the place. And part of why that is, is because it's like what's normal in the movie industry. When you think about movies, there's action films, there's little indie films, there's Ingmar Bergman, depressing Northern European films, there's big action films, there's sci-fi, there are so many kinds of films, right? And think about, how do films get greenlit? How do films get made? Well, there's lots of different ways, from the big Studio process to equivalent of angel investors. In gaming, there's many different ways that games get greenlit, but I'll tell you about one of the most common.

Holly Hester-Reilly:
Okay.

Amy Jo Kim:
In many gaming studios I've worked with, gaming studios will have several projects going at once. The bigger the studio, the more project. And that discovery is when you're like, “Is this a good idea? Should we make this game?” So there's a point in most games lives where there's some sort of demo or some sort of pitch. And that pitch could be static screens, it could be an actual demo of gameplay, it could be a movie that shows how the gameplay might be. There's a lot of ways to do a demo. But a game pitch to get greenlit is going to be early in the game's life, it's some idea of, here's a game we want to do, here's how it would play, here's who it appeals to, here's why we think we should make it. Okay? Pretty basic.
Games will get greenlit, but they'll get greenlit for the next phase. So like, “Okay, we're going to green light this to build a prototype.” That would be an example. And there's milestones. First playable is a very common milestone. It's like an early alpha version. It's something that's actually playable, it's got not all the features, but some of them. And then each milestone will be another green light opportunity because games get canceled all the time when they first playable, “This isn't so good. Uh-oh our competitor just released something almost identical, we're not going to go that direction.” There's different reasons games get canceled or put on hold. But that green light to build the first playable, to build the prototype, to build something is I've been part of that multiple times so I know it inside and out.
Let me give you an example of a game that we worked on that wasn't greenlit and then was greenlit, and I'll tell you what happened. This is like stories from the front lines.

Holly Hester-Reilly:
Yeah, please.

Amy Jo Kim:
There's a game team at a casual gaming company that have been working on idea they were excited about for about six months. And they worked up some art, they worked up some gameplay ideas, they presented it to the heads of the studio, and they said, “This is not innovative enough. This looks fun, but everything in it feels like a copycat.” And the studio was trying to get away from copycat games because they had actually make quite a few of them and they wanted to get into the value you get from original IP. So doing something really original.
So they kept getting dinged because the game wasn't innovative, and that's when I started working with me. So we dug into the genre and to the customers, and we talked to probably 30 very targeted customers. I have a process I call “the superfan funnel” and that's where you find not just average customers, but leading edge customers who are the canary in a coal mine for the next wave of what's going to happen. You can read all about that in my book, how to do it step-by-step. But we found about 30 hot core early customers who are excited about the genre we were developing in. Some of them we interviewed, but then we put together sketch playtest, not fancy art, but the sketches of the experience we wanted to deliver from day zero to day 60. I call those concept scenarios. And it's kind of like a movie storyboard, but it's for a game and it takes out all the fancy art that can wow people and really shows you the experience over time; what's different on day 7, day 30, day 60, than it was on day 1? How is your experience different? What are you doing? What activities are you engaged in as you level up through the game?
So that's what we brought to life, and we were able to really shape that gameplay by listening to the customers and understanding what they wanted that wasn't already in the market. So it's finding that blue ocean in a crowded red ocean market that the game thinking methodology, and particularly the superfan funnel really surfaces. So we interviewed these customers, we play tested probably 18 out of the 30, we play tested them with the sketches of the 60-day experience, refined it based on their feedback, tested it again, went to an iterative cycle, captured video clips of their reactions, and then put all that into our presentation. And we went to the board of this company, the head of it and everybody involved and did a presentation saying, “Here's the game we thought we wanted to build.” Then we went and talked to a bunch of people and we saw that there's this particular kind of activity that they long for, that they are already trying to kind of mock-up with spit and baling wire using other stuff. And that's a really common thing to look for, are there people out there trying to make this happen in some way even though it doesn't exist?
We presented that and said, “Here's this blue ocean opportunity, here's why, here's what people are already doing, here's this competitor but they're not the same, here's this competitor but they're not the same. In between them is this interesting opportunity, that's where we want to go. Here's the game design, here's the rollout plan.” Like we had ... “Here's what it is.” And it got greenlit and the team was ecstatic.

Holly Hester-Reilly:
Yeah. That sounds really exciting.

Amy Jo Kim:
And so a lot of why it was greenlit is it was more innovative, but much more importantly than just innovative, it was meeting a validated market need. And that's the thing that pushed it over the edge.

Holly Hester-Reilly:
Yeah. There's a lot in there that's very similar to I guess Lean Startup principles, right? And Agile and Lean UX.

Amy Jo Kim:
Lean Startup principles are great for somebody... Lean Startup came out of the writings of a guy who had worked for bigger companies that were doing waterfall style development. And it came, it's like, “Wow! You could test something early. You could validate something early.” Those of us in game design have been doing this for decades.

Holly Hester-Reilly:
Oh, yeah.

Amy Jo Kim:
On the Sims, we validated it for a freaking year before we started building. Lean Startup is great. It's very similar to Lean Startup. A lot of people in gaming, when Lean Startup came out there like, “Yeah, we've been doing that for a while.” That said, a lot of people in gaming don't do it. There are plenty of people in gaming that subscribe or came up during the [inaudible] model of gaming where there is a genius who's got an idea and you build the idea. And you see that in tech, you see that in gaming. Every once in a while that works out, but mostly it doesn't. Like if you go and talk to Sid Meier whose one of the great, great game designers of all time. He did Civilization and all the Civilization.

Holly Hester-Reilly:
I was going to say, I was literally playing Civilization just last night.

Amy Jo Kim:
If you talk to Sid Meier about his process, so much of it is similar to what I advocate and what I teach people. You know, start with the core loop, find the fun, we call it in gaming. Which is, what is that core activity that people are going to do over and over and over again? You need to make sure that works, you need to make sure that's fun. That's the lesson that applies to everybody.
We bought that lesson to The New York Times, going back to that story, when we were sitting around the table struggling with which model are you going to go with. One of the things we kept coming back to is, okay let's suppose subscribers come back and they are three months into their subscription, it's day 90, what does the session look like? What are they doing? They come in, they read the paper, are they being completionist and reading every article? Are they just wanting be directed at the articles they like? Does the behind-the-scenes system get smarter over time, so that on day 90, it knows so much more about them than on day one? What's going on? Right? And a lot of that can be improved by always remembering what your core learning loop is. What is it that people are doing, and how does that change over time?
That's been one of the biggest time savers and also the biggest ahas for the people I work with in tech outside of gaming. Because so many people will build beautiful onboarding, and I love onboarding. I've redesigned the onboarding for dozens of companies. Onboarding is incredibly important. But if there's nothing to do on day 30, onboarding isn't going to get you engagement. So a lot of people make the mistake of focusing on discovery and onboarding, and they don't really have a they're there. They don't really have an interesting activity to do overtime. And onboarding can be done anytime. You can make beautiful onboarding right before you launch, you can do onboarding almost any time because onboarding is an arc, you go through it whereas, your daily activity experience is a loop. It's a loop that you go through again and again. And if you don't get that right, the most beautiful onboarding in the world isn't going to save you.

Holly Hester-Reilly:
I hear what you're saying, it sounds like the beginning of the mastery path that you sometimes [crosstalk], right?

Amy Jo Kim:
That's right.

Holly Hester-Reilly:
Can you tell our listeners more about that?

Amy Jo Kim:
Sure. So many of us in tech are familiar with different models for engaging customers. There's the aarg model or the pirate model, aarg, where it's like acquisition et cetera, et cetera. There's a lot of different models.
The model that we find incredibly useful that came out of gaming but it applies to everyone is the mastery path model. And the four stages are discovery, onboarding, habit-building, and mastery.
Discovery is the same as it is everywhere, which is before somebody has signed up, registered, purchased, what is it that they experienced? What's the discovery experience? How do they find out about the product or game? So that's discovery.
Onboarding we understand, onboarding is learning the ropes. Once somebody's gotten involved, how do you teach them the ropes? How quickly can you get them some value going on? When do they start interacting with other people, et cetera. All of that is onboarding.
Habit-building is different than onboarding. A lot of people get them confused. Habit-building is once someone's learn the ropes, they're not learning the ropes anymore, what is it that they do regularly? What's the re-engagement trigger? What is it that pulls them back? And that's habit-building. Like, what are they doing on day 21? That's a good question to ask when you're looking at habit-building.
Mastery is more of a vector than a stage or a state, but mastery comes post habit-building. If habit-building is what you offer for regulars, people that become regulars in your product, mastery is what enthusiast get or experts. So what do you have to offer people on day 90? What do you have to offer people once they've mastered the basics systems? Going back to The New York Times example. In the New York Times, mastery might look like earning the right to have your comments float to the top in The New York Times comments section, or even earning the right to act as a moderator in some sort of social environment would be a mastery thing. Or somebody has become such a good customer and they've gotten so involved and not been a bad actor that they get to have a new kind of experience, a new activity, a new power or impact on the environment.
A really good example of this is League of Legends. League of Legends is a very popular and somewhat controversial multiplayer game. League of Legends is known for having a lot of toxic behavior that happens, and it's very competitive game. The staff has implemented a lot of very creative systems to manage that. And managing toxic behavior in online games is hard. It's like an air bubble under a rug, you think you've got it and then it pops up somewhere else. These people are smart.
One of the things League of Legends did that's a perfect example of going through these stages is in order to help them manage the online behavior and deal with consequences and evaluating the appropriate consequences for people that had committed [inaudible], they started conscripting some of their players to be judges. They called it, The Tribunal. And so players that had not had any bad actor complains against them and had played for a while, I think it was two months, I forget how long it was exactly, were invited to be part of The Tribunal. And if you were in The Tribunal, you got some training and you were essentially first line customer support. And you would get cases assigned to you, minor cases and you would evaluate, does this person need to be booted or banned? Do they just need to be muted? Do they need a 24-hour time out? Like, what's the appropriate consequence for their behavior?
For much more serious ones, they would have their hired customer support. It's essentially a metagame. You can think of it as what we call in gaming, [The Elder Game], which is a new kind of activity that's fundamentally different than what you've been doing so far as you grind your way up the ladder. It's a new kind of activity that opens up to the most loyal players. That's mastery.

Holly Hester-Reilly:
Yeah.

Amy Jo Kim:
And so they instituted this system. I don't know if it's still there. This was a few years ago. And they did something else really smart, which is people cycle through. I think you could be a judge for a few months and then you had to take a break. Really smart because people can get power-hungry and they can get into clicks and helping their friends and all that kind of stuff. So they made sure there were some healthy changing of the guards, so that slots would open up for other people. And what a genius move because if another player is talking to you about your bad behavior, it's really different than an official customer support person. So that's an example of what you can do for mastery that is different than onboarding or habit-building. And that serves your players and also serves the product.

Holly Hester-Reilly:
Yeah. That's a really helpful story. I personally really want to hear a bit about working on The Sims because it's one of my favorite games for a really long time, and it's one of the games for those of us, I feel like, maybe in the industry you might know a lot of game designers' names, but there's a few game designers that stand out sort of as household names outside of that, and Will Wright is one of them. And so I'm curious to hear what your experience was there. And did you get to work with him directly? And what was that like?

Amy Jo Kim:
Yes, I worked with Will really closely on a number of projects, including The Sims. The Sims was amazing. At the very beginning, Will and Bing Gordon and I and a few other people were imagining it, so visioning, right?

Holly Hester-Reilly:
Mm-hmm (affirmative).

Amy Jo Kim:
And that happens at the beginning of every project. There's some form of it. With Will, visioning always involves tinkering and prototyping. So Will and his small team built multiple throwaway prototypes, multiple. This went on and on for months and months. And what we were trying to do is just what I was talking about earlier, find the core loop, find the fun. Is the core activity this? Is the correct activity that? We were experimenting and trying all these different things. Talking about Lean Startup, right? This is exactly the kind of thing they would recommend except with Lean Startup, they recommend a lot of times testing your marketing rather than the core loop.
As a side note, if you're testing a fake landing page, you're testing your marketing message, which is great. But you're testing discovery. When you're testing like Will did, you're testing habit-building.

Holly Hester-Reilly:
Right.

Amy Jo Kim:
Right? And it's much harder to get that right. So we did a long time of all these experiments, and then when one of the experiments was really starting to gel with the interactive. Then started to add some more functionality to it, built it up and do a prototype, got a greenlit, worked on it some more. I dipped in and out of that project. I was part of it at the very beginning, the visioning, and looking at and reacting to those small prototypes. Then I got very involved again as we were heading into beta and doing, one of the things I worked a lot on was onboarding actually, getting the onboarding smooth out. And then some of the just the core interactive systems. And then when we were getting at the end of beta, when we were getting ready to ship, and then post ship, I worked really hard on the website which made The Sims single-player experience into a multiplayer experience.
So I had worked before on SimCity on the SimCity website, creating a social and rating systems because that's my specialty or was at the time. So on SimCity, we had had a system where people could upload their city to the website to show it off and they could upload objects, blah, blah, blah. For The Sims, we let people upload a series of screenshots they had taken in the game and tell a story with it. And then we let other people read and rate those stories. Perhaps you remember this on the website.

Holly Hester-Reilly:
I do remember this. I definitely [crosstalk].

Amy Jo Kim:
That was my system. So myself and my team I was working with designed that, tested it, tuned it, built it. So that was the other thing that we worked on after The Sims. It's natural that I would work on that given that I'm a multiplayer designer, and that makes it multiplayer. It exploded. That feature absolutely exploded. The stories people were telling were unbelievable.

Holly Hester-Reilly:
Yeah.

Amy Jo Kim:
They were about their personal trauma and being adopted and moving around the world and being a foster kid. Like, stories people were telling were unbelievable. And the thing that was crazy was the feature, to take a snapshot in your game automatically with one-click got added near the end of beta. It almost didn't make it into the game for a bunch of reasons. But that feature enabled, that one little feature enabled the storytelling. The website enabled the storytelling to be shared with the world. And it was a huge part of what made The Sims so compelling socially and culturally.

Holly Hester-Reilly:
Yeah.

Amy Jo Kim:
And then I worked with Will later on other games. And I was also, I helped produce Will's masterclass. So I want to make sure that all of your listeners know that the great, brilliant, amazing Will Wright who I adore and I learned so much from has a masterclass, part of masterclass.com. It was produced last summer and I was the onset interviewer. They hired me to help bring it to life. So I watched the whole thing come to life.
I've been working in gaming for a couple of decades. I've worked with Will on three different projects. I learned a ton just sitting there listening to him. I cannot encourage you enough, go check out Will Wright's game design masterclass, if you want to take what I just told you times a million.

Holly Hester-Reilly:
That sounds amazing.

Amy Jo Kim:
It's so good.

Holly Hester-Reilly:
Yeah, that sounds so good.

Amy Jo Kim:
Yeah.

Holly Hester-Reilly:
Awesome. Well, I know we're running out of time so let me switch gears and ask, I always like to ask my guest if they have one piece of advice for someone who's earlier on in their career, what would you tell them?

Amy Jo Kim:
Don't separate testing and design. Try as much as you can to put those two things together. One of the biggest learnings I have from my years working on hundreds of projects and watching some of those projects turn into massive hits and watching many of them fail, is that the best designers are aggressive about finding out what's wrong with their idea as well as what's right with it as early as possible. People that are precious about their ideas, people that just want to breathe life into the idea and love it and show their friends and get money and just like they're so in love with their idea are not the best entrepreneurs.
The way to become an excellent product manager, product designer, entrepreneur, innovator is to be relentless about testing with the right people as early as possible and be joyous when you discover the things that are wrong with your idea as well as right within that. If I had learned that earlier and been less defensive about my ideas, I think I would have gotten even further in my career. And I got pretty far. And now, as a very, very experienced game and product designer, I trust my own ideas even less than I did 20 years ago. Because I know it's not so important what I think, it's what the customers we're designing for think.

Holly Hester-Reilly:
That's right.

Amy Jo Kim:
And one of the biggest problems I see in companies, and I help companies get around this, is siloing research and design. Having the game team do the design and then once it's in beta, having research do user interface testing. That doesn't work very well. What works way better is to put those teams together and really have your customer insights team or just you, if you're a smaller group or team do concept testing early, early, early like I described with those storyboards overtime. The amount of time and money you'll save is unbelievable. And people that have the skill, the skill to put design and testing together have a superpower that's going to get them jobs, prestige, and most of all, successful shipped products.

Holly Hester-Reilly:
That's fantastic. You can't see it, but I'm grinning from ear-to-ear. That's so valuable advice. I really appreciate it.
I know you have to go, but the last thing I wanted to ask you is just I know you have a membership community for rising product leaders called Gschool. Do you want to tell us a little bit about that?

Amy Jo Kim:
I would love to. Gschool is a monthly membership that includes an unbelievable amount of value. It includes our entire course library, The Innovation Sprint, which covers game thinking, cooperative design, community design, progression design, how to build concept scenarios like I told you about, how to build a mastery path teardown so you can do awesome competitive analysis. All of that is included along with expert coaching and one-on-one onboarding with a certified game thinking coach. We run a certification program as well and we've got now 24 certified game thinking coaches from all over the world, and they are part of Gschool.
So for anyone listening, you can go to gamethinking.io/gschool to learn more about Gschool. You can also just go to gschool.io. That's a quick URL. We will be opening it to the public in October. And right now, it's open for early bird signups, which will close at the beginning of September. So any of you cutting-edge people, check it out now. Folks who are a not quite so cutting-edge, October is when our big public launch is going to be.
And I also want to mention that if certification sounds interesting to you, go to gamethinking.io/certification. Our next cohort will be launching in October. It is application only. We don't take everybody. But for any startup CEO or coach or trainer or product manager who wants to dramatically level up your skillset and bring it to your entire organization, our certification program, which is a 6-month program, is awesome. I mean, the glowing reviews speak for themselves. However, Gschool delivers much of what you get there at a slower pace, where you can move at your own pace and really dig in and learn game thinking and level up your skillset at a ridiculously low price. So check it out.

Holly Hester-Reilly:
Awesome. Thank you so much. I really appreciate it. It's been fantastic to talk to you and I hope you have a great day.

Amy Jo Kim:
Wonderful to talk to you as well. Thanks for having me on.

Holly Hester-Reilly:
That was Amy Jo Kim, founder and CEO of Game Thinking Academy. You can find Amy Jo at gamethinking.io. You can also find her on LinkedIn under Amy Jo Kim or on Twitter as well with Amy Jo Kim.
Product Science Podcast is brought to you by H2R Product Science. We teach startup founders and products leaders how to use the product science method to discover the strongest product opportunities and lay the foundations for high-growth products, teams, and businesses. Learn more at h2rproductscience.com.
Enjoying this episode? Don't forget to subscribe so you don't miss next week's episode. I also encourage you to visit us at productsciencepodcast.cam to sign up for more information and resources from me and our guests. If you love the show, a rating or review would be greatly appreciated. Thank you.

‍