- Chapter 1 – The Potential of Flash: The great potential of Flash as a platform and the big question: Why are there so few great Flash games?
- Chapter 2 – Making money: How do Flash developers currently make money.
In this post I’ll cover:
- Chapter 3 – Generating value: How Flash developers currently create ‘valuable’ game for their players?
Chapter 3 – Generating Value
- The problem with short form games
- A new definition of value
- The game mechanics of retention
The problem with short form games
- Players fall in love with the portal: Players start thinking of Addicting Games or Newgrounds as a go to source of entertainment, not NinjaKiwi or Sean Cooper.
- Little long term love for the game: Games are treated as disposable moments in the broader experience of wasting an evening surfing a game portal. Some may provide brief burst of joy, but this just reinforces the appeal of the portal.
- Dominant aggregators exercise editorial control. The terminology is ‘portfolio management’ or ‘selecting titles that match our audience’. The effect is the same. Dominant aggregators often apply effective pressure to developers to make what the aggregators desire and in turn disconnect developers from the real needs of the customers. Though well intentioned, editorial efforts typically results in a reduction of consumer choice, an elimination of innovative outliers and a suppression of disruptive business models. Currently Flash portals are quite open, but these behaviors are beginning to creep into practices of some like Addicting Games and MiniClip.
- Lack of trust in the game developer: When the developer asks directly for money, the customers runs away. It is like the clown at the circus asking you to pay after you already paid an entrance fee. The customer doesn’t know the clown is starving. They naturally assume that they are just part of the show. Clowns asking for money = creepy; Flash game developers asking for money = creepy.
Note on payment services and trust
The concept of paying for Flash games is still new to players. Payment services like HeyZap or MochiCoins see this as one of the major issues to creating microtransaction games. They attempt to solve the problem of developer trust by creating heavily branded and marketed payment services. The implicit message is “You don’t trust the starving clowns, but you can trust us!”
- Game ratings on portals: Players on a particular portal rate the game usually on a scale of 1 to 5 stars. Highly rated games are given more traffic by the portals. With this particular rating system, games with overly long introductions that deliver value late in the play session are at risk of being bailed on by easily bored players. Inevitably these players rate games with a 0. This creates a natural incentive to deliver as much easy value as possible in as short a time as possible. It ends up being cheaper to produce a 3-minute ‘complete’ experience that earns a 5-rating than it is to create a 60-minutes experience that earns the same rating.
- Number of ‘plays’: The other metric developers care about is how many they serve. This metric over emphasizes the importance trial players who click the link, but don’t play the game. The metric spikes up when your game spread throughout the various portals and drops off rapidly there after. Again, there is no incentive to make games with depth. Instead you want a new title with a catchy intro that gets people watching that ad. Putting effort into anything longer doesn’t improve your numbers.
- Weekly and Daily Top 10 lists: Portals put up list that highlight the best new content for the week or day. These acts as a means of letting games bask in the public gaze and are highly coveted both for their traffic and their implied status. However, games quickly fall off these lists and the only way to get back on is by releasing a new game. This encourages developers to release often in order to get as many shots at the spotlight as possible.
Many Flash gamers like web games because they can pop in for a short play session, have a bit of fun and then leave. It is tempted to assume that short play sessions demand short form games that can be completed in a few minutes. This is not the case.
- Fun: Are players having fun? Do they love your game?
- Retention: Are players sticking around and coming back for more?
- Money: Are players willing to pay you for your game?
If you build a game where you can objectively answer “Hell, yes!” to all those questions, you’ve got a game that will pay the bills and delight your players.
- Build metrics into your game that measure Fun, Retention, and Money.
- Gather accurate data from statistically valid samples of actual players.
- Use the information you gather to inform the design of new features.
- Use the information you gather to determine if your new features were successful.
- The player is randomly served the survey during 2 minute intervals. So one player may get the survey at 2 minutes in. Another might get it at 4 minutes. And so forth. Each player gets the survey once.
- Record the player’s answer to the question “How much fun was this game (1 = Not fun, 5 = Very Fun)” This takes only a few seconds and can usually be easily worked into the context of the game.
- An optional step at this point is to ask an open ended question “What don’t you like about this game so far?”.
- Average the ratings for each point in time and the graph the results as a line graph. By using a running average of a few days or a week, you’ll avoid having your results being swamped by old data from old versions of the game.
- By looking at the graph, you’ll easily identify the points in time when players find the game to be enjoyable. It tells you if you need to improve the intro, the body of the game, etc.
- If you are extra smart, you can show the comments for the point in time where your fun rating dips. This gives you qualitative data to help you diagnose why your scores are dropping at that point in time.
- Persist your customer identity.
- Record the percentage of users that return at various intervals (5 minutes, 1 hour, 1 day, 1 week, 2 weeks, 4 weeks, 8 weeks.)
Target: Aim for a weekly retention % over 20%. A good rule of thumb is that player need to play for two weeks before they make a purchase.
- Just take the amount of money you’ve gotten so far and divide it by the number of unique users who have come into the system.
- Release your game to users on a portal. It doesn’t need to be a big portal, but it should be capable of delivering a few hundred to a thousand views a day. Feel free to site lock the game if you worry about eventually selling a sponsorship for your game. If your game isn’t capable of driving even a few hundred views a day, go back to the drawing board and make a better game. For Bunni, we repeatedly put the game up on Newgrounds.com and took it down again.
- Measure the basic metrics mentioned above. This is your baseline.
- Make a change to your game that is targeted at improving one or more of the metrics.
- Measure again. Is the game better or worse? Ask why.
- Repeat steps 3 and 4 until the metrics of game are in a range that meets your target goals.
- Expand your test or kill the game: At this point, you can choose to release the game more broadly by launching it on more portals. Alternatively, a game with poor metrics that isn’t improving can be killed early in the process, freeing you up to climb more fertile creative hills.
Once you start practicing this process, you’ll notice a shift in how you design and build games. You’ve gone from designing in the dark to steering your game towards delivering value using the light of up-to-date, reliable information.
- Random sampling: We tried to avoid using self selected ratings which often are biased towards either those with very strong opinions or a niche portion of the population that enjoys rating things.
- Better defined question: We asked a standardized question that has been used on hundreds of games over the years. This let us compare the score to known baselines. Often portals offer a bar with a number that user can set. Who knows what criteria portal raters think they are offering an opinion on?
- Tie ratings to gameplay: We included time stamp information so we could tie ratings to particular moments in the game. Often there are specific points in time where players experience difficulty. Portal ratings tell none of this information.
The results were fascinating. 200 people had rated the game on Newgrounds. Yet only 40 people had actually played the game. Of the people who played, our average score was 4.22, a rather good number for any game. Interestingly, the player rating actually increased the longer people played the game suggesting our core gameplay was not merely initially fun, but fun for the long haul. Our players were falling in love.
- The core gameplay works quite well and doesn’t need to be changed.
- Something about the initial experience was turning off large numbers of users before they even played the game.
Of the hundreds of design option initially available to us, there was now one obvious feature that needed improvement. We focused on streamlining the sign-in experience so that we weren’t asking for as much personal information upfront. Mere hours later, we initially tested at 3.7 (and stabilized at 4.15) on Kongregate and eventually went on to score a 4.38 on Newgrounds.
- Over analyzing: Some designers worry that all the numbers remove the creativity from the game development process. Use common sense. If you are analyzing the correct color of blue, maybe you’ve gone too far.
- Lack of practice: It takes a bit of practice to learn how to use specific metrics. You need to recognize what is noise and what is a meaningful signal. You need to learn what a ‘good’ rating looks like. This takes time, setting baselines and experimenting.
- Out of date: You have to keep metrics up to date as the design changes. Stagnant or out of date metrics will not be used.
- Inability to dig deeper: Often developers will implement high level metrics and then not have enough flexibility to find out more once an issue is highlighted. At the very least have the ability to segment your stats based off time so you can see how your latest update affected your results.
- Treated as low priority: Developers put off integrating metrics since they don’t seem to contribute directly to the game play. This is dumb. You still turn your lights on before you go driving at night even though it takes you an addition 5 seconds to flip the switch.
- Good to great: If you have made a good game, metrics can help you polish it into a great game.
- Finding the important design levers: Rich feedback lets you quickly focus on changes that make a real difference. You can think of the various variables in your game as levers. Turn the right levers and your game will improve. However, time is limited and some levers have a much greater impact than others. Without metrics, developers turn levels willy-nilly, often making the game worse without knowing. The right metric help you identify the levers that really matter. They often aren’t what you think they are.
- Knowing when to kill a project: If you have a horrible game, metrics won’t turn it into a great game, but they will let you know that maybe you are polishing a turd.
Don’t fear the metrics. You still need to be just as creative and passionate as before, but now you’ve got this wonderful information rich environment that gives you immediate feedback. I think of it as painting in a well lit room versus painting in the dark.
- Flirting: Your game ranks high on fun for the first few minutes. However, weekly retention is close to 0%. Most portal Flash games fall into this category. Ads work well here, but you’ll give the vast majority of your revenue to the aggregators and middlemen.
- Dating; Your game ranks high on fun for the first hour or two. Weekly retention is still low, falling into the 1% range. However, a large percentage of people rate your game a 5. These players are willing to pay you directly. Monetize them by using a content or time gate to get them to pay a one time fee. Most downloadable casual games fall into this category. There are a small handful of Portal Flash games that reach the dating stage. Dating level games also give up the majority of their revenue to aggregators in middlemen.
- Married: Weekly retention is higher with over 20% returning each week. 5% return after after a month. Players have integrated the game into their lives and are willing to spend money on it like any other favored hobby. You’ll find individual players willing to spend hundreds or even thousands of dollars on your game. A long form game that has a larger number of married players is a business that can make good money for years. MMOs and Facebook games fit into this category. There are only a half dozen or so casual and Flash titles that are worth marrying. Games that you can marry are one of the few types of games that lead to long term developer independence and that limit the inexorable dominance of aggregators.
The relationship between fun and retention
- Narrative, story, and cut scenes exhibit “rapid burnout”. In other words, player see them one or twice and then are bored when they see them again. Games that rely on such content have generally low retention metrics. You can mitigate this by releasing new narrative content on a regular basic to keep the product ‘fresh’, but this has a high cumulative cost.
- Linear levels or solvable puzzles also exhibit rapid burnout. Game systems that can be completed or conquered are usually one shot activities. You can layer additional challenges within each level, but often only expert players will be motivated to come back for a second play through.
- Some handcrafted content like text or static images can be refreshed cheaply: The type of handcrafted content you include makes a huge difference on the slope of your increasing costs. New text-based questions in a trivia game are relatively cheap compared to creating new God of War levels. An hour of text-based content is likely several orders of magnitude cheaper to build.
- Social content is low burnout: People will keep interacting with their friends for years. Mechanics that can tap into this often have very high retention rates. Anything that allows players to chat, share and form social identities in a community is pure gold.
- Grinding results in burnout, but it slows the process. Techniques like leveling or purchasing upgrades can dramatically increase the length of the game for very little development and design costs. Think of grinding as method of stretching, but not adding to your content. Grinding techniques only delay the inevitable. They can result in lower fun scores as people feel obligated to play, but aren’t enjoying the process of playing. Since you want people to fall in love, such a reaction can be counter productive to your goals.
- User generated content systems are low burnout: User generated content is ultimately a social system that encourages users to create consumable puzzles. The puzzles themselves may be short lived, but the community of creators can thrive for decades. This solves the problem of the linearly increasing cost of more handcrafted content by apply large numbers of people working for free.
- Algorithmic content has low burnout, but is hard to create and balance: Evergreen mechanics like Bejeweled or random map generation in Nethack keep people playing for hours. However, they are tricky to invent and balance.
An example of a high retention game is one like Puzzle Pirates that has social (avatar, chat, guilds), grinding (levels) and evergreen algorithmic content (puzzles). There is some light narrative in the form of periodic events and very little in the form of conquerable level design. Most games have a mix of all these various types of content and successful services almost always put a portion of their reoccurring revenue towards a steady trickle of low marginal cost handcrafted content. However, a high retention game designs tend to emphasize content with less burnout.
- Are you ignoring bad metrics like portal ratings?
- Are you measuring the holy triumvirate of value: fun, retention, money?
- Are you collecting real customer data?
- Does your game score 4 out of 5 on the fun scale?
- Do players return after a week?
- Is your game design amendable to high retention play?
- Are you iterating on your game and improving your game as measured by internal metrics? Have you figured out the big levers that affect player experience?
- Do you know when you are done? Do you know when you’ve reached the point where your game has proven value to your players?
- Are you willing to bail on the game if it doesn’t show signs of improvement?
- Are you striving to be the primary provider of value to your customers?
- Integrating Google analytics into Flash: http://philprogramming.blogspot.com/
- Implementing Player Fun: http://philprogramming.blogspot.com/2009/08/using-google-events-to-track-fun-rating.html
- Retention in Flash games: https://web.archive.org/web/20090803115737/http://freetoplay.biz/2009/07/23/finding-fatal-flaws-lessons-from-kongregate-cc09/