As I coach new developers, I’ve taken to scribbling out the same useful diagram for visualizing the creative process again and again on coffee-ringed napkins. In order to limit my future abuse of culinary paper wares, I’ve reproduced my images in a more formal fashion in this essay.
The conversation usually starts with the following statement: “Creativity is like a snake swallowing a series of tennis balls.”
And when confused looks inevitably result, I sketch some variant of this odd little picture:
Using this as a starting point, we start chatting about joys and pitfalls of creativity.
- The Brainstorming Phase
- Failures in brainstorming
- The Culling Phase
- Failures in culling
- Failures in cycling
The Brainstorming Phase
We all start with an idea. It could be a small inspiration or a large insight. Immediately, you begin a process of brainstorming and daydreaming. This is a time of infinite possibility and promise. I use the term brainstorming broadly to include any activities that expand the options or possibilities of a project. The traditional image of a group of designer types sitting in a room with a whiteboard is indeed part of the brainstorming phase, but it is really only one of a much broader spectrum of activities.
There are several activities that occur during this phase:
- Ideas: Generate new ideas related to your initial insight. These are proto-experiments. A half-jotted note in a notebook “Solar powered underpants!” is an example of an idea. (This is a real idea. It has been hot in Seattle lately)
- Thought experiments: Invest energy in your ideas to understand how they might be built and thinking through the theoretical impact of each idea. A spec concerning “A Method for inserting a small fan in one’s shorts to reduce ambient temperature” is an example of a thought experiment.
- Real world experiments: Build working models from physical materials or usable code so you can experience your idea first hand. A working model of underpants plus a small metal fan (plus a ready supply of bandages) is an example of a real world experiment.
- Cross fertilization: As you work through ideas, you see new possibilities. I’ve discovered through much trial and error that escaping to a coffee shop with air conditioning is immensely more effective than other attempted alternatives.
Brainstorming is ultimately the act of kick starting experiments. Even when you dream up a completely off-the-wall idea, you are stating that “There is some potential in this direction.” You’ve formed a postulate at the fuzziest possible level. Future steps during the brainstorming involve making more detailed predictions and modeling the results.
When brainstorming is successful, we end up with a portfolio of experiments These go by a variety of different names in software development: features, user stories, use cases and ‘the thing Bob made while screwing around on the weekend.’
Problems with brainstorming
The single most common flaw during the brainstorming period is that creators do not build enough testable experiments. This mistake comes in a variety of flavors.
- The creator already knows what needs to be done
- Experimentation is considered expensive
- The original idea is brittle
The creator already knows what needs to be done
Why waste time on expensive experiments when the right answer is obvious? The flaw in this thinking is that creativity is an iterative process in which you synthesize the final result from a variety of sources and thousands of potential solutions. It is not purely a deductive process with a single right answer.
When you fail to experiment broadly, you are building your solution from an anemic set of mental and technical resources. It is the equivalent of trying to design a bridge when the only material you’ve tested is paper. You can certainly build a bridge, but it will not be nearly as good compared to someone who experimented with a broad range of materials and construction techniques including steel or concrete.
To understand the power of a portfolio of experiments, consider some simple statistics. If 4 out of 5 experimental systems are bound for failure and you create only one experiment, you have a 20% chance of overall success. On the other hand if you create 10 experiments, you have a 89% chance of finding a success. In practice, your chance of success is even higher since ideas cross pollinate. By learning and adapting to your new knowledge you’ll uncover new options that are often far superior than the original set of experiments.
(Note: If the goal of experimentation is hands on knowledge, try including a wide range of participants that can bring a variety of pertinent skills to the table.)
Experimentation is considered expensive
In the example above, a savvy counter of beans might note that 10 experiments is likely to cost 10 times as much as a single experiment. Yet all this extra money only increases the additive chance of success by a mere 450%. So the team compromises and invests in a handful of expensive experiments.
The solution here is to use less expensive experiments, not fewer experiments. What can you make in a day? What can you make in an hour? Instead of using teams of 5 to 10, what can you learn with a team of 1or 2? By focusing on lightweight experimentation and rapid turnover between experiments you can pack more experiments into your brainstorming phase.
One technique I love that keeps experiments small is Post-it note design docs. Since your experiments must fit on a sticky note, you are forced to keep the scope small and easily implementable.
The original idea is brittle
Sometimes you start with an idea that seems brilliant, but as you try to expand upon it, you keep running into walls. You know you’ve found a brittle idea when you try to bend the concept in interesting new directions and it collapses again and again.
Over time, you get learn to recognize brilliant yet brittle ideas early in the creative process. My test is to try to think up 20 or 30 crazy variations on the idea. If I can imagine that most of those variations would be exciting to build, then I know I’ve discovered a robust idea that is worth investing in further. If I can’t, then I have a brittle idea.
It is too easy to invest months, even years of your life trying to “make it work.” Instead there are a couple techniques for making the idea more robust:
Put the idea aside: The single best thing you can do is to put the brittle idea on the back-burner. Over time, if the idea is in fact brilliant, it will find its way back into your creative process. A different perspective, be it brought on by time or new experiences, can be an essential ingredient in softening the idea’s previous constraints.
I wrote up an idea for a game called Cute God a while back, but none of the prototypes really gelled. It was a design with a thousand problems and very few good solutions. Instead of belaboring the point, I consciously stopped redesigning it. Years later, some of the original combinatorics ideas found their way into a game called Triple Town that should be released later this year. Ideas you cull are never really erased. Instead they turn into fertile soil from which the next generation of ideas are grown.
Radical simplification: What is core experience that you are trying to achieve? Write a single sentence or draw a picture that represents that experience. Put that on the wall in front of you. Now brainstorm a dozen different incredibly simple ways of creating the essence of that experience for the user. Toss out all your complex systems and constraints and start over with just the core.
Recently a friend and I went through this exercise on a game concept that was born from a pencil drawing of dozens of stick figures attacking one another. The brilliant, yet brittle design was a Toribash-style fighting game where you could move individual joints for each of the stick figures in an epic multiplayer battle. As an exercise, we went back to the original drawing and asked “What was the simplest way to get that image up on the player’s screen” and “How do we evoke the coolness of a dozen stick figures blasting one another with shotguns.” Out goes the rag doll physics. Out goes the complex UI. Out goes the multiplayer. The resultant idea was robust, easier to implement and still captured the emotional joy of the original inspiration. At the very least, this exercise helped the designer look at the problem in a new light and question their original (brittle) constraints.
This process can all be summed up “If your design is difficult, cheat.”
There is an entire second phase of of the creative process called culling. Inevitably, not all experiments are good experiments. Some show immediate value and others are plagued by obvious flaws.
During the Culling Phase, you need to kill flawed experiments so you can double down on good ones.
No one really talks much about this unpleasant part of creativity. We lionize the eureka moments of brainstorming and end up ignoring the agonizing process of trimming and shaping those meandering experiments into something coherent and valuable. This is a huge mistake.
Critically culling your experiments is an essential step to any successful creative act. Culling focuses the creative act, ensures projects are finish-able and ultimately yields a more powerful final experience. When I start painting, I place down a thousand brush strokes. But only one stroke appears on top and is visible to the viewer. Each previous stroke is an experiment that leads me towards that final visible stroke. Once I’ve learned enough from my experiments, I make an informed decision, place the optimal mark and move on.
Those who fail to cull, fail to create meaningful projects. You can spot a newbie designer from a thousand yards by suggesting they kill a feature that doesn’t seem to be contributing much. Their nostrils flair and their voice rises. A litany of denials, excuses and accusations pour forth. And you know immediately that their project is going to be an incoherent piece of crap. This is a good coaching moment. 🙂
Culling is composed of several activities
- Determining your culling criteria: You need to know how you are culling up front so you don’t end up making arbitrary decisions. The single best way I’ve found of defining culling criteria is to write down what success looks like. For example, in Bunni 2, we say that the game is a ‘social stickerbook’. Anything that doesn’t fit that vision is worth questioning.
- Deciding what to invest in further: Look for opportunities to amplify obvious value in your existing experiments. For example, in Half Life 2, one experiment was this risky concept called a ‘gravity gun’. When a real world prototype was made, players loved it. Valved decided to invest further and tried to figure out how that gun could be used throughout the entire game.
- Deciding what to remove. If something doesn’t fit your culling criteria, it is better to cut it early and spend those resources elsewhere. In Bunni 2, we dabbled with a combat system. However, after we built a simple version, we realized that it really didn’t fit our culling criteria so we put it on the backburner.
Problems with Culling
There are several common issues that come up during the culling phase
- No explicit culling criteria
- Experiments are not tangible.
- Assuming more is better
- Fixing every problem
- Focusing on problems not opportunities
- Judging features not core experiences.
No culling criteria
Often people get so caught up in the amazing optimism of brainstorming, they fail to agree upon explicit culling criteria. Instead the particular politics or opinions of the day hold sway and features are randomly invested in. Culling does occur, but in a haphazard fashion that is just as likely to kill good features as bad.
When you leave your culling criteria vague, you are saying that it is okay for everyone to have slightly different opinions about what is good and what is out. This leads to unnecessary conflicts. You need to bite the bullet and have the hard conversation about what your shared vision for the project should be. I like the term ‘culling criteria’ since it ensures buy off from everyone that, yes, you will mothball experiments for the greater good of the project. The act of explicitly stating a small set of common goals ensures that everyone buys into something bigger than their individual efforts and passions.
I put ‘goals’ at the top of almost every single design document I write. Though this is the shortest part of the design, it is often the section most critical to success. Experiments will blossom in dozens of directions, but the goals keep the project on track.
The experiments are not tangible
Often due to fear of creating expensive experiments, creative folks spend far too much time in the land of ideas and thought experiments. Some call this documentation and invest in it like some religious protection from mistakes. As a result, very few real world experiments are built. With nothing concrete to react to and judge in a critical fashion, it becomes difficult to apply the culling criteria in an objective fashion.
The solution is to make your ideas real as quickly as possible. Paper prototypes, 24-hour game jams, role-playing with a friend…create your idea in the physical world. To get a bit geeky, a vast portion of conscious cognition emerges as a post-processed rationalization of our body’s physical interactions with the world. Feed your subconscious cognition by creating systems you can touch, see and play with directly.
Instead of merely wearing down a single golden path in your mental thought experiments, you’ll accumulate the wisdom that only comes from a thousand real world observations. You’ll see players smile when they stumble. You sense the stickiness of a dialog that asks you to ‘click continue’. You’ll give yourself the richest possible source of information about the problem space in the shortest amount of time.
Recently I wrote out a design for a simple word game called Panda Poet. It was completely obvious how the game played on paper. The rules were crisply defined and I could easily play through the experience in my head and imagine the delight that would result. All it really needed was a quick implementation and the project would be done.
So we implemented the first prototype. The result was completely unplayable. The interface was fundamentally broken. The feedback loops were not functional. In the first 10 minutes of playing a physical prototype, I learned 10 times as much about the problem space than I had learned in the previous days spent designing on paper. Panda Poet was almost a failure, but by building multiple real world prototypes, we learned enough to salvage the design. It should be out later this year.
Assuming more is better
As a creator it is easy to get caught in the belief that more is more. Yet, the people consuming our creations rarely feel this. They have limited time and limited mental resources to spend on our work. Instead, they see all the extra ‘stuff’ as mental noise that actively harms their enjoyment of the project.
Imagine that you have a painting of a duck and you start randomly adding static in the form of blobs from other pictures to it. You can add a lot of static and still tell it is a duck. But the static detracts from the image. The end user would likely be much happier if you just gave them a great picture of a duck.
The same thing occurs when you fail to kill experiments. You are actively adding low quality noise to your creation.
Creators that fear rigorously editing their creations suffer from the sunk cost fallacy. They assume that since they spent effort making something, value will be lost if it is removed. Often they consider their work ‘precious’. However, there is no inherent utilitarian value in a feature simply because it took a lot of effort to build. The user doesn’t see your effort. They only see the messy and imperfect noise that come from not rigorously culling your flawed experiments.
Practice killing your ideas: David McClure has a great saying, “Kill a feature every week”. Culling with wisdom and discipline is a skill worth training much like any healthy habit. You practice and it slowly and steadily gets easier. You begin to see each feature and experiment as a small step in a much larger process, not a rare and precious thing that must be protected.
Be objective: Another technique is to use objective, data-centric criteria. By boiling down decisions to numbers and metrics, you give yourself permission to make emotionally difficult decisions. This can be taken too far, but is particularly useful if you have a group of people that have divergent subjective opinions on a topic. Again, real world experiments facilitate this approach. There is an objective measurable reality to how people react to art. Understand this fact and used it to facilitate your culling.
One implementation of this technique is the use of Fun ratings. I have a survey built into many of the games I work on that asks users how fun the game is on a scale of 1 to 5. It is one thing to tell someone that their game sucks. That statement comes across as a purely subjective and potentially insulting opinion. It is another thing to have 10,000 people say that your game rates 3.1 out of 5 and to know from historical data that you need to reach 3.9 in order to have a chance at financial or critical success.
Fixing every problem
A related issue is that creators often attempt to fixed all the problems with their failed experiments. This is particularly common on projects that are thought of a series of modular features. The creator lists out the problems with each feature and then methodically fixes each in an attempt to bring the feature and therefore the project up to a reasonable level of quality.
The impact of this technique is painful to witness.
- The expense of ‘completing’ the projects blossoms and progress across the board slows to a crawl as multiple objectives are pursued simultaneously. Adding more resources often only slows down the project more (ala the Mythical Man Month.)
- Quality still decreases. It is rare that each feature is completely aligned to the central value of the project. You end up with a project that is being pulled in a variety of directions all at once. The result is like student art where a new artist meticulously polishes every single shape in the drawing, but the end result is a hideously disjointed experience.
A good rule of thumb for game development is that every experimental feature you start takes 30 times as much effort to finish. So one day prototyping yields one month polishing. If you try to polish everything, the surface area of your project grows huge in very short order.
I once worked on a failed online game that suffered from this issue. There were about 5 different project owners, each of which assumed that their job was to point out the flaws in the game and mandate (on threat of death) that the team fix them all. The team would scramble to plug holes one after another. In very short order, the game turned into an incoherent mass of half polished features. In hindsight, a cleaner set of culling criteria that resulted in killing broken features would have resulted in a more focused project of higher overall quality.
The solution is to focus less on the problems and more on the opportunities. You win when you generate value. If you release an unpolished version of a something genuinely interesting and wonderful, it is amazing what people will forgive.
In Steambirds, the levels were tossed together with a semi-random assortment of planes. The writing on the mission text was highly questionable. The graphics were one iteration past the initial prototype art. There was an invisible wall that caused players to die inexplicably. It could have easily turned into a 12-month project.
However, what we did right was polish the heck out of the core mechanic of movement and attacking. Of all our experiments, it was a robust and interesting gem that resulted in a powerful user experience. In the end, by focusing on the heart of the game, none of the other issues really mattered. We saved months of labor by focusing on what worked and killing or ignoring what didn’t.
Brainstorming and culling occurs in an iterative cycle. In each cycle, you create experiments and then cull back to the valuable core. Then you repeat. Each complete cycle spawns a new spurt of ideas and experiments that must be culled in turn.
Each cycle results in accumulating more value for the customers of your labor. When you’ve generated enough value, you stop.
There are a couple issues that occur during this process.
- Not iterating.
- Not delivering value to the customer.
Often creators stop after a single cycle of brainstorming and culling. I see this quite often in groups that come from a waterfall-centric culture. Planning is treated as the equivalent of brainstorming and initial implementation. Culling is treated as scope reduction and bug fixing. After one long cycle, the product releases.
The solution here is shorter, lower cost cycles. Any of the various agile methodologies cover this ground extensively. To facilitate more cycles, I ask the question: How do I decrease cycle time so I can fit more learning cycles into my project? I’ve asked this question for art, for games, for UI design and for application development. In all situations, the more cycles I can pack in, the happier I am with the end result.
Not delivering value to the customer
The exact opposite of not enough iteration is to continue to iterate indefinitely and never release the value you have accumulated to a wider audience. There is always room for improvement and always new value to generate. You can get stuck in the trap of constantly cycling through the creative process and never feeling that your product is good enough to release.
Here are two good solutions I recommend
- Timeboxing: Set a release date and release regardless of how far you’ve gotten. Pixar’s Darla Anderson has a great quote that “we don’t finish our films. We just release them” The predetermined release date is a forcing function that ensure they pack in as much value as possible before they are forced to put something out. It also ensure that you stop working on a flawed experiment. The emotional distance that comes from releasing can be extremely helpful in realizing that you need to take a break from a particular great white whale that is eating your life.
- Release criteria: Another alternative is to set objective goals that trigger a release. For Flash games, I know that when I’ve hit a 3.9 fun rating, I can release the game. Certainly, I could invest further, but it really isn’t worth it.
All these thoughts and pictures can be summarized in a very short list.
- Brainstorm: Create lots of low cost, real word experiments.
- Cull: Rigorously apply agreed upon culling criteria to weed out the weak ideas and reinvest in your most promising experiments.
- Cycle: Repeat the process until you generate meaningful value.
- Practice: Across multiple projects, practice all stages of the creative process so you constantly improve the myriad of skills involved in brainstorming, culling and cycling.
Or if you are a visual learner, just reference this picture:
*blink* … Can't wait to read an internalize all of this!
That was brilliant. Now I only have to find the way to force all of my coworkers to read, analyze and understand this.
This resonates with Tom Wujec's TED talk, https://www.ted.com/talks/tom_wujec_build_a_tower.html … kindergarten kids beat MBAs because they build more prototypes.
Great article danc! I was going to write a similar piece but now that you've got one with great illustrations and research I'll just point everyone here instead. This is a great summary of the fundamental theory tying together agile, scrum, XP, and whatever software development process is cool this week. But, it does it in a way that anyone can understand and apply.
Have you ever played Magic: The Gathering? Nothing teaches the value of focus like deck building in Magic.What the human brain is poor at grasping is: every time you add something it detracts from everything else.Well, really, it pulls focus away from everything else. So if the thing you are adding is totaly rad and cool but doesn't interact well with the rest of your mechanics then it is taking the spotlight off of something that does.More and more I think game design is about elegent subtraction.
\”If I think any other man is able to see things that can naturally be collected into one and divided into many, him I follow after and 'walk in his footsteps as if he were a god'.” – Plato, speaking as Socrates and quoting Homer, Phaedrus 266b
I've been talking recently to people in other artistic fields recently about their creative process. And I think one of the problems with video game development that we face is the size and time of these tennis balls. The larger the game, the larger the tennis balls. We need to find quicker ways to iterate through designs and recognize ideas that are not worth pursuing more quickly. To build anything to the stage of \”how is this going to feel for the player\” is a very time consuming exercise when it fails. At an 80% failure rate, that's a project that takes 5 times longer than it could if the right decisions were made all the time.I think we need faster ways of prototyping ideas that are extremely low-cost that we can measure success with accuracy. There seems to be two areas where an idea can fail. The concept is either wrong or the implementation is wrong. And I think identifying bad concepts early is the key to speeding up the development and creative processes in making a games. What can we prove with sketches and postit notes, and how can we measure if these concepts will resonate with players? Bad implementations can always be iterated on, but it's also important to kill off things that are too difficult to implement when they don't improve on the core concept that was proven. If you can prove your core concept, then you have already defined your culling criterion for implementation. If you start having new ideas that are not part of your core concept, then they create noise. This makes me feel that the majority of brainstorming should happen when developing core concepts. This can reduce noise and the size of your tennis balls.Anyway, it's food for thought 🙂
These diagrams are similar to one I put together in 2007, then wrote more on in May of last year to illustrate a very similar concept (in relation to videogame development project management):http://www.hobbygamedev.com/articles/vol2/resourcefulness-vs-resources/It is not at all my intent to imply that you've ever seen my work or know of it – surely these are ideas and representations that many thousands of people are running into and thinking about every day! – however a former college pointed out the similarity, and I thought it might be fun to share my version of the design space graphs and the potential inflection points.Divergent and Convergent were my terms of choice to classify whether a particular event caused the decision space to swell or constrict, respectively.Nicely done, and thanks for putting this together!
Great post! It helped me realize some stuff that I felt I knew but hadn't conceptualizedThanks for sharing!
It's always amusing to me to read your blog and others like it, and to see people in the gaming industry pseudo-quantify the things my art studio professors used to say (and probably still do).I'm a student at FIEA, and I just finished my first major video game project. A 7 month long, 15 person affair. Not to mention the several smaller projects that preceded that. I had myself a bit of a laugh a few times while reading this, and I'll definitely be sharing it with the people I worked with.Funny how a lot of this stuff doesn't really sink in until you've fallen on your face a few times.
\”Ideas you cull are never really erased. Instead they turn into fertile soil from which the next generation of ideas are grown.\”This is my favourite line in the entire post. I think that understanding the benefits and necessity of culling is the biggest gap between designers and non-design-types (I fall somewhere in between).
Hey Danc,Again a great post! I have sent it to our design team, and try to interest management next week as well ;DEven though the subject matter is not new, I have not yet seen it visualised in such a clear and practically useful way. It will surely help as a mental model to pinpoint opportunities in our processes. I'll let you know what it brings us…Ohw and if you ever get a chance to come to Amsterdam, please give us a visit ^_^ReneW!Games
I'm making my first game and it must be the most insightful article I've stumbled upon… Really helpful.
I am a graphic designer and I rarely find this kind of comprehensive articles on the creative process in the web space. Most of them are just 3-4 superficial paragraphs.In the era of instant gratification, we must remember that sometimes technology has to wait for us, and not the other way around.I can only appreciate this, I will make a mental post-it out of it.
It told clearly how the creative processes are. I can use it as a framework to do my project.Thanks, sir. This really is usefull.
I don't understand how 10 experiment will give u a number of 89%succeed
Think of at as rolling a 5 sided die 10 times, what's the chance of you getting a 5 at least once? The formula is: 1 – (4/5)^10