Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
October 26, 2014
arrowPress Releases
October 26, 2014
PR Newswire
View All
View All     Submit Event

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

by Neils Clark on 02/08/14 07:36:00 pm   Expert Blogs   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.






Is a really common thing to hear, in any online game. The expression originated in EverQuest, whereupon leveling up a player would hear something like a 40-foot-tall bronze gong being torn in half by another giant gong, with that sound being run through someone’s gravelly track & field loudspeaker. It’s not exactly my favorite sound in the world, but then I never got to hear it after days or weeks of grinding, as the harsh audio reward for my long labors.


Players would broadcast their glee, by telling nearby players and guildmates, “Ding!”


To great adulation and congratulation. It became such a cultural artifact, such a hallowed ceremony for the first big player organizations, that they brought the tradition into the games they migrated to later: WarcraftGalaxiesDark Age of CamelotCity of HeroesEverQuest 2. The Ding! has since been repurposed not just for levels, but picking up coveted weapons, new skills, even sometimes a chance to practice rough fights. Lately I’ve even heard people who don’t play many games, using it after getting promoted at work, moving to a nicer apartment, or getting some hot guy’s telephone number.




A great deal of the ding’s power rests securely in its ability to inspire pride. It’s a part of fun. In fact, a lot of our language for fun – in this book – relates back to the ding.


Many of the game design languages built specifically for developers, for instance by designers Dan Cook, or Benjamin Ellinger, relate strongly back to musical notation. They take specific kinds of fun (or engagement, if you prefer) specific rewards, then paint in terms of very subtle dings, and very powerful ones. In this way, structuring rewards is central to design; it’s as key as grammar in developing a writing style, or as vital as pacing in cinematography. Especially in games whose systems ask us to perform actions we’ve never tried before, and to navigate systems that might work in stark contrast to the other realities we’ve lived. We often need to use the rewards, the dings, to teach a player simply how to walk around.


It’s inevitable that the artistic structuring of reward would have some crossover with behavioral conditioning. This is especially tricky ground to walk, because the implications of behavioral conditioning are chilling.


“We want to know why men behave as they do.” Wrote the father of radical behaviorism, Burrhus Frederic Skinner “Any condition or event which can be shown to have an effect upon behavior must be taken into account. By discovering and analyzing these causes we can predict behavior; to the extent that we can manipulate them, we can control behavior."


B.F. Skinner is best known for placing a pigeon, or rat, into a box. Inside these Skinner Boxes, he would eventually place a food dispenser, a signal light, an electrified floor, and a push button. By mixing the dispensation of food, signals, and punishing jolts of electricity, he displayed that it was possible to train pigeons and rats to any number of behaviors. Pigeons were trained to be more effective than early computer algorithms, in guiding missiles towards enemy ships. The only reason they weren’t ultimately used in wartime, was that the notion seemed laughable to officers.


In discussing his training of pigeons, and un-training of them, Skinner noted that they remembered what they’d been conditioned to do "… as long as six years after the response had last been reinforced. Six years is about half the normal life span of the pigeon." Skinner’s primary interest isn’t just placing exacting routines into pigeons. He wants to control the behavior of human beings.


We start this process by basic shaping. A boxed pigeon may wander to the left, wander to the right, but once he tilts his head slightly towards the button? We drop a food pellet. He dallies a bit longer, then takes a step towards it? Another food pellet. He taps the button? Maybe more than one pellet. Like a nice mini-jackpot. He taps it again? We give him pellets for every press, and so shape the behavior that we want. Though we don’t keep going indefinitely, or eventually the pigeon gets full (or  maybe we run out of food pellets).


Thankfully, we don’t need to reward him for every press. So enters the science, in Skinner’s book Science of Human Behavior. He believes that there are ways to structure rewards, so as to simply keep pigeons, or human beings, pressing the proverbial button. Skinner doesn’t believe in free will. In point of fact, he writes that "Man's power appears to have increased out of all proportion to his wisdom." That it’s the duty of better men to shape their lessers to “productive” ends, else they’ll never, ever, be satisfied.


So after shaping, we use Interval Reinforcement, reinforcing a behavior after a certain amount of time, a certain interval. “We are less likely to see friends or acquaintances with whom we only occasionally have a good time,” writes Skinner, “and we are less likely to write to a correspondent who seldom answers. The experimental results are precise enough to suggest that in general the organism gives back a certain number of responses for each response reinforced.”


Rewards based on the passage of time will never quite be so effective at encouraging behavior as Ratio Reinforcement, where rewards which come in response to some behavior. This could be a Fixed Ratio, as when “the schedule of reinforcement depends upon the behavior of the organism itself—when, for example, we reinforce every fiftieth response” He writes that this is essentially getting paid on commission.


What’s much more effective than every 15th press – if our goal is getting a pigeon or player to tap furiously on buttons – is Variable Ratio Reinforcement. When the pigeon doesn’t know quite how many responses it will take, somewhere around fifty, maybe it’s around a thousand. Skinner writes that the “… pigeon may respond as rapidly as five times per second and maintain this rate for many hours.” He continues, “The efficacy of such schedules in generating high rates has long been known to the proprietors of gambling establishments. Slot machines, roulette wheels, dice cages, horse races, and so on pay off on a schedule of variable-ratio reinforcement….The long-term net gain or loss is almost irrelevant in accounting for the effectiveness of this schedule.”


We can also combine schedules, “so that reinforcement is determined both by the passage of time and by the number of unreinforced responses emitted. In such a case, if the organism is responding rapidly, it responds many times before being reinforced, but if it is responding slowly, only a few responses occur before the next reinforcement.” When rats and pigeons are kept in physical boxes, tapping on buttons, it's the ratio schedules, especially variable ratio schedules, that get them tapping in the vain expectation of something. They go from tapping around 50 times – without reward – to thousands. Since they never know when the next food pellet is coming, they keep pressing the button.


Human beings may or may not need better rewards than food pellets, but our rewards can be so much more subjective. Shiny best-sword-in-game may fit the bill. Perhaps it's shiny best-shield-in-game, or shiny best-horse-in-game. Warcraft provides its players no dearth of variety for their (often game-actualizing) pellets. Last I left Warcraft, one friend was killing foxes on the 1/10,000 chance they'd yield a fox kit. Maybe the momma fox is pregnant? They don’t explain it. Anyway, probably fewer than twenty foxes were out in the world at any given time. When I last logged out he'd been at it awhile, with no dice. But if he got one of those kits? Boy howdy it'd be worth some gold.


And there is, always, gold. Skinner noted that with physical internal needs, say food, we can eventually get full. With sex there’s a refractory period. At the very least a Gatorade break. That’s why, when these “better” men went to improve life for their “lessers,” they’d also use Generalized Reinforcers. “The commonest example is money. It is the generalized reinforcer par excellence because, although 'money won't buy everything,' it can be exchanged for primary reinforcers of great variety…the exchange value of money is more obvious than that of attention, approval, affection, or even submissiveness.”


Oddly enough, in near every online world in which I’ve played, near every monster is carrying currency. How did I get seventy-five pieces of silver from that Embittered Dire Wolf? I really, truly, don’t need to know. I’m just glad to have it. I’ll need it for something, down the line, I’m sure.


I’m conditioned.


It's probably not that every big-ticket game designer spends all her time crayoning inside the lines of a BF Skinner coloring book. It's no huge surprise when game experiences wind up looking like even blatant Skinner Boxes, with a designer who knows what passion looks like. Not necessarily because she picked up a psychology degree, but because she lived it. Because, whether or not she has any inkling as to why, her imagination thrives, writhes, and lives within the language of systems.


But there is a difference. Even in our simple language of fun, we had eleven unique ways in which these rewards might speak to a player. I hope that the languages on community, place, challenge, and art provide more. I hope they spark insights on the artful crafting of rewards. When the depth of our world is reduced to the cold, hard, simple Ding!, it’s not about the designer providing something unique. The word “compelling” doesn’t even seem to fit, despite being so tied to its sister “compulsion.” At the point of compulsion, designers are being lazy. When their only goal is retention, they deserve less consideration than a sidewalk con artist peddling fixed shell games. This is why compulsion, especially when used to knowingly and willfully create grinds, is so far beneath fun.


B.F. Skinner was a man for whom culture and the arts were naught but the ringing of some dog’s food bell. "Literature, art, and entertainment,” he wrote, “are contrived reinforcers. Whether the public buys books, tickets to performances, and works of art depends upon whether those books, plays, concerts, or pictures are reinforcing."


Which is offensive. To Skinner it’s all about the science, “only the observable in behavior.” Which is nice in theory, but nobody can accurately observe everything. It’s almost painfully clear, the fissure between Skinner and the vitality and soul of the arts. Science without philosophy is like sex without condoms, and in rejecting free will B.F. notoriously infects his theories with a demented megalomania. Rejecting the philosophies that enrich and color our world gives the impression of a cold interior, to the boy who once wanted to write fiction. Who abandoned it out of the belief that he had nothing to say. Skinner once remarked, "I do not admire myself as a person. My successes do not override my shortcomings."


Gamers have been drunk on the deluge of sights and sounds and feeling, because we were the ones seeing something new come to life. But when it works, and more than that you uncover a new piece of vocabulary – so painstakingly carved – in the language of experience? Suddenly new dimensions of understanding, human understanding, become available. Your world is the richer, and all it took was play. That’s not the product of dumbfuck conditioning, it’s a hallmark of art. The two couldn’t be more different.


Novel systems are the height of game design, but they're unreliable at best. Expensive at best. Rather than capture something essential about the processes of human experience, rather than draw us in with the magic of the unknown – so painstakingly carved – many developers have turned to the more fiscally-responsible twisting of behavioral, motivational, even positive psychology. Some of you designers are my friends. Hear me when I say,


Cut that shit out. Make something fun.




This is a brief blip from my book In Play.


You can start from the beginning, on my blog. 

Related Jobs

Digital Extremes
Digital Extremes — LONDON, Ontario, Canada

University of Central Florida, School of Visual Arts and Design
University of Central Florida, School of Visual Arts and Design — Orlando, Florida, United States

Assistant Professor in Digital Media (Game Design)
The College of New Jersey
The College of New Jersey — Ewing, New Jersey, United States

Assistant Professor - Interactive Multi Media - Tenure Track
Bohemia Interactive Simulations
Bohemia Interactive Simulations — Prague, Czech Republic

Game Designer


Daniel Cook
profile image
First, I appreciate the overview. And arguing for breadth over narrowness is something hardly anyone could find disagreeable. Yet, this essay falls into the category of pop culture fear mongering.

I come from a perspective where I believe that ultimately it *is* possible to model culture and art in a productive manner. As such I interpret Skinner as a scientist looking for better models of why people behave as they do. He went about it in a rigorous fashion, but that is not something to be condemned. You create limited experiments to control for confounding variables not because you are a megalomaniac.

Some details:

1) The term 'reinforcer' can be used narrowly or broadly. Not all feedback need to be 'dings'. For example, this comment (and any subsequent comments) are feedback to you for writing this essay. It is far richer than a ding yet fits within the terminology. So yes, most of art and culture can indeed be viewed from the perspective of systemic reinforcement schedules and honestly, it is a wonderfully insightful exercise. Choosing a moronic interpretation of a rich concept serves your argument but fails to advance the conversation. This text is your 'dumbfuck conditioning' right here. There's a richness to such a 'ding' worth savoring.

2) The idea that there are (large) holes in our current models of human psychology is certainly true and in fact expected. These gaps are the way of science and not knowing is not an excuse for not looking. Instead of attacking Skinner's highly controlled experiments as failing to provide a complete picture of why human work, perhaps it is better to focus on the pop culture simplifications.

Ultimately, it is the poor application of science that seems to cause so much harm, not the fact that science exists. Fight the appropriate enemy. Are there greedy people doing bad things? Why not focus on that? Are there weak ways of using reinforcement schedules? Let's have a rational discussion on that topic. Are there good ways of using reinforcement schedules (because they'll always exist whether you want them or not)? Let's talk about that.

Blindly suggesting people make 'fun' games is not a useful alternative to such a discussion.

We should become wary when arguments swing too far in another direction and hold up art as a near religious artifact of faith. 'Art of the gaps', so to speak. Doing arbitrary things for arbitrary reasons is just as likely to produce harmful work as blindly misusing scientific research. I've witnessed as many (if not more) games and game developers destroyed by 'artistic' opinions held by 'creatives' as I have by folks trying to implement Skinner boxes.

In the end, advocating for 'fun' fails to provide an alternate set of useful tools. At best it is a vacuous feel good statement. Joy, you are using opinion and rhetoric to shred someone else's work...what are you adding that is of verifiably superior value?

Here's a moderate approach:

- Many of the reproducible studies involving human psychology are likely highly applicable to game design.
- There are pieces of culture we don't understand. Let's not give up.
- Instead, we should try to create works that rely on both the existing science of psychology and theories for parts we don't get know.
- Test where possible to see which assumptions worked
- Apply those to the next iteration.

This recent cloud of 'Evil Skinner Box' discussions disturbs me deeply. There's an anti-intellectual, anti-science, anti-craft theme that runs through the current faddish moral panic. Don't be offended by B.F.Skinner. Understand him in the context of what he was attempting. Use the parts that work. Improve upon his efforts. Be a designer informed by science, not one that is fearful of it.

All the best,

Neils Clark
profile image
Notwithstanding that I may have less respect for Skinner's ideals, I don't think that we disagree.

With the Skinner part, my ire is mostly summarized up above. His seeming contempt for the notion of free will, for culture. His science is worth understanding (it's part of the toolbox). So is his personal context.

A design can be made rich in a lot of ways, but I think most well-rounded gamers have started to see one-dimensional hamster wheels as the soap operas of gaming. We might want that familiarity now and again, or if the alternative is cash4gold ads and paid programming. But it ain't no Deadwood.

Those Skinnerian principles are still in the good stuff, but (and hopefully I'm not twisting your words) "far richer than a ding"

The things you ask for are (I think) probably all actually in the book this post is excerpted from. By the time we get here, I've mapped out a language for fun (learning, creation, destruction, unity, etc etc). I play with other languages: space, community, manipulation (guess where Compulsion shows up) challenge, art.

But like, I find it maddeningly awesome you would give such a clear, awesome post.

All the best to you too!

Katy Smith
profile image
From what I gather, the gist of this article is that we as game developers should be going for deeper, more varied, and more meaningful interactions. That's great! I don't disagree with that. However, I think you're painting Skinner and human psychology in a bad light.

Behaviorism was a reaction to psychoanalysis. People like Freud, Alder, and Jung were all about studying the mind as a great, unknowable but interpret-able thing. Behaviorists like Watson and Skinner (who was probably the most radical behaviorist, if not the most vocal) were big on science and measurement. They rejected the belief that the mind was unknowable and focused on breaking things down into small, measurable content. To them, it did not matter why people thought or felt the way they did, only the way they act, react, and behave. That's why the most famous behaviorist studies focused on feeding rats, or pigeons, or doing small menial tasks. They threw out looking at art and philosophy not because it had no meaning, but because the meaning was irrelevant to what they were studying.

Fast forwarding to today, the concepts created by behaviorism still apply. Fixed / Variable Interval / Ratio reinforcement schedules do work. However, it doesn't answer this question: If I'm conditioned to answer the phone when I hear it ring because I will be rewarded with a conversation, why do I sometimes not answer it?

There are holes in all psychological theories, and the field does admit that. Rather than throwing out an entire school of thought because it doesn't answer all questions, it seems much more productive to use parts of each. Saying "skinner boxes aren't fun, make something fun!" doesn't really solve the problem. It just eliminates one of the tools that we know works for shaping player behavior.

Neils Clark
profile image
Thanks for the comment, Katy, and for giving a broader perspective on how behaviorism developed out of the psychoanalytic tradition.

Specific is good.

And this post, taken on its own, is (even intentionally) surfacy. Though when we get to Rogers and Maslow and Vaillant and neuro and everything else - then take them in tandem with today's cultural context - I do think it's worth pointing at Skinner's rejection of a lot of fairly observable stuff.

But yeah, I'm kind of an asshole about it.

Check out my site for more of what I mean by fun. I'm giving away my book piecemeal, but (if you want) ping me via email and I'll send you the whole thing. It's as obnoxiously written as my stuff above, but I nod to designers, architects, and a few more big psychologists.

Luis Guimaraes
profile image
I think it comes down to target audience.

There's a slice of the public that wants to be respected by the designer and which can't be fooled by simplistic dominant strategies and shortcuts. For who's targeting that group, Game Design can't be so easily dumbed down. It wouldn't be any fun if it wasn't challenging.