Gamasutra: The Art & Business of Making Gamesspacer
Why Kinect is the future of game story
Printer-Friendly VersionPrinter-Friendly Version
View All     RSS
April 25, 2014
arrowPress Releases
April 25, 2014
PR Newswire
View All
View All     Submit Event

If you enjoy reading this site, you might also want to check out these UBM TechWeb sites:

Why Kinect is the future of game story
by John Sutherland on 11/03/10 03:34:00 pm   Expert Blogs   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.


The new movement-based consoles from Microsoft and Sony (though Nintendo deserves props for being the advance guard here) have caused a lot of excitement in the industry. This is particularly true of Kinect, because of its empty-handed skeletal tracking. But the initial enthusiasm should be taken with a rock of salt; at least, compared to what these platforms can do down the road.

Having worked on Kinect games well before the public announcement of the platform, there's plenty I'm not allowed to say. Fortunately, what I have to say has nothing to do with confidential information about the project, only the possibilities for projects that haven't been dreamed up yet.

I’ll speak mostly of the Kinect, for a couple of reasons: it’s the one I’m most familiar with, having worked behind the scenes on it, and it’s a fair argument that the complete lack of a controller is the superior (and riskier) technological trick in this field.

I don’t wish to be dismissive of the PlayStation Move, however; it’s only tracking hands, not the full body, and isn’t an enormous leap beyond the Wii. The great news about Move is that players who prefer Sony-like content over Nintendo-like content will now have that available in a movement-based system, and that makes a huge difference when we’re talking about story in games.

But like any new platform, both Kinect and Move are going to go through some common stages of evolution. If we look at the changes all video games went through, or even what moving-picture cameras went through, or (what the hell) what the printing press, or even quills and parchment probably went through, we see certain patterns.

First, it's a marvel. People look at what the new technology can do, and they're amazed it can be done at all. And for a moment in time, that's enough to hold the attention of millions.

Then, it's a toy. People mess with it, see what can be done, try new things. But most likely, they'll try the same kinds of things they're used to from older technologies, because that's the way humans operate. That's where many Kinect and Move games are now: you'll see a mix of new experiments, testing the boundaries. But frankly, we don't know where those boundaries are yet, and you'll see a lot of things that are just translations from the old ways of doing things thrown into the mix. That's normal. Don't panic.

Then, it'll be a brave new world, and those older habits will get filtered out. We'll wonder why we ever did them. The techniques with the platform will be increasingly natural. The games on Kinect will work just the way the players would want them to the first time around. The best of the current Kinect games are touching this phase of evolution, at least in part.

Ultimately, inevitably, the platform will evolve into a story device. Not a story telling device as much as a story experiencing device, at least from the player's point of view. Why? Because after the novelty wears off, people will crave depth.  Story is central to human experience, and both the Kinect and Move systems, even more than video games in general before them, are connected to human experience in a full-bodied way.

We’ve now seen what Kinect is doing in its first generation of games: it’s asking the legitimate question, What can players do with their bodies in a game? And we see the current set of answers: they dance, they exercise, they play sports, they pet their cute baby tigers. You know, the obvious stuff. And we might legitimately ask, Is that all there is? before returning to the amazement that such a thing can work at all.

PlayStation Move’s line-up is, like the platform itself, more cautious: there are sequels to controller-based games, like Kinect, there are sports and there is and dancing. And caution does have its advantages, which I’ll get into later.

But both platforms are in the early phases of evolution, it’s safe to say. The big question is, where are they going?

Glimmers of the future

As the games continue to expand, there will be deep stories, but probably not new ones at first. The best, most immersive stories in the next wave of games will probably not be planned with the game movements woven into the plot from the ground up.

This is okay, temporarily. The announcement of the relationship with between Kinect and the Star Wars franchise, for example, has drool-worthy possibilities. Who doesn’t want to use the Force with their own hands and see the immediate effects? This is one of the areas where Kinect has a strong advantage over the Move. But a different activity in the same franchise leans toward the Move platform: when you play with light sabers, it may well be more satisfying to have something in your hands. Context is everything.

Stepping back to step forward

As these games evolve, I should also observe that, like any evolution, they won’t progress along a direct path. Strangely, it may well be from translations of controller-based games that we can get a real glimpse of the future. Adapting an existing game like Heavy Rain is great practice for developers who will walk this tightrope later without the net of franchise familiarity. It won’t be until the development of movement games becomes second nature that game studios can relax and just create new games without being so self-conscious about using the new platform correctly.

Power to the Player

Every story platform does certain types of stories better than others.  Novels can give lengthy internal introspection, while movies can give colorful flights of visual fancy. The type of stories that are likely to excel on the new movement platforms are likely to be the ones where the protagonist gets to exercise significant power. This is true of all game stories, but will be true in spades here.

The Star Wars example of using the Force is a good demonstration of this. Games that give characters fantastic (in the literal sense) powers are likely to be deeply satisfying. And these are not just powers of destruction, but equally, powers of creation. Can you imagine creating a bridge with a wave of your hand? Snapping your fingers and having a city appear? Solving a game problem by drawing a new object with your finger, and then using it? (Think “Harold and the Purple Crayon” on steroids.)

Dreaming Like Einstein

One of the best ways to create anything new is to imagine the end result you want, and then figure out the steps to get there. This is how Albert Einstein worked: he’d dream of his theories first, and then do the calculations later in the waking hours.

This is wide open. This won’t be based on anything games have done before. This will be the practical realization of someone’s vision of stepping in front of their console camera and vanishing into another world. But this is more likely to happen two or three waves of game development down the road, unless someone gets very brave.

When it does happen, though, all of our talk of immersion with conventional video games will seem laughable compared to the possibilities in Kinect. Research into a variety of somatic movement therapies has demonstrated that things are experienced more deeply in the mind when they are also experienced in the body. Kinect should have a marketing tagline: Immersion – it's not just for the cerebrum any more.

Doing the math, or at least, the script

Now, from a writer's point of view, this is still a story telling device. Everything is. But the way we tell stories has to change and adapt with every new form.

A brief sprint through the major forms will demonstrate what I mean.

Going way back, epic poetry was not even written down at first. That meant it had to have memorable lines linked with rhyme or some other poetic device (alliteration, in the case of Anglo-Saxon) so that whoever was passing on the oral tradition had concrete links in their memory, little stepping stones to keep it from being forgotten.

Written text didn't really count as a popular story form until the printing press came along. Sure, there were monks transcribing things, but the popular stories only really happened in theaters.

With the printing press and the later expansion of literacy, however, printed stories meant that a writer would weave elaborate, vivid scenes that could be written densely, with the assumption that a reader could go over them more than once. Not so with the performed stories in the theater.

A huge shift came in the twentieth century, when movies came along. Writers started to think, and write, visually.  For the first time, we had a performed story with spoken dialog as the minority of the text.  Mostly, it was action and images.

Then we got to video games, a form that is still in its early stages of evolution, even not considering motion-based gaming platforms. Here, writers had to start thinking radically in terms of player choices.

So here we are in the 21st century, with this new platform. How do we fill in the blanks, do the calculations, to get to the new reality of an evolved story vision?

I can speak for the game writers: it’s going to involve a mind shift that’s similar to the one our forebears went through at the advent of movies, when they went from primarily spoken dialog to visual storytelling.

Now that we’ve been living with visual story telling in both movies and games for a few years, and game writers had to change from controlled character action to player-prompted action, we have a new problem: Kinect will force game writers to think in player gestures.

Toward a Language of Gestures

Developing a new language of gestures is a more complex matter than developing a visual language for movies. The truth is, every game writer and game development team will create their own, depending on the needs of the game. Then there will be a period of exchange, and certain tools will make sense in multiple cases, and standards will emerge, just like cuts, zooms, and fades did 100 years ago.

But where controller games have a limited number of inputs that can be defined as a few choices, movement games have a potentially unlimited number of inputs that can be defined any number of ways. Creating the boundaries to impose some kind of order, while still giving the impression of fluid freedom to the player, is the big contradictory challenge.

This will have a variety of effects on game development, not the least of which is the increased interdependence among game writers, game designers, and programmers. Some initial work up front will need to happen among these groups to establish a “box of verbs” for writers to play with.

The Practical Stuff

So the good news is that there are no boundaries, but that’s also the bad news. The correct balance can only be determined by experience. To that end, here are some practical tips from the trenches:

  • Making an enormous set of gestures that are necessary to play the game, especially if they’re not intuitive, will be confusing to players.
  • Making a set of gestures that are similar to each other, with only fine distinctions among them, will be confusing to machines.
  • Always consider the physical toll on the player, and understand that some gestures work better than others. Flying is easy. Running is hard.
  • You should create a small set of required gestures (small, as in three or four), and an array of “background gestures,” things you don’t tell the player, that also work.
  • Avoid situations where the player is waiting for the game to do something, and the game is also waiting for the player. The game has to take the initiative to avoid those moments of “Now what?”

The ideal game will be one in which the player can try any kind of gesture that may occur to them, and get the desired effect (or at least some delightfully surprising effect) in the game.  Not only will this involve the trifecta of writers, designers, and coders; usability engineers will become all the more important. As the array of movements is designed, the reality check with normal people will bring more fluent accuracy to this naturally less structured form.

It will only be when entire teams get comfortable with this new Zen insecurity that real invention of new worlds can emerge. But they will. And when they do, we may well be in for some of the richest experiences of our gaming lives.

The Goal: No Separation

What all Kinect game developers need to keep in mind is the huge advantage over all controller-based systems, including the movement-based controllers like the Wii and Move: there is no separation between the player and the game.

Any story form will have a certain leeway with the willing suspension of disbelief: readers are sitting in a chair holding a book, playgoers are in a theater looking at actors, gamers have traditionally been couch-dwellers holding a controller. But take away that need to forgive the practical limitations of the form, and the story becomes all the richer. When if we could take a patron out of the theater, and put her on the island with Prospero? What if we could make the book disappear, and bring the dream directly to the reader? Kinect is, for the moment, the closest thing to the holodeck. It’s only a matter of time before Hamlet arrives.






Related Jobs

Nexon America, Inc.
Nexon America, Inc. — El Segundo , California, United States

Web Designer - Temporary - 3 month
Darkside Game Studios
Darkside Game Studios — Sunrise, Florida, United States

Mid-Senior Graphics Programmer
Digital Extremes
Digital Extremes — LONDON, Ontario, Canada

Digital Extremes
Digital Extremes — LONDON, Ontario, Canada



Ian Fisch
profile image
Ok seriously relax. It's just a new interface. A clunky one at that.

"And these are not just powers of destruction, but equally, powers of creation. Can you imagine creating a bridge with a wave of your hand? Snapping your fingers and having a city appear? Solving a game problem by drawing a new object with your finger, and then using it? (Think “Harold and the Purple Crayon” on steroids.)"

You can do all of these things with existing interfaces (mice and keyboard, gamepad), and it is often far less frustrating.

You sound like one of those people who said the N64 DD drive would change the face of gaming, not realizing that PC's had rewriteable harddrives for years. "You can cut down a tree and it will grow back slowly over years!!" That's you.

John Sutherland
profile image
You really think making something happen out of thin air is the same experience as clicking with a mouse and typing on a keyboard? Okay. We're just different people, I guess.

Tadhg Kelly
profile image

Amir Sharar
profile image
Things don't have to be doing something inherently new, to revolutionize how people view and use such products.

Sometimes a new interface is all that is needed for it to be a success.

You can do those things with a gamepad, mice or keyboard, but ask your grandma to use them. There is a clear difference in using a physical device with unnatural movements, and in using your body with natural movements.

The Eyetoy could have done this, but sometimes you need a rebranding and repackaging of a device to make an action or behaviour more popular and accepted. We saw plenty of great MP3 players in the market before the first iPod hit. Just like the iPod, these players can play MP3s, videos, and deal with playlists.

Stephen Chin
profile image
The thing though is that the effect "Make something" is different than the action "Push a button". Whether you're making something happen through a gesture or a button press is something different than the effect itself.

When looking at Kinect (and other such things), we should be careful not to confuse what it is with the rest of the system.

That said, as Amir mentions, the interface that Kinect offers could and is full of potential in the sense of a more direct connection between player action and game action, among other things.

Tim Carter
profile image
What I want is a game where, if you play Sim City, you actually have to build large structures in front of your big-screen TV, and you better watch out or you might drop a five ton beam on your little brother.

Prash Nelson-Smythe
profile image
I have a question: How does a lack of controller imply more immersiveness and less separation compared to a physical motion controller? Most real life action with external consequence involves touching an external object, either a tool or the thing you actually want to affect. Surely, performing physical actions onto thin air would be an less immersive because it is so separated from the reality of our human experience? I'm not saying anything about the ultimate potential of such a system for control or enjoyment should people just "get used to it", but I don't see the hook that will get people using it intuitively in the first place.

Having said that here are some actions where a lack of touch would *add* to immersion:

- Dancing without a partner. This could work as it is movement for the sake of it and naturally needs no controller, tool or object to act upon.

- Supernatural powers, such as Star Wars and god games.

- Being a conductor of an orchestra? But conductors often hold batons so a controller wouldn't really hurt. (I'm scraping the bottom of the barrel here)

I can't think of any more but it would be great if anyone could add to this list.

The list of actions where a controller might improve immersion is much longer and includes anything where you would hold a bat, racket, tool, weapon, steering wheel etc.

There's another class of actions where you aren't holding anything, but you are physically interacting with an object/person: e.g. playing volleyball, fighting games. In this case, the Wii/PSMove and Kinect approaches *both fail* to increase immersion through the control method.

I think you've made an assumption about Kinect's implications for immersiveness and I really can't see what any of this has to do with authored stories. Don't forget that humans have a great deal of natural fine motor control in their fingers/hands/arms compared to other body parts. We are also amazing pressure and texture sensitivity in our fingers and hands. To you use a traditional controller effectively already requires much coordination from your upper arms down to your wrists/fingers/hands and detailed pressure/texture detection in your fingers/hands. Getting *fine* control over a game using your arms, say, without a controller would require remarkable coordination in all the large muscles in your body, while staring intently at the screen for realtime feedback. There is a reason that gamepads have evolved the way they have. I accept they have become inaccessible but I think that means they need to be redesigned rather than scrapped. I can't see a future of gaming without some external physical feedback because it just adds so much to the experience.

Prash Nelson-Smythe
profile image
I just realised something, though I've probably been slow on the uptake here: I understand that Kinect's launch titles will require players to stand for it to function properly. I literally have no space to stand in front of any TV in my flat. The bedroom is too small and the living room has a coffee table in front of the TV (which is absolutely essential). I'm currently trying to remember the living rooms of everyone I know. Only about half have standing room in front of the TV, and I can't think of a single one in London. Does this city lack space for Kinect? For those who don't know, we don't really have the spare acres in the UK compared to the US. Basically, everything is smaller here, even outside cities.

Apparently, later games will be able to handle sitting players due to a change in the SDK, but wouldn't this limit the possibilities for movement? Also, would a game say on the box if it's compatible with sitting for those without standing space? Are people willing to rearrange furniture for this? Perhaps the issue will be ironed out over time, but initial launch momentum is important for a platform.

Could anyone who has played the launch titles at home comment on this?

William Collins
profile image
I must admit, I was sleeping on Kinect until I read this article and then my brain started swirling. If you factor in the emerging 3D technology and pair it with Kinect that would be a really interesting experience. If you want to go even further (and the technology isn't here yet), take a video projector and shrink it to something you could pair with the 3D glasses, use any white wall in your house as the screen and we've just made a huge leap in video game immersion. This would also solve the space issue the one fellow in the UK was concerned about. Wow, the Xbox could actually be something literal as in a sort of box you surround yourself in, whited out on the inside (top missing for light to come in) and the goggles projecting 3D images all around you. Any estimates on how long it'd take for this tech to get to the market? And I wonder if any of the big three already have a patent for something like that. hmm

David Wesley
profile image
As the experience improves, I would expect a convergence between controller and gesture based gaming. You mentioned that the Star Wars partnership "has drool-worthy possibilities." A Star Wars type game would be perfect for the use of a wand type controller along with voice and gesture controls. If game developers are not working on multiple controller gaming already, they better get to it soon. The possibilities are endless.

John Sutherland
profile image
You make a good point, David, and several other people here are with you: sometimes, it's more immersive to have some tactile sensation in your hands. My point was only about traditional game controllers; since it's unlikely that the game story involves that very object, it's likely you'll have some disbelief to suspend. But a wand controller that can be translated, mentally, into a light sabre? Yes, that would be a good thing, and I wouldn't be surprised to see it as a compromise solution in a Kinect Star Wars game. Any good developer should recognize that whatever makes the game better is the direction to go.

Jordan Carr
profile image
I feel that Kinect games will often fall short in connecting gamers to stories because of a psychological effect similar to the uncanny valley phenomenon. I obviously don't have hard evidence of this, but I think there is a similar experience going on.

Tactical feel is an important part of our kinetic memory that I don't think you are taking to mind. As someone who's done many years of martial arts from different schools and taught, generally styles that focus on actually performing moves on opponents fair better and easier for students to remember than styles that just have "forms" without any interaction with other people/objects.

That and when interacting with media, I find the controller allows me to more easily to get "immersed" into the games. I've lost myself into so many great RPGs of past, from Baldurs Gate to Chrono Trigger. I find the wii mote (and even the PC mouse) distracting in my ability to become immersed, because I find myself having to think about movement as opposed to simple button pressing or thumb-pressing an analog stick. Again, this is anecdotal evidence.

That being said, I am excited for some of it's prospects.

Jason Wilson
profile image
I'm more concerned with the user-interface applications for Kinect, but I'm sure they will come up with more interesting games than the Wii-type variety. (Personally, I don't see the point of playing a FPS designed for a traditional controller with Kinect or MOVE. It would prove akward and the difficulty would need to be dumbed down -- a Time Crisis-type game with a dedicated, force-feedback enabled gun would be better.)

How about a (possibly first-person) mode in Madden Football (or's hoping) where you play an eligible receiver and your friend (present or on XBL) tries to cover you as a defensive back?

David Clair
profile image
I'm pretty sure that a controller could/can be used in addition to (or as part of) Kinetic, so to me the possibilities of blending of the two schemes seem like it could be really interesting.

So with that in mind, there would be nothig to prevent a "Star Wars game" featuring a specific lightsaber controller (that would have some normal controller functions) and also Kinteic compatability as well..

Brad Borne
profile image
The biggest problem with interacting with 3d displays with Kinect is, the more direct of an interaction you're trying to simulate, the worse that input lag (and stupid modern TVs' image processing lag) is going to get.

Guh, Microsoft, processors will always get cheaper, but you can't recode the entire Kinect software library to add that co-processor later.

It's funny, though, when you think about it, Kinect is a huge step back from the rumble pack...

Rick Kolesar
profile image
The minute you get someone up in front of a screen, acting out the motions they have to do to play a game, their imagination will kick in. Once that happens, they will loose themselves and feel more immersed in that game. Buttons or not, it's getting us to act like kids again and just have fun.

It's called imagination and Kinect is bringing it back.

J Thorne
profile image
First of all, thanks John for an awesome thread. I have been playing with the Kinect the whole day yesterday so I've had first hand experience with its technology. I am a keen beleiver that the Kinect will change the way games are made and will take the gaming industry into a complete new level. For all those Sony or Wii fans that have been commenting negatively about the Kinect, good luck grabbing your controller and moving your arms to perform the limited functions that they can offer.. As always Xbox is one step ahead of its competitors. Thanks again.

james albert
profile image
Sometimes a new interface is all that is needed for it to be a success.

You can do those things with a gamepad, mice or keyboard, but ask your grandma to use them. There is a clear difference in using a physical device with unnatural movements, and in using your body with natural movements.

The Eyetoy could have done this, but sometimes you need a rebranding and repackaging of a device to make an action or behaviour more popular and accepted. We saw plenty of great MP3 players in the market before the first iPod hit. Just like the iPod, these players can play MP3s, videos, and deal with playlists.

Alex Snel
profile image
While I think that yes Kinect could do some interesting things. I also firmly believe that it will be missing that tactile feedback that gamers love. The problem I see is that Kinect will be basing too much feedback on just visual and auditory. Its a funny thing, I remember getting my first rumble pack and thought it was pretty cool that you could feel a shock when your character was hit or rumble when driving my pod racer. When playing a racing game now and I don't have that tactile feed back I feel like something is missing. I love hand shaking rumbling that you get with explosions and so forth in games, I find that far more engaging than being able to put my hands in the approximate position of a steering wheel or other objects.

So now if Microsoft want to get this tactile feed back they either have the choice of selling a collection of peripherals and controllers so players can enjoy the games more or they can implement a system where players can hold household objects that are roughly shaped like the game ones. But on the latter point, I see a lot more broken tvs.