Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
October 31, 2014
arrowPress Releases
October 31, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

The Holodeck may be possible, but it's going to need game designers
The Holodeck may be possible, but it's going to need game designers
November 13, 2012 | By Mathew Kumar

November 13, 2012 | By Mathew Kumar
More: Console/PC, Serious, Design

Science fiction for many is mere fantasy and escapism, but Lee Sheldon—a writer and producer on Star Trek: The Next Generation and now an associate professor at Rensselaer Polytechnic Institute—sees the fantasy of Star Trek’s Holodeck as entirely possible.

With a seed grant, he and his team are working towards creating the Emergent Reality Lab, using modern technologies to create a “first-generation Holodeck.”

Alternate reality simulations have long been created for military and medical uses, Sheldon said at a talk at the Montreal International Games Summit, admitting that his team are “not the first ones to do this.”

He was, however, particularly dismissive of a recent Microsoft patent with similar aims, mocking the patent image’s unrealistic living space—“If Microsoft wants the patent on that couch, they’re welcome to it.”

For attendees who were skeptical about the possibilities of a Holodeck using modern technologies, Sheldon summarised recent individual progressions in virtual reality, such as the “Cyberwalk” treadmill that lets the friction of the users feet move it allowing them to “walk” while stationary (“I love this—one of the problems with it is however that it costs ten million dollars”) and the Montreal-local McGill University’s haptic floor tiles.

Such technologies when combined, could work towards a Holodeck experience, Sheldon argued.

“You have 360 panoramic screens, 3D stereoscopic projection, surround sound, motion tracking interfaces that include everything from Kinect at the low end, intelligent virtual agents, omi-directional treadmill floors, compliant surfaces, adjustable air flow, temperature gradients, and Smell-O-Vision... why can’t we bring back Smell-O-Vision?”

That’s just technology, however. Sheldon appealed to the fact that game designers would add the important factors that would tip the combination into virtual reality: sustained narrative and contextual play.

“The Holodeck was not about the technology,” he said, "but the narrative immersion.”

Although the first Emergent Reality Lab has not yet been completed, Sheldon took the attendees through the lab’s “dress rehearsal,” an eight week pilot with a class of beginner Mandarin students that involved sets, actors and Kinect-based play all within a single classroom.

Four weeks into the class, the classroom was transformed into a Beijing airport, where they were told to expect an RPI representative to greet them and help them through customs, getting a taxi cab and checking into a hotel and other tasks. Instead they found only a crumpled piece of paper implying the representative had disappeared and were left facing native Mandarin speakers playing roles such as a custom agent who would not respond in English.

“This was designed for an hour and fifty minutes of play, and the students completed it in half an hour. From that point on there was emergent play, they started to ask questions to the Mandarin actors relating to the RPI rep’s disappearance,” Sheldon explained.

Other days included a visit to a tea room, which included interaction with a virtual actor via Kinect.

“We served real tea,” Sheldon said. “you have to be careful mixing this space with real furniture and projected furniture, make sure people aren’t putting a cup down on a table that doesn’t exist.”

Sheldon described that his earlier experience designing ARGs helped him in preparation for designing a Holodeck-like experience.

“The real world is totally unforgiving. There’s no patching reality. In an ARG things change from moment to moment; you can’t think of everything and you have to be able to adjust to how your players play the game.”

“Your players are smarter than you,” He continued. “They’ll figure things out faster than you, like the emergent play in our airport. You have to be very careful, you have to be very flexible.”

“Forget linear storytelling, we have to come up with new paradigms for this. We cannot tell stories in the same way to make this thing work; in fact we have to give up some authorial control. But not all of it, give up just enough that the players /think/ they are in control. That’s most important.”

Related Jobs

Rensselaer Polytechnic Institute
Rensselaer Polytechnic Institute — Troy, New York, United States

Assistant Professor in Music and Media
The College of New Jersey
The College of New Jersey — Ewing, New Jersey, United States

Assistant Professor - Interactive Multi Media - Tenure Track
Next Games
Next Games — Helsinki, Finland

Senior Level Designer
Magic Leap, Inc.
Magic Leap, Inc. — Wellington, New Zealand

Level Designer


Joshua Darlington
profile image

What are the holography spex?

Nice Kinect use.

“Forget linear storytelling, we have to come up with new paradigms for this. We cannot tell stories in the same way to make this thing work; in fact we have to give up some authorial control. But not all of it, give up just enough that the players /think/ they are in control. That’s most important.”

IMO the best entertainment designers/balancers are the players themselves. They have better real time information about player interest level, engagement profile and etc. The trick is to offer them dynamic generative feedback systems to play with (esp other people). If a player is fatigued or is feeling passive - an increasing amount of linear top down design can be a failsafe.

Thom Q
profile image
Good. When it's finished, first holo-novel i'll be playing is Dixon Hill.

[User Banned]
profile image
This user violated Gamasutra’s Comment Guidelines and has been banned.

Michael Pianta
profile image
Exactly what I've always said. In the show you see people sitting down in chairs or couches, or other wise handling objects that don't actually exist and speaking with (and touching) people who aren't there. I question if that is in any way possible. Maybe with nanobots that reconfigure themselves into the relevant size shape and color? Otherwise, if we just want a visually convincing illusionistic space, it's not that hard to imagine. In some ways we already have that - ever since linear perspective was discovered in the renaissance.

Sarah Johnson-Bliss
profile image
Yet another Star Trek technology becoming a reality. Things are getting seriously cool lately. I mean, a NASA scientist has seriously brought warp drives (nearly identical in operation to Star Trek's) into a reasonable realm of possibility. We have PADDs already (iPad). We can get faster than light communications with Quantum Entanglement. There's some students working on cloaking. Another group of students are working on what seems to be anti-gravity. Australian scientists are working on teleportation. There's also talk of tractor beams. And now, a sort of holodeck system.

Put these together, and we'll be set to travel the stars!

Where do I sign up?
(Let's just hope that the Klingons, Romulans, Borg, Dominion, etc. are all figments of our vivid imagination... not premonitions!)

Rob B
profile image
Well I hate to be a party pooper but,

'We can get faster than light communications with Quantum Entanglement.'
Im afraid we cant. Aspects of quantum mechanics appear to go faster than light, no information is ever and can ever be sent using it though.

As for the Alcubierre drive, the power requirements were probably the least of the hurdles to get over. The primary issue is that it requires exotic matter that may simply not exist. After that there are a host of possible effects that may make using it impossible and/or lethal to use. The research is fascinating and should lead to increasing our understanding of relativistic effects but its not going to get us in to deep space for the foreseeable future.

Exciting times in science, but often sensationalised.

k s
profile image
Do you have a link to this research regarding the warp drives as that subject really interests me.

Ilya Belyy
profile image
k s,

I believe it's this one

Whilst it lowers the energy requirement, there are other fundamental problems left to overcome, as Rob noted.

Rob B
profile image
2011016932.pdf You can go right to source, most of it doesnt require in-depth knowledge.

Incase you were wondering where this 'exotic matter' malarky comes in, thats what the references to negative energy density are about.

k s
profile image
Thanks Ilya and Rob, I do hope the obstacles in this problem are over come but for now we have to wait :(

Joshua Darlington
profile image
I expect that we will see many other changes in display before we get to this.

For example - video walls are being made easier and easier with plat panel tech becoming paper thin. You will know that this tech is getting close when all store signs becomes video signs. Dome theaters are another display innovation on the horizon - there is a faster internet being rolled out to the scientific community to support telescopes - so its based on dome display spec.

Right now, I would be happy with a lo rez, low frame rate, horrible lag time, monochromatic hologram for my kinect.

tony oakden
profile image
I hated the Holo deck episodes on Star Trek. They where contrived and, even by star trek standards, implausible.

Philip Michael Norris
profile image
For whomever designs it, be sure to include a 'private browsing' function. I don't need people seeing my holodeck history. I can only imagine what it would look like.

Christian Kulenkampff
profile image
For me the cool part in holodecks never was the holodeck stuff, but the AI. AFAIR the AI often is the game designer in those holodeck stories, so when holodecks are possible we may not even need game designers...

Tommy Lee
profile image
I'm building a team.. still need a neurologist, quantum physicist, expert in anti-gravity and a genius in spacial relativity. Let me know.. thx :)