Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
July 29, 2014
arrowPress Releases
July 29, 2014
PR Newswire
View All
View All     Submit Event





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Coding "To The Metal" is a dangerous ideal.
by Simon Roth on 05/22/13 05:37:00 pm   Expert Blogs   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Edit: It seems a lot of people have really taken this light hearted jab the wrong way, and quite a few others need to perhaps actually read the words that I have written.

This article is not a technical discussion, but one of a business point. People are losing money and shipping games late. Platform specific code is expensive and often thrown away. There will never be another console requiring the amount of fine tuning as the PS3 did. Pushing the limits of hardware is important, but it is an ideal. Something that we should reconsider when we are in an industry of constant layoffs, crippling budgets and failed projects.


The new console generation has reared its ugly, ugly head and the misinformation has come out swinging. Any moment now a "veteran developer" will wander out into the blinding light of the PR storm and start to make bizarre comments about how games development would be far better if we were closer to the metal of the console. If only they had more direct hardware access and less API's choking their creativity.

They mean well, but the excitement of the moment has left them dazzled and confused. The grunts in the trenches, the people actually making games, do not want this.

I am not a clever man. I am however, a pragmatist. And, I am tired of this silly fetishization of doing shit the hard way.

I can see the attraction, the raw speed of blitting to a hardware framebuffer, the power and convenience of direct memory access, it's carnal, sexy, badass. But it doesn’t ship games.

Low level code is worthless..

Firstly, it's a maintenance nightmare. People working on the code will require an acute knowledge of the target platform. That's not something we have time for anymore. Nobody has time for that anymore. Not AAA nor indie. Specific hardware expertise is hard to find and is, more often than not, expensive. Learning to work with a unique machine, especially towards the end of it's cycle is demoralizing for lots of developers. (Anyone who has put their back out recently whilst trying to throw a PS3 dev kit through a window knows what I'm talking about) Some will rise to the challenge, but nobody who likes sunshine or fresh air.

My second, and perhaps more important point is that it locks you to the hardware. Even the laziest beatnik indie outfits are multiplatform nowadays, code has to be portable. Really portable. Even a project built for a single specific platform can find itself deployed on an entirely different architecture at a moment's notice. Opportunities come and go fast, if you can't deploy a vertical slice within a week, you might find yourself out of the game.

Maia Game

In my opinion, Direct hardware access is a bug. It's not the nineties any more and you shouldn't be able to accidentally brick a machine with a few junk pointers. I don't trust my drivers, but I trust them more than a bunch of overworked developers in their thirtieth consecutive week of crunch.

Moreover, it is so often just a security hole waiting to happen. Modern day gamers, be they on phones, tablets, Windows, Linux or even the smelliest of consoles, will have passwords, personal data, perhaps even credit card information on their machines. Software can never be fully bulletproof, and security comes at a price: We need to start planning for more layers of abstraction, with higher latencies and overheads.

If you have time to spare, or programming resources to allocate, you should be working on your tool set. The largest cost in games development this generation will be artist time. Sit down with your artists and ensure they have the tools they need to do their jobs to the best of their abilities. Your audience probably wont care that you got a 2% performance increase with some nasty low level hacking, but they definitely will care if your game ships half a decade late because of the development hell you're heading towards.

I've found that when you accept that squeezing blood from transistors is bad programming, and bad business, you focus on the more important things. Last week I shipped an early alpha of Maia to backers and am currently deploying our latest milestone to three platforms simultaneously. The API's I use to do this are are slow, bloated, have terribly latency and have an abysmal memory footprint.

And I fucking luv 'em.

Maia Alpha 0.25


Related Jobs

FitGoFun
FitGoFun — Mountain View, California, United States
[07.29.14]

Unity 3D Programmer
Activision
Activision — Seattle, Washington, United States
[07.29.14]

Software Engineer - Activision Seattle
Treyarch / Activision
Treyarch / Activision — Santa Monica, California, United States
[07.29.14]

Senior Environment Concept Artist - Treyarch (temporary)
Vicarious Visions / Activision
Vicarious Visions / Activision — Albany, New York, United States
[07.29.14]

Software Engineer-Vicarious Visions






Comments


Johnathon Swift
profile image
For extreme high performance programmers that know what their doing, low level coding is a godsend. It might not be an indie thing, but if you're trying to get the absolute most out of a system when writing the next Crysis or Gears title, then low level access is great.

Simon Roth
profile image
Maybe, but the last Crysis title cost $66 million and failed to make that back. We must learn to push boundaries without constantly facing financial ruin.

Amir Barak
profile image
@Simon
Yes, but for every game written in Unity that "makes it" on the market there're thousands that sink so I don't think it's "low level code" that damaged Crysis...

Michael Ford
profile image
@Amir
I could be very much mistaken, but I don't think Simon is specifically talking about the end of the scale of entire packaged engine and development environments such as Unity3D.

Whilst Unity3D is certainly a possible example of the further end of the scope of the things mentioned in the article, there is still a significant amount of 'in between' area, that if focused on a bit more by the right kind of people with the creative capacity and resources to do so, could really come up with something a lot more of the best of both sides to make it much less of a challenge when as Johnathon mentioned "writing the next Crysis or Gears title" without having to resort to such low level coding as a core necessity.

That's the impression I got from what Simon was trying to lay out in the above article anyway... less about the what is a necessity now, and more what could be the case if the right focuses were adopted to make it happen. ;)

Robert Walker
profile image
While I agree that with the crazy processing power of today's systems, even on mobile devices, there is much more room for someone who might not even be a programmer to go wild throwing things into their games and making things, I have to wholeheartedly and seriously disagree that this negates the need for or importance of low-level programming. I'm also not sure what your definition of "low-level" is when you state that it makes your code non-portable. If you are talking about hand-rolling byte-code for a specific processor, then sure, but it is certainly possible (and with proper planning, not all that difficult) to create and maintain a multiplatform codebase.
To discount something so useful as wholly without merit is very telling of inexperience and youth, which is not in and of itself a bad thing. This naivete toward what already exists and why it exists is a thing that brings us neat new inventions and ideas. It takes both sides of the coin. And, like I said before, at this time we have so much spare processing power that there's opened up this interesting gap where people who do not care about optimization or speed can get in and create something. All in all, whichever part of the spectrum you are on, you are necessary.

Robert Walker
profile image
Also, please don't just read "inexperience and youth" and go into a frenzy. I do not mean it as an insult and I'm well aware that you've released a few titles.

Matt Walker
profile image
I like what cousin Robert has to say.

Amir Barak
profile image
In the end everything we use to develop games (and software in general) are tools in our belt. If you've just spent 2 weeks trying to optimize high level code where 2 days in assembly would have sufficed then writing high level code just wasted your time.

Code doesn't have to be portable, why should it, code needs to be working, correct and coherent.

Doesn't ship titles? are you kidding me? what do you think high level code turns into? not to mention you conveniently just forgotten about 30 years of video games...

James Podesta
profile image
I guess that article is from a very "business" point of view. if you have X dollars to spend on a title, spending it on tool chain enhancements and designing a game that avoids 'requiring' low level type speed is probably good sense. I'm not sure think he must have low technical knowledge though. you aren't going to make cutting edge 60fps animation and physics that is on par with the competition if you don't have some serious low level coding in there somewhere. And customers WILL care if their game is chuggy, even if they don't understand why.

Of course, if you're using an off-the-shelf engine (meaning someone ELSE wrote your lowlevel code for you) and you can design a game where the highlevel entity count isn't going into the 10,000s, then I don't see much place for lowlevel code.

David Klingler
profile image
There is definitely value in low-level code, although not all of your programming has to be based on that. What is also significant is thinking low-level while programming high-level. We have powerful platforms now, but there is still a need for optimization.

I think I understand where Simon is coming from, and I don't completely disagree with what he has to say when comparing high level and low level coding, but I disagree on the point that low-level code is worthless. You probably shouldn't be worrying about coding as close to the hardware level as possible when you're making a breakout clone (just an example), but as said in previous comments, low-level code still has a lot of value when working on the next Crysis. The idea that low-level coding is "bad business" as he mentioned is somewhat true however in some cases.

Jim Rollins
profile image
Hey Simon, while we're on the subject of how higher level programming can save the world, I hope a garbage collector snags your naive and wanky little opinion piece!

Christian Nutt
profile image
You could probably have come up with a more constructive critique than this -- a comment is an opportunity to showcase your opposing viewpoint. Please keep things civil.

Kelvin Bonilla
profile image
Please keep these sort of comments in less professional environments; we have places like Reddit and Kotaku for this.
You could have expressed the same message without the use of profanity.

Andrew Grapsas
profile image
This article makes a whole lot of assumptions without experience (I am assuming the author has no triple-A or large xplat team experence, that is, I apologize if I am wrong). When you optimize at a "low level", you are not limiting your ability to be xplat. Additionally, not all engineers on the team spend their time doing these optimizations. You're talking about a limited set of systems engineers that are there to 1) help with best practices 2) push each platform to the extreme so the game takes full advantage of the unique environment.

If you've used any open source or available engine, you've experienced the benefit of having low level systems engineers optimize to each platform you deploy to. We do not write generalized "one solution to rule them all" code. That's just never going to happen. Each platform is a unique flower and, as such, requires considerations.

Jane Castle
profile image
So much inexperience, assumptions, gross over simplifications and general lack of skill and\or knowledge about low level programming in this article. I don't even know where to begin to start correcting you.

Hell the rebuttal would be just as long as the original article. That's how erroneous your "opinion" is....

If Michael Abrash were dead he'd be rolling in his grave right now....

Karl E
profile image
Instead of pointing out Robin's inexperience and naivety, how about some hard data:

What is the proportion of low-level coders among game developers?

Is it going up or down?

If down, what are the main reasons? Is it just multiplatform development and the decreased need for games to push the limits of hardware, as Robin says? Or is it simply more use of third-party engines and better re-use of code within companies?

Benjamin Quintero
profile image
I'd wager this number is falling. Technology is just getting expansive. Even down-to-metal guys like Carmack are vocal about the fact that what he does just isn't that important anymore. Typically you can have complete garbage tech with pretty textures and it will appeal to a broader audience than amazing technology with programmer art.

In short, with platforms like Unity you can have an entire game company without a single engineer... =(

Jane Castle
profile image
Unity you can have an entire game company without a single engineer... =(

That is until Unity is bought out by a larger entity, they change the licensing terms to squeeze more money out of customers and\or drop support and key employees start leaving or get laid off.

Don't think it can happen?, I worked at a company the licensed the Renderware engine... Things "changed" after they were bought out by EA..... Funny how that happens.....

Things with Unity are not as "rosy" as you picture them. Using Unity is not without it's own special pitfalls and aggravations. As for your comment about not needing a single engineer, well I'll assume that is just grandstanding on your part. ;)

Unity has A LOT of investors and eventually they will want a return..... And by return I don't mean the money they get for being the standard for indie developers....

Benjamin Quintero
profile image
Jane, those are different things. Yes, they will probably sell and things will change but that doesn't change the here and now. You are talking about "what if" and "someday" in a world where someone could release something new tomorrow that is even better.

Maybe the artists I know are more technically minded. I know plenty of artists who copy-paste their way to piecing Unity shaders together and grabbing game components they don't fully understand from the community but you know what? They got it working. The same can't be said for an engineer trying to piece art together other than grabbing some clipart; in which case my original comment still stands.

Jane Castle
profile image
Really? please name a "quality" "ground breaking" Unity title that has come out that proves your point.

Most unity titles I see are just derivative amateurish works which prove the point that it takes more than some artists just slapping stuff together.

Benjamin Quintero
profile image
Jane I just dont know where you going with this argument. You have to stick to one topic at a time please. Now we are talking about quality ground breaking (in quotes. Did I say that?).

Jane Castle
profile image
Sure we'll stick to one topic. You naively think that you can make games by slapping "art" and some scripts together and without the need of a single engineer.

I asked you to provide examples where this has been the case and produced a successful product. You have not delivered an example.

Your statement is similar to saying you can have an entire game company without a single artist. Which is of course a complete fallacy just like your statement is.

As for bare to the metal guys, sure the number is falling and most developers don't care that much anymore. Of course this has also led to the prevalence of bloated software, that has a target fps goal of just 30fps.

Benjamin Quintero
profile image
Jane, I dont dare list examples because it doesnt matter. Any indie on this site knows that success is relative, $100 or $1M and both are a success to someone out there even if not to you or I. Even if one copy was sold and that person was the most diehard fan of that "failure" of a game then it may have still been a success by another measure. So for an example I say pick a card, any card.

Just because YOU dont know any companies doesnt mean they dont exist. Ive never held a $1M in my hands but I know it exists. I dont need to lie to a stranger on a forum to try and be the winner of a debate of opinions :) Im just saying what I know.. Thats it. Choose to believe it or let it go, dont care.

Ditto on 30fps...

Andrew Traviss
profile image
This isn't about what the headline claims. Somebody has to code to the metal. You're just arguing that it's a waste of time for game developers to do it; we should leave that to 3rd party engine developers. If they made any mistakes, we should put the faith in them to fix those mistakes within our schedules.

Using an API versus working against the metal is a tradeoff, not an outright upgrade. When you're working against something higher level, you can and will run into instances where the engine developer made decisions that conflict with exactly what your game needs. The complexity of understanding the quirks of the hardware platform are replaced with understanding the quirks and intricacies of the engine software, and you're still probably going to have to understand some hardware quirks because no cross-platform solution is perfect.

You definitely gain a lot by letting someone else handle coding to the metal, but if you are stressing the limits of the underlying platform, I really don't think you're going to eliminate the problems you've outlined, just change what part of the equation is creating them and which minority group on the team can solve them.

Mike Rentas
profile image
This, exactly. You're trusting the engineers at Unity, Apple, wherever to have written and optimized that code for you. Somebody still needs to code the actual APIs you're calling, and that stuff still needs to be highly optimized. By sinking your game's foundations higher up in the stack, you get to skip a lot of the grunt work, but you're building on the shoulders of giants, and you have little or no control over where they're going to wander off to, or how much they'll care about fixing a specific bug you've come across. If you want proof, just do a search for "iCloud" and "CoreData".

Alexander Jhin
profile image
I mostly agree with this post. But as with most things in life, it's not so black and white.

High level code works well... until it doesn't. But wait until it doesn't work well enough before dipping into highly optimized low level code.

Often, straw men don't have half a brain. But even without Oz's gifts, they can still be helpful to Dorothy.

Daniel Smith
profile image
I'm kind of confused by this article - where are you drawing the line for low-level/"to the metal"? you talk about bypassing drivers, which i think 99.9% of developers would(should) rather not do anyway. I can't believe any significant number of programmers are really advocating this. Nor have i heard many requests for direct framebuffer access in the last 10 years.

If this is your argument then... sure... but not sure who you are arguing against.

If, instead, you are insisting we all stick to high level managed languages (as opposed to C/C++) then i would have to completely disagree. You don't say this directly, but it's implied by saying you use (And love!) slow, bloated APIs. The performance overhead of writing your game in C#/Lua/java/whatever *is* significant. And, on mobile devices especially, keeping to 30/60fps is still a struggle even for moderately complex projects.

Joseph Willmon
profile image
Is it too late to get in on this thread where we misinterpret the author's point and take umbrage at the phantom point we've decided to imagine in its place? Because that seems like some good fun.

Simon, what you're really saying is patently true: Low-level code is absolutely vital, but it's just one tool in the developer's toolbox, and one many developers unfortunately employ far more often than makes practical sense.

Marc Audouy
profile image
There are plenty of great technical innovations we wouldn't have if we didn't have a bunch of crazy nerds that love low-level access and optimizing for the hardware. if you want your game to look good and do stuff nobody else does there isn't much choice. Obviously you don't need all your coders to know everything, just some good engine engineers make a huge difference though. And if they do their job the gameplay programmers will not have to worry about these "details".
Take one example: From Dust. this game wouldn't have existed if there wasn't an engine programmer that optimised the hell out of the simulation to make it run on current consoles. It wasn't necessarily revolutionary but it was inspiring.

I agree that optimising your code for the hardware doesn't guarantee a good game, but in the AAA space as everybody does it you have to do it too or the game will look like shit in comparison.

You are free to ignore it, but graphics and technology still sell games, I don't see that changing anytime soon.

Joseph McNulty
profile image
I agree wholeheartedly here, but I think it speaks to a truth that has been know for a very, very long time, and that truth is that video games don't need to look beautiful. It's that simple. Now, that's not to say that they shouldn't look beautiful. For some games, this is the selling point, or at least a major part of it, such as the Call of Duties and the Tomb Raiders. However, for the great majority of games, as this article points out, focusing too much effort in this aspect can so quickly ruin a project or cause so much bloat in project planning that the best course of action from the get-go is to ignore it. If you have millions, so much positive reputation that you could break even on the die-hard fans alone, and you have reason to believe that the success of your game rides on its visuals, then by all means, put in this effort. Even then, and certainly for everyone else, focusing on building a game whose mechanics and systems that thrive well within the limitations of an existing API is a much more sensible investment of man-hours. Of course, I think the primary reason visuals are seen as low-hanging fruit is because despite the huge efforts involved in bypassing APIs (huge compared to none) are that it's less taxing, creatively, to make a computer render a primitive more smoothly and do it faster than to come up with something interesting to do with the ugly looking one.

Marc Audouy
profile image
low level optimisation are not just for graphics, if for instance your game running on middleware can handle 100 agents at the same time, but a low-level optimisation on a custom engine allows you to have 10000, it can open up completely different gameplays.

Matthew Carr
profile image
If nothing else the article at least generates discussion, but I don't really understand the point being made. If it is one of "programmers should prioritise where they spend their time and not go overboard on optimising for negligible gain in cases where that time could benefit the project overall elsewhere" then in general I agree. Otherwise I would suggest the author has not worked on something that requires what I can only imagine is being referred to by "low level" programming.

I'm actually months in on a project using Unity right now that requires what would imagine would be "low level" programming relative to standard Unity development. This is because of the design of the game and its extreme use of procedural generation and large amount of memory usage/change. So for significant sections of the code I handle memory as best I can in C# myself (Marshal, 'unsafe' code, etc) to avoid getting the GC involved. I'm also doing a number of other optimisations through writing code in C/C++ DLLs for example among others. "Low level" is going to be relative to the platform/environment and I'd say that's what I'm doing with Unity.

The game design would not be possible to achieve effectively (would have significant GC collection spikes, processes that could not run fast enough to keep up with the game speed, etc) without these optimisations. Does that mean the game design should be changed to suit the limitations of the "high level" qualities of the engine, or if it's possible to achieve the vision by writing perhaps less portable, lower level code and in so doing perhaps stand out from the crowd technologically, does that not make sense to do. I think you'll find that the worthwhile low level code in the industry is exactly for this purpose. It allows games and engines to push the boundaries of what would otherwise be possible. Games would not be where they are today without it and it's naive to think we are at a point now where it doesn't still have value.

Final thought, the knowledge a programmer gains through low level coding/optimisations, whether that be understanding more about the memory management, SIMD extensions, virtual method tables, bitwise operations, etc, should make the programmer better overall in how they approach future technical design and allow them to incorporate essential optimisations as they write new code for the first time.

Kevin Simpson
profile image
From reading this article I think we all got a very different definitions of the meaning of low level code.

Managing memory is fine and if code is to be multiplatform wouldn't you HAVE to manage memory?

It seems to me like it's a much more general thing. For example, if you are working on a game for the Xbox 360 and PS3 and write a general engine for both, to further push performance on both you have to deeply explore the architecture. When you are spending the next two months writing SPU optimizations, something went wrong. I think that is what the article was trying to get at. Not managing memory is just bad programming. Taking your time to individually look at each platform and hack your way to 100% efficiency is bad practice.

And at any rate, I think knowing about and worrying about low level optimizations are two different things entirely. I think the argument should be unless there is a very compelling reason, you shouldn't worry too much about very low level aspects of development because it will quickly raise cost.

Kevin Simpson
profile image
I can agree with this article in a lot of ways and think it should be taken as such.

Really ask yourself why you are programming what you are programming. For beginners they often want to program EVERYTHING because if you don't you aren't a real programmer or some other absurd nonsense and for more experienced programmers a lot of time I find very complicated solutions to problems simply because, technically, it's the most "right" solution in a given context even though their is a much more general alternative that which is not as optimized and right, is much more readable, maintainable, and portable.

Alan Boody
profile image
I think his point, in a round about way, is we keep reinventing the wheel with low-level engines, components, etc. for every new game we create. He has a point here. Why would you continue to reinvent the wheel when there's plenty of wheels?

Michael Thornberg
profile image
Because not all wheels fit every vehicle.

Michael Thornberg
profile image
Good tools. And by that I mean tools that makes the path between idea to realization as short as possible. That and having realistic release dates. Those areas play more into why games are missing their dates more than someone tinkering with that extra piece of shader being added, or that extra polygon being drawn. Those don't matter when the content is lacking.

Amir Barak
profile image
You can't kill the metal
The metal will live on
Java tried to kill the metal
But they failed, as they were smite to the ground

C# tried to kill the metal
But they failed, as they were stricken down to the ground
JavaScript tried to kill the metal Ha,hahahahaha
They failed, as they were thrown to the ground

---
Terrible puns aside, for a second I actually did think, from reading the title, that Simon was going to explain that listening to metal while programming is dangerous :D

Alexander Radkov
profile image
I like your attitude, a lot of stuff should be kept simple for as long as you can.

Saw your page, the screenshots look cool.

Jon Hayward
profile image
It's good to see you advocating focusing a little more time on the artist toolchain, especially as content is now king. In my experience even spending a small amount of time improving the in-house tools can result in a markedly better result from the rest of the team.

And once you've implemented something, you can always optimise a little more in the right areas for a massive difference rather than everywhere at once through development.


none
 
Comment: