Obligatory FXAA Post

I did a quick search through AltDev and I don't think anyone else has really talked about FXAA on AltDevBlogADay yet, but the time is long overdue. I'm a little late to the game trying out Timothy Lottes' post-process anti-aliasing technique, but I had been noticing people saying great things about it when I finally decided to give it a serious look over 4th of July weekend. I had already been thinking about writing about my experiences with it when I saw Eric Haines had posted up a glowing review of it over on realtimerendering.com. So if you don't trust the judgement of some intern that doesn't even have a college degree completed, please refer to all the more qualified people that are having similar experiences to me.

The Problem at Hand

For those of you that might be reading this and haven't drank the graphics programming Kool-Aid, I'm going to fill you in a little bit as to why we care about anti-aliasing.

In our line of work, the end result of a rendered scene is typically a 2-dimensional array of colors that is displayed on the user's monitor. Just like how audio (which is originally analog), is quantized when it is made digital, the image is built into a discrete number of pixels during the process of rasterization (i.e. filling in each triangle). The measure of how many pixels are used to display the image is of course known as the resolution, which is why things become increasingly blocky as you lower a game's resolution, due to there being fewer dots per inch.

Even at high resolutions, individual pixels can still often be identified by the viewer, because the lines and edges of triangles form distracting "jaggies". This is noticeable along the outline of an object being displayed due to rasterization, but can also occur on interior surfaces for other reasons such as shadows and texture map resolution. The user typically picks the resolution their monitors are set at, but most console games provide 720p or 1080p images to the TV, and monitors tend to follow suit (my laptop is set to ~720p). Other than relying on hardware to pack more pixels into the same physical area by increasing the dots per inch (which is something that Apple is claiming with it's retina displays, but check out this post for an interesting look at that), we have to find ways to smooth the transition between pixels. Perhaps the simplest strategy to dealing with aliasing is to render into a texture twice as large as the target resolution and then downsampling it to actual output resolution. This allows you to take the average of every four pixels, softening out the boundaries by averaging the edge colors together. However, this also just straight up sucks for performance. You end up processing 4x as many fragments from the higher resolution, and you have 4x the memory usage for the buffer. This is way too high a cost to pay for some smooth edges.

The Hardware Option

There is a hardware accelerated method, MSAA (Multi-Sample Anti-Aliasing). This operates by computing additional samples when rendering the frame instead of rendering additional pixels and downsampling. The fragment shader is only run once for each group of samples. The problem is that it doesn't work with the increasingly popular deferred rendering techniques, because you can't really take more samples of a buffer you've already rendered. In short, the damage is already done, the data has been discretized to a particular resolution. Furthermore, I've always found that MSAA is still pretty expensive (but then again, the only way to make your rendering take 0 ms is to quite doing rendering and switch to a job in finance).

New Maps of AA-land

The desire to use deferred rendering has pushed alternate forms of anti-aliasing to get a lot of attention (I mean who doesn't need more acronyms, right?). Perhaps the most prevalent that you may have heard of is MLAA, but most of the techniques that I'm referring to here are post-processing techniques that rely on detecting and softening edges. By doing anti-aliasing as a post-process, it will work seamlessly with deferred rendering, and pretty much anything else for that matter. If the technique only needs the color buffer access, then it can even be applied to video/screenshots/whatever of existing games, which I think researchers in this area have found as a great method of showing off how their work.

MLAA was originally a CPU based technique developed by Intel, that has since then been adopted to be done on the GPU. This has been outlined in GPU Pro 2, Game Developer Magazine, and around the net, but if you're not familiar with it I'll make a few brief points about it. It essentially boils down to storing edges in a texture with different colors indicating which side of the pixel the edge is located at, and an additional buffer is used to calculate the blending weights for the blurring.

Impressively, they found results falling somewhere between 4x and 8x MSAA, while being 11x faster than 8x MSAA. The memory footprint of the technique is 1.5x or 2x the size of the back buffer depending on the hardware (2x for the Xbox 360 for all you console devs who probably have already heard everything I'm saying). That's pretty good for something that solves the problems encountered with deferred rendering at the same time. This is undoubtably why the technique has garnered so much attention, and I really recommend the article in GPU Pro 2 if you want a clear view of all of the details.

Enter: FXAA

As you may know from reading other posts, my big side-project/hobby/thing is that when I give a new technique a try, I do it in Unity because it a) is usually not straightforward and requires actually understanding the technique to get it working and b) can be evaluated using the many projects I've done in Unity previously. From looking over the details of MLAA, I knew it would probably take a full weekend to get it right, and I had been procrastinating quite a bit with getting around to doing it.

When the third iteration of FXAA rolled out I decided to take a look at what it entailed. I knew in the back of my mind that FXAA was a strictly luminosity based technique, which is interesting to me. The authors of MLAA recommend using depth to determine edges for best results and performance. However, luminosity based techniques offer the advantage/disadvantage of smoothing boundaries that exist in places other than depth, such as with aliasing on texture maps and with shadows. The downside is that this can produce results that are too blurry in places you don't want blur, such as with text on a prop in 3D space. I once tried a very, very simple luminosity based AA filter, that resulted in too many cons (especially with blurry text) for me to seriously use. I was curious if FXAA would give me similar problems.

With the intention of just looking over the code briefly, I suddenly found myself staring at a very simple and easy to use code base offering a ton of well explained pre-processor options for target platform and quality. Being around midnight when I started looking, I quickly decided that porting the higher-quality PC version of the HLSL code to CG/Unity would be fun. There were two steps involved:

1) At the end of all the other post-processing, calculate Luminosity and slam it into the alpha channel (super easy to do).
2) Perform the FXAA pass. Porting mostly involved fixing texture look-up syntax.

No extra buffers involved. This cuts down on the extra memory needed for MLAA, and the code seemed simple enough that it would probably be pretty fast. I ported all the code within 2 hours and then went to sleep. The next morning I finished setting it up for use in Unity... and was blown away by the speed and the results. Here's a breakdown of what I got running in the Unity editor at ~720p, using Dust, a previous project of mine, to test it. These are PNG's cropped down at native resolution:

Shot 1: No Anti-aliasing

Shot 2: FXAA3

Shot 3: 6x MSAA

It takes FXAA3 only ~1 ms [Note: corrected from an erroneous order of magnitude type when I first posted this article] on my laptop (MacBook Pro with an NVIDIA GeForce 320M) to be completed (including the luminosity calculation), and as I mentioned, no additional memory either. I'll pay that millisecond any day of the week for that quality of anti-aliasing. Furthermore, the blurriness on text was much more acceptable than the fast blur I had tried previously. Note that this text is not really meant to be read, but rather recognized to match the voice over so the player can understand that the voice is that of the journal's author. I wish I could credit the sources where I put together the fast blur from, but it's been more than a few months since I implemented/ditched it. Here's a comparison:

Shot 1: No Anti-aliasing

Shot 2: FXAA3

Shot 3: Fast Blur

This is getting a bit more to the personal opinion end of things, but even though FXAA3 does blur the text, I feel like the fast blur makes it almost uncomfortable to look at. I get the sensation that I'm getting tested for a new prescription of glasses, and have to recognize out of focus letters. To me, that seems like a pretty good indication that your luminosity based AA technique isn't up to snuff if you get that type of blurring. FXAA3 on the other hand seems acceptable enough that it'll definitely be enabled in future builds of Dust that get pushed up onto my website.

What did we learn?

Hopefully we learned that FXAA is both fast and doesn't require additional memory, and is trivially simple to implement or port. There are versions for the 360, PS3, and PC, as well as HLSL and GLSL variants. It's a luminosity based solution, and comes with the associated pros/cons those techniques have, but I've found FXAA to minimize the cons. It should literally take you at most a few hours to get it up and running in your game and evaluate if it's a good fit for what you're doing. Furthermore, it is in the public domain (credit to the post by Eric Haines for this twitter snippet):

Although at this point, I'd the amount of quality you're getting for the effort of implementation, it might as well be under the beer license. Finally, I would point out that Timothy Lottes is still improving on the code, and has already released version 3.9 in the time since I first touched it. You can find the latest source links and updates on his blog: http://timothylottes.blogspot.com/.

What do we really want from our rendering?

As of late on Twitter and elsewhere, I've had/read a few discussions about the goals of the time and energy poured into our pretty graphics, and I realized it might make an interesting blog post. I can still remember sitting in my first graphics programming class and hearing the professor talk about how it is the more often the goal to simulate film and photography than real life. This is a pretty broad generalization that I've found to actually hold true more often than not.

Even after I decided to write about this, I spotted this part of a fitting conversation between Emil Persson, John Carmack, and Stephen Hill:

This is actually a dilemma that is difficult to answer, and bears in mind questions about what other disciplines and groups have in mind. To me, the problem is that both approaches could be perfectly acceptable. In fact, I would go so far to argue that as broad generalization, that game players almost *want* to be put into a movie when they play a video game, often demanding increasingly cinematic experiences. On the other hand, I think a lot of gamers imagine the future as being "like you're actually in the game," which strays more toward the argument that we want to mimic the human eye over the camera lens. Maybe thinking about what players want isn't the right way to go about it, players want to have fun, and they don't always want to understand why. I have a feeling designers would care even less about this issue, caring more about effectively immersing the player in their levels and plots, which is not entirely relevant here because though because this is more heavily dependent on the actual content of the game. We don't care about content so much, we care about what the "eye" is.

The argument here is very much a discussion of whether or not rendering programmers are trying to simulate a film camera or a human eye, with a very significant part of that being the lens. If you're not familiar, some effects used in video games such as depth of field and lens flares are largely due to issues with capturing images by using a camera, instead of directly with a human eye. Consider that these are some of the things that gamers rave about over every time a big title puts out a new trailer or demo, yet if it seems a little crazy that these things we sometimes try very hard to reproduce, are actually artifacts. I believe there are a couple factors at play here, a big one being that filmmakers have employed the classic "it's not a bug, it's a feature!" mentality and have often used these "flaws" to great effect, which ties directly into what Stephen Hill is arguing in that Twitter conversation. Depth of field in particular jumps to mind as an effect useful for storytelling, and even things like the poor film quality simulated in Tarantino and Rodriguez's Grindhouse movies play a role in setting the mood, and I'm sure that such an effect could play a noticeable role in the right game. On the other hand, there's this amusing little comparison between video games and photography that shows just how ridiculous this copying of camera flaws can get:

Yet one has to admit that HDR bloom tends to get a good "wow" factor from players, which leads to programmers running around on the internet sarcastically shouting for "MOAR BLOOM." After several experiences with team members boosting effects way too high (not to say I haven't done this myself, the key is to expect to adjust it back down), I've begun to understand why so many graphics programmers hold the physical basis of what they are creating to be of upmost importance. That being said, is it a physical basis of a camera or a human eye? I've read papers that discuss the physical basis of how an effect occurs in a camera, and I've read papers that discuss how an effect occurs in an eye, so really either could apply. However, that reminds me of a similar discussion...

Part 2: Reality or Art?

A related and perhaps more interesting discussion, is whether what looks "good" or what is actually "correct" is more important. This is something that when I started making games, I wasn't familiar with how technical this issue could get, but I think it stems from more than a few core deviations between artists and programmers. When it comes to visuals, everyone wants to have the "best," but just like the debate of whether or not that means duplicating camera artifacts among rendering programmers, there is the question of whether or not the best result is the one that is most realistic or the one that the artists think is best.

In a simple world the answer is easy when both parties agree. However, as technology capabilities grow, there are many more chances for rendering programmers to implement systems that do a much better job of modeling how these effect happen in real life. As our graphics improve, art content may start to look different in fundamental ways. Shifting our techniques to draw from a more accurate physical basis can allow us to create even more stunning visuals, but when I was reading through the course notes for Naty Hoffman's "Crafting Physically Motivated Shading Models for Game Development," I wondered if suddenly changing the way the specular highlight behaves might have a traumatizing effects on artists. I can easily recall several artists and designers that I've worked with that would take the side that a physical basis is not nearly as important as achieving a "good look." It doesn't help that programmers are not artists, which I know can make a lot of artists doubt their credibility if they suddenly think an "improvement" makes their art look worse. And speaking as a programmer myself, it doesn't help that sometimes we can be pretty stubborn ourselves, especially when the math has been proven out that the change makes for a more accurate simulation of real life.

This issue is something that I've become increasingly aware of from the excellent work of Naughty Dog's John Hable, from which I'd like to draw a couple examples of where this debate can come into play. In Hable's 2010 GDC presentation about some of the rendering techniques used in Uncharted 2, he discussed the importance of doing calculations in linear space. One consequence of this is that the diffuse falloff will be much sharper when calculations are done in the correct space. This is something that artists might initially might be turned off by, because have soft falloffs is supposed to be a good thing, right? But this is really just a case of not knowing what we really want, and Hable points out the lack of soft falloffs in the film Avatar, which as you may recall, was highly praised for its visual quality. In the realm of skin rendering, something that Hable is also involved in, it is often noted that as we account for subsurface scattering, we need to use detailed textures as input. Artists may have had the inclination to blur details in the textures themselves from wanting to account for the subsurface scattering. However, once programmers account for the scattering in code, they need to become aware that they no longer need to soften texture details to achieve the best results. If you're interested in checking out some of Hable's work for yourself, I highly recommend you visit his website www.filmicgames.com.

There will always be a great deal of approximations and fudge work involved in real-time rendering, but where the fudge work happens is a constantly moving target as graphics hardware continues expand the possibilities for what can be properly simulated. As this target moves, it also creates an environment that reminds me of something a graphic designer once told me about people that are truly pro status at using Photoshop: the people who really know what they're doing can make a dozen smaller tweaks that are hard to notice on their own, but when put together the end result is stunning. I think the same can be said about the direction that rendering is going. Just remember that improvements are not always improvements in everyone's eyes, let alone easy to notice, and it doesn't help that we all mistakes and sometimes our "improvements" are actually in the wrong. It's all fun and games until someone gets stabbed over chromatic aberration.

Death of a Project

This is actually the follow up to my article on my adventures with Off-Screen Particles in Unity, but being that it ended with me deciding to kill the feature, I've decided that my process towards doing so would make a much more interesting post. I feel that knowing when to throw in the towel or radically change approaches is a good skill to have when developing any game, especially because this is the type of engineering endeavor where “seems right” and “ships on time” is so much more important than things like “is actually right” or “bridge you're engineering doesn't collapse when the wind blows”.

Context

An important thing to keep in mind when you're working on a feature is the context of why you're doing it. This includes everything from how much time you can allocate for working on it to how much it impacts the user experience in your final product. In my case I was trying out a potential optimizations to hopefully allow an early prototype from a designer become feasible, which was not due to his heavy use of particles to create a dust storm, causing performance issues from heavy overdraw. This in itself is an important thing to keep in mind when doing an optimization, is whether or not what you're optimizing is actually a bottleneck in your game.

In my case, I knew the overdraw was becoming an issue because I: 1) reviewed possible reasons why our tons of particles could hurt our performance (to be fair, I was already a little familiar with the issue: knowledge is power!), 2)tested the effects of changing things like particle size, count, screen resolution, and 3) attempted to kill the designer for not using Unity' nifty overdraw visualization in the first place (Side note: attempting to kill a designer should only happen if you work in a strictly unprofessional environment, like a University lab, and will rarely have a direct impact on perf).

After identifying the problem, the next thing to do was decide how I could address it. Because the particles had only a minor impact on gameplay, the team cut it at the time and focused our efforts elsewhere for our deadlines at the time. However, I knew that being able to ramp up the particle density would really enhance the look and feel of the game, so I kept the problem as a side project.

Ramping Up

The project was slow to get started, due to it being on the back burners for some time. I started to really make progress on it when we were transitioning to hopefully ramp up our work on the game once again after many other projects that had been distracting team members for some months began to wind down (this is a continual problem in a University setting, but that's a story for a different day). I knew that the feature would be important if the proposed design changes happened, and that the work would be wasted if the designers went ahead and designed new zones without the feature at their disposal early on. With this in mind, the project quickly rose to a much higher level of significance. I spent a lot of time when I wasn't with the team cracking away at my work, and putting extra hours in at this point can often pay off.

Putting in that extra effort at the right time can really pay off. I quickly began to see measurable results and progress. The implementation that I discussed last time does have a substantial impact on a scene I brewed that's mostly particles at a density similar to what design wanted, resulting in a greater than 10 ms performance increase in my test scene at a 1280x720 screen resolution when rendering particles to a buffer smaller by 2x. I could do even better than that when using a 4x smaller buffer. That clearly identified two things: 1) that the feature warranted further refinement and 2) that I was right about the bottleneck, as the increase in perf proved that I was not wasting time going down the wrong path. However, in my first post I detailed artifacts that are noticeable in our particular game, and that I intended to explore the mixed resolution rendering as a solution to those issues.

Spotting the Wall

I knew that going to mixed resolution would require substantially more resources for the rendering and compositing of multiple passes, but would also allow me to get away with using a 4x decrease in our smaller buffer. So I started out with the intention of laying it out as simply and as efficiently as possible, and then tune from there.

After a little more work, I had it: mixed resolution off-screen particles. They had some room for quality improvements, but they definitely did a lot to fix the artifacts from before. I had also gained something else: my offscreen particles rendered with almost no increase in perf. I had lost all of my savings from my earlier work, which also meant that I may have moved my bottleneck as well.

I had a couple options at this point: 1) tediously work to increase perf and quality at the same time, 2) scrap mixed resolution and try to find another solution, or 3) kill it. The first option bears the weight of what would most likely be a lot of work on a project that I only have a limited amount of time to work on, and is especially difficult to do without source level access. I did try a couple quick and dirty thoughts on the second option, but going back to the drawing board has its own costs associated with it.

So I Killed It

It's not easy to kill something, especially when early work shows promise. However, it's an important thing to be able to do. I have other things to be poking around at in my spare time, and a feature that the game can live without is certainly not worth it. I had invested a reasonable amount of time into the feature already, but that didn't mean it was worth further pursuit. I suspect further work on it would be roughly equivalent to doubling the scope of the feature. Your time is always one of your most valuable resources, especially when code familiarity in itself makes you a valuable asset to your team. Getting stuck with your wheels spinning on a feature that isn't worth the benefits for the work isn't going to help anything, no matter what discipline you're in.

Game Programming Interviews and Tests: Entry Level Edition

I'm starting to realize that most of my blog posts begin with me making excuses about something as well as linking to somebody else's post or article. Spoiler alert, today will be no different. The excuse is that this post is not my follow-up to my last one about my initial adventures with off-screen particles, due to a lack of time to work on it while moving into a new place and also some disappointing results with mixed resolution rendering. So what I'm talking about instead is my experiences with interviewing for programming intern positions throughout this past Spring. A while back, Jaymin did a great post about programming tests and demos, but I feel like I might have some insights for aspiring entry-level game developers.

The Hunt

I started my hunt for getting professional development experience by testing the water by sending out a few intern applications during my second year of college. The first step to getting any job is to actually apply for it. Typically this happens online these days, especially for intern positions. This brings me to my first point: looking good on paper counts for something. No matter what, you have to look appealing enough for a company to think about investing the time to actually talk to you. This was evident because, even though I was completely unqualified at the time, I scored a programming test for the Insomniac core team. I'm sure the only reason that happened was because I didn't let my grades slip during my first two years of college (proving that even the most painful "general education" class in existence still counts for something), and I also had experience working on several game development projects around the University. In short, I looked good on paper. Having a well thought out resume and portfolio are crucial, and the evidence adds up both from responsiveness to online applications and responsiveness from studio representatives at job fairs, such as the annual career pavilion at GDC.

After making it past the first hurdle, there are typically a two things that happen next. You will almost always need to do a technical interview and a programming test, although the order that they happen can vary. The Insomniac test was proof that I wasn't qualified yet, but at least it showed me what I needed to do to get past that initial challenge of getting noticed in the ever growing pool of fresh applicants looking for work in game development. So what exactly is in a programming test and how should you prepare?

The Test

For an intern, most tests are emailed to you, and you typically have a specified period of time to send it back. I've had several tests structured as a small programming project, in the style of typical work required to make a video game from the ground up. Anything that requires a substantial amount of work for a single problem is usually very generic in nature, so don't expect heavy AI or Graphics problems if the position isn't specifically for that discipline. Some of the more “small project” style questions that I've had have included memory management, binary file I/O, basic 2D collision detection, and string parsing. Almost every test requires C/C++ to be used, so make sure you've had practice! Also, take Jaymin's advice about showing a smart answer instead of a correct answer. Most problems have many solutions, is yours clean and optimized, and does it handle a good spread of use cases?

Some tests are limited to just a few hours, and I tend to find those to be more towards the stressful end. Perhaps the hardest test that I ever took was a 17 page short answer onslaught crammed into the span of 3 hours. It had required sections on bug finding and solving (things like what to do if the game crashes in release only, in a soak test with no memory leaks, etc), short answer code writing (think bit twiddling, list reversing, basic assembly writing, etc), and math problems (involving vectors, matrices, and trig) together forming the first 12 pages, and then 4 optional sections on gameplay, graphics, core, and networking programming (the remaining 5 pages). For me, it highlighted just how different a test could be from the “small project” style test.

Interviews

A technical interview tends to feel different in nature than a programming test, which was surprising to me the first time I had one. One thing that you should know before any interview is the difference between a struct and a class in C++, this is a question that gets asked over and over again, partly to break the ice. More than three quarters of my interviews have started with that question. Be prepared to know about inheritance, virtual functions, dot products, cross products, const correctness, cache use, and game engine design. Also make sure that you're prepared to talk about game projects you've worked on, especially those you have on your resume or portfolio. Typical questions about your work include asking about your individual contributions to group work, what kind of difficult problems have you overcome, and if what is something (maybe just part of a project) you've worked on that you wish you could go back and redo.

Something telling of being a programmer is that no one has ever asked me if I have played games made by their company, where as I know designers that have had tons of questions about redesigning games made by the company they were interviewing with. Don't sweat it if you haven't had a chance to pick up a company's latest game, spend your time refreshing yourself about the details of multiple inheritance and less commonly used C++ keywords instead. If anything, you might be asked what games you've been playing recently, just to make sure you play some games.

Rejection

Be prepared to not have everything go smoothly the first couple jobs you interview for. I've botched tests and interviews alike, sometimes from things completely out of my control, but the important thing was that I kept working to improve myself as a programmer afterwards. Have you read Jason Gregory's excellent Game Engine Architecture? Do you have projects where you can show how you specifically contributed? Can you get involved with a mod group or a student club or a something to get experience working with others? Have you done projects completely on your own?

I've heard for a long time that game industry jobs require personal connections and luck, but to be honest you just have to be at a certain level of experience as programmer and be able to prove it. I've always maintained that the key is to get a feel for what that level is, and then set your goals accordingly. Make sure you feel like you're continually getting closer to that end goal of landing a job (and in a work environment that suits you best!). If it seems hard, that's because it's certainly no cake walk to get there in 3 or 4 years time. This is especially because until you have a game development job, you usually have to spend time doing things like going to class or working a non-games related job. I attended a talk at my university last Fall about interviews and tests for general software engineering jobs, and I left knowing that non-game industry interviews are laughably easy in comparison, even for "top-tier" companies. It's not easy, so keep at it!

Adventures with Off-Screen Particles

This is hopefully going to be a two part post because I didn't get to fully explore it as much as I wanted to, as I've been losing my mind during another eventful week of final exams in college. I considered postponing this and pulling something out of my ass for today... but then I saw fellow contributor Wheezie had already stolen that spot with his post “What Did I Do Today?” If you haven't already, you should check it out to at least read the amazing comic that he put at the end of it. Especially because half of the time I don't think people who aren't designers understand what designers actually do. I mean, it's pretty clear that graphics programmers spend all their time inventing new acronyms for anti-aliasing techniques except for when they're tightening up the graphics on level 3, but designers- they're a whole different mystery. Wait, what was I actually going to be talking about in this post? Particles, that's right...

Particles

Particle systems are cheap, flexible, and easy to set up, especially if you're using Unity3D, as is often the case with the projects happening at my University. This is why on a lot of my projects that I've been on (small student projects with tight deadlines), a designer often picks up the task to reduce the workload of other team members. This was the case with one such project that I was working on the fall. That project was Dust.

As you might guess from the name, there's a lot of dust in Dust. So much so that it takes place in a desert. Some of the designers on the project were tasked with helping to build the ambience for the game with some particle systems. An innocent enough task, but when it came time to integrate the work into our initial prototype, it was clear something was awry.

The problem with video game content creation is that people often have to learn the hard way how their work can impact performance. In this case, we wanted substantially more particles than anything other than a top of the line computer could accommodate (like the one we had been working on). And to be fair, a game like Dust should have as many particles as we can manage to have without blowing perf. One of the biggest impacts on performance from large numbers of particles is overdraw, especially when the particles are filling large sections of the screen:

[caption id="attachment_5183" align="aligncenter" width="484" caption="Massive overdraw from dust particles"][/caption]

This is a shot of the Unity editor's visualizer for overdraw. The bright areas are spots where many pixels are being drawn over and over again. This is particularly an issue with particles because the system contains many overlapping quads, where pixels will be drawn over many times. Keep in mind that the given shot is from the current version of Dust, his systems were originally much, much heavier on overdraw.

Doing it Offscreen

A solution to this particular consequence of particle effects is presented in the fabulous GPU Gems 3 by Iaian Cantlay. The technique boils down to reducing the number of pixels being rasterized by rendering the particles to a texture that is then composited back into the main image. The rendering is done after a depth buffer has been formed, so that the pixels in the particles that fail the depth test can be discarded properly as you render them. This means that the color can be applied directly back into the scene, which is especially easy if your particles are additive like ours were.

I started considering it as a possible solution to allow for thicker particles in future iterations of the project when I read through Shawn White's post on his implementation in Unity for Off Road Raptor Safari HD. It didn't take too long for me to adapt my own implementation a week or so ago, but there are definitely several issues that come to light very quickly (many of which are discussed in the GPU Gems 3).

Problems / Solutions

[caption id="attachment_5189" align="aligncenter" width="450" caption="Zoomed in view of cracks from point sampled depth"][/caption]

Some of the issues that I immediately encountered were visible halos between solid objects (notably the boat that the player controls in Dust) and particles after compositing. One solution to this problem is to have the depth buffer, which is being down-sampled due to the low-res target, take the minimum or maximum depth to be used when downsampling. As noted in the gems article, this is really just a rule of thumb, but it did indeed fix issues with cracks.

[caption id="attachment_5190" align="aligncenter" width="450" caption="Zoomed in view of depth farthest heuristic"][/caption]

However, it made it apparent that the much more serious issue is the aliasing that occurs due to the buffer being a lower resolution. It's very obvious along the edge of the sail in Dust. My goal was to get the particles to render at quarter res, but right now only half res comes close to an acceptable quality. One thing I could do is render the edges at full res in yet another pass, but the big question in my mind is whether or not two passes for particles would still result in a performance increase. We already have to create a depth buffer specifically for the purpose of the off-screen particles, because we don't do deferred rendering, it's not used for much else.

Next Time

Hopefully by the time my next post rolls around (and I'm done with finals), I'll have some more results (and perf numbers) and give some insight onto whether or not we decide to actually use it in the game. On the plus side, we modified Dust's design recently to be more separated, so potentially we can split it so that it's only in effect where the particles are prevalent enough to warrant its use. Also, I'll try to set up some fresh test scenes that show the effects better, considering that upsampling the in game shots went horribly.