The naive approach to adding anti-aliasing to the renderer is to do super-sampling. Super-sampling will sample the scene multiple times per pixel giving you sub-pixel accuracy. This is equivalent in complexity to rendering the same scene at a higher resolution. The problem with this approach is that the number of rays being traced increases dramatically. If we wanted a decent anti-aliasing of 16x we would need to sample a 512x512 image with 4,194,304 rays.
The result of having no anti-aliasing is shown above. The classic stair-steps are present. This happens because when only one ray is traced per pixel, that pixel only gets one color. The ray, being infinitely thin, can only strike one object. So, each pixel get its full contribution from a single object (ignore transparency for a now).
Anti-aliasing works by blending multiple samples together. It gives nice smooth edges. However, the cost for the normal super-sampling is very high. Instead of sampling the entire scene 16x I decided to find the areas where quality would suffer. This is the approach of identifying areas where we need super-sampling, and then only doing the extra samples there.
The first step is to calculate an edge map based on the original single-sampled scene. This edge map shows where color changes will cause us to notice the under-sampling that occurs. The areas where there aren't edges won't need the extra samples because we will not be able to detect the effects of a single ray.
The edge map is calculated at the same resolution as the original color surface. White are areas where extra sampling will be needed. We can pass this image back into our raytracing engine and ask it to resample the scene. The raytracer will perform multiple samples into the scene wherever the edge map is white and it will perform no extra samples (retaining the original color from the first render) where the edge map is black. For these images the system was set to perform 16x super-sampling, so 16 new rays were cast wherever the edge map is white. Those samples are randomly jittered at the sub-pixel level and then averaged together for the final color.
This is the smooth, final result. The important aspect to this new method of super-sampling is the speed. Instead of performing 16x sampling for every pixel on the screen, only those areas where extra samples are needed get super-sampled. Everywhere else get only a single ray. The difference for this test scene is staggering. This smooth result used 7.7% of the rays needed to produce the equivalent naive image (super-sampling every pixel).
You can compare this result to the image in the previous post, which had no anti-aliasing. The image still took about 1 second to render and save. Still, wait for some more advanced effects before getting too excited about the speed.
