Texture Compression

Bitmaps Sprites and Textures discusses why pictures are important to Video Games, and how it's important to choose between detail and size. Texture compression saves space in Video RAM and allows the textures to be transferred more quickly, allowing the use of even more.

Texture compression requires special algorithms and hardware that work extremely quickly, owing to the real-time nature of the application. Things like PNG and JPEG are just too slow; to churn out 60 frames per second, the graphics card has to draw everything in view in around 16 thousandths of a second. This means accessing hundreds of textures billions of times.

Furthermore, decompressing textures really needs to be done piecemeal. You need to be able to take a finite quantity of a texture and decompress it by itself. Normal image compression formats simply don't allow that.

The most common texture compression system is S3TC (or some variation thereof), developed by S3 Technologies. It's the most widely used format, having been used on the PC for many years, and in many consoles. It's also used in handhelds capable of 3D graphics such as the PSP and DS. There are several variations on S3TC.

Unlike in typical compression algorithms such as PNG and JPEG, the resulting compression ratio is always the same. If you throw a 32MB texture at the algorithm, you'll always get an 8MB texture as a result, for a specific format of compression. Now you can use four textures instead of one, but still only require the same amount of Video RAM.

The precise details of its operation, as with all compression schemes, are somewhat complicated. But the principle behind S3TC isn't too hard to understand: For each 4x4 texel block of the texture, the compressor selects two colors A and B that can be mixed to approximate the texels in the block. The rest describes in which proportion A and B are mixed to form each texel: all A, all B, or two-thirds of one and one-third of the other. The 16 proportions fit in the same space as two texels, resulting in a constant reduction of 16 original texels to the size of four (A, B, and the proportions). (Remember that a texel is not necessarily the same as a pixel of color; it's just a piece of arbitrary data which can be used for anything - such as storing the data relating to proportions.)

It should also be noted that the Game Cube and Xbox used S3TC, but the PlayStation 2 could not use it directly. There were games that developers could play to use various forms of compression, but because they were software solutions, they were often a tradeoff between some quantity of performance and the resulting memory savings. That is why the PS2 could make detailed graphics, but not as much as the other two (as the port of Resident Evil 4 showed). So this is very important for high end graphical details.