Last time, we learned the basics of the Game Boy hardware, and tried a simple strategy for rendering games with high-res graphics. Sometimes this simple approach worked well:
but other times it didn’t work at all:
In this part we’ll look at the timing of the Game Boy display hardware, and see how this interferes with our simple scheme.
Computing the appropriate color for a pixel, given the background and sprites, takes time. As such, the graphics hardware does not compute colors for the entire screen simultaneously, but instead render pixels one at a time via raster scanning.
If we imagine how an old-fashioned CRT display scans out an image, there is an “electron gun” that scans each row of pixels from left to right, and proceeds down the screen through the rows. While the GameBoy uses an LCD screen rather than a CRT, it employs a similar notion of how the screen is scanned. Pixels are scanned from left to right in each horizontal line, and lines are scanned from the top down.
After the each line has been scanned, it takes time for the “electron gun” to move back to the start of the next line; this time is called the horizontal blank. At the end of the last line, it takes an even longer time to scan back to the start of the first line; this time is called the vertical blank.
During the two blanking periods, no rendering takes place. Because the rendering hardware is idle during this time, it is possible for software to safely change the values of hardware registers and affect how the next line will be drawn. For example, in Super Mario Land, the level scrolls from right to left as the player progresses, but the status/score display does not. In order to achieve this effect, the game changes the horizontal background-scroll register at the end of line 15.
A Less Simple Renderer
Now it should be clear why our simple rendering approach has problems: it only looks at the values of the hardware registers at the end of a frame (vertical blank), but those registers might have changed on each and every scan line (horizontal blank). Once we realize this, though, it is really quite simple to work around
First, we need to “snapshot” the state of the graphics registers at the end of each scan line. A clever implementation might only take a new snapshot when one or more registers actually change, or might store compact deltas, but my implementation takes a pretty naive snapshot at the end of each line.
Next, instead of rendering a textured quad for every tile, we can simply render a one-scan-line tall textured quad for every tile for every scan-line. On each line we render the tile using the state snapshot for that line.
This approach is somewhat wasteful of CPU and GPU time, but it rectifies the problems we saw before:
Try It Out!
I’ve gone ahead and thrown the code (such as it is) up on GitHub, just to prove I have nothing up my sleeves. You can also download a pre-built binary of the emulator for Mac (I haven’t had time to work on a Windows build yet, sorry).
Keep in mind that this is only a proof of concept; the controls are hard-wired, there is no support for save states or save slots, and it only supports a handful of Game Boy games.
My goal here was to show that this kind of graphics replacement was possible in theory, and I hope I’ve achieved that. I don’t have the kind of time it would take to turn this project into a real, usable emulator, so if you want to fork the code and run with it, feel free.
Scaling this approach up to a more powerful console would bring some interesting challenges, so I’ll throw out a few notes for anybody who might be interested in trying. Readers should feel free to bail out now if none of this makes sense:
- I dealt with palettes in the easiest possible fashion: I use a 4-channel image to store weights for each of the colors in a 4-color palette (effectively treating the palette as a basis). For 16-color palettes this would involve 16-channel images, which is still pretty tractable. Beyond that, you’ll probably need a new technique.
- Because the GameBoy is pure grayscale, it was easy to “project” arbitrary grayscale images into this basis by expressing them as a weighted sum of colors from the palette. For more complex palettes this runs the risk of being over- or under-constrained, so you’d likely want a tool to directly edit replacement images in a paletted fashion.
- More accurate emulation is often required for more interesting consoles. In order to handle mid-scan-line effects on things like the NES, you might need to draw the replacement graphics on a per-pixel basis, rather than the per-line approach I use. In that case, it would help to detect when the graphics registers change, and only do the more expensive rendering when/where required.
- I’ve ignored relative sprite priorities in my emulator, but I know that subtle priority details often have big effects (e.g., the clipping for item blocks in Mario 3). The key to getting priority right while still using replacement graphics is to cast the pixel-compositing operations usually applied into something like the Porter-Duff algebra, which will naturally scale to images and alpha transparency.
- Many SNES games use HDMA to make rounded shapes (e.g., the Power Bomb in Super Metroid) or wavy distortion effects (e.g., Phantoon). If these kind of scanline effects get scaled up, they will become blocky. Ideally, an emulator should be able to interpolate between the state of registers at different scanlines to smooth out these shapes. The challenge then is figuring out when not to interpolate.