How Computers Display Works : A Software Perspective
Without monitors, we may never be able to use computers the way it is now. But, how do they work? How can they produce such beautiful interfaces we see today? How is your current computer/mobile phones can display this website to you?
In this article, I will take you down to the fundamental of how computers can display stuff at the monitor. Due to my education in computer science, I will explain it mostly on the software-side. I will touch a little bit on the display technologies, though, but, it may not be that comprehensive and deep. So, don’t expect me to explain how the actual chemical and physical properties of the monitor, so that they can emit various lights, that will be another person’s job to do.
Some of you who are reading this post may already knew the concept of pixels. Yes, they are the building blocks of digital images. Images consist of pixels of varying colours that are arranged in a rectangular shape so that they will form the complete image. This pixel concept is also essential here, since the computer will send pixel data to the monitor to display.
But, how do a normal monitor display image(s)?
Do a monitor instantenously display all pixels sent by the computer? No. Due to current technology protocol standards, which all of them communicate in a serial manner, this will not be possible, since some pixel data will arrive at the monitor quicker, and some will arrive later. Even though, serial protocols have a very major advantage that makes it very common in many current protocol standards (not just for displays), and that it can be very fast. As to why, it will be beyond the talking points of this article (probably a future post?).
Since monitors receive pixel data in this manner, they have to wait for the whole data to arrive. Doing this will make the monitor response time to be very slow, since it has to wait for the whole data to arrive, in order to display the current image. Current monitors opt for another way, just display the data it currently receives instantly.
Computers, on the other hand, will process pixels orderly. They will send the top-left pixel first, and the the pixel on the same row on the right. After all pixels in this row is done, it will send the pixels on the next row, starting again from the leftmost pixel. In turn, the monitor will also display the top-left pixel first, down to the bottom-right pixel. You probably think, well, why am I still seeing this as if the monitor is displaying the pixels instantly? That’s because, our eyes are weak!. That’s probably a bit harsh, but that’s the truth. Since the data is streamed and displayed so fast, our eyes cannot see that process. And because it is so fast, that’s why monitors can display many images in a second, that for us, it looks like a smooth video.
A CRT display under action, recorded with a high-speed camera (Source: The Slow Mo Guys, in a Hackaday article)
Framebuffer?
Up to this point I always refer the thing that is displayed to the monitor as images. Now, let’s refer it as frames, since that is the more common terminology for that.
In order to accomodate the monitor, computers will allocate a special place (usually) in the main memory (RAM, random access memory). This special memory region is then called as a framebuffer. The size of the memory allocated depends on the current resolution of the display and the number of bits it needed to represent a single pixel. Common size for a single pixel is 1-bit (monochromatic, can only display black and white), 8-bit (grayscale, can display gray), 16-bit and 24-bit. 24-bit is the most common one, which can cover a lot of color combinations. The amount of memory needs to be allocated, then, can be calculated simply by multiplying each pixel size with the width and height of the display resolution. For example, a 1920x1080 display with 24-bit pixel framebuffer will need approximately 49.7 Mb (megabits) of memory, or around 6.2 MB (megabytes, 1 byte = 8 bit).
This framebuffer data will be used directly by the monitor to display the current frame. Applications that needs to change something in the display will need to access this framebuffer. They either do it by modifying the framebuffer directly, or use other components provided by the operating system and/or third-party libraries. Windows does not prevent direct framebuffer modifications, therefore all Windows applications use other components, such as Direct3D, DirectX, or OpenGL to draw stuffs. Linux allows direct framebuffer manipulation, given if the user has the correct permissions.
Screen tearing, public enemy #1
By my last explanation, then it became clear how computer applications display frames on monitors. In order to view a series of frames, you just spam that framebuffer with frames, right? Yep, that’s almost the perfect way to do it, if only not for this one small problem…
Example of a frame affected by *screen tearing* (source: Wikipedia)
Every time an application finishes with a frame, it will then move to work on the next frame, writing to the framebuffer. At the same time, the monitor will start to read the framebuffer. When the monitor finishes reading the framebuffer, it will read it again from the start, hoping that now, the framebuffer contains the new frame. If the timing is not right, the monitor will start to read the framebuffer while the application is still processing the frame. This means that the monitor will display a combination of the new frame and some of the old frame. This will create a visual artifact that seems like the current frame is sliced. This artifact is referred to as screen tearing.
In order to prevent screen tearing, a single framebuffer is not enough. A second buffer is then introduced to the system, called double-buffering. Early implementation of double-buffering follows this two simple steps:
- Applications manipulate the second buffer (also called backbuffer)
- After the applications done with the current frame, the backbuffer is then copied to the framebuffer (also called frontbuffer). This process of copying is usually synced with the current state of the monitor (remember that monitor displays from top-left to bottom-right).
Double-buffering illustration (source: National University of Colombia) (whoops, the original link is down & I don't have a backup image...)
This steps, however, requires more memory and CPU time in order to copy the memory and synchronizing with the display. This way also reduces performance of graphical-intensive programs, since the program needs to wait for the backbuffer to finish copying to the frontbuffer before the next frame can be written.
The next step in improving double-buffering is to eliminate the copying process altogether. Let’s say we can have a pointer that can point to the address of the buffer. If we can alter this pointer back and forth between front and back buffer, we can get away with not copying data from back to front buffers. This technique is called page-flipping. Applications will write on the buffer (second buffer) that’s not currently pointed by the pointer (first buffer). When it finishes, the pointer then changed to the second buffer, and applications will modify the first buffer for the next frame. Although very fast, this technique has a problem, that is not every program/application have the ability to change this pointer.
Page-flipping illustration (source: National University of Colombia) (whoops, the original link is down & I don't have a backup image...)
So, if two is good, then three must be better! And yes, current implementations uses triple-buffering techniques to draw on the screen. The two backbuffers is written alternately, like in the page-flipping technique. The first buffer is copied into the framebuffer, while the second one is manipulated by the program for the next frame. When the copy process is finished and the next frame is ready, the application will start manipulate the first buffer, and the second buffer is then copied to the framebuffer. This improves performance significantly, as the display and the software is now independent of each other. The application can go at it’s own pace to generate the next frame, and the display can just display whatever in the backbuffer, since there will always be one finished frame in either backbuffer to be displayed.
Now you know the simple explanation into how computer can display images on screen. There are more wonders of display technologies, either in the hardware-side and the software-side. There are still more display problems beyond screen tearing that are present in current display technologies, either on the hardware or the software, and there are still more stuff that can be discussed. Monitor technologies are still evolving, so, probably there will be new technologies that are better and faster in the near future.
Writer’s Note : As promised in the last post, here it is! A pretty different topic than usual. Fun fact: this post is actually in my post drafts since last January. I forgot to post this, so, yay…
Also another fun note, this is actually not the planned article that I will write. That other topic I planned turns out to be a bit tricky to write. It is about my personal project/experiment that I made, and, well, let’s say, the results are not favorable. Probably I will write about that too in the near future, I guess?
Other than that, have you heard about Temtem? The one that is advertised as MMO-Pokemon? It looks interesting and fun. I will probably dive into the game to see for myself, soo, expect me to write about that!