Bob Currier, Synthetic Aperture
Compression is both the boon and bane of computer video. Boon because without it today's computers could not keep up with video data rates; bane because it modifies the video image in ways difficult to predict, complicating the job of the video professional.
Many professionals resist the idea of any compression, fearing that the material they worked so hard to create will be distorted and end up looking like a cheesy special effect. However, remember that even our beloved D-1 uses 2:1 compression on the chroma channel, and that any broadcast signal is severely bandwidth limited, which is merely another name for compression. So rather than objecting to compression as such, such professionals are really objecting to the unknowns of digital compression.
To try to make sure that the good outweighs the bad, many different approaches have been developed for everything from traditional video to multimedia PC-based video. The trick is to understand the differences, and be able to choose the right compressor for the job.
Digital compression is the process of reducing the number of bits needed to represent a given set of data. It does this by exploiting redundancies in the data.
In some cases this is simple: if we have a data file of nothing but binary zeros, obviously we do not need to store the whole file, but could instead simply store an integer representing the number of zero bits. Through clever mathematics, we can expand compression to more complex--and realistic--cases.
By reducing the number of bits used to store a set of data, we not only reduce the space required to store it, we also reduce the bandwidth needed to transmit it. In the world of digital video, we need both. We typically speak of a compressor reducing the bandwidth or data rate of a signal, with the understanding that it also reduces storage requirements.
So why do compression and video go together? Because digital video is data intensive. A standard CCIR 601 digital video signal has a data rate of over 20 MByte/second, even if only the active image area is used; well beyond the computational and storage capabilities of standard computers. Compression is required simply to make computer handling of video economically feasible.
All digital compression is a gamble. It is betting on the fact that there is redundancy in the data that can be "compressed out." Fortunately, any normal video picture has a significant amount of such redundancy. However, it is always possible to come up with a video image, realistic or not, which can overwhelm a compressor to the point where it cannot maintain a quality image. It is this kind of failure that has given compression a bad reputation in some circles.
Because compression is not much use without a way to decompress and recover the original data, we normally speak of compressor-decompressor pairs, or codecs.
Lossless vs. Lossy Codecs
All codecs can be divided into two categories: lossless and lossy. Lossless codecs completely preserve the data they compress: you get out exactly what you put in. Because of this, we can compress anything with a lossless compressor without fear of distorting the data, or in the case of video, the image. Unfortunately, lossless compressors cannot achieve the level of compression we need for video. Lossless compressors are hard pressed to deliver even a 2:1 compression ratio.
Lossy codecs, on the other hand, only return an approximation of the data that went into the compressor. How close the approximation is determines the quality of the codec. Typically there is a direct tradeoff between the quality of the output of a codec and the level of compression it is able to achieve. Many lossy codecs can achieve 10:1 compression or more without visible degradation of the image.
Video codecs typically take advantage of characteristics of the human perception system. For example, because our eyes are more sensitive to luminance than they are to color, we can dedicate relatively less bandwidth to the color components, achieving greater compression. This type of compression is similar to the 2:1 chroma sub-sampling used in 4:2:2 digital video.
There are more sophisticated tricks one can play when encoding color, again taking advantage of limits in human perception. Some of these have the effect of changing--and usually reducing--the color gamut. This can sometimes lead to unpleasant color shifts.
This reliance on the characteristics of perception leads to a somewhat uncomfortable change for the video engineer: the need to rely on the monitor rather than test instruments. After years of doing exactly the opposite, digital compression changes things so that the subjective quality of the image is more important than its quantitative quality. This should lead to some lively "discussions" during post production!
As we noted before, compressors work by removing redundancy from data. Spatial, or intraframe, compression takes advantage of similarities within a video frame. For example, an expanse of blue sky varies little from pixel to pixel. There is no need to store the same number of bits for such an area as are stored for an area with great amounts of detail.
Because spatial compression is looking for similarities, video noise and film grain can reduce the effectiveness of the compression.
Many codecs also add the concept of temporal, or interframe, compression. This allows the codec to take advantage of similarity between successive video frames. If two successive frames have the same background, there is no need to store the background again. Instead, only the differences between the two frames need be stored.
To achieve further compression, some codecs incorporate motion prediction. Rather than simply comparing two successive frames, this technique notices moving objects in a frame and predicts where they will be in the next frame. Then only the difference between the prediction and the actual location need be stored.
Because temporal compression makes one frame depend on another, it makes editing of temporally compressed video difficult or impossible. Editing may require that video be decompressed and recompressed, often creating significant compression artifacts in the process. For this reason, temporal compression is best suited for final delivery.
Temporal compression can yield significant data reduction. However, its effectiveness is reduced by common cinematic techniques such as pans, slow zooms or dollies, and the current rage, handheld camera work. Like spatial compression, it is sensitive to noise and grain.
Data Rate Limiting
Some compressors have a feature known as "data rate limiting." Such a codec can dynamically control the amount of compression--and therefore the quality of the image--while maintaining a constant rate of compressed data. This is useful when the compressed data stream will be played back from a storage device with a fixed data rate, such as a CD-ROM.
Choosing a Codec
With all these differences, choosing the correct codec can be a difficult task. You need to choose the level of compression you need to meet your data rate limits, the level of quality you want and whether or not you need real-time compression.
Next time we will sample some of the more popular codecs and look at their specific characteristics.
Bob Currier is President of Synthetic Aperture, a multimedia production company specializing in digital video and QuickTime VR. He also serves as Sysop of the Macintosh Multimedia Forum on CompuServe.
He can be reached at firstname.lastname@example.org. Be sure to visit the Synthetic Aperture web site at <http://www.synthetic-ap.com/> for more tutorial information, sample content, and information on new media services.
This article orignally appeared in a slightly different form in Computer Video Production magazine.
Tips & Articles | About Us | What's New | Press Room