A cell of an image is segmented into two or more subcells. According to one embodiment, the nullity of each of the subcells is classified and the cell is encoded in accordance with the nullity of the subcells. According to another embodiment, the quantization level of each of the subcells is classified and the cell is encoded in accordance with the quantization levels of the subcells.

The performance of video encoding is characterized both over the short term (e.g., based on the previous encoded frame) and over the long term (e.g., based on the previous n encoded frames). An encoding parameter (e.g., the global quantization level) is selected for the current frame based on the short-term and long-term performance characterizations and the current frame is encoded using that selected encoding parameter. In a preferred embodiment, the global quantization (Q) level is permitted to change for only certain (i.e., adjustable) frames. In addition, the global Q level for key frames is generated using special processing which tends to allow the key frames to be encoded using more bits.

A current level of a virtual transmission buffer is maintained and used to select video frames for compression. The current level is compared to a threshold to determine whether to compress the current frame. The threshold is dynamically maintained between an upper "full" level and a lower "clear" level. If the current level exceeds the full level, the video compression processing is changed to a decline mode in which the threshold is gradually decremented towards the clear level. In addition, if the current level exceeds a "watch" level for too long, then the video compression processing is also switched to the decline mode. The present invention allows the virtual buffer to absorb variability in compressed frame size from frame to frame without resulting in undesired latency.

For each image, an initial set of quantization (Q) levels is selected based on the pixel data. The pixel data are transformed to blocks of transform coefficients. The initial set of Q levels is incremented and decremented. The incremented Q levels are used to generate a low estimate of bitrate for the transform coefficients and the decremented Q levels are used to generate a high estimate of bitrate for the transform coefficients. The low and high bitrate estimates are compared to a specified target bitrate to generate Q-level adjustments. The Q-level adjustments are used to select the Q levels to encode fully the quantized coefficients of the current image in the sequence of images.

A heterogeneous multimedia stream has interleaved samples of two or more different substream types. For example, a heterogeneous video stream may have interleaved intra, predicted, and bi-directional frames. In addition to a global bit bank, a different bit bank is maintained for each different frame type. A global quantization level is selected for each frame based on the global bit bank and the bit bank for the corresponding frame type. The frames are encoded using the selected global quantization levels. In a preferred embodiment, the number of bits available for encoding frames differs based on the frame type. In addition, any unused bits from encoding the current frame are shifted from the bit bank corresponding to the current frame type to a bit bank of a different frame type.