An integrated method for both super-resolution and multi-frame demosaicing includes an image fusion followed by simultaneous deblurring and interpolation. For the case of color super-resolution, the first step involves application of recursive image fusion separately on the three different color layers. The second step is based on minimizing a maximum a posteriori (MAP) cost function. In one embodiment, the MAP cost function is composed of several terms: a data fidelity penalty term that penalizes dissimilarity between the raw data and the super-resolved estimate, a luminance penalty term that favors sharp edges in the luminance component of the image, a chrominance penalty term that favors low spatial frequency changes in the chrominance component of the image, and an orientation penalty term that favors similar edge orientations across the color channels. The method is also applicable to color super-resolution (without demosaicing), where the low-quality input images are already demosaiced. In addition, for translational motion, the method may be used in a very fast image fusion algorithm to facilitate the implementation of dynamic, multi-input/multi-output color super-resolution/demosaicing.

Using image segmentation in video compression brings about a limitation in video quality when a segment moves in position from frame to frame. The limitation arises because color contributions (bleeding or blurring) naturally occur between neighboring segments. The above-identified problem is overcome by providing solutions to compensate for color contributions between neighboring segments. In accordance with one embodiment, a method and apparatus de-blurs an image segment. De-blurring involves removing from the segment approximate color contributions from neighboring segments. This results in a segment that is approximately independent of color contributions from neighboring segments. In accordance with another embodiment, a method and apparatus re-blurs an image segment. Re-blurring involves adding to the segment approximate color contributions from a new arrangement of neighboring segments. This results in more realistic rendering of the segment, as it is located in the new arrangement.

Disclosed is a method for characterizing the content of a digital image. A component of the method is the homogeneity measure, which calculates a value or values based on the local similarity of portions of a sample area. A further component is a means of optimally adjusting the shape and offset of the local sample area to improve the resulting measure. The homogeneity measure may be applied at a higher level to segment an image, by grouping together larger portions of an image whose sub-components are locally similar. A strongly homogeneous region is a contiguous region composed of elements exhibiting local homogeneity better than that of surrounding elements, and may be applied to identify the scale and other characteristics of a portion of an image. This may be used to facilitate algorithms that separate lighting and texturing information, and in identifying the composition of an image.

A method for segmenting video objects in a video sequence that is composed of frames including pixels first assigns a feature vector to each pixel of the video. Next, selected pixels are identified as marker pixels. Pixels adjacent to each marker pixel are assembled into a corresponding a volume of pixels if the distance between the feature vector of the marker pixel and the feature vector of the adjacent pixels is less than a first predetermined threshold. After all pixels have been assembled into volumes, a first score and descriptors are assigned to each volume. At this point, each volume represents a segmented video object. The volumes are then sorted a high-to-low order according to the first scores, and further processed in the high-to-low order. Second scores, dependent on the descriptors of pairs of volumes are determined. The volumes are iteratively combined if the second score passes a second threshold to generate a video object in a resolution video object tree that completes when the combined volume or video object is the entire video.

Lossless video data compression is performed in real time at the data rate of incoming real time video data in a process employing a minimum number of computational steps for each video pixel. A first step is to convert each pixel 8-bit byte to a difference byte representing the difference between the pixel and its immediate predecessor in a serialized stream of the pixel bytes. Thus, each 8-bit pixel byte is subtracted from its predecessor. This step reduces the dynamic range of the data. A next step is to discard any carry bits generated in the subtraction process of two's complement arithmetic. This reduces the data by a factor of two. Finally, the 8-bit difference pixel bytes thus produced are subject to a maximum entropy encoding process. Such a maximum entropy encoding process may be referred to as a minimum length encoding process. One example is Huffman encoding. In such an encoding process, a code table for the entire video frame is constructed, in which a set of minimum length symbols are correlated to the set of difference pixel bytes comprising the video frame, the more frequently occurring bytes being assigned to the shorter minimum length symbols. This code table is then employed to convert the all of the difference pixel bytes of the entire video frame to minimum length symbols.