Enhanced video data compression in intelligent video information management system

What is claimed is:

1. A method of representing video data in compressed form, comprising the steps of:

dividing into rectangular blocks a present image to be compressed, the present image being formed in an image plane, each of said rectangular blocks consisting of n pixel blocks, each of said pixel blocks corresponding to an m pixel by m pixel segment of said image plane; n being an integer greater than or equal to 1; m being an integer greater than 1;

comparing each of said rectangular blocks of the present image with a corresponding rectangular block of a reference image to generate comparison data for the block of the present image;

selecting for representation the rectangular blocks of the present image for which the generated comparison data meets a selection criterion;

generating mapping data indicative of locations in the image plane which correspond to the selected rectangular blocks for the present image; and

representing the present image in compressed form by use of:

(a) header data for identifying the present image;

(b) said mapping data generated for the present image; and

(c) image data representative of the selected rectangular blocks of the present image;

wherein:

said comparing step includes applying an orthogonal transformation to at least one pixel block of the n pixel blocks of each rectangular block of the present image to form coefficient data, and comparing the coefficient data with corresponding coefficient data for a corresponding pixel block in the reference image.

2. A method according to claim 1, wherein said comparing step includes comparing a coefficient which represents a DC component of the coefficient data for at least one pixel block of the present image with a coefficient which represents a DC component of the coefficient data for the corresponding pixel block of the reference image.

3. A method according to claim 2, wherein said comparing step includes comparing a coefficient which represents a frequency component of the coefficient data for at least one pixel block of the present image with a coefficient which represents a corresponding frequency component of the coefficient data for the corresponding pixel block of the reference image.

4. A method according to claim 1, wherein said comparing step includes comparing a coefficient which represents a frequency component of the coefficient data for at least one pixel block of the present image with a coefficient which represents a corresponding frequency component of the coefficient data of the corresponding pixel block of the reference image.

5. A method according to claim 1, wherein said representing step includes applying quantization to the coefficient data formed for each selected rectangular block to obtain quantization data, and applying Huffman encoding to the quantization data to form said image data representative of the selected rectangular blocks of the present frame.

6. A method according to claim 5, wherein the orthogonal transform is the discrete cosine transform (DCT), and the application of said DCT, said quantization and said Huffman encoding are in accordance with the JPEG image data compression standard.

7. A method according to claim 1, wherein m=8.

8. A method according to claim 7, wherein n=4.

9. A method according to claim 8, wherein a rectangular block is selected for representation if any one of the four pixel blocks of the rectangular block meet a selection criterion.

10. A method according to claim 8, wherein each of said rectangular blocks consists of a horizontal sequence of four 8.times.8 pixel blocks.

11. A method according to claim 10, wherein four 8.times.8 luminance pixel arrays and exactly two 8.times.8 chrominance pixel arrays are formed for each rectangular block of the reference image.

12. A method according to claim 11, wherein one of the two 8.times.8 chrominance pixel arrays consists of U pixels and the other of said two 8.times.8 chrominance pixel arrays consists of V pixels.

13. A method of representing video data in compressed form, comprising the steps of:

dividing into rectangular blocks a present image to be compressed, the present image being formed in an image plane, each of said rectangular blocks consisting of n pixel blocks, each of said pixel blocks corresponding to an m pixel by m pixel segment of said image plane; n being an integer greater than or equal to 1; m being an integer greater than 1;

applying an orthogonal transformation to at least one pixel block of the n pixel blocks of each rectangular block of the present image to form coefficient data;

comparing the coefficient data with corresponding coefficient data for a corresponding pixel block in a reference image to generate comparison data for the rectangular block which includes the pixel block of the present image;

selecting for representation the rectangular blocks of the present image for which the generated comparison data meets a selection criterion; and

generating compressed image data only for the selected rectangular blocks of the present image.

14. A method according to claim 13, wherein said step of generating compressed image data includes processing the coefficient data formed by applying said orthogonal transformation to the pixel blocks of the selected rectangular blocks of the present image.

15. A method according to claim 14, wherein said processing includes quantizing said coefficient data and Huffman encoding the quantized coefficient data.

16. A method according to claim 15, wherein said orthogonal transformation and said processing of the coefficient data are performed in accordance with the JPEG image data compression standard.

17. A method according to claim 13, wherein n=4 and m=8.

18. Apparatus for storing compressed video data, comprising:

means for receiving video signals representing a dynamic sequence of images;

means for converting the video signals into a sequence of video data fields;

processing means for compressing said video data fields; and

means for storing the compressed video data fields;

said processing means being programmed to:

divide each of said video data fields into pixel blocks each corresponding to an m.times.m segment of an image plane, m being an integer greater than 1;

apply an orthogonal transformation to each of the pixel blocks to form coefficient data;

as to one of said images selected to be a reference image, store at least some of the coefficient data as reference data;

as to said reference image, quantize the coefficient data to form quantization data and Huffman-encode the quantization data to form first storage data;

cause the means for storing to store the first storage data;

as to one of said images subsequent to said reference image, compare the coefficient data corresponding to at least some of the pixel blocks of said subsequent image to the reference data for the corresponding pixel block of the reference image;

on the basis of said comparison, select pixel blocks of said subsequent image for further processing and select portions of the reference data for updating;

update the selected portions of the reference data using coefficient data corresponding to the selected pixel blocks of said subsequent image;

quantize the coefficient data corresponding to the selected pixel blocks and Huffman-encode the resulting quantized coefficient data to form second storage data; and

cause the means for storing to store the second storage data.

19. Apparatus according to claim 18, wherein said processing means includes a first processor for dividing the video data fields into said pixel blocks and a second processor for processing said pixel blocks formed by said first processor.

20. A method of performing data compression with respect to image data, said image data being representative of pixels arrayed in an image plane, the method comprising the steps of:

dividing said image data into a plurality of basic data blocks, each of said basic data blocks representing a rectangular part of said image plane, all of said basic data blocks being of equal size;

detecting a spatial frequency characteristic of each of a contiguous plurality of said basic data blocks;

in dependence upon a result of said detecting step, selectively combining said contiguous plurality of said basic data blocks to form a combined data block;

subsampling said combined data block to form a subsampled data block that is equal in size to said basic data blocks; and

applying a transformation to said subsampled data block to form a block of coefficient data.

21. A method according to claim 20, wherein said detecting step includes:

applying a transformation to each basic data block of said contiguous plurality of basic data blocks, to form a respective array of coefficient data, some of said coefficient data representing respective spatial frequency components of the respective basic data block; and

detecting a characteristic of at least one of said coefficient data representing spatial frequency components.

22. A method according to claim 21, wherein said transformation applied to each basic data block of said contiguous plurality of basic data blocks is a two-dimensional DCT.

23. A method according to claim 20, wherein said transformation applied to said subsampled combined data block is a two-dimensional DCT.

24. A method according to claim 20, wherein each of said basic data blocks consists of an 8.times.8 array of pixels.

25. A method according to claim 24, wherein said combined data block consists of an 8n.times.8n array of pixels, where n is an integer greater than 1, and said subsampling step comprises subsampling said combined data block by a factor of n in each of a horizontal direction and a vertical direction.

26. A method according to claim 24, wherein n=2.sup.p, p being an integer equal to or greater than 1.

27. A method according to claim 20, further comprising the steps of:

combining a first contiguous plurality of said basic data blocks to form a first combined data block;

subsampling said first combined data block to form a first subsampled data block that is equal in size to said basic data blocks;

applying a transformation to said first subsampled data block to form a respective block of coefficient data therefrom;

combining a second contiguous plurality of said basic data blocks to form a second combined data block that is larger than said first combined data block;

subsampling said second combined data block to form a second subsampled data block that is equal in size to said first subsampled data block; and

applying a transformation to said second subsampled data block to form a respective block of coefficients therefrom.

28. A method of performing data compression with respect to image data, said image data being representative of pixels arrayed in an image plane, the method comprising the steps of:

dividing said image data into a plurality of m.times.m pixel blocks, m being equal to 2.sup.p, p being an integer equal to or greater than 1;

detecting a spatial frequency characteristic of each of a contiguous plurality of said m.times.m pixel blocks;

in dependence upon a result of said detecting step, selectively combining said contiguous plurality of m.times.m pixel blocks to form an (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, q being an integer equal to or greater than 1;

processing said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block to form a processed m.times.m data array representative of said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block; and

applying a transformation to said processed m.times.m data array to form a block of coefficient data.

29. A method according to claim 28, wherein said processing step includes subsampling said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block.

30. A method according to claim 28, wherein said detecting step includes:

applying a transformation to each m.times.m pixel block of said contiguous plurality of m.times.m pixel blocks, to form a respective array of coefficient data, some of said coefficient data representing respective spatial frequency components of the respective m.times.m pixel block; and

detecting a characteristic of at least one of said coefficient data representing spatial frequency components.

31. A method according to claim 30, wherein said transformation applied to each m.times.m pixel block of said contiguous plurality of m.times.m pixel blocks is a two-dimensional DCT.

32. A method of performing data compression with respect to image data, said image data being representative of pixels arrayed in an image plane, the method comprising the steps of:

dividing said image data into a plurality of m.times.m pixel blocks, m being equal to 2.sup.p, p being an integer equal to or greater than 1;

combining a first contiguous plurality of said m.times.m pixel blocks to form an (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, q being an integer equal to or greater than 1;

processing said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block to form a first processed m.times.m data array representative of said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block;

transform-encoding said first processed m.times.m data array to form compressed image data representative of said (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block;

combining a second contiguous plurality of said m.times.m pixel blocks to form an (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block, r being an integer equal to or greater than 1, r.noteq.q;

processing said (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block to form a second processed m.times.m data array representative of said (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block; and

transform-encoding said second processed m.times.m data array to form compressed image data representative of said (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block.

33. A method according to claim 32, wherein m=8.

34. A method according to claim 32, wherein each of said processing steps includes subsampling the respective pixel block.

35. A method according to claim 32, wherein each of said transform-encoding steps includes applying a two-dimensional DCT to the respective m.times.m data array to form coefficient data, quantizing the coefficient data, and Huffman-encoding the quantized coefficient data.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

The above-referenced parent patent application discloses a digital video recorder which has intelligent video information management capabilities.

In any digital video recording apparatus, the large quantity of data inherently present in streams of video signals tends to require difficult trade-offs to be made between the storage capacity to be provided by the apparatus and the image quality to be available upon reproduction of the recorded video signals. The trade-offs can be particularly critical when it is desired to provide random access to the recorded video signal, as is the case in the intelligent video information management (IVIM) recorder disclosed in the parent patent application. In a preferred embodiment of the IVIM recorder, recording of video signal streams generated by up to 16 cameras, with random access playback, is provided on one or more hard disk drives. To achieve an adequate recording duration on the hard disk drives, a number of compression strategies were proposed in the parent patent application.

The present inventors have developed additional features related to the compression strategies disclosed in the parent patent application so as to provide more efficient utilization of data storage capacity and/or improved image quality, albeit with requirements for additional processing during compression-encoding operations.

OBJECTS AND SUMMARY OF THE INVENTION

It is an object of the invention to provide a method and apparatus for compressing, with a high degree of efficiency, video data representing streams of dynamic video images.

It is a more particular object of the invention to provide improvements upon, and alternative embodiments of, the "conditional refresh" video data compression strategy disclosed in the parent patent application.

According to a first aspect of the invention, there is provided a method of representing video data in compressed form, including the steps of dividing into rectangular blocks a present image to be compressed, the present image being formed in an image plane, each of the rectangular blocks consisting of n pixel blocks, each of the pixel blocks corresponding to an m pixel by m pixel segment of the image plane, n being an integer greater than or equal to 1, m being an integer greater than 1; comparing each of the rectangular blocks of the present image with a corresponding rectangular block of a reference image to generate comparison data for the block of the present image; selecting for representation the rectangular blocks of the present image for which the generated comparison data meets a selection criterion; generating mapping data indicative of locations in the image plane which correspond to the selected rectangular blocks for the present image; and representing the present image in compressed form by use of (a) header data for identifying the present image, (b) the mapping data generated for the present image, and (c) image data representative of the selected rectangular blocks of the present image; wherein the comparing step includes applying an orthogonal transformation to at least one pixel block of the n pixel blocks of each rectangular block of the present image to form coefficient data, and comparing the coefficient data with corresponding coefficient data for a corresponding pixel block in the reference image. The comparing step may include comparing a coefficient which represents a DC component of the coefficient data for each pixel block of the present image with a coefficient which represents a DC component of the coefficient data for the corresponding pixel block of the reference image, and may also include comparing a coefficient which represents a frequency component of the coefficient data for at least one pixel block of the present image with a coefficient which represents a corresponding frequency component of the coefficient data for the corresponding pixel block of the reference image. The orthogonal transformation to be performed as part of the comparing step may be the discrete cosine transform (DCT), and the image data to be used in representing the present image in compressed form for the selected rectangular blocks may be formed by quantizing and Huffman-encoding the coefficient data produced by the DCT.

According to a second aspect of the invention, there is provided a method of representing video data in compressed form, including the steps of dividing into rectangular blocks a present image to be compressed, the present image being formed in an image plane, each of the rectangular blocks consisting of n pixel blocks, each of the pixel blocks corresponding to an m pixel by m pixel segment of the image plane, n being an integer greater than or equal to 1, m being an integer greater than 1; applying an orthogonal transformation to at least one pixel block of the n pixel blocks of each rectangular block of the present image to form coefficient data; comparing the coefficient data with corresponding coefficient data for a corresponding pixel block in a reference image to generate comparison data for the rectangular block which includes the pixel block of the present image; selecting for representation the rectangular blocks of the present image for which the generated comparison data meets a selection criterion; and generating compressed image data only for the selected rectangular blocks of the present image.

According to a third aspect of the invention, there is provided apparatus for storing compressed video data, including circuitry for receiving video signals representing a dynamic sequence of images, circuitry for converting the video signals into a sequence of video data fields, processing circuitry for compressing the video data fields, and a storage device for storing the compressed video data fields, wherein the processing circuitry is programmed to divide each of the video data fields into pixel blocks each corresponding to an m.times.m segment of an image plane (m being an integer greater than 1), apply an orthogonal transformation to each of the pixel blocks to form coefficient data, as to one of the images selected to be a reference image, store at least some of the coefficient data as reference data, as to the reference image, quantize the coefficient data to form quantization data and Huffman-encode the quantization data to form first storage data, cause the storage device to store the first storage data, as to one of the images subsequent to the reference image, compare the coefficient data corresponding to at least some of the pixel blocks of the subsequent image to the reference data for the corresponding pixel block of the reference image, on the basis of the comparison, select pixel blocks of the subsequent image for further processing and select portions of the reference data for updating, update the selected portions of the reference data using coefficient data corresponding to the selected pixel blocks of the subsequent image, quantize the coefficient data corresponding to the selected pixel blocks and Huffman-encode the resulting quantized coefficient data to form second storage data, and cause the storage device to store the second storage data.

According to a fourth aspect of the invention, there is provided a method of performing data compression with respect to image data, the image data being representative of pixels arrayed in an image plane, the method including the steps of dividing the image data into a plurality of basic data blocks, each of the basic data blocks representing a rectangular part of the image plane, all of the basic data blocks being of equal size, detecting a spatial frequency characteristic of each of a contiguous plurality of the basic data blocks, in dependence upon a result of the detecting step, selectively combining the contiguous plurality of the basic data blocks to form a combined data block, subsampling the combined data block to form a subsampled data block that is equal in size to the basic data blocks, and applying a transformation to the subsampled data block to form a block of coefficient data.

According to a fifth aspect of the invention, there is provided a method of performing data compression with respect to image data, the image data being representative of pixels arrayed in an image plane, the method including the steps of dividing the image data into a plurality of m.times.m pixel blocks, m being equal to 2.sup.p, p being an integer equal to or greater than 1, detecting a spatial frequency characteristic of each of a contiguous plurality of the m.times.m pixel blocks, in dependence upon a result of the detecting step, selectively combining the contiguous plurality of m.times.m pixel blocks to form a (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, q being an integer equal to or greater than 1, processing the (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block to form a processed m.times.m data array representative of the (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, and applying a transformation to the processed m.times.m data array to form a block of coefficient data.

According to a sixth aspect of the invention, there is provided a method of performing data compression with respect to image data, the image data being representative of pixels arrayed in an image plane, the method including the steps of dividing the image data into a plurality of m.times.m pixel blocks, m being equal to 2.sup.p, p being an integer equal to or greater than 1, combining a first contiguous plurality of the m.times.m pixel blocks to form an (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, q being an integer equal to or greater than 1, processing the (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block to form a first processed m.times.m data array representative of the (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, transform-encoding the first processed m.times.m data array to form compressed image data representative of the (m.multidot.2.sup.q).times.(m.multidot.2.sup.q) pixel block, combining a second contiguous plurality of the m.times.m pixel blocks to form an (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block, r being an integer equal to or greater than 1, and r not being equal to q, processing the (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block to form a second processed m.times.m data array representative of the (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block, and transform-encoding the second processed m.times.m data array to form compressed image data representative of the (m.multidot.2.sup.r).times.(m.multidot.2.sup.r) pixel block.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a video recording/personal computer (VR/PC) unit provided in accordance with the invention.

FIG. 2 is a summary block diagram of electronic components of the VR/PC unit of FIG. 1.

FIG. 3 is a summary block diagram representative of analog signal multiplexing and processing circuits that are part of a front end processing/compression component shown in FIG. 2.

FIG. 4 is a block diagram representation of signal conversion and digital signal processing circuitry that is part of the front end processing/compression component shown in FIG. 2.

FIGS. 5A and 5B represent, respectively, raster-scan and rectangular tile formats of an image plane (field) of video data formed in the VR/PC unit.

FIG. 6 illustrates a rectangular tile (pixel block) format employed in the VR/PC unit.

FIG. 7 represents the format of a minimum coding unit (MCU) employed in the VR/PC unit.

FIG. 8 represents a 4:1:1 color video data format employed in the VR/PC unit.

FIG. 9 is indicative of a color data format used for each MCU.

FIG. 10 is a representation of a data format used for compressed video data generated by the digital signal processing circuitry of FIG. 4.

FIG. 11 is a pictorial representation of a technique used in the VR/PC unit for reconstituting a field of compressed video data.

FIG. 12 is a flow-chart representation of an improved process for selecting pixel blocks for inclusion in a field of difference data included in the data format of FIG. 10.

FIG. 13 represents a coefficient data array generated by conventional application of a two-dimensional DCT to an 8.times.8 pixel block.

FIG. 14 is a high-level flow-chart representation of a process for carrying out image data compression utilizing adaptive sizing of pixel blocks.

FIG. 15 is a pictorial illustration of the construction of pixel blocks of various sizes.

FIG. 16 is a pictorial representation of a process for converting an over-sized pixel block to a standard-sized array of image data.

FIG. 17 is a flow-chart representation of processing carried out in connection with the construction of over-sized pixel blocks in accordance with the invention.

FIG. 18 is a flow-chart representation of processing carried out in connection with an "evaluate complexity" step shown in FIG. 17.

FIG. 19 is a flow-chart representation of a transform-encoding process applied to over-sized pixel blocks in accordance with the invention.

FIGS. 20A and 20B together show a data format that may be employed for compression-encoding mapping data that is part of the image data format of FIG. 10.

FIG. 21 is a pictorial illustration of another compression-encoding technique that may be applied to the mapping data of FIG. 10.

DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 is a perspective view of an integrated device which combines digital video recording, random access retrieval of recorded video data, and user-friendly personal-computer-based functionality. Reference numeral 23 generally indicates the integrated device, which may be referred to as a video recording/personal computer or "IVR/PC" unit. The VR/PC unit is adapted to receive streams of video signals generated by one or more video cameras. A preferred embodiment is configured to receive simultaneous input from 16 cameras (which are not shown in FIG. 1). The VR/PC unit also provides an output video signal, either live from one or more video cameras, or reproduced from video data storage facilities provided within the VR/PC unit, to drive one or more display monitors, which also are not shown in FIG. 1.

The internal components of the VR/PC unit 23 are contained within a molded plastic housing 22. As will be seen, the internal components include control and signal processing circuitry, as well as a number of data recording devices. For example, integrated within the VR/PC unit there are preferably two or more fixed or "hard" data recording drives of the magnetic type, as well as at least one drive for a removable data storage medium. A preferred embodiment includes both a floppy disk drive and a digital audio tape (DAT) drive. The floppy drive may be used for loading software programs; the DAT drive may be used to store video data, retrieved from the internal hard drives, for permanent or archival storage on a magnetic tape formatted in accordance with the standard DAT format. Access to the drives for removable media (which are not shown in the drawing) may be had via a hinged dust-shield 24 provided at a front elevation 26 of the housing 22. Also provided at the front elevation 26 is a front panel 28 on which a plurality of switches are mounted. The switches permit the user to control various functions such as selecting input cameras for recording, setting a format for a displayed video signal, and controlling playback operations.

A commercial embodiment of a VR/PC unit, of a type in which the present invention may be applied, is currently being sold under the trademark "INTELLEX" by the assignee of the present application, Sensormatic Electronics Corporation, Boca Raton, Fla.

An overview of the internal components of the VR/PC unit will now be provided, with reference to FIG. 2. As seen from FIG. 2, primary components include a motherboard 40 and front end processing/compression electronics 42.

The motherboard 40 provides the overall intelligence and control for the VR/PC unit. Preferably, the motherboard 40 is constructed in accordance with conventional architecture for personal computer motherboards. The central processing unit for the motherboard is preferably constituted by a conventional microprocessor 44, which may, for example, be one of the models of the well-known Pentium line of microprocessors.

The motherboard 40 controls, and exchanges data with, data storage devices such as the above-mentioned hard disk drives, DAT drive and floppy disk drive. The motherboard is also adapted to receive user control signals, which may be input via the front panel 28 (FIG. 1) or via conventional user input devices (not shown) such as a mouse and/or a keyboard. The motherboard 40 also exchanges data with the front end processing/compression electronics 42 while receiving an input digital video signal from the front end electronics 42, and providing an output video signal to a display monitor, which is not explicitly shown in FIG. 2.

It should be understood that the motherboard 40 includes conventional features such as program memory, working memory, input and output data communication ports, data signal bus connections, interfaces for recording medium drives, and video interface ports. All of these are preferably of conventional construction.

The front end electronics 42 provide signal processing with respect to input video signals received via a back panel 46. The arrangement of the front end electronics 42 may, in general, be like that incorporated in the above-mentioned INTELLEX video recorder and/or as disclosed in the above-referenced parent patent application.

FIGS. 3 and 4 together show some details of the front end electronics component 42 of FIG. 2. Certain features of the front end electronics 42 relating to analog signal multiplexing and processing are schematically illustrated in FIG. 3. At the input side of the front end electronics 42 is a multiplexer 50 which selects ones of the input camera streams for recording and/or display by the VR/PC unit. Any one of the input video signal streams may be assigned to any one of three video signal locking channels 52-1, 52-2 and 52-3. At the output side of the signal locking channels, a channel-selection multiplexer 54 is arranged to assign any one of the three signal locking channels to either one of two signal conversion channels 56-1 and 56-2.

FIG. 4 provides an overall view, in functional block form, of analog/digital conversion and digital processing circuitry included in the front end electronics component 42. Major functional blocks related to digitizing and digital processing include an analog-to-digital conversion, buffering and data formatting block 60, a control and compression processing block 62, a live video display processing block 64 and a live video image analysis block 66. Also included is an interface 68 for providing a connection with a PCI bus which interconnects the digital processing circuitry of the front end electronics 42 with the motherboard 40 (see FIG. 2).

Continuing to refer to FIG. 4, block 60 receives two channels of analog video acquired through and selected by the analog multiplexing and processing circuitry of FIG. 3, digitizes the two analog channels, and buffers fields of the digitized video data in buffers 70 and 72, respectively corresponding to the two channels. Control signals to be output to the analog multiplexing and processing circuitry of FIG. 3, and signals indicating the status of the analog circuitry, including sync signals, are received and transmitted through the block 60. In addition, the block 60 controls a video data bus 74 and distributes the buffered fields of video data to the blocks 62, 64 and 66. A control/status bus 76 interconnects the control block 62 and other blocks shown in FIG. 4 and permits the control block 62 to control the other blocks and to receive signals indicative of the status of the other blocks. Control and status signals ultimately transmitted to or from the analog circuitry of FIG. 3 are also carried on the control/status bus 76.

In addition to providing overall control of the functions of the circuitry of FIGS. 3 and 4, the block 62 also performs data compression processing with respect to the video data output on video bus 74. Block 64 provides display processing of the video signals carried on video bus 74 and outputs a processed video signal, including overlay information and image plane allocation, to the motherboard 40 and to display monitors (not shown). The block 66 performs moving image analysis with respect to the video data carried on the bus 74 to permit the VR/PC to detect characteristics of the images represented by the incoming video data.

Additional details relating to the blocks 60, 62, 64 and 66 may be found in the above-referenced parent patent application. As disclosed therein, the block 60 outputs fields of video data twice, once in a line-scan format, and a second time in the form of rectangular pixel blocks (`tiles`). Preferably, as disclosed in the parent application, the line-scan and tile data for each field are interleaved with each other via time-division-multiplexed operation of the video bus 74.

The processing circuitry of FIG. 4 may include four or more programmable digital signal processing devices (DSP's), including one each corresponding to blocks 62, 64 and 66 and a fourth processing device included in block 60 and programmed to establish the line-scan and tile video data formats described above. However, it is also contemplated to combine some of the processing represented as separate blocks in FIG. 4 for performance by a single processing device. For example, two DSP's may be programmed to carry out, between them, all of the live display processing, live video data analysis, front end control, and video data compression functions.

There will now be described aspects of the format in which the block 60 causes the video data to be transmitted on the video bus 74.

FIGS. 5A and 5B each show an image plane 80, which is divided, for internal data representation purposes, into 240 lines in the vertical direction, and each line is divided into 640 pixels in the horizontal direction. The image plane is also represented in the form of tiles or pixel blocks, each pixel block measures 8 pixels in both the horizontal and vertical directions (FIG. 6). The image plane is thus divided into 80 tiles in the horizontal direction and 30 tiles in the vertical direction (FIG. 5B).

A 4:1:1 data format is employed whereby a group of four pixels is represented by four bytes of luminance data and two bytes of chrominance data. In effect, each line is divided into discrete groups of four pixels each, and for each such group four luminance pixel bytes are provided, as well as one byte of U-chrominance data and one byte of V-chrominance data (FIG. 8). This format contrasts with conventional 4:1:1 formats, in which each chrominance data byte corresponds to a two pixel by two pixel area of the image plane. The format utilized herein and illustrated in FIG. 8 helps to minimize "smearing" of the color information in the vertical direction and lessens any adverse effect upon image quality that may result from allocating only 240 horizontal lines to each video data field.

For compression encoding purposes, the image plane is divided into discrete groups of four horizontally-arrayed eight-by-eight tiles (FIG. 7). Each group of four horizontally sequential tiles constitutes a "minimum coding unit" (MCU). The data required to represent each MCU is made up of four eight-by-eight pixel blocks of luminance data, and one eight-by-eight block each of U data and V data, as shown in FIG. 9.

"Conditional Refresh" Data Compression

A description of the format of the compressed video data, as formed at block 62 of FIG. 4, will now be provided with reference to FIG. 10.

After compression, a stream of video images produced by a given camera is represented as a sequence of data fields 100. There are two types of fields in the sequence of fields: "reference" image fields 102 and "difference" image fields 104. The reference fields 102 occur at regular intervals in the sequence of data fields. For example, in a preferred embodiment of the present compression technique, every thirty-third field in the sequence is a reference field; that is, thirty-two "difference" fields 104 are provided between each sequential pair of reference fields 102. The reference fields 102 are each compression encoded without reference to any other image. On the other hand, each of the different fields 104 is compression encoded with reference to one or more preceding images.

Each of the reference fields 102 begins with a field header (not shown in FIG. 10). The field header may include information for identifying the field, such as data identifying the camera which is the source of the field, an indication that the field is a reference image field for the purpose of data compression processing, an indication that the field is a reference image field for the purpose of image analysis, an indication whether the field is generated while the VR/PC is operating in a single camera mode such that only video information from one camera is being captured, an indication whether the field corresponds to even- or odd-numbered scanning lines, and an indication as to whether frames or fields of the video signal are being captured. Preferably, the field header also includes data to indicate the date and time at which the field was generated.

The balance of the reference field 102 is made up of compressed video data corresponding to every one of the 2400 tiles of the image plane (see FIG. 5B). In particular, the compressed video data making up the balance of the reference field is formed by processing every one of the 3600 eight-by-eight pixel blocks (2400 luminance pixel blocks and 1200 color information pixel blocks; six pixel blocks in all for each four tiles in the image plane; see FIG. 9). Compression-encoding is applied to each of the 3600 pixel blocks by the control/compression signal processing block 62 of FIG. 4. According to a preferred embodiment of the invention, the compression encoding of the pixel blocks of the reference field 102 is performed in accordance with the well-known JPEG compression-encoding standard. JPEG encoding entails conventional steps such as orthogonal (DCT) transformation, quantization of coefficient values, and Huffman (run-length) encoding.

On the other hand, as to the "difference" fields 104, compressed video data is provided only for selected tiles that represent "changes" relative to a preceding image. When there is little or no motion in the video image stream, very few, or none, of the tiles of the image plane are represented by image data in the respective difference field 104. In this way, a high degree of compression is realized.

As seen from the lower portion of FIG. 10, a typical difference data field 104 is made up of a field header 106, followed by map data 108 which indicates the portions of the image plane which were considered "changed" in the present image relative to a preceding image, and are accordingly represented by image data in the present difference data field 104. For example, the map data may consist of one bit for each of the 600 minimum coding units (MCUs) in the image plane, it being recalled that an MCU corresponds to a discrete horizontal sequence of four tiles in the image plane. For example, a "1" bit corresponding to a particular MCU would indicate that the MCU has been found to be "changed" and is represented by JPEG-compressed video data corresponding to the six pixel blocks for the MCU.