Interactive camera for network processing of captured images

We claim:

1. A system for processing and correcting raw, digitized image information sets and converting each processed and corrected image information set into a display image format comprising:

an electronic still camera having image capture means uncompensated for image capture defects for capturing a scene image as a raw, digitized image information set and digital memory for storing one or more information set, the camera having an identification code keyed to an image defect correction map derived at manufacture to map and correct for image capture defects and a data communication controller for transmitting the camera identification code and the raw, digitized image information sets from said memory to a designated receiver; and

an interactive network remote from said camera further comprising:

a receiver for receiving said transmitted, raw, digitized image information sets and said identification code and for communicating on the interactive network to transmit or receive data, including transmission of said raw, digitized image information sets;

a repository on the interactive network for receiving the image defect correction map derived at manufacture keyed to the camera identification code;

processing means responsive to a communication from said receiver for retrieving the image defect correction map unique to the camera identification code and for correcting and processing said each raw, digitized image information set into a display image format for display or reproduction of the captured image; and

means for utilizing the display image format for displaying or reproducing the captured image.

2. The system of claim 1, wherein:

said interactive network comprises an interactive, multimedia, telecommunications network with distributed processing and storage of information between user's residences and programming sources on a subscription basis with service providers; and

said receiver comprises a user's home set-top box or HIC for interacting with the telecommunications network upon subscribing to a service on the network.

3. The system of claim 2 wherein said processing means comprises a local control node of the interactive network accessible to and from the user's home set-top box.

4. The system of claim 3 wherein said processing means is loaded with processing algorithms for the particular camera type to process the raw, digitized image information sets into the display image format and to utilize the image defect correction map specific to the camera identified by the identification code in the processing of the image information sets.

5. The system of claim 2 wherein:

said processing means comprises the user's home set-top box or HIC; and

said set-top box or HIC is downloaded with processing algorithms for the particular camera type and the image defect correction map for the particular camera identified by the camera identification code to process the raw, digitized image information sets into the display image format and to utilize the downloaded image defect correction map specific to the user's camera in the processing of the image information sets.

6. The system of claim 2 wherein:

said processing means comprises a local control node of the interactive network accessible to and from a print maker authorized by a user to make reproductions of the captured images represented by raw, digitized image information sets.

7. The system of claim 6 wherein said local control node is downloaded with processing algorithms for the particular camera type and the image defect correction map for the particular camera identified by the camera identification code to process the raw, digitized image information sets into the display image format and to utilize the downloaded image defect correction map specific to the user's camera in the processing of the image information sets.

8. A method of processing and correcting raw, digitized image information sets and converting each processed and corrected image information set into a display image format comprising:

providing an electronic still camera to a user for capturing a scene image as a raw, digitized image information set, the camera having an identification code keyed to a camera specific image defect correction map derived at manufacture from test of the camera to correct for image capture defects and a data communication controller for transmitting the camera identification code and the raw, digitized image information sets to a receiver of an interactive network;

transferring said image defect correction map to a repository of an interactive network accessible through a service;

transferring color correction and processing software algorithms to a repository of the interactive network accessible through a service;

transferring the camera specific defect correction map and the color correction and processing software from the repository to an image information processing system upon receipt of a request for processing raw, digitized image information sets acquired by said camera, the request including the camera identification code;

transferring raw, digitized image information sets acquired by said camera to said image information processing system through said service; and

in the image information processing system, processing and correcting raw, digitized image information set into a display image format set.

9. The method of claim 8 further comprising the step of:

transferring the display image format sets to a user defined recipient on the network.

10. The method of claim 8, wherein:

said interactive network comprises an interactive, multimedia, telecommunications network with distributed processing and storage of information between user's residences and programming sources on a subscription basis with service providers; and

said defined recipient comprises a user's home set-top box for interacting with the telecommunications network upon subscribing to a service on the network.

11. The method of claim 10 wherein said image processing system comprises a local control node of the interactive network accessible to and from the user's home set-top box.

12. The method of claim 10 wherein said processing system comprises the user's home set-top box.

13. The method of claim 10 wherein said processing system comprises a local control node of the interactive network accessible to and from a print maker authorized by a user to make reproductions of the captured images represented by raw, digitized image information sets.

14. A system for processing sets of digitized image information and converting each processed image information set into a display image format, said system comprising:

an electronic camera that generates a camera identification code and sets of digitized image information corresponding to captured images, said camera including a data communications means for transmitting the camera identification code and the sets of digitized image information when instructed by a camera user;

an interactive network remote from said camera, said network comprising:

means for communicating with the interactive network to transmit or receive data, including transmission of the digitized image information sets and the identification code;

a receiver for receiving the digitized image information sets and the identification code transmitted from said camera by the data communications means;

a repository on the interactive network for receiving one or more processing algorithms keyed to the camera identification code;

means for downlinking said one or more processing algorithms on the interactive network to the repository from a remote source distinct from the camera user; and

processing means responsive to a communication from said receiver for retrieving said one or more processing algorithms keyed to the camera identification code and for processing each digitized image information set into a display image format for display or reproduction of the captured image.

15. The system of claim 14 wherein the remote source of the algorithms distinct from the camera user is a manufacturer of the camera.

16. The system of claim 14 wherein said camera includes an image capture array composed of photosites and a memory for storing the camera identification code, and wherein said data communication means includes means for retrieving and transmitting the camera identification code with the stored sets of digitized image information.

17. The system of claim 16 wherein the camera identification code is correlated to a photosite defect map identifying any defective photosites of the image capture array by row and column number.

18. The system of claim 16 wherein the camera identification code is correlated to a color correction map compensating for the degrees of any non-uniformities in color responses of the photosites to color of a test image captured by the image capture array by row and column number.

19. The system of claim 18 wherein the color correction map compensates for gamma and individual photosite or overall white balance errors.

20. The system of claim 18 wherein the camera includes an imaging lens system and the color correction map compensates for optical aberrations in the imaging lens system affecting the photosite response.

21. The system of claim 18 further comprising:

means for providing artificial illumination of the captured images; and

wherein the color correction map compensates for imbalances in the artificial illumination keyed to the photosite locations affecting the photosite response.

22. The system of claim 14 wherein said means for communicating with said interactive network comprises a communication link between the data communications means and the receiver.

23. The system of claim 14 wherein said means for communicating with said interactive network further comprises a cellular telephone communications link.

24. The system of claim 14 wherein said means for communicating with said interactive network comprises a personal communications system link.

25. The system of claim 14, wherein:

said interactive network comprises an interactive, multimedia, telecommunications network with distributed processing and storage of information between user's residences and programming sources on a subscription basis with service providers; and

said receiver comprises a user's home interface controller (HIC) for interacting with the telecommunications network upon subscribing to a service on the network.

26. The system of claim 25 wherein said processing means comprises a local control node of the interactive network accessible to and from the user's HIC.

27. The system of claim 26 wherein said processing means is loaded with the processing algorithms keyed to the camera identification code to process the digitized image information sets into the display image format and to utilize the processing algorithms keyed to the identification code in the processing of the image information sets.

28. The system of claim 25 wherein:

said processing means comprises the user's HIC; and

said HIC is downloaded with the processing algorithms keyed to the camera identification code to process the digitized image information sets into the display image format and to utilize the downloaded algorithms in the processing of the image information sets.

29. The system of claim 25 wherein:

said processing means comprises a local control node of the interactive network accessible to and from a print maker authorized by a user to make reproductions of the captured images represented by digitized image information sets.

30. The system of claim 29 wherein said local control node is downloaded with processing algorithms for the particular camera identified by the camera identification code to process the digitized image information sets into the display image format and to utilize the downloaded algorithms specific to the user's camera in the processing of the image information sets.

Description Submit all comments and votes

CROSS-REFERENCE TO RELATED APPLICATIONS

Reference is hereby made to commonly assigned, co-pending U.S. patent application Ser. Nos.: 085,516, filed Jun. 30, 1993, in the names of K. Parulski et al. and titled SINGLE SENSOR COLOR CAMERA WITH USER SELECTABLE IMAGE RECORD SIZE; and 167,756, filed Dec. 15, 1993, in the names of K. Parulski et al. for PORTABLE ELECTRONIC CAMERA WITH COMPUTER DOCKING ADAPTOR PROVIDING COMPUTER CAPTURE CONTROL.

FIELD OF THE INVENTION

This invention relates to the field of electronic imaging employing an interactive image capture system with distributed processing of image information, and more particularly to a simplified digital camera and centralized image processing system.

BACKGROUND OF THE INVENTION

Electronic still cameras are known in the prior art for capturing images of scenes and providing the captured images to a computer for display on a computer monitor. In the above-referenced '516 and '756 applications, electronic still cameras are described for capturing images and displaying the captured images on the monitor employing a computer docking station and interface to download image data to computer memory for display and more permanent storage of the video image data. In those cameras, the camera on-board memory may then be employed to capture further images until it is again filled.

In the '756 patent, the camera is either operable by itself in a stand alone mode to store images in on-board memory or may be tethered to the computer in a docked mode to directly transfer the video image data to the computer memory. In the '516 patent, an electronic still camera is described having a resolution mode switch for selecting the pixel resolution.

Electronic still cameras of the type described in the '516 and '756 applications employ two-dimensional, charge-coupled-device (CCD) arrays of CCD photosites. The CCD photosites are overlaid with a three color filter in a checkerboard pattern. The red, green and blue analog, raw image pixel signals of the CCD photosites are clocked out in a video frame format. The raw image analog pixel signals are digitized, and luminance and chrominance pixel information is developed from the red, green and blue raw image pixel signals. Typically, the luminance signal is derived from the green filtered photosite pixel signals, and chrominance is developed from the red and blue filtered photosite pixel amplitude difference signals, with a green signal interpolation. The digitized video frame signal sets are processed within the cameras to correct for defective photosites within the array and for color balance.

The corrected digital pixel signals are compressed and formatted into a video frame signal for display on a monitor or television at the same or a later time. Typically, the video frame signals for each image captured are stored in flash EEPROM or other memory for later read-out to a personal computer (PC). A central processing unit governs the operations of the circuitry and hardware under the control of programmed operating routines.

Currently, a PC electronic still camera is being marketed by Apple Computer, Inc. which operates in high and low resolution modes to take a limited number of pictures for display through a PC interface. Such electronic still cameras are relatively complex and expensive when constructed to have high resolution and image storage capabilities.

In U.S. Pat. No. 5,220,420, an interactive multimedia system with distributed processing and storage of video frames and associated data and sound in nodes disposed throughout a cable television distribution system is described. An interface to allow users to input video images from a camcorder or VCR into a video to RF modulator in a Home Interface Control (HIC) is also described. The ability to display photographic quality images in both still frame and short full-motion segments on a home television receiver is attributed to the system, although the available video camcorders or VCRs do not support photographic quality images.

An RF to video demodulator and a video digitizer are provided in the node coupled to the HIC to demodulate and digitize the video data input from a camcorder or VCR to forward it to a regional processing center. The regional processing center converts and normalizes incoming digitized video frames into a system standardized format and stores them in an object database in each of the system nodes for selective retrieval by a viewer. No specific camcorder or video camera type is described in the '420 patent, but it is clear that a conventional analog video frame output utilizing the traditional NTSC format is supplied to the HIC to be converted to a digital format by the video digitizer 164.

PROBLEMS TO BE SOLVED BY THE INVENTION

The expense and complexity of the conventional high resolution electronic still image camera inhibits its acceptance in the market for conventional, non-network image capture and storage. The cost of participating in an interactive system is magnified by the expense of acquiring suitable high resolution video image capture cameras.

SUMMARY OF THE INVENTION

It is therefore a primary object of the invention to simplify and make less expensive a high resolution electronic still image camera system for use in conjunction with an interactive home information system.

It is a further object of the present invention to provide an interactive system for use with a simplified electronic still camera for capturing high resolution digitized images and storing the captured images for viewing on a subscriber's television or computer display monitor.

The present invention preferably employs an electronic still camera for use in association with a system for providing signal processing of raw, digitized image information representing the color and intensity value of each pixel of a captured image for a specified display or reproduction format, the camera comprising: a camera controller responsive to camera control signals for controlling operations of the camera; means for storing a camera identification code; a solid state, image capture array of color filtered, pixel photosites; an imaging lens system for focusing the image to be captured on the image capture array; image capture means responsive to an exposure control signal for exposing the image capture array to the focused image through the imaging lens system for a predetermined exposure interval in order to develop a raw, analog image information set for each color from which image luminance and chrominance signals may be derived for each captured image; driver means responsive to the camera controller for clocking out the raw, analog image information set from the image capture array upon completion of the exposure; means for digitizing the raw, analog image information set; data storage means responsive to the camera controller for temporarily storing at least one of the raw, digitized image information sets, each representing a captured image; and data communication means for retrieving and transmitting each stored raw, digitized image information set to further processing equipment.high resolution electronic still camera for use in association with an interactive system for providing centralized signal processing of raw, digitized image information representing the color and intensity value of each pixel of a captured image for a specified display or reproduction format.

The camera does not include on-board signal processing for processing the raw, digitized image information set to account for dark signal current variances or defects in individual CCD photosites of the array that are unavoidable with current CCD technology at a reasonable price or for other filtered photosite or camera related defects.

In a further aspect of the invention, such a camera is employed in a system for processing the raw, digitized image information sets to correct for such variances or defects remotely from the camera at the user's HIC or upstream in the interactive network accessed from the user's HIC for providing a corrected image data set to the user or others designated by the user. Other designees include a print or CD maker or other user's Set-Top Boxes for making print or CD reproductions or for distributing images to others.

The camera having an identification code keyed to a camera specific image defect correction map derived at manufacture from test of the camera to correct for image capture defects and a data communication controller for transmitting the camera identification code and the raw, digitized image information sets to a receiver of an interactive network is preferably used in a system and method of processing and correcting raw, digitized image information sets and converting each processed and corrected image information set into a display image format comprising the means for and steps of: transferring the image defect correction map to a repository of an interactive network accessible through a service; transferring color correction and processing software algorithms to a repository of the interactive network accessible through a service; upon receipt of a request for a processing raw, digitized image information set, the request including the camera identification code and entered in the network, transferring the camera specific defect correction map and the color correction and processing software from the repository to an image information processing system; in the image information processing system, processing and correcting each raw, digitized image information set into a designated display image format; and transferring the display image format sets to a user defined recipient on the network.

ADVANTAGES OF THE INVENTION

In accordance with the present invention, and due to the various features thereof, an inexpensive, high resolution camera may be employed with an interactive network subscribed to by a purchaser to enable electronic image capture and video display of the captured images. The expected capabilities of the emerging interactive networks and services to be provided on such networks are taken advantage of in economizing and adding flexibility to the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, advantages and features of the invention will become apparent from the detailed description given hereinafter in relation to the accompanying drawings, in which:

FIG. 1 is a schematic illustration of the components of an electronic still camera in which the present invention may be practiced;

FIG. 2 is a schematic illustration of the interrelationship between the camera of FIG. 1 and an interactive network for communication through an HIC;

FIG. 3 is a schematic illustration of the interrelationship between the camera of FIG. 1 and an interactive network providing signal communication through a PCS or cellular phone link; and

FIG. 4 is a schematic illustration of the interconnection of an interactive network leg with the simplified electronic still camera through the HIC or the alternative links.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

As stated above, the present invention preferably employs a high resolution, but simplified, electronic still image camera for capturing an image as a set of raw, digitized image information corresponding to the pixel intensities of an image capture CCD element array. FIG. 1 is a schematic block diagram of one such camera 10 for providing the raw, digitized image information along with a camera serial number or identification (ID) code to one of the systems described in detail below for processing the image information for color correction and correction for CCD element pixel defects.

The camera 10 may be a simplified version of the cameras disclosed in the above-referenced co-pending '516 and '756 applications, incorporated herein by reference. The camera 10 may be operated in a stand-alone mode or a "docked" mode as described in the '756 application. The simplified camera 10 comprises the camera microcontroller 14 responsive to camera control signals, e.g. depression of the image capture button by the user, from control block 22 for controlling operations of the camera 10. The microcontroller 14 also controls the exposure parameters and illumination by processing output signals from photometer 40 and operating flash unit 42 when measured ambient light conditions require artificial illumination. A control liquid crystal display 44 is also provided in which camera operating and status messages may be provided to the user by the microcontroller 14 in a manner well known in the art.

A trichromatic, solid state image capture array 20, e.g. a CCD array of the type described above, is located behind an imaging lens system 24 and an optional blur filter for focusing the image to be captured on the image capture array 20. A shutter and aperture mechanism 26 is responsive to an exposure control signal from microcontroller 14 for exposing the image capture array 20 to the image focused thereon by the lens system 24 in a conventional fashion. The image capture array 20 is responsive to a driver 28 in order to the develop trichromatic, raw, analog image information which is amplified and supplied to an A/D converter 30. The array driver 28, operated by a timing generator 32 in response to the microcontroller 14, clocks out the raw, analog image information to the A/D converter after the exposure of the array 20 is complete. The A/D converter 30 digitizes the raw, analog image information as an information set and supplies it to a buffer memory 34. The information set is then retrieved by the microcontroller 14 and optionally subjected to compression before being stored in non-volatile memory 16 along with the camera ID code. The camera serial number or ID code is stored in ROM 12 at manufacture.

The memory 16 is preferably a non-removable flash, electrically-programmable read only memory (FLASH EPROM), although a hard disk drive, a floppy disk drive, a removable memory IC card or a similar device could be used. Different types of compression could be employed, e.g. a conventional adaptive differential pulse code modulation (ADPCM) technique.

The contents of the memory 16 are read out through a data communications controller 46 enabled by the microcontroller 14 in response to a user initiated camera control signal. The stored sets of the raw, digitized image information each representing a captured image are thereby transmitted to an image processing system and converted into a display or reproduction image format for display or for use in making a print or creating a Photo CD of the captured image.

The ID code for the camera 10, stored in memory 12 at manufacture, is read out by a camera microcontroller 14 and appended to the raw, digitized image information set for each image frame that is transferred to non-volatile memory 16. The camera 10 is tested during manufacture to determine which of the CCD array photosites of the CCD array 20 are defective. A pixel defect map associated with the ID number is created that identifies the defective photosites by row and column number. In addition, a color correction map is created that corrects for non-uniformity in the checkerboard pattern, color filter array over the photosites as well as color sensitivity variations in the photosites. The color correction map compensates for gamma and individual photosite or overall white balance errors in a well known manner. Optical aberrations in the lens system 24 of the individual camera 10 could also be measured and stored with the color correction map. In addition, imbalances in the flash illumination could also be measured at manufacture of the camera and stored as an illumination correction map keyed to the photosite locations. In accordance with the present invention, the ID code retrieved from memory 12 and stored with the raw, digitized image information sets is used to in turn retrieve the correction maps of that same camera for use in processing the information sets.

In this regard, the raw, digitized image information set for each captured image is transmitted remotely of the camera 10 with the camera ID code to complete color processing and to construct a corrected image information set in the display image format that is returned to the user for display on a video monitor. The corrected image information may be formatted to an NTSC standard for display on a conventional television or formatted in a reproduction format for a particular color printer for printing as a color print or as a CD.

More particularly, at the remote processing and correction location, the raw, digitized image information is processed in a conventional fashion to derive luminance and chrominance. A green layer interpolation is effected, and color difference generation of Red-Green and Blue-Green is effected.

The image capture defect or correction maps stored for the particular camera are retrieved from a source using the camera ID code. In the examples described below, the correction maps are stored in a central or somewhat distributed repository (data bank) of any suitable storage medium and format and accessed in a variety of ways. When accessed, white balance normalization is effected employing the color correction map. Color correction is accomplished by shifting or scaling any or all pixel values that require normalization.

The pixel defect map is employed to identify false data bits and to initiate interpolation of scene content from surrounding valid pixel values. A processing algorithm may be employed that takes into account color information of adjacent pixels to generate suitable replacement color information for the defective pixel location in a manner well known in the art. Similarly, optical aberrations and artificial illumination hot spots could also be corrected for, if the camera specific correction maps are derived at manufacture and stored in the reposit