 |
Description  |
|
|
BACKGROUND OF THE
INVENTION
This invention relates to computer-aided design systems, and in particular to a system and method for providing enhanced image color processing in such a two-dimensional design system.
In a computer-aided design system, high resolution graphic capabilities provide the ability to enter images with a digital camera or allow the designer to create images using "paintbox" programs. When the image on the graphic CRT is completed,
an output camera or printer may produce a hard-copy of the image. Because of the manner in which the components of a computer-aided design system interprets color, errors are introduced that cause the hard-copy image to appear differently from the image
as it appears in the real world or as displayed on the graphic CRT.
SUMMARY OF THE INVENTION
The present invention is directed to providing a system and method for correcting the colors in an image in a computer-aided design system such that the image produced from the output of the system will have the desired appearance. The present
invention provides the user of a computer-aided design system with tools and techniques for replacing the colors of an image with proper colors before the image is reproduced on an output device so that the color-distorting effects caused by the output
process, or other colors distorting process, may be corrected. According to one aspect of the invention, a method for correcting color errors introduced in computer-generated images includes creating a color chart having one or more colors and
performing the error-inducing process on the chart such that the color chart is distorted by the process. The method further includes determining the error that has been introduced by the process in adjusting the colors and images before performing the
error-inducing process on the images as a function of the determined error such that the image will have correct colors after the process is performed.
While the above aspect of the invention is capable of correcting the color of images that are considered a flat-color image made up of primary colors only, many images processed in a computer-aided design system, such as photographs and images
entered into the system through video cameras and the like, are composed of a myriad of differently colored pixels having (R, G, B) values which vary continuously over a large spectrum. With such continuously-colored images, it is virtually impossible
to identify all of the colors in the image and to determine the correction to the colors that will produce the desired appearance of the image at the output of the system. Therefore, according to another aspect of the invention, each pixel in a
continuously-colored image to be corrected is examined and, if its color is identical with one of the original colors, for which a corrected value has been determined, the color of the pixel is changed to that of the corrected value. For image pixels
whose color is not identical with an original color, an interpolation from other corrected values is made in order to construct a corrected value to be applied to the pixel.
These and other related objects, advantages, and features of this invention will become apparent upon review of the following specification in conjunction with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is an illustration of CAD hardware system useful in the present invention;
FIG. 2 is a flowchart illustrating a portion of the CAD system according to the present invention;
FIG. 3 is a flowchart illustrating the color palette editing functions of the present invention;
FIG. 4 is a flowchart illustrating the color adding function of the present invention;
FIG. 5 is a flowchart illustrating the color deleting function of the present invention;
FIG. 6 is a flowchart illustrating the color replacing function of the present invention;
FIG. 7a is a flowchart illustrating the color modification using hardware feedback functions of the present invention;
FIG. 7b is a flowchart illustrating the calculate corrected color function of the present invention;
FIG. 7c is a flowchart illustrating the double exposure function of the present invention;
FIG. 8 is a flowchart illustrating the color modification with user judgment functions of the present invention;
FIG. 9 is a flowchart illustrating the display color chart based on registered color flowchart according to the invention;
FIG. 10 is a flowchart illustrating the display color chart based on corrected color function of the present invention;
FIG. 11 is a flowchart illustrating the register corrected color function of the present invention;
FIGS. 12a and 12b are a flowchart illustrating the color correction function of the present invention;
FIG. 13 is a flowchart illustrating the defining a corrected color space function of the present invention; and
FIG. 14 is a flowchart illustrating the interpolation function of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
For the purpose of the following description, there are a few assumptions made about the nature of images that are displayed on standard graphic CRTs of CAD systems. As set forth in detail in "Fundamentals of Interactive Computer Graphics",
Foley, J. D. and Van Dam, A. (pp. 611-616 Addison - Wesly 1982), the disclosure of which is hereby incorporated herein by reference, the image, which is displayed in two-dimensional space, is composed of a rectangular array of pixels, the smallest unit
of the CRT; and the color of each pixel is represented by its red, green, and blue components, each of which is variable between zero and 255. Thus, the (R, G, B) value determines the color of the corresponding location currently displayed on the CRT.
The (R, G, B) value for each pixel of an image may be stored in the computer's memory, and retrieved when desired. The two-dimensional image may be entered as an input to the CAD system through a video camera, scanner, or by development by the user with
a standard CAD software.
(I) Description of the Hardware
The hardware requirements to support the present invention are basically those needed to support a standard two-dimensional CAD system. An illustration of such a system is given in FIG. 1. The hardware includes a graphic visual input/output
device such as a CRT 10 or 10', with a standard light pen 12 or mouse 14 as a locating instrument, a keyboard 16, a central processing unit 18 that will support program control of the individual pixels displayed on the graphic CRT 10 or 10' through use
of a graphics command library, and a data storage unit 20, preferably a hard disk storage unit. If the images used in the system are to be input through a video input, a video camera 22 and standard associated frame capture hardware (not shown) are also
required. For input of printed images, scanner 21 is provided. Images can also be inputted with a standard two-dimensional digitizing drawing device 32. Data storage unit 20 is needed for storage of the data files supporting the two-dimensional
software system, including the digital graphic images. Another consideration is the number of simultaneously displayable colors possible on the graphic CRT 10 or 10' of the computer system; a minimum of 256 simultaneously displayable colors is required. Adequate facilities can be found in the Engineering Workstations currently available from several venders such as Silicon Graphics Inc., Apple Computer Inc., and SUN Microsystems.
The present invention is preferably carried out with a Silicon Graphics Inc. IRIS workstation with a three-button mouse and a graphic CRT having a viewable resolution of 768 by 1024 pixels, a central processor having 2 megabytes of main memory
and a 170 megabyte hard disk for supporting 16 million simultaneously displayable colors on the graphic CRT.
The output of the system can be directed in known manner to a video tape recorder 24, an output camera 26 capable of producing color slides or photographs of various sizes, a hard-copy printer 28, or a plotter 30.
(II) Description of the Software
In this specification, the following definitions apply to these terms:
______________________________________ 1) Color: Representation in (R, G, B) space where R(red), G(green) and B(blue) are scalar values in the range 0.0-255.0. 2) Registered Color: Any color selected by user for correction; designated (rR,
rG, rB). 3) Corrected Color: The "correct" color of a registered color; designated (cR, cG, cB). Is applied to the image prior to processing the image to an output device such that the image from the output device has a desired appearance. 4)
Color Palette: A disk file containing a set of registered colors and their corrected colors (if any). More than one color palette may exist in the system. However, the software can use only one palette at one time.
______________________________________
Referring to FIG. 2, system 15 provides the tools for a user to replace the colors of an image with proper colors before the image is developed on an output device, such as a camera 26 or a graphics printer 28, using a two-dimensional CAD system
programmed in accordance with the invention. Using such programmed 2-D CAD system, the user first creates and edits (34) a color palette by adding, deleting and replacing registered colors. "Correct" colors may then be found for registered colors in a
palette using color analyzing functions employing hardware feedback (36) or color displaying functions which are applied by the user to allow the user to select corrected colors (38). Finally, a color palette of registered colors and corresponding
corrected colors is applied in a function for applying color correction (42) to an image created in known manners at 40. Since most CAD systems are menu driven, the following will describe how a user would execute the functions by selecting from a menu
displayed on the graphic CRT of the system.
(a) Creating Color Palette
The first step is to create and edit a color palette so that the system can utilize registered colors in the palette to correct the colors of an image. Basic capability for creating and editing color palette (34) can be provided in a menu with
the following choices:
______________________________________ ADD registered color(s) DELETE registered color(s) REPLACE registered color(s) ______________________________________
The "ADD" function permits user to add (register) a new color to the color palette by typing the (R, G, B) or by picking a color on the screen using a locating instrument such as light pen 12 or mouse 14. This new color then becomes one of the
registered colors in the palette. The "DELETE" function permits user to remove registered color(s) from the palette along with the corrected color(s) by selecting the registered color(s) to be deleted using a locating instrument. The "REPLACE" function
permits user to change the (R, G, B) of an existing registered color.
FIG. 3 illustrates the main flowchart for the color palette generation and editing functions 34, and outlines a function that allows the user to select various subfunctions for performing color palette editing. Once invoked from the
two-dimensional CAD system (44, 46), the user is prompted to enter the name of the color palette of interest (48) using keyboard 16. If the specified palette is not found in the storage unit 20, then a new palette is generated (50, 56). If the
specified palette is found, then its registered colors and corrected colors are retrieved from the storage unit 20 and stored in a global variable and an array of colored squares, representing existing registered colors in the selected color palette, are
displayed (52) in the menu area of the graphic CRT. The system then displays a menu for the user to select (54, 58) among the individual subfunctions 60-64 provided for palette editing. Each of the subfunctions 60-64 that may be invoked by a user
selection performs a portion of the palette editing function. Once each of these subfunctions 60-64 finishes its task, control is returned to the palette generating and editing function indicated by the labels R1-R3. Control is then passed back to the
menu selection (US) 70 so that the user may proceed to the next function. When the "exit" function 66 is selected, this function terminates and all the parameters stored in global variables are to the storage unit 20 for later retrieval 68. Control
then passes back to the two-dimensional CAD system (43, 44).
FIGS. 4, 5 and 6 are flowcharts for the subfunctions for creating and editing a color palette. After the user selects the "Add color" function 60 (FIG. 4), the system receives an (R, G, B) value from user (72) by prompting the user to pick a
color from the images current displayed on the graphic CRT 10 with a locating instrument or by entering the (R, G, B) value of the new color via a keyboard 16. The received color is examined (74) to see if an identical color already exists in the color
palette. If so, an error message is generated (78) and this color will not be registered in the color palette. If the received color does not already exist, then the system creates a new global variable and stores (76) this (R, G, B) value which
becomes one of the registered colors of the current palette. The new registered color joins other registered colors displayed on the menu area of the graphic CRT. The user can continue to add new colors to the color palette until the user indicates to
the system that no additional colors are to be added (80).
FIG. 5 is a flowchart illustrating the "Delete color" function. Once this function is selected, the user is prompted to select (86) the registered color to be deleted from the color palette displayed on the menu area using a locating instrument. The system examines (88) the (x, y) coordinate obtained from the user and determines (90) if the color selected by the user is existing in the palette. If not, then an error message 94 is issued and no color is deleted. Otherwise, the selected color is
deleted (92) from the current color palette and the global variable it occupied is cleared. The user can continue to delete registered colors from the color palette until the user indicates to the system that no additional colors are to be deleted (96).
FIG. 6 is a flowchart illustrating the "Replace color" function 100. In a manner similar to the "Delete color" function 62, the user is prompted to select a color and the system determines if the selected color exists for the palette (102, 104,
106, 112). Once an existing registered color is selected, the user is prompted to pick a new color on the graphic CRT using a locating instrument or to enter a new (R, G, B) value with the keyboard (108). The (R, G, B) value for this new color then
replaces (110) the original one stored in the global variable. The user may repeat this function to replace additional registered colors until the user indicates to the system that no additional colors are to be replaced (114). When the "exit" function
66 is selected by the user, all the global variables associated with the registered colors are stored (68) to the storage unit 20 and the control passes to the two-dimensional CAD system 44.
(b) Finding Corrected Colors
1. Hardware Feedback
When the user is satisfied with the registered colors in the color palette, the next step is to determine the corrected color for each registered color in the palette. There are two functions that provide the user with tools for finding the
corrected colors. The first function (36) provides the user with tools to send a color chart displayed in graphic CRT 10 containing the registered colors to an output device, such as camera 26, and then input the
resulting photograph of the color chart back into the system to allow the system to determine the color "error" created by the process of developing a hard-copy of the color chart by the output device. The system would then modify the (R, G, B)
values of the registered colors based on the "error" to provide corresponding corrected colors which will more closely approximate the registered colors offering processing to the hard-copy. To execute the first method, the following menu is provided:
______________________________________ Display registered color chart Calculate corrected color Display corrected color chart Modify corrected color Display colors over 255 ______________________________________
In order to produce an image on the graphic CRT 10 of the registered colors in a palette selected by the user, the user selects the "Display registered color chart" function. The system will display all the registered colors in a color chart
containing an array of colored squares, one for each registered color. The user then instructs the system to send the chart to an output device using conventional methods. The hard-copy of the image produced by the output device is then fed back to the
system using an input device such as scanner 21 or the like. The user then selects the "Calculate corrected color" function to find the corrected color for each registered color. The function is accomplished by the system scanning the registered colors
and determining the changes in (R, G, B) of each registered color caused by the output/input cycle. These changes then are added to the original (R, G, B) of each registered color to compensate for the "error" occurred during the output/input cycle.
These new (R, G, B) values are then designated corrected colors and are stored along with their corresponding registered colors in the appropriate color palette.
The flow diagram for the function which determines corrected colors using hardware feedback (36) is illustrated in FIG. 7a. When this function is selected (124), the user is prompted to enter (126) the name of the desired color palette using
keyboard 16. The system then retrieves the registered colors and corresponding corrected colors, if any, from this palette file in the storage unit 20 and stores them in a set of global variables (130). If the specified palette does not exist in the
storage unit (128), this function is terminated and control passes back to the two-dimensional CAD system (44). The system displays a menu (132) for user to select (134) various subfunctions 136-144.
If the "Display registered color chart" function 136 is selected, the system displays all the registered colors of the current color palette in an array of colored squares on the graphic CRT. The user instructs the system to send this color
chart to the output device to obtain a photograph or printout, and then reenters this photo, graph, or printout into the system using an input device such as scanner 21 or video camera 22 using conventional techniques. Once the previous step has been
completed, the "Calculate corrected color" function (Level 1 correction) function 140 is selected. The flowchart for the Level 1 correction function is illustrated in FIG. 7b. The system will prompt the user to select an area of the color chart via an
input device such as mouse 14 or light pen 10 to be corrected and obtain (250) the (X, Y) coordinates of the area selected by the user. The system then scan-converts the selected area by reading the (R, G, B) values of each pixel (252-262) of the
inputted hard-copy. After reading in the new (R, G, B) values, the system calculates the "error" caused by the output/input cycle (256) and finds a corrected color (258) for each registered color.
Assuming that an existing registered color has an (R, G, B) value (rR, rG, rB), and the new (R, G, B) value after the output/input cycle is (nR, nG, nB), then the "error" is: (eR, eG, eB)=(rR-nR, rG-nG, rB-nB). Adding this "error" to the
original color (rR, rG, rB), the corrected color is:
The corrected colors determined during this function are stored in the global variables along with their associated registered color.
Once each registered color in a palette has a corrected color, the user can apply this color palette to an image in a manner that will be set forth in detail in Section C (Correcting Image). If the image produced is still not satisfactory, then
the user may fine-tune the corrected colors by selecting the "Display corrected color chart" subfunction to display all of the corrected colors in an array of squares, and the same output/input procedure as applied to the original registered colors is
repeated. After the output/input cycle, the corrected color chart is again displayed on the screen and the user selects the "Modify corrected color" function (Level 2 Correction) to cause the system to determine the composite error between the modified
corrected colors and the associated registered colors, in the same manner as in the "Calculate corrected color" function, by comparing the (R, G, B) values of the previous corrected colors with those just entered. In this manner, a new set of corrected
colors is generated to replace the old one.
To fine-tune the corrected color, the "Display corrected color chart" function 138 and "Modify corrected color" (Level 2 correction) function 142 are selected. Functions 138 and 142 are essentially the same as the "Display registered color
chart" function 136 and "Calculate corrected color" function 140 respectively except that the corrected colors previously determined by functions 136 and 140, not registered colors, are displayed and analyzed. If the new (R, G, B) value after the
output/input cycle of the corrected color (cR, cG, cB) is (nR, nG, nB) and let:
then the new corrected color (ncR, ncG, ncB) is
While performing functions 138 and 142 to display corrected color, it is possible that one or more of the corrected colors' (R, G, B) component values will exceed 255, which is the maximum value that the graphic CRT 10' can display directly.
When the "Display corrected color chart" function 138 is selected, the system will give an indication on the graphic CRT, such as "Displayed colors over 255", to apprise the user if certain displayed color values exceed the maximum. Along with providing
this indication, the graphic CRT will display the excessive color values at a maximum (255) allowable intensity and will prompt the user to send this first color chart to the output device such as camera 26 as the first exposure. All colors that are
below maximum intensity are displayed on the graphic CRT in their actual (R, G, B) values. After the first exposure is completed, the system prompts the user to reinsert the same film cartridge into output camera 26 for the second of a double exposure
and to select the "Display colors over 255" or "Double Exposure" function 144.
The "Double Exposure" function is illustrated in FIG. 7c. This function alters the color chart display on the graphic CRT by scan-converting the color chart (264-272) and deleting all color components less than 255 from the image of the color
chart by subtracting 255 from each of the "overflow" color components i.e., those that exceed 255 (268). The user is then prompted to send this modified chart to the output camera (274) for a second exposure on the same film. The "overflow" color
components compound the color values on the film from the first exposure to produce the correct color.
Each time the "Display corrected color charts" and "Modify corrected color" functions 138 and 142 are selected, a new set of corrected colors is generated in the manner previously set forth to replace the previous ones. Repeated use of these two
functions should achieve optimal corrected colors. While the hardware feedback function may be applied to essentially any input/output interface, it is very important that the same input/output device be used throughout the correction procedure to
obtain accurate results. If the registered colors are registered from an image entered from an input camera but the photograph for the registered colors is reentered using the scanner, then there is an additional "error" between the scanner and camera,
that adversely affects the accuracy of the results. When "Exit" 146 is selected by the user, all the global variables associated with the registered and corrected colors are stored to the storage unit 20 in the file for the current palette and control
passes back to the two-dimensional CAD system (43, 44).
2. User Judgment
The second method (38) for finding corrected colors for registered colors provides tools to the user for creating and selecting possible corrected colors. The user, not the system, determines the desirable corrected colors. When the "Select
Corrected Color" function 38 is selected, a plurality of colors having (R, G, B) values which vary slightly from a registered color are displayed as squares in a color chart and the user is prompted to send the chart to the output device. The user then
selects from the hard-copy the square having the color that most closely matches the registered color displayed on the graphic CRT.
To execute the second method, the following menu is provided:
______________________________________ Display color chart based on registered color Display color chart based on corrected color Register corrected color ______________________________________
When "Display color chart based on registered color" is selected, the user is prompted to specify the name of the color palette of interest, and the system displays all registered colors for the palette in an array of squares in the menu area and
prompts the user to pick the registered color of interest with a locating instrument. After a registered color ("current registered color") is selected, the system then prompts user to select the range of the colors to be displayed by selecting a range
number. The smaller the number entered, the smaller the difference among the colors of the displayed chart. Once a range is selected, the system then displays a new color chart on the graphic CRT which is composed of colors whose (R, G, B) components
are obtained by slightly varying the current registered color. Each chart normally has 30 to 60 different colors with the original registered color displayed as a large area in the middle of the chart. The user is then prompted to send the color chart
to the output device. By comparing each color on the hard-copy photograph or print with the original registered color displayed on the graphic CRT, the user judges which colored square produces the color on the photograph that most closely matches the
registered color. This color will then be selected as the first corrected color candidate for the current registered color.
The "Register corrected color" function allows user to select the color square that will be designated the corrected color for the current registered color. The system will prompt the user to pick the corrected color from the color chart on the
graphic CRT according to its location on the hard-copy using a locating instrument, and prompts the user to pick the corresponding registered color from the menu area. The registered color and its new corrected color are both stored as a pair.
Once an initial corrected color is selected, the user may fine-tune the corrected color by selecting "Display color chart based on corrected color" function. This function is identical to the "Display color chart based on registered color"
function, except that the colors displayed in the chart are variations of the presently selected corrected color of the current registered color. By sending the displayed chart to the output device and evaluating the hard-copy, the user again selects a
color that the user judges to be closest to the displayed corrected color. By initially using a larger range number and gradually decreasing the range number while repeating this function, the user can quickly find the desired corrected colors
efficiently.
When the "Select Corrected Color" function is selected, steps 150 to 156 (FIG. 8) are performed in a manner that is identical to steps 124 to 130 in FIG. 7. A desired color palette will be retrieved from the disk and the parameters for the
registered and corrected colors of the palette will be loaded into global variables. The menu choices for this function are displayed (158) and the user is prompted (160) to select a subfunction from the menu. There are three subfunctions 162-166 for
user to select, plus an "Exit" function.
FIG. 9 illustrates the flow diagram for the "Display color chart based on registered color" function 162. When this function is selected, the user is prompted to select (172) the registered color of interest from the color palette using a
locating instrument. Once a valid registered color is selected, the user is prompted to enter (174) a range number such as 1, 2 or 3 via the keyboard. The larger the number entered, the bigger the difference among the colors of the displayed chart.
The system then generates (176) 30 to 60 colors to be included in a color chart by increasing and decreasing each of the (R, G, B) components of the registered color as a function of the range number selected. The generated color chart is displayed
(176) on the graphic CRT with the registered color in the center. The user is then prompted to instruct the system to send the displayed chart to the output device for producing a hard-copy photograph or print.
FIG. 10 illustrates the flow diagram for the "Display color chart based on co | | |
