Method and system for enhancement and detection of abnormal anatomic regions in a digital image

What is claimed as new and desired to be secured by Letters Patent of the United States is:

1. A method for automated detection and indication of an abnormal anatomic region using a digital image, comprising the step of:

generating a single digital image of an object;

storing said single image;

filtering said stored single image to remove anatomic background derived from normal anatomic structure and thereby to enhance in the resulting filtered image an abnormal pattern corresponding to an abnormal anatomic region;

searching said filtered image to determine a region having said abnormal pattern in said digital image; and

indicating the position of said abnormal anatomic region in connection with said digital image.

2. The method as defined by claim 1, wherein said step of filtering the stored digital image includes a step of signal-to-noise ratio (SNR) suppressing filtering of said stored image signal, and a step of SNR enhancing filtering of said stored image signal, a step of producing a difference image between said SNR-suppressed image and said SNR-enhanced image.

3. The method as defined in claim 2, wherein said searching step comprises:

determining which of the pixels of the difference image exceed a predetermined amplitude threshold value;

identifying contiguous pixels determined by said determining step as discrete islands;

determining at least one of the circularity and the size of said islands; and

identifying an abnormality by comparing at least one of the circularity and size of said islands against predetermined criteria.

4. The method as defined in claim 2, wherein said searching step comprises:

determining which of the pixels of the difference image exceed a predetermined amplitude threshold value;

identifying contiguous pixels determined by said determining step as discrete islands;

determining which of the identified islands include at least a minimum predetermined number of contiguous pixels; and

determining whether the islands which are determined to have at least said minimum number of contiguous pixels meet predetermined clusterization criteria.

5. The method as defined in claim 2, wherein said step of searching comprises:

determining which of the image pixels of the difference image have a value exceeding a predetermined threshold value;

measuring predetermined features of contiguous image pixels identified in said determining step; and

identifying the abnormal anatomic region based on the features measured in said measuring step.

6. The method as defined in claim 5, wherein said measuring step comprises:

measured predetermined geometric parameters of said contiguous image pixels identified in said thresholding step.

7. The method according to claim 6, wherein the predetermined geometric parameter measured in said measuring steps include circularity and size.

8. The method as defined in claim 2, wherein said step of searching comprises:

determining which of the image pixels of the difference image have a value exceeding a predetermined threshold value;

repeating said thresholding step at varied predetermined threshold values;

measuring, for each performance of said determining step, predetermined features of contiguous image pixels identified in each said measuring step;

determining variations in said predetermined features as a function of variation of said predetermined threshold values; and

identifying the abnormal region based on the variations in said predetermined features determined as a function of variation of said predetermined threshold values.

9. The method as defined in claim 8, wherein said measuring step comprises:

measuring predetermined geometric parameters of said contiguous image pixels.

10. The method as defined in claim 9, wherein the predetermined geometric parameters measured in said measuring step include circularity and size.

11. A system for automated detection and indication of an abnormal anatomic region from a digital image of an object, comprising:

means for generating a single digital image of said object;

means for filtering said single digital image to remove anatomic background derived from normal anatomic structure thereby to enhance in the resulting filtered image an abnormal pattern corresponding to an abnormal anatomic region;

means for searching the filtered digital image to identify said abnormal pattern in the filtered digital image; and

means for indicating the location of the abnormal anatomic region based on the location of a region of the filtered digital image in which the abnormal pattern is identified.

12. The system as defined in claim 11, wherein said filtering means comprises:

first means for producing a signal-to-noise ratio (SNR)-suppressed image;

second means for producing a SNR-enhanced image; and

third means for producing a difference image based on the difference between said SNR-enhanced and SNR-suppressed images.

13. The system as defined in claim 12, wherein said searching means comprises:

means for thresholding image pixels of the difference image to identify all image pixels having a value greater than a predetermined threshold value; and

means for measuring predetermined features of contiguous image pixels identified by said thresholding means; and

means for identifying the abnormal anatomic region based on the features measured by said measuring means.

14. The system as defined in claim 13, wherein said measuring means comprises:

means for measuring predetermined geometric parameters of said contiguous image pixels identified by said thresholding means.

15. The system as defined in claim 14, wherein said measuring means includes means for measuring circularity and size of said contiguous image pixels identified by said thresholding means.

16. The system as defined in claim 12, wherein said searching means comprises:

means for thresholding image pixels of the difference image to identify all image pixels having a value greater than a predetermined threshold value;

means for repeating said thresholding at varied predetermined threshold values;

means for measuring, for each performance of said thresholding, predetermined features of contiguous image pixels identified in each repeated performance of said measuring;

means for determining variations in said predetermined features as a function of variation of said predetermined threshold values; and

means for identifying the abnormal anatomic region based on the variations in said predetermined features determined by said determining means.

17. The system as defined in claim 16, wherein said measuring means comprises:

means for measuring predetermined geometric parameters of said contiguous image pixels identified by said thresholding means.

18. The system as defined in claim 17, wherein said measuring means includes means for measuring circularity and size of said contiguous image pixels identified by said thresholding means.

19. The system according to claim 16, wherein said searching means comprises:

means for determining which of the pixels of the difference image exceed a predetermined amplitude threshold value;

means for identifying contiguous pixels determined by said determining step as discrete islands;

means for determining which of the identified islands include at least a minimum predetermined number of contiguous pixels; and

means for determining whether the islands which are determined to have at least said minimum number of pixels meet predetermined clusterization criteria.

20. An apparatus for automated detection and indication of an abnormal region in an object, comprising:

image generating means for generating a digital X-ray image signal of the object;

storing means for storing said digital image signal generated by said generating means;

first processing means for filtering said stored digital X-ray image signal and producing a signal-to-noise ratio (SNR)-suppressed image;

second processing means for filtering said stored digital X-ray image signal and producing a signal-to-noise ratio (SNR)-enhanced image;

third processing means for producing a difference image between said SNR-suppressed image and SNR-enhanced image;

means for searching said difference image, and extracting and determining abnormal regions in the difference image,

locating means for locating said determined regions with abnormal features, and producing a location signal corresponding to the abnormal region in said digital X-ray image signal; and

indicating means for displaying the indication of the abnormal region responsive to said location signal in connection with said original X-ray image.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to radiographic systems, and more particularly to the processing of X-ray images using feature-extraction techniques.

2. Discussion of Background

Detection and diagnosis of abnormal anatomical regions in radiographs, such as cancerous lung nodules in chest radiographs and microcalcifications in women's breast radiographs, so called mammograms, are among the most important and difficult tasks performed by radiologists.

Recent studies have concluded that the prognosis for patients with lung cancer is improved by early radiographic detection. In one study on lung cancer detection, it was found that, in retrospect, 90% of subsequently diagnosed peripheral lung carcinomas were visible on earlier radiographs. The observer error which caused these lesions to be missed may be due to the camouflaging effect of the surrounding anatomic background on the nodule of interest, or to the subjective and varying decision criteria used by radiologists. Underreading of a radiograph may be due to a lack of clinical data, lack of experience, a premature discontinuation of the film reading because of a definite finding, focusing of attention on another abnormality by virtue of a specific clinical question, failure to review previous films, distractions, and "illusory visual experiences".

Similarly, early diagnosis and treatment of breast cancer, a leading cause of death in women, significantly improves the chances of survival.

X-ray mammography is the only diagnostic procedure with a proven capability for detecting early-stage, clinically occult breast cancers. Between 30 and 50% of breast carcinomas detected radiographically demonstrate microcalcifications on mammograms, and between 60 and 80% of breast carcinomas reveal maicrocalcifications upon microscopic examination. Therefore any increase in the detection of microcalcifications by mammography will lead to further improvements in its efficacy in the detection of early breast cancer. The American Cancer Society has recommended the use of mammography for screening of asymptomatic women over the age of 40 with annual examinations after the age 50. For this reason, mammography may eventually constitute one of the highest volume X-ray procedures routinely interpreted by radiologists.

A computer scheme that alerts the radiologist to the location of highly suspect lung nodules or breast microcalcifications should allow the number of false-negative diagnoses to be reduced. This could lead to earlier detection of primary lung and breast cancers and a better prognosis for the patient. As more digital radiographic imaging systems are developed, computer-aided searches become feasible. Successful detection schemes could eventually be hardware implemented for on-line screening of all chest radiographs and mammograms, prior to viewing by a physician. Thus, chest radiographs ordered for medical reasons other than suspected lung cancer would also undergo careful screening for nodules.

On radiographs, the presence of nodules is obscured by overlying ribs, bronchi, blood vessels, and other normal anatomic structures. Kundel et al. (in) Optimization of chest radiography, HHS Publication (FDA), 80-8124, Rockville, Md., 1980, introduced the concept of conspicuity to describe those properties of an abnormality and its surround which either contribute to or distract from its visibility. Kelsey et al. in the same publication investigated factors which affect the perception of simulated lung tumors and found that the visibility of lesions varied with their location on chest radiographs. Thus, a computerized search scheme would have to be capable of locating nodules that have varying degrees of conspicuity (i.e., nodules immersed in backgrounds of various anatomic complexity).

Research on computerized nodule-search methods has been limited. Of those attempted, geometry-based detection schemes (such as edge detection methods) were applied to the original image, or to a high-frequency enhanced image, without elimination of the structured background of the normal lung anatomy. Basically, none of the prior methods known to the inventors has been sufficiently successful to warrant large-scale clinical trials.

Several investigators have attempted to analyze mammographic abnormalities with digital computers. However, the known studies failed to achieve an accuracy acceptable for clinical practice. This failure can be attributed primarily to a large overlap in the features of benign and malignant lesions as they appear on mammograms.

The currently accepted standard of clinical care is such that biopsies are performed on 5 to 10 women for each cancer removed. Only with this high biopsy rate is there reasonable assurance that most mammographically detectable early carcinomas will be resected. Given the large amount of overlap between the characteristics of benign and malignant lesions on mammograms, computer-aided detection rather than characterization of abnormalities may eventually have greater impact in clinical care. Microcalcifications represent an ideal target for automated detection, because subtle microcalcifications are often the first and sometimes the only radiographic findings in early, curable, breast cancers, yet individual microcalcifications in a suspicious cluster (i.e., one requiring biopsy) have a fairly limited range of radiographic appearances.

The high spatial-frequency content and the small size of microcalcifications require that digital mammographic systems provide high spatial resolution and high contrast sensitivity. Digital mammographic systems that may satisfy these requirements are still under development. Digital radiographic systems with moderately high spatial resolution are made possible by fluorescent image plate/laser readout technology. Currently, digital mammograms with high resolution can be obtained by digitizing screen-film images with a drum scanner or other scanning system. The increasing practicability of digital mammography further underlines the potential ability of a computer-aided system for analysis of mammograms.

SUMMARY OF THE INVENTION

Accordingly, an object of this invention is to provide an automated method and system for detecting and displaying abnormal anatomic regions existing in a digital x-ray image.

Another object of this invention is to provide an automated method and system for providing reliable early diagnosis of abnormal anatomic regions.

A further object of this invention is to provide an automated method and system for selecting and displaying abnormal anatomic regions by eliminating structured anatomic background before applying feature extraction techniques.

Yet another object of this invention is to minimize patient exposure to x-ray radiation by providing an automated method and system for detecting and displaying abnormal anatomic regions based on the digital information provided in a single x-ray image of the anatomy under diagnosis.

These and other objects are achieved according to the invention by providing a new and improved automated method and system in which prior to feature extraction, a single projection x-ray image is processed to obtain signal-enhanced image data with a maximum signal-to-noise ratio (SNR) of a suspected abnormal region and is also processed to obtain signal-suppressed image data with a suppressed SNR. Then, according to the invention, difference image data are formed by subtraction of the signal-suppressed image data from the signal-enhanced image data to remove low-frequency structured background, which is basically the same in both the signal-suppressed and signal-enhances image data.

Further according to the invention, once the structured background is removed, feature extraction, based on for example thresholding, circularity and size is performed. Threshold levels are varied and the effect of the variation on circularity and size is used to detect abnormalities, such as lung nodules. Another feature extraction technique is to test for clusters of suspected abnormalities, such as mammographic microcalcifications.

BRIEF DESCRIPTIONS

A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

FIG. 1 is a schematic diagram illustrating the automated system for nodule detection according to the invention;

FIGS. 2a and 2b are histograms of the original image and the difference image, respectively, obtained according to the invention, with the pixel value of the nodule indicated by an arrow;

FIG. 3 is an illustration of the effective diameter and degree of circularity of an island;

FIG. 4 is a graph illustrating dependence of island size on threshold level for a nodule and a non-nodule;

FIG. 5 is a graph illustrating the dependence of island circularity on threshold level for a nodule and a non-nodule;

FIG. 6 is a graph illustrating variation of island circularity and size for various threshold levels for a nodule and a non-nodule;

FIG. 7 is a schematic block diagram illustrating in more detail the automated system for nodule detection shown in FIG. 1;

FIG. 8 is a schematic block diagram of the automated system for detection of microcalcifications in mammograms according to a second embodiment of the invention;

FIG. 9 is a graph providing a schematic illustration of a contrast-reversal filter;

FIGS. 10a and 10b are respectively a histogram of an unprocessed mammogram and a histogram of the difference image obtained from the matched filter (3.times.3)/contrast-reversal filter (n.sub.A =9, n.sub.B =3) combination;

FIG. 11 is a graph illustrating the dependence of detection accuracy on the kernel size of a contrast-reversal filter for a matched filter/contrast-reversal filter combination;

FIG. 12 is a graph illustrating the dependence of detection accuracy on the kernel size of a median filter for a matched filter/median filter combination;

FIG. 13 is a graph illustrating the dependence of detection accuracy on the kernel size of a matched filter for a matched filter/median filter combination;

FIG. 14 is a graph comparing the performance of three image-processing methods in combination with local thresholding;

FIG. 15 is a graph comparing the performance of three image-processing methods in combination with global thresholding;

FIGS. 16a and 16b are graphs respectively illustrating the dependence of true-positive detection rate of microcalcifications on local thresholding level and the dependence of false-positive detection rate of microcalcifications on local thresholding level;

FIG. 17 is a graph illustrating the dependence of detection accuracy on the contrast of microcalcifications; and

FIG. 18 is a schematic block diagram illustrating in more detail the automated system shown in FIG. 8.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views, and more particularly to FIG. 1 thereof, a schematic diagram of the nodule detection scheme is shown The technique begins with an attempt to increase the conspicuity of nodules by eliminating the "camouflaging" background of the normal lung anatomy This is accomplished by obtaining a single-projection digital chest image (step 10) and creating two images from a single-projection chest image: in one of the two, the signal-to-noise ratio (SNR) of the nodule is maximized, (step 20) and in the other, the nodule SNR is suppressed (step 30), while the background remains essentially the same. Then the difference is obtained (step 40). The difference between these two processed images consists of the nodule superimposed on a relatively uniform background in which the detection task is greatly facilitated. This difference image approach differs fundamentally from conventional subtraction techniques (e.g., temporal or dual-energy subtraction) in that the two sets of image data, i.e., SNR enhanced and SNR suppressed, are obtained from the same single-projection chest radiograph.

With the SNR-maximizing filter, the goal is to enhance those characteristics of a nodule which are different from the characteristics of the normal anatomic background. A spatial filter which is matched to the two-dimensional profile of a given nodule is expected to yield, upon processing, a maximal response at the location of that nodule (the position of maximal correlation). However, the use of multiple filters, each matched to one of an infinite number of conceivable nodule sizes and shapes, is quite impractical and probably impossible. Thus, it is necessary to find a few matched filters, or perhaps just one, which will enhance, to some degree, nodules of various sizes and shapes. Therefore filters were investigated, each of which was matched to the profile of some simulated nodule; i.e., the filter was proportional to the Fourier spectrum of a simulated nodule of a given size and contrast. This matched filter did not take into consideration the background noise in the radiographic image. Three matched filters which corresponded to simulated nodules having diameters of 6, 9, and 12 mm were investigated.

In particular, the effect of the three matched filters on a 512.times.512 section of a chest image containing two real and seven simulated nodules (ranging from 6 mm to 15 mm in diameter and from 35 to 65 in contrast in terms of digital pixel value) have been examined. It was found that the SNR-maximizing filter that was matched to a 6 mm diameter nodule was too sensitive to small, high-contrast portions of rib edges and thus yielded many false-positives. On the other hand, the filter that was matched to a 12 mm diameter nodule did not have a sufficient high-frequency content, and thus small nodules were missed in the detection process. Therefore, an SNR-maximizing filter that was matched to a 9 mm nodule was used.

The SNR-suppressing filter is intended to reduce the predominance of the nodule in the image while producing a background similar to that obtained with the SNR-maximizing filter. The "SNR-suppressed" image is produced from the original digitized chest image by means of a two-dimensional spatial-smoothing filter (linear or non-linear). Linear filters examined included uniform rectangle functions (which correspond to sinc functions in the spatial-frequency domain) and Gaussian functions having standard deviations of 6, 9, 12, 24, and 36 mm. Non-linear filters examined included median filters and modified median filters. The modified median filter differs from the conventional median filter in that the pixels which are used in determining the median value about some pixel location are not immediately adjacent to each other, but rather lie along a circumference at a given radial distance from the pixel location in question.

With the SNR-suppressing filters, it was found that uniform rectangle and Gaussian functions with the same rms size yielded similar results. The modified median filter and the conventional median filter appeared promising in that thresholding of the difference image yielded many islands which corresponded to nodules. However, the resulting islands in the difference image had jagged edges which gave misleadingly low circularity measurements. The combination of the 9 mm matched filter with the 12 mm by 12 mm uniform rectangle function yielded the highest number of nodule islands and the lowest number of non-nodule islands during thresholding of the difference image.

After the two filtered images are obtained from the original image, a difference image is computed. In one evaluation performed 512.times.512 portion of an original chest image was used. A 10 mm simulated nodule was positioned in the middle of the lung field and partially overlapping a rib. The difference image was obtained using a matched filter corresponding to a 9 mm nodule with a contrast of 65 in terms of digital pixel value for maximizing the SNR of the nodule. Both filtering operations were performed in the frequency domain with the aid of a fast-Fourier-transform (FFT) algorithm. The conspicuity of the nodule in the difference image was thereby increased and the complexity of the normal lung background was reduced, although the overall structure of the lung was still visible.

Histograms of the original image and of the difference image above described are shown in FIGS. 2(a) and 2(b), respectively. Since only one quarter of the chest is analyzed, the range of the histogram, i.e., the dynamic range, of the original chest image is only approximately 500 pixel values. The pixel value of the nodule is indicated by an arrow on the histograms. In the original image, it is apparent that the pixel value of the nodule is comparable to those of other lung structures. If the location of the nodule is varied relative to the other lung structures, then the pixel value of the nodule may vary within the dynamic range shown. However, the pixel value of the nodule in the difference image is always located at the high end of the histogram and is isolated from most of he other structures. Also, the histogram of the difference image is very narrow. These histograms demonstrate that, with the difference image approach, one can successfully eliminate the effect of the unwanted anatomic background.

Next described are feature extraction techniques including circularity, size and growth tests (step 50) employed to detect a lung nodule (step 60), schematically illustrated in FIG. 1.

Once the difference image is obtained from the "SNR-maximized" and "SNR-suppressed" images, feature-extraction techniques are used to isolate possible nodules while disregarding other structures. Because of the difference in the spectral contents of the SNR-maximized and SNR-suppressed images, the backgrounds resulting after filtering are not identical, and thus the "structured noise" is not completely eliminated in the difference image. However, the conspicuity of the nodule is increased, and therefore extraction of the nodule from the simplified background becomes easier than that from the original, complex anatomic background. The nodule is extracted by thresholding the difference image and performing tests for circularity and size and evaluating their change with variation of threshold level; the latter being referred to as a "growth" test.

Thresholding on the difference image is performed at various pixel values (threshold levels). The pixel values above a given threshold level correspond to a specific upper percentage of the area under the histogram. For example, the threshold level corresponding to the upper 4% of the histogram area in FIG. 2(b) is 525. It should be noted that, as the percentage increases, the threshold level decreases.

Pixel values below the threshold level are set to a constant background value, giving rise to an image of "islands." As the pixel threshold level is lowered so that a greater number of the pixel population as a % of the histogram exceeds the threshold, i.e., as the histogram percentage is increased from 4% to 8%, the islands grow and their shapes vary. The invention utilizes the way in which the various islands grow with decreasing threshold levels as a means of characterizing and distinguishing between those islands that result from nodules and those that arise from non-nodules (i.e., normal lung structures). At each threshold level, the islands are loaded automatically with simple computer searching techniques and then submitted for shape and size testing.

FIG. 3 schematically illustrates the measures for the size and circularity of a given island. The area of the island corresponds to the number of connected pixels at and above the threshold level. The effective diameter is defined by the diameter of a circle having the same area as that of the island. The degree of circularity is defined as the ratio of the area of the island that lies within the equivalent circle, which is centered about the centroid of the island, to the area of the island.

The growth of each island is monitored at discrete intervals of the threshold level. FIG. 4 demonstrates the dependence of island size on the threshold level for a nodule and a non-nodule. The threshold level is varied in increments of 1% of the area of the difference image histogram. The size of the island is expressed in terms of the effective diameter in mm. It should be noted that, as the threshold level decreases, i.e., as more pixels are included in difference image displayed, the nodule island gradually grows in size as compared to the non-nodule island. The sudden increase in the effective diameter of the non-nodule island, which is caused by a merging of the island with another non-nodule island, is typical of non-nodules in the peripheral region of the chest. A typical example of the non-nodule is a rib edge.

FIG. 5 illustrates the dependence of island circularity on the threshold level for a nodule and a non-nodule. The circularity of the nodule island remains above approximately 0.85 as the threshold level is changed over a wide range. However, the circularity of the non-nodule island decreases. The sudden decrease in circularity for the non-nodule island indicates the merging of the island into another island.

The variation of island circularity and size for various threshold levels is shown in FIG. 6. These growth characteristics of islands, as demonstrated in FIGS. 4-6, are used in accordance with the invention in order to distinguish between nodules and non-nodules. An island is rejected if the size and circularity do not remain at predetermined levels for a certain number of consecutive threshold levels (in increments of 1% of the histogram). The inventors have used a two-choice criterion that an island had to satisfy in order to be considered a nodule. The island must either (1) have an effective diameter between 3 and 18 mm and a circularity of at least 0.85 for 10 consecutive threshold levels or (2) have an effective diameter between 9 and 18 mm and a circularity of at least 0.75 for 4 consecutive threshold levels. The two-choice criterion was used in order to detect both small and large nodules; the first criterion being for small and medium-size nodules and the second criterion for large nodules. Usually, non-nodule islands are small initially, grow relatively quickly because they merge with other non-nodule islands, and have a low degree of circularity when their effective diameters become greater than 9 mm. However, in order to detect very small and very large nodules, a multiple-test criterion has been used.

It should be noted that once the original digital chest image is input to the computer, the nodule detection process is totally automated. After the distinction between nodules and non-nodules has been made automatically, the detection results can be presented to a radiologist for the final decision.

FIG