 |
Description  |
|
|
The present invention relates to a method for evaluation of image
collection system performance in automated machine vision instruments.
More specifically, the method for evaluation characterizes linearity,
frequency response, signal to noise and pixel correction limits during
operation for automated cytology instruments.
BACKGROUND OF THE INVENTION
Automated analysis of biological specimens requires a high degree of
performance and consistency from the optical and electronic imaging
systems. Image processing analyses of biological specimens use various
segmentation algorithms and morphological operations that depend on
consistent imagery for accurate and repeatable results. Performance of
image collection systems comprised of imaging optics and detector
electronics and a digital processor can be primarily characterized by
three measurements. These measurements include modulation transfer
function or frequency response, linearity and the signal to noise ratio.
Image processing systems used for biological analysis may be used to make
or help make diagnoses regarding the state of infections or disease of
human subjects. Therefore, these diagnoses must be of the highest degree
of integrity possible. Accordingly, the imaging systems used to capture
images for processing must be checked frequently during operation to
ensure highly consistent and accurate performance. The present invention
provides techniques and apparatus used to characterize such systems during
operation.
SUMMARY OF THE INVENTION
An automated apparatus for checking automated cytological system image
collection integrity in an automated cytological system is provided
including an imaging apparatus controlled by a digital processor. The
automated apparatus includes linearity checking apparatus, coupled to the
imaging apparatus and the digital processor. The modulation transfer
function (MTF) of the image collection system is checked, where the MTF
may be determined beyond one half the sampling frequency of the image
collection system. A system signal to noise ratio is also checked for
specific regions of the frequency spectrum.
This invention comprises a suite of tests to characterize the performance
of the imaging system during operation. The test methods of the invention
discussed herein are specifically directed by example to an automated
machine visioning system having a pulsed arc lamp, biological microscope
objectives and a CCD imaging device. However, the concepts of the
invention may be employed to check other illumination sources, image
collection devices and image capture electronics such as LASER sources,
reflective optics, tube cameras, TDI sensors, PIN diodes and
photomultiplier tubes.
It is one object of the invention to provide an improved means to
characterize modulated transfer function of an imaging system.
It is another object of the invention to provide an improved means to
characterize linearity of imaging system.
It is yet a further object of the invention to provide an improved means to
characterize signal to noise of an imaging system.
It is still a further object of the invention to provide an improved means
to characterize and evaluate the acceptability of an imaging system for
automated cytological analysis.
It is yet another object of the invention to provide an improved means to
characterize and evaluate the acceptability of an imaging system for any
automated vision interpretation system.
It is yet another object of the invention to provide a runtime means to
characterize and evaluate the acceptability of an imaging system for
automated cytological analysis.
It is still a further object of the invention to provide an improved
runtime means to characterize and evaluate the acceptability of an imaging
system for any automated vision interpretation system.
It is yet another object of the invention to provide an improved means to
characterize modulated transfer function of an imaging system with an
undersampled detector.
It is still a further object of the invention to provide an improved means
to characterize modulated transfer function of an imaging system
simultaneously at numerous points in the field of view.
It is yet another object of the invention to provide an improved means to
characterize modulated transfer function of an imaging system with a bar
target that is easy to fabricate with low frequency patterns.
It is still a further object of this invention to provide an improved means
to characterize modulated transfer function of an imaging system with a
square wave bar pattern.
It is yet another object of this invention to provide an improved means to
characterize the signal to noise ratio of an imaging system for specific
regions of the frequency spectrum.
Other objects, features and advantages of the present invention will become
apparent to those skilled in the art through the description of the
preferred embodiment, claims and drawings herein wherein like numerals
refer to like elements.
BRIEF DESCRIPTION OF THE DRAWINGS
To illustrate this invention, a preferred embodiment will be described
herein with reference to the accompanying drawings.
FIG. 1A and FIG. 1B show an automated cytology system and the placement of
a calibration and test target into an optical path of an automated
microscope as employed by the method and apparatus of the invention.
FIG. 2 schematically shows an automated microscope of the type used in
automated cytological system having a calibration plate mounted on a
movable stage.
FIG. 3 shows one example of a calibration and test target or plate as used
in one aspect of the invention.
FIG. 4 shows an example of a fiducial marking.
FIG. 5 shows a linearity profile for an automated cytology system.
FIG. 6 schematically illustrates one example of a system apparatus used for
testing linearity.
FIG. 7 is a linearity plot of an illumination sensor reading on the
abscissa axis and the image collection system detector reading on the
ordinate axis.
FIG. 8 shows an example of a modulation transfer function as employed in
one embodiment of the invention.
FIGS. 9A, 9B, 9C and 9D show bar patterns of progressively increasing
spatial frequency and an intensity profile of those bar patterns in an
image plane.
FIG. 10A shows a square wave plot for a theoretically perfect square wave.
FIG. 10B shows the Fourier transform of a perfect square wave.
FIG. 11 shows one example of an FFT foldback for MTF determination beyond
the detector sampling frequency.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
In a presently preferred embodiment of the invention, the camera system
disclosed herein is used in a system for analyzing cervical pap smears,
such as that shown and disclosed in U.S. patent application Ser. No.
08/571,686, Dec. 13, 1995 which is a continuation of abandoned U.S. patent
application Ser. No. 07/838,064, entitled "Method For Identifying Normal
Biomedical Specimens", by Alan C. Nelson, et al., filed Feb. 18, 1992;
issued U.S. patent application Ser. No. 08/179,812 filed Jan. 10, 1994 now
U.S. Pat. No. 5,528,703 which is a continuation in part of abandoned U.S.
patent application Ser. No. 07/838,395, entitled "Method For Identifying
Objects Using Data Processing Techniques", by S. James Lee, et al., filed
Feb. 18, 1992; U.S. patent application Ser. No. 07/838,070, now U.S. Pat.
No. 5,315,700, entitled "Method And Apparatus For Rapidly Processing Data
Sequences", by Richard S. Johnston, et al., filed Feb. 18, 1992; U.S.
patent application Ser. No. 07/838,065, now U.S. Pat. No. 5,361,140
entitled "Method and Apparatus for Dynamic Correction of Microscopic Image
Signals" by Jon W. Hayenga, et al. filed Feb. 18, 1992; and U.S. patent
application Ser. No. 08/302,355, filed Sep. 7, 1994 entitled "Method and
Apparatus for Rapid Capture of Focused Microscopic Images" to Hayenga, et
al., which is a continuation-in-part of abandoned application Ser. No.
07/838,063 filed on Feb. 18, 1992 the disclosures of which are
incorporated herein, in their entirety, by the foregoing references
thereto.
The present invention is also related to biological and cytological systems
as described in the following patent applications which are assigned to
the same assignee as the present invention, filed on even date herewith,
and which are all hereby incorporated by reference including pending U.S.
patent application Ser. No. 303,179, filed Sep. 8, 1994, entitled
"Cytological System Illumination Integrity Checking Apparatus and Method,"
to Ortyn et al.; pending U.S. patent application Ser. No. 309,130, filed
Sep. 20, 1994, entitled "Cytological System Autofocus Integrity Checking
Apparatus," to Ortyn et al.; U.S. Pat. No. 5,499,097 entitled "Automated
Cytology System Position Integirty Checking Method and Apparatus" to Ortyn
et al.; and pending U.S. patent application Ser. No. 08/309,249, filed
Sep. 20, 1994, entitled "Biological Specimen Analysis System Processing
Integrity Checking Apparatus" to Ortyn et al.
Now refer to FIGS. 1A and 1B which show a schematic diagram of one
embodiment of the apparatus of the invention for checking illumination
integrity for an automated microscope. While the method and apparatus of
the invention will be discussed in terms of an example herein related to
an automated cytology apparatus, it will be understood that the invention
is not so limited. The features and principles of the invention may be
applied to check urine analysis processes, semiconductor process defects,
liquid crystal devices and other types of processing systems employing,
for example, continuous arc lamps, filament lamps, laser sources, tube
cameras, PIN diodes and photomultiplier tubes.
The apparatus of the invention comprises an imaging system 502, a motion
control system 504, an image processing system 536, a central processing
system 540, and a workstation 542. The imaging system 502 is comprised of
an illuminator 508, imaging optics 510, a CCD camera 512, an illumination
sensor 514 and an image capture and focus system 516. The image capture
and focus system 516 provides video timing data to the CCD cameras 512,
the CCD cameras 512 provide images comprising scan lines to the image
capture and focus system 516. An illumination sensor intensity is provided
to the image capture and focus system 516 where an illumination sensor 514
receives the sample of the image from the optics 510. In one embodiment of
the invention, the optics may further comprise an automated microscope.
The illuminator 508 provides illumination of a slide. The image capture
and focus system 516 provides data to a VME bus 538. The VME bus 538
distributes the data to an image processing system 536. The image
processing system 536 is comprised of field-of-view processors 568. The
images are sent along the image bus 564 from the image capture and focus
system 516. A central processor 540 controls the operation of the
invention through the VME bus 538. In one embodiment the central processor
562 comprises a Motorola 68030 CPU. The motion controller 504 is comprised
of a tray handler 518, a microscope stage controller 520, a microscope
turret controller 522, and a calibration slide 524. The motor drivers 526
position the slide under the optics. A bar code reader 528 reads a barcode
located on the slide 524. A touch sensor 530 determines whether a slide is
under the microscope objectives, and a door interlock 532 prevents
operation in case the doors are open. Motion controller 534 controls the
motor drivers 526 in response to the central processor 540. An Ethernet
(.TM.) communication system 560 communicates to a workstation 542 to
provide control of the system. A hard disk 544 is controlled by
workstation processor 550. In one embodiment, workstation 542 may comprise
a Sun Sparc Classic (.TM.) workstation. A tape drive 546 is connected to
the workstation processor 550 as well as a modem 548, a monitor 552, a
keyboard 554, and a mouse pointing device 556. A printer 558 is connected
to the Ethernet (.TM.) network system 560.
During image collection integrity checking, the central computer 540,
running a real time operating system, controls the automated microscope
and the processor to acquire and digitize images from the microscope. The
flatness of the slide may be checked, for example, by contacting the four
corners of the slide using a computer controlled touch sensor. The
computer 540 also controls the microscope stage to position the specimen
under the microscope objective, and from one to 15 field of view (FOV)
processors 568 which receive images under control of the computer 540.
Referring now to FIG. 2, there shown is placement of a calibration and test
target 1 into an optical path of an automated microscope 3 having a turret
22. The calibration and test target may be mounted on a stage 521
substantially in a horizontal X,Y plane which intersects the optical path.
The stage 521 is movable in the X,Y plane as well as along a Z axis which
is perpendicular to the X,Y plane and which is parallel to the optical
axis of the automated microscope. The turret 22 may comprise multiple
objective lenses as is well known in the art. The microscope turret
control 522 provides signals in a well known manner for positioning a
selected objective lens into position for viewing a slide, for example.
It is to be understood that the various processes described hereinabove
with respect to checking system linearity, system frequency response and
system signal to noise ratio may be implemented in software suitable for
running on a digital processor. The software may be embedded, for example,
in the central processor 540.
Referring now to FIG. 3 one example of a calibration and test target is
shown. Several of the processes employed by the present invention require
a calibration and target plate. In the case of a transmission microscope,
the calibration and test target 1 may comprise a piece of glass
approximately 1.45 mm thick. The calibration and test target
advantageously comprises specified clear areas 34 and/or image primitives,
including periodic structure, such as horizontal and vertical bar targets
36. Other types of image primitives, such as fiducial markings, may also
be used. FIG. 4 shows an example of a fiducial marking. Such calibration
and test target plates may be used for most transmission microscopes to
simulate the optical path difference effects introduced by the substrate,
coverslip and specimen media. In some embodiments of the invention, the
calibration and test target may be advantageously mounted onto a
conventional cantilever arm for ease of placement onto and removal from
the stage.
System Linearity Test
Referring now to FIG. 5, a linearity profile for an automated cytology
system is shown. The profile is plotted with respect to a horizontal axis
representing energy in generic units and a vertical axis representing
voltage output in millivolts. Linearity of response is the change in
voltage out of a system with respect to the light level input. Note the
characteristic saturation curve 50. In the region below 600 mV,
corresponding to an input energy of about 70 units, the profile indicates
that the system responds fairly linearly to a change in input energy. For
voltages above 600 mV, the system appears to generate less of a voltage
output for each additional energy input. After about 110 energy units of
input, the system is saturated and no longer produces a change in voltage
for a corresponding change in energy input.
Many components influence the linearity of an image capture system, such as
an automated cytology analysis system, including the detector and
subsequent electronics, optical components, stray light baffling and other
elements. Most systems are designed for operation in the linear region.
However, if a system is allowed to operate slightly into the nonlinear
region, some dynamic range can be gained. This is often the case in many
designs. In these types of systems it is critical that the system operate
with the same characteristic linearity curve over time and temperature.
Therefore, it is highly desirable to test system linearity.
One example of a system linearity test apparatus is shown in FIG. 6. The
linearity testing apparatus 60 comprises an illumination source 508, a
camera 512, a specimen plane 62, a first lens 63, a second lens 64, a beam
splitter 74, a photodetector 76, a neutral density wedge 66, a third lens
72, and a fourth lens 70. The first lens 63 may comprise an objective of
an automated microscope, where the objective is selected to have a
predetermined magnification and is positioned over the nominal clear area
34 of the calibration and target plate 1. It is advantageous to run the
linearity test using objectives having high power and low power
magnifications.
In one example using an automated microscope having two objectives,
20.times. and 4.times. magnifications are tested. The neutral density
wedge 66 is positioned to yield a desired illumination level. A single
image is acquired by the imaging apparatus and a mean pixel intensity is
computed to characterize the camera response at the selected illumination
level. The neutral density wedge 66 is positioned at another illumination
level. A single image is again acquired and a second mean pixel intensity
is computed to characterize the camera response at the second selected
illumination level. This process is repeated for six different regions, in
one example, to characterize the linearity profile of the system.
The embodiment of FIG. 6 employs a double beam system in which the
illumination level is set by rotating the circular linearly variable
neutral density wedge 66. Feedback for setting a selected illumination
level is provided by the sensor 76 that receives light split off the main
optical path 110 before it illuminates a specimen at specimen plane 62.
FIG. 7 is a linearity plot which is essentially a plot of the illumination
sensor reading on the abscissa axis and the camera reading on the ordinate
axis. Table 2 shows the data plotted in FIG. 7 in tabular form. The sensor
is previously calibrated to ensure its linearity. Therefore, the linearity
plot is fundamentally a linearity plot of the system. The limits of the
linearity test are dynamic and may be determined by the formula below.
Limits on the camera response may be calculated as a function of the
illumination or sensor response level. The sensor response level is the
sensor response measured at a certain illumination as a percentage of the
sensor response measured at 100% illumination.
The formula for calculating the camera response limit is given as
limit=(sensor response level)*slope+intercept
Using this formula, the camera response limits may be determined with the
information provided in Table 1.
TABLE 1
______________________________________
Linearity Limits
Sensor Response
Minimum Maximum
Level Slope Intercept Slope Intercept
______________________________________
0-10% 2.60 4.00 3.00 6.00
10-20% 2.50 5.00 2.70 9.00
20-35% 2.47 5.67 2.47 13.67
35-50% 2.13 17.33 2.40 16.00
50-75% 2.08 20.00 2.24 24.00
75-105% 1.88 35.00 2.04 39.00
______________________________________
TABLE 2
______________________________________
Linearity Test Data, AP300
20X Camera Linearity
Illumination Level
Camera Resp.
Minimum Maximum
Illuminance Response Low Limit Upper Limit
______________________________________
0.00% 5.07 4 6
16.51% 52 46.27 53.58
24.21% 71.34 65.48 73.48
39.62% 108,89 101.73 111.1
56.14% 143.62 136.77 149.75
77.05% 184.92 179.86 196.19
99.07% 230.3 221.25 241.1
100.17% 232.31 223.32 243.35
______________________________________
Modulation Transfer Function Test
A Modulation Transfer Function (MTF) test characterizes the frequency
response of the system. Modulation transfer functions are well known and
typically comprise a curve of contrast in the image plane versus spatial
line frequency of a sinusoidal input in the object plane. See, for
example, Smith, Modern Optical Engineering, pp. 308-323, McGraw-Hill Book
Company, 1966. FIG. 8 shows an example of an MTF. As the line frequency of
the object increases, that is, as objects get smaller and closer together,
the ability of an optical system to provide contrast in the image
decreases. FIGS. 9A, 9B, 9C and 9D show bar patterns of progressively
increasing spatial frequency. Also shown are intensity profiles 80, 82, 84
and 86 of those bar patterns in the image plane. As line frequency
increases the contrast in the image plane decreases. Beyond a
predetermined cutoff frequency, the contrast is zero (i.e. there is no
modulation in the image).
Modulation is defined as follows:
Modulation=(max-min)/(max+min)
where: max and min are the maximum and minimum intensity values in the
image plane.
There are typically two methods for generating an MTF curve. The first
method involves conducting a series of contrast measurements over a set of
discrete bar patterns. The contrast is measured at each bar pattern and a
pseudo MTF curve is gradually generated. The first method does not
actually generate an MTF curve because a true MTF test has a sinusoidal
input. Sinusoidal targets are very difficult to generate and usually
cannot be generated at even modestly high frequencies. Therefore, a bar
pattern, which generates a square wave, is usually used. Although this is
not true MTF, it is common practice. Another problem with the first method
is that bar patterns, even square wave patterns, are difficult to generate
at very high frequencies such as those above 250 lp/mm (i.e., 2 micron
line widths). The problem is that many optical systems have a cutoff
frequency around 2000 lp/mm. Therefore, this method can only test the
pseudo (square wave) MTF in the lower part of the MTF curve up to 250 line
pairs per millimeter.
Another method u | | |
