 |
Description  |
|
|
FIELD OF THE INVENTION
This Invention relates to Associative Memory Systems and in particular to
Associative Memory Systems using spatial light modulators (SLM) such as
liquid crystal light valves (LCLV).
BACKGROUND OF THE INVENTION
The speed and computational accuracy of modern digital computers are
well-known. However, all digital computers solve problems in a sequential
fashion through the use of numerical computation. While the processing
unit contained in a simple pocket calculator can easily out-perform the
human brain in number crunching tasks, digital computers are able to
accomplish this sophisticated numerical analysis only on a step-by-step
basis. Digital computers exhibit their best abilities when presented with
a serially programmable algorithm. Digital computers are not capable of
sophisticated parallel processing, such as that required when a human
undertakes the task of pattern recognition. Problems such as comparing the
fingerprint found at the scene of a crime with a data base full of
fingerprints is the sort of practical and necessary problem that arises
and yet is not easily solved by a digital computer. To the extent that
digital computers have been programmed to match the fingerprint found at
the scene of the crime with an existing fingerprint in the files, lengthy
serial searches of memory are required to digitally achieve accurate
pattern recognition.
A matrix algebra based on an associative memory model was described by J.
J. Hopfield in his paper "Neural Networks and Physical Systems with
Emergent Collective Computational Abilities," proceedings of the National
Academy of Science U.S.A., 1982, Vol. 79, pp. 2554-2558. The Hopfield
model utilizes feedback and nonlinear thresholding to force the output
pattern to be the stored pattern which most closely matches an input
pattern presented to the associative memory system. A digital emulation of
this model requires large storage and computational effort for the
manipulation of an association matrix used in the model. For example, in
order to store two-dimensional image patterns consisting of N.times.N
pixels, the model requires a matrix with N.sup.4 entries be used.
A natural implementation of an associative memory model would be one which
uses optical technology. Optical associative memory systems store
information as patterns so that, upon the introduction of a stored pattern
to the system, the system is able to recall the stored pattern and perform
a match. These optical systems achieve massive parallel processing. The
ability of an optical associative memory to perform such a function has
wide application in the fields of pattern recognition and image
understanding. Used in conjunction with a laser beam, specially treated
photosensitive film or plates act as holograms. A hologram is a frozen
"picture" of an object wherein the image of the object is recorded on the
film plate as an interference pattern between a reference beam of plane
waves (which is directed only at the photographic film) and an object wave
front (which is created by reflection from the object, where the object
wave front is made by the same coherent source that produced the reference
beam). Holograms are characterized as having extremely good spatial
coherence. The light used to produce the hologram, normally laser beam,
exhibits a high degree of temporal coherence. In order to view the
recorded holographic image, one redirects coherent light along the same
path as the reference beam which originally recorded the hologram. A
viewer views the hologram along the same line of sight that connected the
object and the hologram during its recording. Directing a new reference
beam on the hologram causes an image to appear which, in a lensless
environment, gives rise to a three-dimensional image. The lifelike
dimensionality of a lensless image produced in a hologram is due to the
fact that, unlike a photograph, a hologram stores not only amplitude
changes but also records phase changes as interference fringes resulting
from the interaction between spatially coherent object and reference
beams.
Holograms are characterized by very precise and lifelike imaging. In
addition, a hologram, when viewed from different angles, produces
different views of the recorded image. The hologram is programmable for
use in storing a plurality of images, by varying the angle of the
reference beam used to record the image. The information stored within a
hologram is recorded throughout the holographic medium; even a portion of
the hologram retains the complete record. It therefore can be seen that
holograms are quite useful in parallel processing systems. Furthermore,
holograms are inherently useful for optical pattern recognition
mechanisms.
Among the types of holograms known in the art are the volume, Fresnel, and
Fraunhofer holograms. The volume holograms have a thickness and can be
used to record either amplitude or phase modulated images without the
generation of both primary and conjugate waves that is inherent with thin
holograms. Fraunhofer holograms are characterized as holograms that record
distant objects. Larger and closer positioned objects produce Fresnel
holograms.
The Fourier transform hologram uses a lens and is adaptable for memory
storage purposes. As is well known in the numerical analysis arts, the
Fourier transform is a mathematical tool wherein any function may be
broken up into a sum of sinusoidal superimposed patterns. This manner of
dividing a function into its Fourier components is known as defining the
Fourier transform of a function. In Fourier transform holography, an
object wavefront holographically captured, after it has undergone a
Fourier transformation. To do this, a photographic holographic plate is
placed at the back focal plane of the lens. A flat object, such as a
transparency, is placed at the same distance in front of the lens as the
photographic plate is behind it. The object's wave front, when it reaches
the plate, has been Fourier-transformed by the lens. The holographic
pattern produced as an image is quite unlike the original object. If the
object is illuminated only by coherent light, such as a laser beam, and if
a reference beam is provided at an angle to the plate, the Fourier
transform can be recorded as a hologram.
Pattern recognition has used Fourier transform holograms in another fashion
to perform the operation of convolution. The best way to understand
convolution is to look at an example. If one were to convolve a first
transparency having three dots with a second transparency having one
triangle, using a holographic Fourier transform, one obtains three
triangles, one at each position of the dots. A related operation
mathematically similar to convolution is correlation. The result of
correlating two identical objects is a sharp peak at a position
corresponding to the shift value which superimposes the two objects. The
peak is greatly reduced if the two objects are not identical, making
correlation useful in pattern recognition.
To correlate two transparencies (also referred to as objects) one simply
positions a first object one focal length in front of a lens and a Fourier
transform hologram of a second object one focal length back of this lens.
A second lens is positioned in back of the Fourier transform hologram of
the second object. The correlation of the first and second objects appears
one focal length behind the second lens.
An example of optical pattern recognition using correlation would be where
in a printed page of text, one could recognize a particular word or letter
at some position on a page. Wherever the particular word appears in the
text, a bright spot of light highlights the word in the correlated image.
Wherever the word occurs on the page, there will be a corresponding bright
spot of light in the correlated image called a correlation peak. Thus, the
nature of holograms and lenses combined in an optical system using a
coherent light source allows the operation of pattern recognition to
occur. Such a device has been characterized as an optical neural computer.
The term "neural" is derived from the fact that the parallel processing of
a hologram to provide an associative memory is similar to that of a human
brain's neural system in that the stored information is not localized.
Heretofore, one such optical associative memory has been proposed by
Abu-Mostafa and Psaltis in Scientific American, vol. 256, no. 3 in an
article entitled "Optical Neural Computers," at page 88 (March, 1987). In
that article an optical thresholding device and a pinhole array were used
as part of a double hologram associative memory system.
The applicants have previously disclosed (as co-inventors) an associative
memory system in U.S. Pat. No. 4,739,496, entitled "Associative
Holographic Memory Apparatus Employing Phase Conjugate Mirrors", which
issued on Apr. 19, 1988 on patent application Ser. No. 06/786,884, filed
Oct. 11, 1985. Also, the applicants are co-inventors in U.S. Pat. No.
4,750,153, "ASSOCIATIVE HOLOGRAPHIC MEMORY APPARATUS EMPLOYING PHASE
CONJUGATE MIRRORS IN A TWO-IMAGE WAVE MIXING CONTRA-DIRECTIONAL COHERENT
IMAGE AMPLIFIER", issued on June 7, 1988 on patent application Ser. No.
06/821,237, filed Jan. 22, 1986. (The disclosures contained in both
applications are hereby incorporated by reference.) Hughes Aircraft
company, the assignee of this application, is also the assignee of these
two pending applications. These systems also employ primarily all-optical
elements.
As indicated above, optical elements, such as the hologram, make excellent
associative memory storage devices. When a distorted input image is
presented to a system which includes at least one hologram (containing a
clear representation of that image), the system processes light through
its components in such a manner as to correlate and match the distorted
input image with one of the images stored on the hologram. The sharper the
correlation peaks, the better the match. All optical systems are excellent
parallel processors but generally may not be shift-invariant and
furthermore, they may exhibit optical and gain losses in the system as the
image is processed. In order to achieve a good match, an optical
associative memory must have good thresholding and gain so that the
correlation peak which reconstructs the reference beam (when the image is
to be reconstructed) is sharp and bright. Losses of light intensity in the
system are inevitable as the light is processed through an optical system
as disclosed in the above-incorporated applications or as that disclosed
in the Abu-Mostafa article, supra. Additionally, reconstruction and phase
conjugation of the reference beam in the all-optical systems described in
U.S. Pat. No. 4,739,496 and U.S. Pat. No. 750,153 is achieved inherently
by use of phase conjugate mirrors, (PCMs) using for example BaTiO.sub.3
material. In such systems, thresholding is determined by physical
processes in the PCMs and is neither easily alterable nor readily
adjustable. Also, such optical systems heretofore have required at least a
second for the PCM to respond. BaTiO.sub.3 --based optical techniques are
relatively slow, in a computer sense. Phase conjugate mirrors of an
all-optical component system may be used to fully reconstruct and return
an image to its point of origin to achieve pattern recognition.
Nonlinearities in the phase conjugate mirrors are used to select those
stored objects which exceed a threshold, based on the overlap of computed
integrals of the object input with the stored objects. Although,
experimentally, store-and-recall of two objects with shift invariance, was
achieved, the gains achieved by phase conjugate mirrors were not enough to
overcome hologram losses. Additionally, the non-linearities of the phase
conjugate mirrors were difficult to control.
It is therefore an object of this invention to provide a system which makes
use of the pattern recognition properties of a hologram but in such a
manner that optical losses are kept to a minimum, thresholding with gain
achieved, and sharp correlation of images at the hologram accomplished,
with shift invariance. U.S. Pat. Nos. 4,546,248 and 4,556,986, both issued
to Glenn D. Craig and assigned to the United States (NASA), disclose
electro-optical systems used to process images with incoherent light
sources. The systems represent attempts to vary spatially the optical gain
of signals without thresholding or enhancement of optical images. Such
references show the state of the electro-optical art, but do not in
themselves advance the achievement of the objects of this invention to
provide an associative memory system.
Liquid crystal light valves (LCLVs) suitable for use in the present
invention include, for example, those shown in the following U.S. patents
all assigned to Hughes Aircraft Company, the assignee of the present
invention:
U.S. Pat. No. 3,824,002, "Alternating Current Liquid Crystal Light Valve",
issued to T. D. Beard, on July 16, 1974; U.S. Pat. No. 4,019,807,
"Reflective Liquid Crystal Light Valve with Hybrid Field Effect Mode",
issued to Boswell et al. on Apr. 26, 1977;
U.S. Pat. No. 4,018,509, for "Optical Data Processing system with
Reflective Liquid Crystal Light Valve", issued to Boswell et al. on Apr.
19, 1977;
U.S. Pat. No. 4,378,955, for "Method and Apparatus for a Multimode Image
Display with a Liquid Crystal Light Valve", issued to Bleha et al. on Apr.
5, 1983;
U.S. Pat. No. 4,239,348, "High Resolution AC Silicon MOS--Light Valve
Substrate" issued to J. Grinberg et al. on Dec. 16, 1980;
U.S. Pat. No. 4,443,064 "High Resolution AC Silicon MOS--Light Valve
Substrate" issued to J. Grinberg et al. on Apr. 17, 1984; and,
U.S. Pat. No. 4,127,322, "High Brightnesss Full Color Image Light Valve
Projection System", issued to Jacobson et al., on Nov. 28, 1978.
SUMMARY OF THE INVENTION
An associative memory system capable of recalling a complete and
undistorted stored image when the memory system is provided with an input
image which is distorted (or is a part of the complete stored image or
both) is disclosed. Such an input image is hereinbelow referred to as a
"distorted image." The associative memory system of this invention
includes a holographic means for recording and reconstructing a first
object transform of a first object. Image transforming apparatus, such as
a lens, provides a second transform of a second object or set of second
objects to the holographic means. The holographic means forms the product
of the transforms of the first object, second object, and the first
reference beam used in recording the hologram. This composite product,
known as a distorted second reference beam, is transformed by a
correlation lens into a correlation function which is the transform of the
product. The correlation function is threshold limited and conjugated by
spatial light modulator means which feeds back the threshold limited
conjugate correlation back to the holographic means for reading out the
first object transform stored therein. An image then appears in the output
plane as a reconstructed first object.
In present preferred embodiment, the spatial light modulator (SLM) means
comprises a liquid crystal light valve (LCLV) means.
The liquid crystal light valve (LCLV) means is electronically controllable
and modulates a readout beam in accordance with the correlation function
incident on the photoconductor side of the LCLV. The modulated readout
beam is then fed back to the holographic means.
A second leg of the system may be provided wherein a second LCLV receives a
focused image from the hologram. This second LCLV modulates a second
readout beam using a polarizing beam splitter. When the beam splitter is
illuminated by the bright readout beam, an enhanced, and amplified object
beam resonates within the memory system, providing a truer match on the
holographic plate.
The processing of data, for pattern recognition purposes, accomplished by
the associative memory of the present inventive system may be generalized
as an associative memory system where a first body of data is transformed
and is recorded within a memory storage means, such as a hologram, using
transforms (plane or spherical waves) of a reference set of data (delta
functions) and the memory system is presented with a second incomplete
body of data (that is, the distorted input image). For example, the nature
of the transformation may be a Fourier transform. A second transform set
of data corresponding to the second body of data is provided to the
hologram. The hologram, in conjunction with a correlation device,
generates the correlations of the first and second bodies of data. Such a
generalized model also includes a liquid crystal means for conjugating and
thresholding this correlation function. This improved correlation function
is transformed as an enhanced reference beam which is then presented to
the data storage area where the data stored in the system is kept. This
results in the reconstruction of the first body of data.
An associative processing occurs wherein the transformed data storage
combines and correlates these first stored and transformed set of data
with the second transformed input data and the reference data to form the
composite product set of data, whereby a first stored body of data is
associated with a second incomplete body of data. In applicant's presently
preferred embodiment, the parallel processing and pattern recognition are
accomplished optically, and thresholding and gain are accomplished
electro-optically using LCLV mean.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a simplified first embodiment of one presently preferred
embodiment of the associative memory of this invention using a liquid
crystal light valve.
FIG. 2 (a) shows the configuration of the grid component 47 like the grid
of FIG. 1 in greater detail.
FIG. 2 (b) shows a plan view of the grid 33 of FIG. 1 in enlarged detail.
FIG. 3 shows a resonator associative memory system in accordance with a
preferred embodiment of this invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
With reference to FIG. 1 the preferred embodiment of the invention of this
application is generally shown and may operate in a first recording mode
and a second reconstruction mode. This description details the associative
memory system operating in he reconstruction mode. A hologram 18 has
stored within it at least two coherent wave amplitudes generated by at
least two different objects. The hologram 18 carries within its
light-sensitive medium both phase and amplitude information with regard to
the objects stored therein. The particular hologram 18 which is described
in detail in the following description is a Fourier transform hologram.
However a Fresnel or a volume hologram can also be used. Utilizing a
Fresnel or a volume hologram will, however, result in the loss of shift
invariance, which is an important advantage of the Fourier transform
hologram. By "shift invariance", it is meant that an object will be
recognized and reconstructed regardless of its position in the input
plane.
When the hologram 18 is irradiated by a complex wave front which is a
distorted version of the stored image, the hologram 18 may be used in
conjunction with the other components of the system, to match the
incomplete input image with the stored image on the hologram 18.
The distorted image is provided to the associative memory system by the
object input plane 12. Light from the object input plane 12 is directed at
the beam splitter 14. The beam splitter 14 redirects light from the object
input plane 12 to the Fourier transform lens 16. It will be noted that the
object input plane 12 is one focal length distance in front of the Fourier
transform lens 16. The hologram 18 is one focal length in back of the
Fourier transform lens 16. A composite product beam (which is a collection
of distorted reference beams, hereinafter referred to as "distorted
reference beam") is generated by the object wave front incident on the
hologram, and is provided through the correlation lens 32. The correlation
lens 32 is in back of the hologram 18.
The composite product or distorted reference beam is provided to the beam
splitter 20. This distorted reference light beam is then provided through
the alignment grid 33 to the photoconductor side 29 of the liquid crystal
light valve (LCLV) 28, which is one focal length in back of the
correlation lens 32. The image received by the LCLV 28 may be
characterized as the correlation of the stored image of the object, stored
within the hologram 18, and the distorted input image from the object
input plane 12.
In simplified mathematical terms, let
a=an object "a";
A=a Fourier transform of the object "a" stored within the hologram 18;
b=a reference "b", the Fourier transform of which B, is also stored in the
hologram 18.
The quantities A and B are in general complex. Then, the amplitude
transmittance of the hologram 18 is proportional to the magnitude squared
of the sum of A and B or .vertline.A+B.vertline..sup.2.
If now a distorted input image a' is provided by way of the Fourier
transform lens 16 to the hologram 18, a transformed input image A' is
presented to the hologram 18. A Fourier transform hologram 18, when
arranged in a system as shown in FIG. 1, gives rise to the correlation of
the distorted image a' with the quantity a which is in turn convolved with
b. It is well known that the convolution of two functions in the spatial
domain equals the product of the Fourier transforms of each function in
the spatial frequency domain. The symbol "*" is used herein to indicate
convolution, then, b * (a' .circle. a) means b convolved with the
quantity consisting of the correlation ( .circle. ) of a' with a.
In the spatial frequency domain, the amplitude transmitted by the developed
hologram is proportional to the expression
A'.vertline.A+B.vertline..sup.2. If one were to expand this product, one
would derive the following expression:
A'(A.sup.2 +AB+B.sup.2 +BA).
Rearranging these terms, one would obtain the following expression:
A'(A.sup.2 +B.sup.2)+A'(BA)+A'(AB),
where A is the complex conjugate of A and B is the complex conjugate of B.
The first two terms of this expression, are respectively, the zero order
term and the -1 order term, neither of which are of direct concern to this
invention; however, the last term, A' (AB) is one which is important to
this invention. B is a tilted plane wave, and this plane wave is derived
from a reference b, which is a shifted delta function. Therefore, in the
spatial domain, the quantity b*(a' .circle. a) represents a shifted
version of the correlate of the distorted input image a' with the stored
a. FIG. 1 shows that the above quantity is present simultaneously at the
liquid crystal (31) and photoconductor 29 sides of the liquid crystal
light valve (LCLV) 28.
The above mathematical results are in keeping with what is experimentally
observed in optical systems. If a first object is placed one focal length
in front of a Fourier transform lens and a Fourier transform hologram of a
second object is placed in back of the same Fourier transform lens by one
focal length; then, a second lens is positioned one focal length behind
the Fourier transform hologram of the second object, a screen which is one
additional focal length behind the second lens will produce a correlated
and convolved image of the first object and the second object which is
stored in the Fourier transform hologram. If the stored image transform B
is the Fourier transform of a delta function b, (i.e. a tilted plane
wave), then the remaining terms of interest A' (A) in the frequency domain
correspond to the spatial domain correlation of input image a' with the
stored image a. These relationships of association arise intrinsically
when a Fourier transform hologram 18 is used within an associative memory
as shown in FIG. 1. The correlated images, as stated hereinbefore, are
provided by the beam splitter 20 to the photoconductor side 29 of the
liquid crystal light valve 28. This correlation image is then presented as
writing light to a spatial light modulator, which is a liquid crystal
light valve 28 in the preferred embodiment. The liquid crystal light valve
is an optical-to-optical image transducer that is capable of accepting a
low-intensity input light image and converting it, in real time, to an
output image with light from another source. The device is designed so
that the input and output light beams are completely separated and
non-interacting. Other spatial light modulators such as magneto-optic
modulators or multiple quantum well heterostructure modulators may also be
used.
One of the significant aspects of the invention and system disclosed in
this application is the manner in which the correlated image provided to
the thresholded and conjugated liquid crystal light valve 28 is enhanced
by the feedback loop 21. A feedback arrangement of mirrors and lenses such
as that described by U. H. Gerlach et al., ("Single-Spatial Light
Modulator Bistable Optical Matrix Device Using Optical Feedback", Optical
Engineering, Volume 19, No. 4, July-August 1980, pp. 452-455) may be
suitably modified and used to feedback the modulated readout beam to the
hologram.
The LCLV is preferably operated in the forward slope mode and a positive
feedback loop is established using the feedback arrangement which images
the liquid crystal side 31 of LCLV 28 onto the photoconductor side 29 with
unity magnification and in registration. Lens 32, which is one focal
length away from both the mirror 92 and the LCLV photoconductor 29, images
the Fourier transform of the input on mirror 92 and the photoconductor 29.
The Fourier transform can be assumed to consist of a set of spots. Because
of the positive feedback loop, spots above the threshold level will latch
at the full intensity level of the readout light. These spots will be
output colinearly with the input. A large gain between the input 12 and
the output 34 is possible.
The phase of the Fourier transform of the distorted reference will be lost
in this configuration. However, it is not important in an associative
memory system application since each referenced beam reads out the
hologram 18 separately. Each of the superimposed holograms is incoherent
with respect to other recorded images. Thus, this invention can function
in the reference leg (FIG. 1 and 71 of FIG. 3) of the associative memory
as a high gain pseudo-conjugator with an adjustable threshold. The light
in the feedback loop 21 (FIG. 1) is polarized so the threshold level can
be adjusted by varying the polarizer 86 orientation.
The image presented to the photoconductor 29 side of LCLV 28 by the
correlation lens 32 and the beam splitter 20 has partially distorted
spurious light associated with this correlated image due to the input of
an imperfect real time image from the object input plane 12. In order to
remove distortions and provide a threshold value of amplitude intensity of
the optical signal to the system, the distorted correlation image incident
on photoconductor side of the LCLV 28 modulates a readout beam 36 incident
on the liquid crystal side 31. The modulated readout beam is fed back
through a feedback arrangement of mirrors and lenses. The modulated
readout beam proceeds from the polarizing beam splitter 30 as an amplitude
modulated signal onto mirror 80. This modulated readout beam diverges
until it reaches imaging lens 82. Imaging lens 82 causes this signal to
converge as the modulated readout beam is reflected off mirror 84. The
readout beam converges at imaging point 57 where the image is inverted and
begins to diverge as it passes through adjustable polarizer 86. As the
beam passes through adjustable polarizer 86, it is reflected off the
surface of mirror 88 and fully diverges onto imaging lens 90. Imaging lens
90 then causes the iterating modulated readout beam to converge onto point
P.sub.0 of the mirror 92. It is important that the mirror 92 be orthogonal
to a line connecting the mirror 92 and the imaging lens 90. The beam
splitter 20 then receives the reflected image from point P.sub.0 and
passes that reflected image onto the hologram 18, by way of the
correlation lens 32, as well as back to converging point P.sub.1 for
another feedback iteration through loop 21. This enhanced beam then serves
to phase modulate a readout beam 36. This phase modulated beam, when
passed through polarizing beam splitter 30, is amplitude-modulated process
of positive feedback through the iteration loop 21 is repeated again. Each
iteration assures yet a more enhanced and stable output signal, which may
be read at the output plane 34. The light intensity of the correlation
image presented to the photoconductive side 29 of the LCLV 28 which is
above a threshold intensity will be enhanced by feedback, whereas portions
of the correlation image with an intensity below threshold is eliminated
or made negligible. In other words, if the image on the hologram LCLV 28
is partially distorted, such as that of a circle having flares or wings,
these flares or wings at the outer portions of the correlation are
clipped, so that the adjustable thresholding provided by the LCLV 28
results in a smooth, round correlation image. The liquid crystal light
valve (LCLV) 28 operates to modulate the phase of the readout beam 36
according to the control provided by the writing light. The polarizing
beam splitter 30 directs the high intensity coherent readout beam 36 to
the LCLV 28 (liquid crystal side 31) and converts the phase modulation of
the reflected modulated readout beam into amplitude modulation.
The polarizing beam splitter 30 provides half its signal back to the
hologram 18 along the feedback loop 21 to the hologram 18 shown in FIG. 1.
By operation of the liquid crystal light valve 28 (LCLV), a stronger
signal is | | |
