Method and apparatus for image recognition using invariant feature signals

What is claimed is:

1. A method of operating an image recognition system comprising:

providing a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix;

providing a display including an indicator, said display being electrically connected to said neural network;

initializing the indicator to an initialized state;

obtaining an image of a structure;

digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space;

transforming the input object space to a feature vector including a set of n scale-, position- and rotation- invariant feature signals, where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where I.sub.k is the set of n scale-, position- and rotation- invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the plurality of input intensity cells, .OMEGA. is an area of integration of input intensity cells, and h(k,I(x,y)) is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k;

transmitting the set of n scale-, position- and rotation- invariant feature signals to the plurality of input neurons of the neural network;

transforming the set of n scale-, position- and rotation- invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality of output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network;

transforming the set of structure recognition output signals to a structure classification signal; and

transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

2. The method of claim 1 wherein transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where ##EQU9## and .DELTA.I is a quantization level.

3. The method of claim 1 wherein transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where h(k,I(x,y)) is a Bessel function.

4. The method of claim 1 further comprising providing a reference database, wherein transforming the set of structure recognition output signals to a structure classification signal includes updating the interconnection weight matrix based on a comparison of the set of structure recognition output signals with the reference database.

5. The method of claim 1 wherein h(k,I(x,y)) decreases as k increases.

6. The method of claim 4 wherein transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where ##EQU10## and .DELTA.I is a quantization level.

7. A method of operating an image recognition system comprising:

providing a camera;

providing a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix;

providing a display including an indicator, said display being electrically connected to said neural network;

initializing the indicator to an initialized state;

obtaining an image of a structure from the camera;

digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space;

transforming the input object space to a feature vector including a set of n scale-, position- and rotation- invariant feature signals, where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h[k,I(x,y)ldxdy , where I.sub.k is the set of n scale-, position- and rotation- invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the plurality of input intensity cells, .OMEGA. is an area of integration of input intensity cells, and h(k,I(x,y)) is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k;

transmitting the set of n scale-, position- and rotation- invariant feature signals to the plurality of input neurons of the neural network;

transforming the set of n scale-, position- and rotation- invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network;

transforming the set of structure recognition output signals to a structure classification signal; and transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

8. The method of claim 7 wherein providing a camera includes providing an infra-red filtered camera and the image of the structure is an infrared filtered image.

9. The method of claim 7 further comprising providing a pulsed infrared illuminator electrically connected to the camera,

wherein providing a camera includes providing an infra-red filtered camera and the image of the structure is an infra-red filtered image.

10. The method of claim 7 wherein transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where ##EQU11## and .DELTA.I is a quantization level.

11. The method of claim 7 wherein transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where h(k,I(x,y)) is a Bessel function.

12. The method of claim 7 further comprising providing a reference database, wherein transforming the set of structure recognition output signals to a structure classification signal includes updating the interconnection weight matrix based on a comparison of the set of structure recognition output signals with the reference database.

13. The method of claim 5 wherein h(k,I(x,y)) decreases as k increases.

14. An image recognition system comprising:

a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix;

a display including an indicator, said display being electrically connected to said neural network;

means for initializing the indicator to an initialized state;

means for obtaining an image of a structure;

means for digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space;

means for transforming the input object space to a feature vector including a set of n scale-, position- and rotation- invariant feature signals, where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where I.sub.k is the set of n scale-, position- and rotation- invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the plurality of input intensity cells, .OMEGA. is an area of integration of input intensity cells, and h(k,I(x,y)) is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k;

means for transmitting the set of n scale-, position- and rotation- invariant feature signals to the plurality of input neurons of the neural network;

means for transforming the set of n scale-, position- and rotation- invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network;

means for transforming the set of structure recognition output signals to a structure classification signal; and

means for transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

15. The image recognition system of claim 14 further comprising an infra-red filter optically connected to the means for obtaining an image,

wherein the means for obtaining an image of a structure comprises an infra-red filtered camera and the image of the structure is an infra-red filtered image of a structure.

16. The image recognition system of claim 15 further comprising a pulsed infra-red illuminator synchronized with the infra-red filtered camera.

17. The image recognition system of claim 14 wherein the means for transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes means for normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where ##EQU12## and .DELTA.I is a quantization level.

18. The image recognition system of claim 14 wherein the means for transforming the input object space to a feature vector by extracting the set of n scale-, position- and rotation- invariant feature signals from the input object space includes means for normalizing the input object space according to the set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h(k,I(x,y)) dxdy, where h(k,I(x,y)) is a Bessel function.

19. The image recognition system of claim 14 further comprising means for providing a reference database electrically connected to the neural network, wherein the means for transforming the set of structure recognition output signals to a structure classification signal includes means for updating the interconnection weight matrix based on a comparison of the set of structure recognition output signals with the reference database.

20. The image recognition system of claim 14 wherein said means for obtaining an image of a structure includes a camera and h(k,I(x,y)) decreases as k increases.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field of Use

The present invention relates generally to the field of image recognition. More particularly, the present invention concerns methods and apparatus for recognition of facial images. Specifically, a preferred embodiment of the present invention is directed to a method and apparatus for automatic face recognition by scale, position and rotation (SPR) invariant feature extraction. The present invention thus relates to methods and apparatus for face recognition of the type that can be termed invariant feature extractive.

2. Description of Related Art

Within this application several publications are referenced by arabic numerals in parenthesis. Full citations for these references may be found at the end of the specification immediately preceding the claims. The disclosures of all these references in their entireties are hereby expressly incorporated by reference into the present application for the purposes of indicating the background of the invention and illustrating the state of the art.

An important concern at security facilities is access control. This is a particularly acute problem at highly classified facilities where hundreds of employees must be identified as they enter. Identification (ID) cards are commonly checked by security personnel in such limited access areas. Because of human error, subjectivity, bias and even conspiracy, it would be technically and economically advantageous to automate this process. Indeed, during peak hours an automatic door keeper would reduce frustrating employee lines, as well as problems with stole and lost ID cards. In addition, the installation of several gatekeeping systems inside a limited access facility would permit better protection against intrusion and tighten access to designated areas within the facility.

Accordingly, an automatic face recognition system has long been sought by security agencies, law enforcement agencies, the airline industry, the border patrol, local authorities, and many other organizations. Examples of other potential applicatons are entry control to limited access areas, such as secure facilities in industry, banks, and various private institutions, secure access to computers and video communications, including video telephones, and user verification for automated teller machines (ATM) and credit cards.

The class of techniques that use biological features to classify a person's identity are biometric techniques. Face recognition is such a biometric technique.

Face recognition has an important advantage over other biometric techniques. Face recognition can be both non-invasive and unnoticeable to the subject under investigation. In contrast, fingerprinting and retinal pattern analysis do not share these advantageous features.

Automatic face recognition techniques have a unique place among automatic pattern recognition (APR) technology. Existing APR technology, in general, cannot yet match the performance of a human operator in dealing with a limited number of objects to be classified under varied and frequently noisy conditions. In contrast, APR techniques can deal with a very large number of objects, such as faces, whose classification is beyond the capacity of a human operator simply because of inability to memorize many names and faces, especially after only a single learning exposure.

Heretofore, techniques have been developed in the prior art in an attempt to analyze and identify human faces. For example, an eye blinking method was proposed to recognize individuals by the location, shape and distance between a given set of eyes.sup.(1). The nose, the mouth, and the outline of the face have also been used to identify faces.sup.(1-4). Color image segmentation and the K-L transformation have been used to extract facial features.sup.(5). Neural network classifiers have also been used to perform robust pattern recognition operations.sup.(1,6).

Further, the below-referenced prior patents disclose techniques that were at least in-part satisfactory for the purposes for which they were intended but which had disadvantages. The disclosures of all the below-referenced prior patents in their entireties are hereby expressly incorporated by reference into the present application.

U.S. Pat. Nos. 5,331,544, 5,012,522 and 4,975,960 disclose digitizing data for further processing. U.S. Pat. No. 5,274,714 discloses the use of a frame grabber for digitizing data to be subsequently processed by a neural network. U.S. Pat. Nos. 5,263,097 and 5,255,347 disclose feature extraction for subsequent processing with a neural network.

The above and other techniques, share several common problems. Some of the techniques may take several stages of complex operations to extract features. Some of the techniques require intensive computation which becomes an obstacle to system speed. Many of the recognition techniques are not invariant to position, tilt, and distance and require the individual to place his/her head in a certain position, thus prohibiting the use of such techniques in portal control applications. Many of the techniques require the storage of high resolution face images and/or complex feature vectors in a database. Any one of these disadvantages creates both speed and memory space problems for the use of these techniques in large database applications.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a method of operating an image recognition system comprising: providing a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix; providing a display including an indicator, said display being electrically connected to said neural network; initializing the indicator to an initialized state; obtaining an image of a structure; digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space; transforming the input object space to a feature vector including a set of n invariant feature signals, Where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h[k,l(x,y)]dxdy where I.sub.k is the set of n invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the plurality of input intensity cells, .OMEGA. is an area of integration of input intensity cells, and h[k,I(x,y)] is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k, and optionally decreases as k increases; transmitting the set of n invariant feature signals to the plurality of input neurons of the neural network; transforming the set of n invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality of output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network; transforming the set of structure recognition output signals to a structure classification signal; and transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

In accordance with this aspect of the present invention, a method of operating an image recognition system is provided comprising: providing a camera; providing a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix; providing a display including an indicator, said display being electrically connected to said neural network; initializing the indicator to an initialized state; obtaining an image of a structure from the camera; digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space; transforming the input object space to a feature vector including a set of n invariant feature signals, where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h[k,I(x,y)]dxdy , where I.sub.k is the set of n invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the plurality of input intensity cells, .OMEGA. is an area of integration of input intensity cells, and h[k,I(x,y)] is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k, and optionally decreases as k increases; transmitting the set of n invariant feature signals to the plurality of input neurons of the neural network; transforming the set of n invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality of output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network; transforming the set of structure recognition output signals to a structure classification signal; and transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

Further in accordance with the above aspects of the present invention, an image recognition system is provided comprising a neural network including a plurality of input neurons, a plurality of output neurons and an interconnection weight matrix; a display including an indicator, said display being electrically connected to said neural network; means for initializing the indicator to an initialized state; means for obtaining an image of a structure; means for digitizing the image so as to obtain a plurality of input intensity cells and thereby define an input object space; means for transforming the input object space to a feature vector including a set of n invariant feature signals, where n is a positive integer not greater than the plurality of input neurons, by extracting the set of n invariant feature signals from the input object space according to a set of relationships I.sub.k =.intg..sub..OMEGA. .intg.I(x,y)h[k,I(x,y)]dxdy , where I.sub.k is the set of n invariant feature signals, k is a series of counting numbers from 1 to n inclusive, (x,y) are the coordinates of a given cell of the plurality of input intensity cells, I(x,y) is a function of an intensity of the given cell of the input intensity cells, and h[k,I(x,y)] is a data dependent kernel transform from a set of orthogonal functions, of I(x,y) and k, and optionally decreases as k increases; means for transmitting the set of n invariant feature signals to the plurality of input neurons of the neural network; means for transforming the set of n invariant feature signals at the plurality of input neurons to a set of structure recognition output signals at the plurality of output neurons according to a set of relationships defined at least in part by the interconnection weight matrix of the neural network; means for transforming the set of structure recognition output signals to a structure classification signal; and means for transmitting the structure classification signal to the display so as to perceptively alter the initialized state of the indicator and display the structure recognition signal for the structure.

Other aspects and objects of the present invention will be better appreciated and understood when considered in conjunction with the following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The advantages and features of the present invention will become more readily apparent with reference to the detailed description which follows and to exemplary, and therefore non-limiting, embodiments illustrated in the following drawings in which like reference numerals refer to like elements and in which:

FIG. 1 illustrates a schematic diagram of an automatic door keeper at multiple entrance corridors according to the present invention.

FIG. 2 illustrates a schematic diagram of a real-time face recognition system according to the present invention.

FIG. 3A illustrates a conventional kernel transform model appropriately labeled "PRIOR ART";

FIG. 3B illustrates a class of kernel transform model according to the present invention;

FIG. 4 illustrates a schematic diagram of a complete automatic face recognition identification system according to the present invention;

FIG. 5A illustrates recognition with a head in an upright position with an ordinary facial expression;

FIG. 5B illustrates recognition with a head in a tilted position with a changed facial expression;

FIG. 6A illustrates training and testing a gray scale interpattern association neural network with a noise level; and

FIG. 6B illustrates training and testing a gray scale interpattern association neural network with different images and with a higher noise level than FIG. 6A.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention and various aspects, objects, advantages, features and advantageous details thereof are explained more fully below with reference to exemplary, and therefore non-limiting, embodiments described in detail in the following disclosure and with the aid of the drawings. In each of the drawings, parts the same as, similar to, or equivalent to each other, are referenced correspondingly.

1. Resume

All the disclosed embodiments are useful in conjunction with face recognition systems such as are used for the purpose of restricting access at the entrances to, or within, limited access areas, or for the purpose of user verification at video telephones, automated teller machines, or the like. All the disclosed embodiments can be realized using conventional components and procedures without undue experimentation.

2. System Overview

The present invention is capable of performing as an automatic face recognition and identification (AFRAID) system which performs human face identification in real time. Among the many scenarios in which the AFRAID system could be exploited to secure access to classified facilities, a preferred embodiment is as a portal or gate control system for a building. Every face entered into a system according to the present invention can be used to update the database so that gradual changes in human appearance are accommodated. Only such extreme measures as plastic surgery or the shaving of a beard would require the intervention of security personnel.

3. First Embodiment

Referring to FIG. 1, an AFRAID system according to the present invention is preferably capable of operating in two modes. In a first identification modality, employees entering the security gate 10, are screened by standard video cameras 20. During the employee's walk through the entrance corridor 30, the AFRAID system can make a decision regarding the person's right to enter the building. If the system decides an individual should be barred from the facility, the system shuts the door 40, or blocks a turnstile, at the end of the corresponding entrance corridor 30, as shown in FIG. 1.

4. Second Embodiment

For enhanced security, the AFRAID system can perform verification in a second modality. In this modality, a person can be required to position themselves in the view of a camera and enter his/her personal identification number (PIN) in order to initiate the face verification process. While the first modality requires a search of all faces in a database, the second modality is limited to a simple verification of the identity of an individual face, with a high rejection rate (i.e., a low false acceptance rate) of all other faces.

5. Detailed Description

It is clear that a robust, high confidence rate face recognition system is required to successfully replace security personnel at an entrance gate. Most of the hardware needed to accomplish automated doorkeeping is already in place. Surveillance cameras and computers (e.g., personal computers or workstations) are typical components of any security installation. The only additionally required hardware is an image digitizer (i.e., a frame grabber) computer board. Therefore, an AFRAID system according to the present invention can be inexpensive. The centerpiece of the present invention is an innovative face recognition algorithm and its software implementation to operate the hardware.

The present invention allows for robust and high-speed identification of a specific face from among many faces resident in a large database. The present invention is based on a unique combination of a robust face feature extractor and a highly efficient artificial neural network. A real-time video image of a face can serve as the input to a high-speed face feature extractor, which responds by transforming the video image to a mathematical feature vector that is highly invariant under face rotation (or tilt), scale (or distance), and position conditions. While not being bound by theory, this highly invariant mathematical feature representation is a major reason for the extremely robust performance of the present invention. A feature extractor according to the present invention is advantageously capable of the rapid generation of a mathematical feature vector of at least 20 to 50 elements from a face image made up of, for example, 256.times.256 or 512.times.512 pixels. This represents a data compression of at least 1000:1. The feature vector is then input into the input neurons of a neural network (NN), which advantageously performs real-time face identification and classification.

Such an effective feature extractor is capable of dramatically reducing the dimensions of the input object space by carrying compressed information. The resulting features make the search space less complex, which in turn significantly increase the convergence speed of the neural network's learning procedure.

In addition to reducing the number of features required for facial image representation, the other major task for the extractor is to generate a set of invariant features. For face identification, that means scale-, position- and rotation- (SPR) invariance. The importance of invariance can not be overstated because it eliminates the necessity of a fixed face location, thus permitting noncooperative face recognition at a single pass. While it is clear that out-of-plane head movements (i.e., different aspect angles) form essential