There is disclosed herein a recognize only embodiment of a recognition matrix comprised of a forward matrix and a reverse matrix each having a plurality of contacts which cause convergence responses on target lines when an input signal is received by said contact. Learning is performed by changing the characteristics of the contacts to alter the convergence responses they cause in accordance with a learning rule involving the comparison of total convergence response on each target line to a convergence threshold. The contacts are not programmed ad hoc in the field as events are individually learned. Instead each contact is programmed permanently by the user for a class of events which is fixed and which can never change. The user typically performs the learning on a computer simulator for all the events which a particular system is to be used to recognize. The patterns of convergence responses and contact structure characteristics which cause these convergence responses for the class of events as a whole are then examined and optimized for maximum recognition power and minimum confusion. This pattern of convergence responses or contact characteristics is then permanently programmed in the contacts of the forward and reverse matrices. A no-confusion embodiment is also disclosed whereby an array oif recognition machines are each programmed to recognize only one event, and all are coupled in parallel to an input bus carrying the signals characterizing the event to be recognized. The outputs are or'ed together.
BACKGROUND OF THE INVENTION
This is a continuation in part application of U.S. Patent Application entitled "Brain Learning and Recognition Emulation Circuit and Method of Recognizing Events", Ser. No. 870,241, filed 06/03/86 and assigned to The Meno Corporation, a California Corporation. The inventor of the new embodiments disclosed herein, Joe Sukonick, contributed the new matter discussed herein as the ROM Recognize Only Embodiment and the No-Confusion Recognition System each of which is discussed under separate heading below. All the other embodiments were invented by the inventors named in the parent application.
A parallel distributed processing network of the back propagation type is disclosed in which the weights of connection between processing elements in the various layers of the network are determined in accordance with the set of steady solutions of the stiff differential equations governing the relationship between the layers of the network.
A neural network system comprises a memory for storing in binary code the synaptic coefficients indicative of the interconnections among the neurons. Means are provided for simultaneously supplying all the synaptic coefficients associated with a given neuron. Digital multipliers are provided for determining the product of the supplied synaptic coefficients and the relevant neuron states of the neurons connected to said given neuron. The multipliers deliver their results into an adder tree for determining the sum of the products. As a result of the parallel architecture of the system high operating speeds are attainable. The modular architecture enables extension of the system.
A Neural Network using interconnecting weights each with two values, one of which is selected for use, can be taught to map a set of input vectors to a set of output vectors. A set of input vectors is applied to the network and in response, a set of output vectors is produced by the network. The error is the difference between desired outputs and actual outputs. The network is trained in the following manner. A set of input vectors is presented to the network, each vector being propogated forward through the network to produce an output vector. A set of error vectors is then presented to the network and propagated backwards. Each Tensor Weight Element includes a selective change means which accumulates particular information about the error. After all the input vectors are presented, an update phase is initiated. During the update phase, in accordance with the results of the derived algorithm, the selective change means selects the other weight value if selecting the other weight value will decrease the total error. Only one such change is made per set. After the update phase, if a selected value was changed, the entire process is repeated. When no values are switched, the network has adapted as well as it can, and the training is completed.
An artificial neural network has a plurality of output circuits individually perturbable for memory modification or learning by the network. The network has a plurality of synapses individually connecting each of a plurality of inputs to each output circuit. Each synapse has a weight determining the effect on the associated output circuit of a signal provided on the associated input, and the synapse is addressable for selective variation of the weight. A perturbation signal is provided to one input, while data signals are provided to others of the inputs, so that perturbation of each output circuit may be controlled by varying the weights of a set of the synapses connecting the perturbation signal to the output circuits. An output circuit may be selected for perturbation by loading an appropriate weight in the synapse connecting the perturbation signal to the output circuit while zeroing the weights of the synapses connecting the perturbation signal to other output circuits. Where the weights are provided by devices incapable of repeated cycles of zeroing and reloading, each synapse connecting the perturbation intput to an output circuit has an addressable switch which is closed for perturbation of this output circuit and which is open at other times. Perturbations of different output circuits may be balanced by varying the weights of the set of synapses connected to the perturbation input or by varying the weights of another set of the synapses connected to one of inputs which receives a balancing signal.
An artificial neural network having analog circuits for simultaneous parallel processing using individually variable synaptic input weights. The processing is implemented with a circuit adapted to vary the weight, which may be stored in a metal oxide field effect transistor, for teaching the network by addressing from outside the network or for Hebbian or delta rule learning by the network itself.
