A learning algorithm for the N-dimensional Coulomb network is disclosed which is applicable to multi-layer networks. The central concept is to define a potential energy of a collection of memory sites. Then each memory site is an attractor of other memory sites. With the proper definition of attractive and repulsive potentials between various memory sites, it is possible to minimize the energy of the collection of memories. By this method, internal representations may be "built-up" one layer at a time. Following the method of Bachmann et al. a system is considered in which memories of events have already been recorded in a layer of cells. A method is found for the consolidation of the number of memories required to correctly represent the pattern environment. This method is shown to be applicable to a supervised or unsupervised learning paradigm in which pairs of input and output patterns are presented sequentially to the network. The resulting learning procedure develops internal representations in an incremental or cumulative fashion, from the layer closest to the input, to the output layer.

A Neural Network using interconnecting weights each with two values, one of which is selected for use, can be taught to map a set of input vectors to a set of output vectors. A set of input vectors is applied to the network and in response, a set of output vectors is produced by the network. The error is the difference between desired outputs and actual outputs. The network is trained in the following manner. A set of input vectors is presented to the network, each vector being propogated forward through the network to produce an output vector. A set of error vectors is then presented to the network and propagated backwards. Each Tensor Weight Element includes a selective change means which accumulates particular information about the error. After all the input vectors are presented, an update phase is initiated. During the update phase, in accordance with the results of the derived algorithm, the selective change means selects the other weight value if selecting the other weight value will decrease the total error. Only one such change is made per set. After the update phase, if a selected value was changed, the entire process is repeated. When no values are switched, the network has adapted as well as it can, and the training is completed.

Trajectories are generated in response to an input label by using a reverse time delay neural network. The reverse time delay neural network comprises an input layer, a plurality of hidden layers, and an output layer, all arranged in succession so that the number of frames per layer increases as the network is traversed from the input layer to the output layer. Additionally, the number of features decreases as the network is traversed from the input layer to the output layer. Features of the trajectory are created from the input label so that a time series of frames can be output by the network. Frames generally relate to particular epochs of time or time units and a frame includes a plurality of features. Interconnection between layers is accomplished using differential neuron units. In the differential neuron unit, standard neuron weighting, summing, and nonlinear squashing functions are performed on the inputs thereto. Moreover, the output of the differential neuron unit includes a contribution from the value of the corresponding feature in the previous frame in the same layer as the differential neuron unit output.

Disclosed are two modifications of the Tank-Hopfield circuit, each of which enables the deconvolution of a signal in the presence of noise. In each embodiment, the Tank-Hopfield circuit is modified so that the equation for total circuit energy reduces to one term representing convolution and another information theoretic (or Shannon) entropy. Thus, in finding its global minimum energy state, each modified circuit inherently identifies an optimal estimate of a deconvoluted input signal without noise.

An analog neural network composed of an array of capacitors for storing weighted electric charges. Electric charges, or voltages, on the capacitors control the impedance (resistance) values of a corresponding plurality of MOSFETs which selectively couple input signals to one input of a summing amplifier. A plurality of semiconductor gating elements (e.g. MOSFETs) selectively couple to the capacitor's weighted analog voltage values received serially over an input line. The weighted voltage on the input line are periodically applied to the proper capacitors in the neural network via the gating elements so as to refresh the weighted electric charges on the capacitors, and at a multiplex rate that maintains the voltages on the capacitors within acceptable tolerance levels.