 |
Claims  |
|
|
I claim:
1. Apparatus for monitoring the operation of a device to detect improper
operation thereof in response to the application of a series of test
monitoring the operation of a device to detect improper operation thereof
in response to the application of a series of test vectors applied
thereto, said device generating information signals and operation signals
in response to each applied test vector, said apparatus comprising:
predicting means for predicting correct operation of the device by
generating expected information and operation signals in response to the
application of the series of test vectors applied thereto;
comparing means responsive to the operation of the device and to said
predicting means for comparing the operation signals of the device with
the expected operation signals in response to the same test vector;
detection means responsive to said comparing means for detecting improper
operation of the device; and
storage means coupled to said device and to said prediction means
responsive to said detection means for storing information signals
representative of the operation of the device and the expected operation
signals for the same application of test vectors, said storage means
includes:
first storage means coupled to the device responsive to
said detection means for storing information signals
descriptive of the operation of the device subsequent to the detection of
improper operation of the device in response
to the corresponding operation and expected operation
signals; and,
second storage means coupled to the predicting means
responsive to said detection means for storing expected information signals
descriptive of the predicted correct
operation of the device subsequent to the detection of
improper operation of the device in response to the
corresponding operation and expected operation signals.
2. Apparatus as recited in claim 1 wherein said predicting means further
comprises:
program means for modeling the operation of the device;
digital computer means for performing said program means so that the
expected information and operation signals of the device is determined;
and,
prediction storage means responsive to said digital computer means for
storing the expected information and operation signals of the device.
3. Apparatus as recited in claim 1, wherein said comparing means further
comprises a plurality of single comparing means for comparing a single
signal from the device with a corresponding single signal from said
predicting means.
4. A method for monitoring the operation of a device to detect the improper
operation thereof in response to the application of a series of test
vectors applied thereto, said device generating information signals and
operation signals in response to each applied test vector, said method
comprising the steps of:
predicting the correct operation of the device by generating expected
information and operation signals in response to the application of each
of the series of test vectors;
repeatedly comparing the operation signals of the device with the expected
operation signals to detect improper device operation in response to each
of the test vectors;
detecting improper device operation by detecting comparison errors; and
repeatedly storing information signals representative of the operation of
the device and expected information signals representative of the
predicted correct operation of the device in response to the detection of
improper device operation.
5. A method for monitoring the operation of a device to detect the improper
operation thereof as in claim 4 further comprising the step of terminating
the storing of information signals representative of the device and
expected information signals in response to the detection of improper
device operation.
6. Apparatus for monitoring digital information produced by a digital
device, to detect improper operation thereof, comprising:
predicting means responsive to the digital device for producing digital
expected information and operation signals representative of the digital
information and operation signals expected to be produced by the digital
device;
comparing means responsive to digital information signals produced by the
digital device and to the expected digital information signals produced by
said predicting means for detecting a difference therebetween;
detecting means responsive to said comparing means for producing a signal
in response to the detection of a difference between the digital
information signals produced by the digital device and the digital
expected information signals produced by said predicting means;
storage means responsive to the signal produced by said detecting means,
coupled to the digital device and to the predicting means for storing
digital operation signals of the digital device and for storing digital
expected operation signals produced by said predicting means.
7. Apparatus as recited in claim 6, wherein said predicting means further
comprises
storage means for storing digital expected information and operation
signals representative of the digital information expected to be produced
by the digital device;
control means coupled to said storage means and responsive to said digital
device for sequentially extracting from said storage means the digital
expected information and operation signals expected to be produced by the
digital device.
8. Apparatus as recited in claim 6, wherein said predicting means
comprises:
determining means for producing the digital expected information and
operation signals expected to be produced by the digital device;
counter means responsive to the digital device for producing a cyclic
sequence of binary numbers;
memory means coupled to said counter means for storing and reproducing the
digital expected information and operation signals expected to be produced
by the digital device; and,
control means responsive to the digital device and said counter means for
storing in and extracting from said memory means the digital expected
information and operation signals expected to be produced by the digital
device.
9. Apparatus as recited in claim 6, wherein said comparing means further
comprises a plurality of comparing devices, each of which comparing
devices compares a single first signal with a single second signal, and
produces an output signal if the first signal does not agree with the
second signal.
10. Apparatus as recited in claim 9, wherein said comparing device is an
EXCLUSIVE-OR gate.
11. Apparatus as recited in claim 9, wherein said detecting means further
comprises an OR gate having a plurality of inputs thereto, with each input
coupled to the output of one of the plurality of comparing devices.
12. Apparatus as recited in claim 6, wherein said storage means further
comprises means for storing digital operation signals of the digital
device and for storing digital expected operation signals produced by said
predicting means in response to the signal produced by said detecting
means.
13. Apparatus as recited in claim 6, wherein said storage means further
comprises means for continuously storing digital operation signals of the
digital device and for continuously storing digital expected operation
signals produced by said predicting means, and terminates the storing of
said operation signals in response to the signal produced by said
detecting means.
14. Apparatus as recited in claim 6, wherein said storage means further
comprises:
counter means coupled to the digital device for producing a binary count;
digital storage means coupled to the digital device, the predicting means
and said counter means for storing digital operation signals of the
digital device and for storing digital expected operation signals produced
by said predicting means; and
control means coupled to said counter means and responsive to said
detection means for controlling the counting of said counter means.
15. Apparatus as recited in claim 14, wherein said control means further
comprises means for inhibiting the operation of said counter means
responsive to a preselected binary count being produced by said counting
means.
16. Apparatus as recited in claim 14, wherein said control means further
comprises means for inhibiting the operation of said counter means
responsive to the signal produced by said detecting means.
17. A method for testing the proper performance of a device, said method
comprising the steps of:
applying test criteria to a simulator of the device for producing expected
information and operation signals therefrom;
applying at least on of the test criteria applied to the simulator to the
device to be tested for producing at least one set of information and
operation signals therefrom;
comparing the expected operation signals from the simulator with the
operation signals from the device to the same test criteria to detect
errors in the performance of the device; and
repeatedly storing information signals of the device and the expected
information signals from the simulator when an error is detected in the
comparing step.
18. A method for testing the proper performance of a device as in claim 17
wherein the performance comparing step includes the step of terminating
the testing when at least one of the responses from the device does not
compare with the expected responses from the simulator.
19. Apparatus for testing the proper performance of a device comprising:
a simulator means for modeling the desired performance of the device;
first means for applying test criteria to the simulator means for producing
expected information and operation signals therefrom;
second means for applying at least one of the test criteria applied to the
simulation means to the device to be tested for producing at least one set
of information and operation signals therefrom;
third means for comparing the expected operation signals from the simulator
means with the operation signals from the device to the same test criteria
to detect errors in the performance of the device; and
means for repeatedly storing information signals of the device and the
expected information signals from the simulator means when when an error
is detected by the third means.
20. Apparatus for testing the proper performance of a device as in claim 19
wherein the third means includes means for terminating the testing when at
least on of the operation signals from the device does not compare with
the expected information signals from the simulator means.
21. Apparatus for monitoring the operation of a device to detect improper
operation thereof in response to the application of a series of test
vectors applied thereto, comprising:
predicting means for predicting expected information and operation signals
of the device in response to the application of the series of test vectors
applied thereto;
comparing means responsive to the operation signals of the device and to
said expected operation signals of the predicting means for comparing the
operation of the device with the predicted correct operation in response
to the same test vector;
detection means responsive to said comparing means for detecting improper
operation of the device; and
storage means coupled to said device and to said predicting means
responsive to said detection means for storing information signals of the
device and the expected information signals, said storage means includes:
first storage means coupled to the device and responsive to said detection
means for storing information signals of the device prior to the detection
of improper operation of the device; and
second storage means coupled to the predicting means and responsive to said
detection means for storing expected information signals prior to the
detection of improper operation of the device. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND
This invention relates to monitoring devices, and more particularly to the
real time monitoring and comparing of the operation of a device with
analytical predictions of correct or desired operation of the device.
Broadly speaking, a difficult and time consuming task associated with
improper operation of a device relates to the identification of problems
responsible for improper operation. Once a problem has been clearly
identified, well known techniques may be applied to correct the problem.
While the task of identifying the source of problems does occur in a
repair process, the more demanding and elusive problems seem to appear
during the design phase of a device. In such an environment, numerous
problems of varying scope are encountered. Frequently, an observed problem
may result from several problems, with a somewhat complicated interaction
among the various individual problems.
There are a number of considerations associated with the problem
identification process. Fundamental to the process is the requirement of
being able to monitor an operational environment associated with a device
without disturbing the monitored environment; or in the alternative, to
minimize the disturbance on the monitored environment by the monitoring
process. This requirement has been long recognized, and by way of example,
can be observed in the usual high input impedance, or capacitance
compensated devices normally associated with typical monitoring devices.
An additional consideration associated with the process of identifying
problems associated with devices relates to the complexity of the design
of a device. Broadly speaking, the greater the complexity of a design, the
greater will be the problems of identifying problems associated therewith.
One method of dealing with such problems is to reduce a large design down
into a number of smaller units, and thereafter to individually consider
problems associated with the various units. While such an approach may be
applied in those circumstances when the nature of a design permits such
reduction, such an approach does not provide insight into problems of an
interactive nature among the various units.
A further consideration associated with the process of problem
identification relates to the speed at which a device may operate. In
particular, with the higher speeds which are frequently associated with
advanced device designs, further considerations are introduced. In
particular, in high speed operations the time between the occurrence of
events can become quite small, and the number of events which occur quite
large. As a result of the foregoing, the location of problems in such an
environment can result in the collection of large amounts of data. The
subsequent analysis of such prodigious amounts of information can in
itself present problems requiring sophisticated and often time consuming
analysis techniques. An alternate way of dealing with such a situation
involves reducing the speed at which a device operates. However, such an
approach introduces a disturbance into the monitored environment by
creating an artificial environment differing from the normal operational
environment. While such an altered environment may be acceptable in some
situations, the dynamic aspects of many designs frequently cannot be
supported during a reduction of operating speed. In addition thereto,
altering an environment by changing the speed at which a process occurs
can introduce further changes which may distort the representation of a
problem. A yet further consideration associated with the process of
identifying problems associated with devices relates to the manner in
which a device may be implemented. In particular, when a new design is
implemented as an integrated circuit, or includes devices implemented as
integrated circuits, significant restrictions on the availability of
information related to the overall operation may be introduced. By way of
example, a particular device which performs a given function may have
associated with it's internal operation any number of related signals
which in themselves represent information of various events indirectly
related to the final output of the particular device, but not otherwise
generally of interest. However, in the process of identifying the source
of incorrect operation of a system which is implemented either as an
integrated circuit, or includes integrated circuits in it's design, the
aforementioned signals which occur internal to the integrated circuit
could provide revealing information with respect to associated problems.
Unfortunately, such signals internal to the integrated circuits are not
generally available for direct monitoring. In short, the advantages which
are gained through the use of integrated circuits, such as compactness,
price, and/or speed of operation, carry with them the disadvantages of
unavailability of certain information which may necessarily be generated
internal to the integrated circuit. Consequently, while the use of
integrated circuits may offer significant advantages from both a technical
as well as econommic standpoint, their use may likewise presents a
significant impediment to the location of associated problems,
particularly during a design phase.
In the past, the foregoing problems have been addressed in numerous ways.
In the most fundamental approach, corrective action is based entirely upon
observation of symptoms, and depends heavily upon a thorough understanding
of the operation of the device, and a degree of intuitiveness on the part
of the observer. However, with increasingly complex designs, such an
approach frequently does not provide an efficient manner in which to
determine the source of observed problems.
A second approach to the problem of identifying the source of malfunctions
is based upon the use of a known good device against which the operation
of the device in question is compared. Broadly stated, such an approach
involves the application of identical signals or conditions to both the
device whose operation is questioned, as well as to the known good device.
Thereafter, the resultant signals from each device are compared, and used
to provide guidance in the determination of the cause of a malfunction.
While such an approach can provide an approach to the problem
identification process, it nevertheless has a number of serious
shortcomings. Fundamental among the shortcomings is the requirement of a
known good device. From a practical standpoint, the acquisition of a known
good device for such use commonly presents the problem of either acquiring
the necessary known good device from a supply source, or in the
alternative requires building such a device. In the case when the needed
device is of the nature of an existing pre-fabricated product, such as a
printed circuit card, the problem can be minimal. However, when the device
in question is a highly complex integrated circuit, such as a
microprocessor or in particular, a custom integrated circuit, the task of
acquiring a known good device can become more involved. The problem
becomes further acute when signals present within the device which are
necessarily generated in the process of producing the final output of the
integrated circuit are of interest. In such a situation, it may well
become necessary to construct such a device from discrete parts or from
other integrated circuits. Such a situation can easily become extremely
costly, not only in terms of time, but also in terms of the resources and
expertise necessary for such a project.
In addition to the foregoing, other techniques which are employed with
respect to microprocessor based designs include the use of emulators and
monitor programs. Broadly stated, such approaches provide a limited amount
of information with respect to the internal operation of microprocessor
based devices, but generally involve some degree of degradation of
performance of the device or the associated operational environment. Such
degradation is generally unavoidable as it represents an overhead
necessary for the performance of the monitoring function. In particular,
the use of emulators or software monitor programs generally degrade the
time transparency of the monitoring tool. While the impact of such
degradation is application dependent, it is undesirable, and a more
preferred approach would include either the further minimization or the
complete elimination of such side effects.
The foregoing considerations can be illustrated with respect to a
microprocessor based designed by the following. Generally speaking, the
signals associated with a microprocessor can be broadly classified into
three groups: control signals, address signals and data signals. These
signals are used to electrically interface a microprocessor with the
components associated therewith. Consequently, monitoring these signals
can provide some indication of the operations occurring within the
microprocessor. A number of techniques have been employed in the past
based upon this approach.
In one technique, the address signals from a microprocessor are segregated
into two groups. Each address signal present within each group is then
assigned a particular binary weight. Thereafter, a corresponding analog
signal is derived for each of the signal groups having a magnitude
proportional to the binary weight of the signal present on the address
lines within each of the two groups. This is generally accomplished
through the use of a digital-to-analog converter device. Thereafter, the
two analog output signals are used to control the horizontal and vertical
deflection plates on an oscilloscope. The resulting display on the
oscilloscope will consequently provide some information with respect to
the location in memory wherein the microprocessor is currently active.
While such an approach does offer the advantage of not disturbing the
operational environment within the microprocessor, the information so
provided with respect to the internal operation of the microprocessor is
very limited.
In yet another technique employing a visual monitoring device, each unique
location in memory for a designated portion of memory is assigned a
particular location on a visual monitoring device, and the contents of the
associated memory location are displayed therein. Consequently, by
observing the resulting visual display, it is possible to gain some
insight regarding the activities of the microprocessor with respect to the
memory.
The foregoing examples are but a few of the approaches which have been
employed in the past to gain insight on the internal operations of a
microprocessor by monitoring the various signals associated therewith. It
will be observed that while each approach shares the common advantage of
not disturbing the operation of a program within a microprocessor, the
information provided by the monitoring technique has been somewhat
limited, failing to provide comprehensive information relating to the
internal operations of the microprocessor.
In yet a different approach to the monitoring problem, the microprocessor
has been used to monitor itself. In such an approach, a series of programs
are used to direct the microprocessor to monitor the contents of various
registers and the status of events occurring internal to the
microprocessor, and thereafter to report the results.
In one such approach the microprocessor operates under the control of a
control program, often referred to as a "monitor program". Operating under
the control of the monitor program, the microprocessor performs selected
instructions present in a particular program of interest under the strict
control of an operator. In such an approach, it is possible for the
operator to exercise greater control over the activities of the
microcprocessor than would otherwise be possible. Such control activities
would normally include control over the execution of sequence of
instructions, e.g., executing instructions on a one-by-one basis, or
executing instructions in groups. At any point in the execution process,
it would be possible to monitor the internal state of the microprocessor,
and report the status to the operator. Other control activities would
include the ability to individually examine, and if necessary change, the
contents of any location in an associated memory unit. By employing the
approach of using the microprocessor to monitor itself, much greater
control and monitoring of the activities of the microprocessor is
possible. By using such an approach, the monitoring of parameters internal
to the microprocessor is frequently limited only by the complexity of the
monitor program itself.
However, notwithstanding the advantages presented by using the
microprocessor to monitor itself, such an approach has an inherent short
coming which is fundamental to the process. In particular, the basic
environment existing during the execution of program instructions is
disturbed. This follows from the fact that the microprocessor must not
only execute instructions associated with a program of interest, but must
also execute the instructions associated with the monitor program. By so
executing the monitor program, the microprocessor is performing tasks that
it otherwise would not do if it were executing the program alone. Such a
short coming can present disadvantages of varying scope. One such
disadvantage is in the time required for the execution of the program of
interest. Since the microprocessor having to execute both the program of
interest as well as the monitor program, additional execution time is
clearly required. While this may be acceptable under some conditions, it
can be most undesirable under others. In particular, in an application in
which it is desired to monitor an environment wherein the microprocessor
must not only execute a program of interest, but must also respond to
events which are occurring external to the microprocessor, the imposing
upon the microprocessor of the additional task of executing the monitor
program can significantly alter the basic environment which is desired to
be monitored. This result follows from the fact that the time which the
microprocessor would otherwise dedicate to the program of interest and
responding to events which are occurring external to the microprocessor is
reduced by the amount of time required to perform the appropriate portions
of the monitor program. Consequently such an approach has the serious
short coming of disturbing the environment which is desired to be
monitored.
From the foregoing it is observed that while the first basic approach
involving the monitoring of the signals from a microprocessor functions in
a manner transparenerious short coming of disturbing the environment which
is desired to be monitored.
From the foregoing it is observed that while the first basic approach
involving the monitoring of the signals from a microprocessor functions in
a manner transparent to programs which the microprocessor may be
executing, such an approach provides very limited information regarding
the particular operations occurring internally to a microprocessor.
In a similar manner, it is observed that while the second basic approach
involving using a microprocessor to monitor itself does provide more
detailed information regarding the internal operations of a
microprocessor, such an approach has the disadvantage of disturbing the
environment within the microprocessor in which programs execute.
From the foregoing it is apparent that there are a number of shortcomings
in the prior art in determining the source of problems associated with
improper device operation. Undesirable effects present with such
approaches either violate a fundamental consideration of monitoring
techniques by disturbing the monitored environment, or burdening available
resources by requiring the construction of models.
SUMMARY
In accordance with the present invention, a method and apparatus are
disclosed with provide for the real time monitoring of the activity of a
device without disturbing the operating environment associated therewith.
Broadly stated, in accordance with the present invention, responses of a
device to a selected set of test conditions are compared against predicted
correct responses. Improper responses are thereby determined in real time
immediately upon the occurrence thereof, with the corresponding
information stored for subsequent analysis. Thereafter, by comparing the
predicted correct response and the observed actual response to a set of
selected test conditions, insight is gained as to the source of a problem.
In accordance with the present invention, an analytical model of a device
is first constructed. Broadly stated, the model serves as a means to
predict the correct operation or response of the device to a given
condition or set of conditions. Consequently, by choosing conditions
appropriate to provide for comprehensive testing of either all or a
selected group of features of the actual device, the anticipated responses
of the actual device to such conditions may be determined. The set of
chosen test conditions are generally referred to as test vectors. In
accordance with the present invention, the response of a device to a
chosen set of test vectors as determined through the use of a model of the
device are determined. The responses are then stored for subsequent
comparison purposes.
Consequently, through the use of a model to predict the response of a
device to a selected set of input conditions, the corresponding set of
expected outputs may be determined. Thereafter, by applying the selected
set of input conditions to the actual device, and recording the resultant
responses of the devices thereto, while simultaneously comparing the
observed response of the device with the response predicted by the model,
improper responses from the device may be immediately detected.
Thereafter, the input conditions, i.e., the test vectors, as well as the
desired and observed responses may be reviewed for indications with
respect to the locations of problems with the device.
DESCRIPTION OF THE FIGURES
FIG. 1 is a conceptual representation of a method and apparatus for the
improved monitoring and detection of improper device operation in
accordance with the teachings of the present invention.
FIG. 2A is an illustration of a microprocessor based Engineering Prototype
Unit.
FIG. 2B is an illustration of signals associated with the Engineering
Prototype Unit of FIG. 2A.
FIG. 3 is an illustration of an implementation of an Analytical Prediction
Unit in accordance with the teachings of the present invention.
FIG. 4 is an illustration of an implementation of the Comparators
illustrated in FIG. 1.
FIG. 5 is an illustration of the OR GATE of FIG. 1.
FIG. 6 is an illustration of an implementation of a history Data Collection
Unit in accordance with the teachings of the present invention.
FIG. 7 is an illustration of an alternate implementation of a History Data
Collection Unit in accordance with the teachings of the present invention.
DETAILED DESCRIPTION
In accordance with the present invention, apparatus and method are
disclosed which provide for the improved monitoring and detection of
improper device operation. Broadly stated, the expected operation of a
device in response to a selected set of test conditions is first
determined through the use of a model. The response of the device as
predicted by the model is then stored for subsequent comparison purposes.
Thereafter, the selected test conditions are applied to the device, and
the responses thereto are compared against the responses predicted by the
model. Improper device operation is thereby determined by comparing the
response of the device with the response predicted by the model.
For a general understanding of monitoring and detection apparatus
incorporating the teaching of the present invention, reference is now had
to FIG. 1. Engineering Prototype Unit 10 may represent any of a broad
range of devices for which the operation thereof is desired to be
monitored. Broadly stated, Engineering Prototype Unit 10 has a number of
types of signals associated therewith. A first type of signal associated
with Engineering Prototype Unit 10 represents signals containing
information associated with the operation of Engineering Prototype Unit
10. Signals of this nature are indicated in FIG. 1 by m signals 12. An
additional type of signal associated with Engineering Prototype Unit 10
are signals from which improper operation of Engineering Prototype Unit 10
may be determined. These signals are indicated generally in FIG. 1 as
signals 14, and collectively are comprised of signal 1 thru signal n. The
specific nature of m signals 12 and signals 14 associated with Engineering
Prototype Unit 10 will depend upon the specific functions performed by
Engineering Prototype Unit 10, and are consequently application dependent,
as will be more fully discussed hereinafter.
Broadly stated, Analytical Prediction Unit 16 contains information of a
predictive nature with respect to the operation of Engineering prototype
Unit 10, and functions to produce signals representative of the the
correct or desired signals produced by Engineering Prototype Unit 10 in
response to selected set of input conditions. As was the case with respect
to Engineering Prototype Uint 10, Analytical Prediction Unit 16 will
generally have a number of types of signals associated therewith. A first
type of signal associated with Analytical Prediction Unit 16 generally
represent information associated with the correct operation of Engineering
Prototype Unit 10. Signals of this nature are indicated generally in FIG.
1 by p signals 18. An additional type of signal associated with Analytical
Prediction Unit 16 are signals which may indicate the improper operation
of Engineering Prototype Unit 10. These signals are indicated generally in
FIG. 1 as signals 20, and are comprised of signal a thru signal z. The
specific nature of p signals 18 and signals 20 associated with Analytical
Prediction Unit 16 will depend upon the specific functions performed by
Engineering Prototype Unit 10, and are consequently application dependent,
as will be more fully discussed hereinafter. It should further be
understood that the signals 12 and 14 associated with Engineering
Prototype Unit 10 and signals 18 and 20 associated with Analytical
Prediction Unit 16 may vary in number depending upon the particular
implementation of Engineering Prototype Unit 10 and Analytical Prediction
Unit 16. Comparator 1 thru Comparator N represent a plurality of identical
comparing devices, each of which operates in the following manner. Each
comparator has two inputs signals and one output signal associated
therewith. The output signal can assume one of two possible states: a
first state indicating agreement between the two input signals to the
comparator, and a second state indicating disagreement between the two
input signals to the comparator. OR GATE 22 is a device having a plurality
of input signals and one output signal associated therewith, and functions
to produce an output signal in response to the appearance of a signal on
any one of the plurality of input signals thereto. The number of input
signals associated with OR GATE 22 is determined by the number of
Comparators, and is consequently determined by a particular application.
Broadly stated, History Data Collection Unit 24 functions as a storage
device. In particular, pursuant to a trigger signal 23 supplied thereto,
History Data Collection Unit 24 functions to store information from from
Engineering prototype Unit 10 associated with the operation thereof as
contained within m signals 12 as well as information from Analytical
Prediction Unit 16 representing correct operation of Engineering Prototype
Unit 10 as contained in p signals 18. The foregoing information is stored
by History Data Collection Unit 24 for subsequent analysis, as will be
discussed more fully hereinafter.
The foregoing described apparatus is configured in the following manner. A
set of pre-selected test vectors are simultaneously coupled to Engineering
Prototype Unit 10 and Analytica | | |
