A method for indicating the presence of speech in an audio signal

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

This invention relates to voice-triggered switching and more particularly to a method and apparatus for producing a speech indication signal in response to detection of voice information in the presence of extreme spurious background signals. A voice operated switch is useful for voice-triggered control of equipment such as telephone and radio transmitters as well as an element of a speech enhancement apparatus requiring separation of time frames containing speech from time frames containing undesired audio information in extremely noisy environments.

Prior voice operated switches have employed various techniques and primarily analog signal detection techniques.

Poikela U.S. Pat. No. 4,625,083 describes a two-microphone voice-operated switch (VOX) system which seems to suggest autocorrelation of signals in an analog sense through the use of a differential amplifier for comparing the signals from the two microphones. This technique is reminiscent of noise cancellation microphone techniques and is not particularly pertinent to the present invention.

Mai et al. U.S. Pat. No. 4,484,344 is a syllabic rate filter-based voice operated switch. It employs input signal conditioning through an analog low-pass filter to limit examination of signal content to below 750 Hz.

Luhowy U.S. Pat. No. 4,187,396 describes an analog voice detector circuit employing a syllabic rate filter. It uses a hangover time function operative as an envelope detector.

Jankowski U.S. Pat. No. 4,052,568 describes a digital voice switch using a digital speech detector and a noise detector operating on broad spectrum speech signals. It also teaches the hangover time function and dual threshold detection.

Sciulli U.S. Pat. No. 3,832,491 describes an early digital voice switch wherein a digital adaptive threshold is employed based on the number of times the amplitude of talker activity exceeds an amplitude threshold per unit time.

SUMMARY OF THE INVENTION

According to the invention, a voice operated switch employs digital signal processing techniques to examine audio signal frames having harmonic content to identify voiced phonemes and to determine whether a selected segment contains primarily speech or noise. The method and apparatus employ a multiple-stage, delayed-decision adaptive digital signal processing algorithm implemented through the use of commonly available DSP electronic circuit components. Specifically the method and apparatus comprise a plurality of stages, including (1) a low-pass filter to limit examination of input signals to below about one kHz, (2) a digital center-clipped autocorrelation processor which recognizes that the presence of periodic components of the input signal below and above a peak-related threshold identifies a time invariant frame as containing speech or noise, and (3) a nonlinear filtering processor which includes nonlinear smoothing of the frame-level decisions and incorporates a delay, and further incorporates a forward and backward decision extension at the speech-segment level.

The invention will be better understood by reference to the following detailed description taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an apparatus employing a voice operated switching means in accordance with the invention.

FIG. 2 is a block diagram of a preprocessor according to the invention.

FIG. 3 is a block diagram of a VOX processor in accordance with the invention.

FIG. 4 is a detailed block diagram of a first level decision means according to the invention.

FIG. 5 is a third level decision means according to the invention.

DESCRIPTION OF SPECIFIC EMBODIMENTS

The invention may be realized in hardware or in software incorporated in a programmed digital signal signal processing apparatus. For example, the voice operated switch may be realized as an element of other devices employing digital signal processing techniques. It is contemplated for specific applications that the invention is realized in a dedicated device constructed around a microprocessor such as a Motorola 68000 enhanced by supplemental digital signal processing components such as a TMS 320 Series device from Texas Instruments. Realizations employing other components are contemplated without departinq from the spirit and scope of the invention.

Referring to FIG. 1 there is shown a block diagram of a voice operated switch (VOX) controlled apparatus 10 illustrating the major functions of a voice operated switch according to the invention. The VOX controlled apparatus 10 comprises a signal conditioning means 12 coupled to receive audio signal input through an audio channel 14 and to provide controlled attenuation signals to the next stage. The next stage is an analog to digital converter (ADC) 16 for converting analog signals to digital samples. The output of the ADC 16 is coupled to a first in first out buffer (FIFO) 18 which adds a delay needed for reliable operation of subsequent stages. Outputs from the FIFO 18 are coupled to a preprocessor 20 and to a variable delay 22. The output of the variable delay 22 is coupled to a digital to analog converter (DAC) 24, the output of which is coupled to a channel switch 26. The output of the channel switch is provided to an output audio signal channel 30. When the voice operated switch control is invoked, voice switched audio is generated. Otherwise the audio channel simply passes a conditioned audio signal containing speech and noise.

Voice operated switching is implemented by processing information extracted by the preprocessor 20, the output of which is provided to a VOX processor 32. The preprocessor 20 and VOX processor 32 may considered together as constituting a voice operated switch. Two control outputs are provided from the VOX processor 32, a first or delay control output 34 and a second or speech decision control output 36.

Referring now in greater detail to the signal conditioner 12 in FIG. 1, the signal conditioner 12 is preferably an automatic gain control apparatus having approximately 50 dB dynamic range. For example the AGC may comprise an array of attenuators whose attenuation is controlled interactively based on estimates of the peak energy during signal intervals. The AGC may be more tightly controlled by basing the attenuation decision only on those intervals determined by the VOX processor to contain speech.

The ADC 12 may be a conventional linear 12-bit converter with an anti-aliasing filter or it may be an A-law or MU-law codec as employed in digital telephony. A sampling rate of 8000 samples per second is suitable for speech processing. The DAC 24 is for reconstruction of the analog signal for utilization and is of a form complementary to the form of the ADC 16.

The FIFO 18 is a digital delay line introducing a delay of approximately 1/4 second (250 ms). The preprocessor 20, as explained hereinafter, conditions the samples and groups them in an overlapping sequence of frames for use in the VOX processor 32. The VOX processor 32, as explained hereinafter, renders the speech/no-speech decision.

The variable delay 22 is provided to account for changes in parameters affecting the delay introduced by the VOX processor 32. The channel switch is closed by the VOX processor 32 to pass speech segments and is opened to block non-speech segments.

The apparatus of FIG. 1 is intended to be descriptive and not limiting as to specific features of the invention, and it illustrates one embodiment of a device considered to be a voice operated switch. The actual switching decision is incorporated into the elements designated as the VOX processor 32.

Referring to FIG. 2 there is shown a block diagram of a preprocessor 20 in accordance with the invention. The preprocessor 20 prepares the digitized input signal for processing in the VOX processor 32. According to the invention, the VOX processor 32 makes preliminary decisions on the presence of speech in an audio signal on the basis of pitch information in invariant voiced speech segments of about 16 ms duration, and then it accounts for limitations of this decision technique by compensating over extended look-forward and look-backward periods to provide for continuity and for leading and trailing unvoiced speech.

The preprocessor 20 comprises a low-pass filter 38, a down sampler 40, a center clipper 42 and a frame segmenter 44. The low-pass filter 38 is coupled to receive digital signals from an selected stage of the FIFO 18 and to pass a filtered digital signal to the down sampler 40. The down sampler 40 is coupled to the frame segmenter 44. The frame segmenter 44 output is coupled to the input of the center clipper 42. The output of the center clipper 42 is coupled to the input of the VOX processor 32 as hereinafter explained.

The low-pass filter 38 is a digital filter having a cutoff frequency of less than 1000 Hz and preferably of 800 Hz in order to improve signal-to-noise characteristics of the useful pitch in the spectrum of 50 Hz to 500 Hz where most of the pitch frequencies of a voiced phoneme are known to be in real-time conventional speech.

The down sampler 40 is a mechanism for decimating the resultant filtered signal. No longer is it necessary to retain a resolution of 8000 samples per second, since the effective bandwidth is only about 800 Hz. Hence the the down sampler 40 functions to discard for example three out of every four samples while retaining sufficient information on which to render the desired decision on a signal of the remaining bandwidth. The complexity of the signal processing is also thereby reduced. (However, the filtered but undecimated signal may be retained for use in selected precision processing, such as autocorrelation.)

The frame segmenter 44 implements a segmentation process in order to segment the stream of digital audio samples into useful processing frames. Specifically, the digital audio samples are assembled in the frame segmenter 44 into frames containing preferable 50% overlap between successive intervals. Frame length is selected to be 256 samples or 32 ms in length in the preferred embodiment. A frame level decision is generated every 16 ms. Because of the overlap the transitions to and from voiced speech segments are handled more smoothly, and second level decisions have available to them twice as many frame level decisions.

The center clipper 42 is a spectrum flattener operative to remove the effect of the vocal tract transfer function and to constrain each harmonic of the fundamental to approximately the same amplitude. The specific procedure comprises finding the peak amplitude during the first third of the segment (i.e., the 32 ms speech segment) and during the last third of the segment and then setting the clipping level at a fixed percentage of the minimum of these two measured maxima. The clipping level input 43, which is a parameter provided by the VOX processor 32 is preferably set to about 0.65 of the lower maxima. A detailed description of the center clipping technique is given in the book by L.R. Rabiner and R.W. Schafer, Digital Processing of Speech Siqnals, pp. 150-154, 1978, (Prentice-Hall, Inc, Englewood Cliffs, N.J. 07632).

To understand the need for a center clipper it is useful to review the classical model of speech generation. Speech generation is considered to involve an excitation of the vocal cords which causes vibration for voiced speech and "white-noise"-like sounds for unvoiced speech. When the vocal cords vibrate at the pitch frequency, they generate an impulse train at the pitch frequency which can be described in terms of a vocal tract transfer function introducing frequency selective attenuation. The corresponding power spectrum is concentrated primarily at discrete frequencies which are harmonics of the fundamental pitch frequency, and the envelope of the spectrum exhibits peaks and valleys. The peaks of the spectrum are known as "formant frequencies", and they correspond to the resonant frequencies of the vocal tract.

According to the invention, the VOX processor 32 capitalizes on the presence of pitch within voiced speech to render its decision about the presence or absence of speech within an audio signal. However, if the excitation or pitch is to be emphasized to enhance its detectability, it is preferable and believed necessary to remove the formant frequency structure from the speech spectrum prior to detection. In the particular type of VOX processor employed, a short-time autocorrelation function is used to detect for the periodicity of the pitch, so that other signal peaks in the voiced speech spectrum are extraneous and will cause false readings because the autocorrelation peaks due to periodic oscillation are higher than the autocorrelation peaks due to the periodicity of vocal excitation, particularly where the readings are based on selection of the highest peak in a segment. To minimize this problem it is desirable to process the speech signal so as to make the periodicity more prominent while suppressing the peaks due to other factors. Hence the spectrum flattening technique of a center clipper is employed according to the invention as explained hereinabove.

Referring to FIG. 3 there is shown a block diagram of a VOX processor 32 in accordance with the invention. The VOX processor 32 is best described in terms of the algorithms of the corresponding software implementation of the invention. The VOX algorithm employs first level decision means 50, second level decision means 52 and third level decision means 54. The first level decision means 50 operates on the single overlapping frame to estimate whether the frame is voiced speech in a first category or unvoiced speech, noise or silence in a second category. The first level algorithm employs pitch as an indicator to determine whether the input frame comprises (1) voiced speech V or tone T, or (2) unvoiced speech U or noise N or silence S, providing the binary decision to a first element 56 of the second level decision means 52. The first level decision means 50 also extracts pitch information P and supplies the extracted tone T to a delayed tone detector element 58 of the second level decision means 52. The first element 56 receiving the VT/UNS decision is a median smoother 56, that is, a nonlinear filter used for smoothing decisions and for passing decisions indicative of sharp, consistent transitions. The delayed decision tone detector 58 is a detector for detecting the presence of a constant frequency tone in the 50 Hz to 500 Hz range having a duration of more than several frames. The output of the median smoother 56 and the delayed decision tone detector 58 are coupled to a decision combiner 60 wherein the decision is made to block the voice decision if the tone output decision T of the tone detector 58 coincides with the voice/tone output decision VT of the median smoother 56.

The third level decision means 54 operates over several frames. Hence all second level decisions are stored in a decision storage means 62 to provide for the delay necessary for third level decisions. The decision storage means interacts with a decision extender/modifier 64 which provides the final speech or no speech decision for each overlapping frame. The decision extender/modifier 64 is intended to eliminate extremely short speech segments, indicative of false detection of speech, to extend second-level decision making such that unvoiced speech segments are included in the decision if adjacent to voiced speech segments, to fill in short silence gaps, and to provide hang-time delays and the like. A synchronizer 66 is employed to assure that equivalent delays are provided between the FIFO 18 and the VOX processor 32. The synchronizer 66 controls the variable delay 22.

Referring to FIG. 4 there is shown a detailed block diagram of a first level decision means 50 according to the invention. The first level decision means 50 comprises an autocorrelator (ACF) 68, an ACF normalizer 70, a positive peaks detector 72, an audio signal presence detector 74, a first peak decision processor 76, a second peak decision processor 78, a periodicity detector 80, a periodicity function processor 81, selected weighting functions 82, 84 and 86 and multipliers 88, 90 and 92, a summer 94 for summing the weighted combination of the outputs of the first peak decision processor 76, the second peak decision processor 78 and the periodicity function processor 80, a comparator 96 and a decisions combiner 98.

The autocorrelator 68 in the preferred embodiment is coupled to receive from the frame segmenter 44 of the preprocessor 20 a 32 ms long overlapping frame of 256 samples decimated to 64 samples, to calculate the non-normalized autocorrelation function between a minimum lag and a maximum lag and to provide the resultant autocorrelation function ACF(k), k=min,...,max, to the ACF normalizer 70 and the audio signal presence detector 74. The preferred minimum lag is 4, corresponding to a high pitch of 500 Hz, and the preferred maximum lag is 40, corresponding to a low pitch of 50 Hz. The ACF at lag zero (ACF(0)) is known as the "frame energy."

The audio signal presence detector 74 employs as a parametric input a minimum energy level (4-5 bits of a 12 bit signal) to detect for a "no audio" condition in the frame energy (ACF(0)). Indication of an audio/no audio condition is supplied to the decision combiner 98. This is the only stage in the decision process where signal level is a criterion for decision.

The ACF normalizer 70 receives the autocorrelator 68 output signal and normalizes the energy and the envelope. Energy normalization is effected by dividing the normalization function output for k=min lag to k=max lag by the frame energy ACF(0). Envelope normalization is effected by multiplication of the ACF by an inverse triangle factor which results in a rectangular envelope to the ACF instead of a triangular envelope rolloff characteristic of an ACF.

The positive peaks detector 72 detects for a preselected number of peaks in excess of a normalized threshold and then calculates more precisely the value of the ACF and the lag of each peak. A preferred normalized threshold is in he range of 0.1 to 0.2. The output, in the form of a list of peaks with ACF values and lags, is provided to the first peak decision processor 76, the second peak decision processor 78 and the periodicity detector 80

The first peak decision processor 76 receives as its input the value of the maximum ACF peak and renders a positive decision output if the value exceeds a preselected threshold P1MAX-T, indicating the presence of a pitch in the signal. A nonlinear function is applied to reflect the probability that pitch is present at various levels of P1MAX. Typical values for P1MAX-T is 0.4 to 0.6, with decreasing values increasing the probability of detection of speech and of false alarms.

The second decision processor 78 is an identical nonlinear function to the first decision processor 76 except that it receives as input the second highest ACF peak and uses as its threshold P2MAX-T between 0.35 and 0.55, that is, a threshold scaled for the second ACF peak.

The periodicity detector verifies the periodicity of the ACF peaks. For a voiced frame, the lags of the ACF peaks should form an arithmetic sequence with zero as the first element and the difference between each element in the sequence corresponding to the pitch period. A lag tolerance accounts for the difference between an ideal sequence and a detected sequence. The periodicity detector 80 provides as output the following values: (1) The theoretical number of peaks computed by dividing the maximum lag by the lag of the first peak (TNPKS); (2) The actual number of peaks forming an approximated arithmetic sequence (less the peak at zero lag) (ANPKS); and (3) a pitch period estimate or sequence difference. The pitch period estimate is passed to the pitch consistency detector (a tone detector) of the second level decision means 52 while the other values are provided to the periodicity decision processor 81.

The periodicity decision processor 81 accepts the above output parameters and assigns a value to each combination from a lookup table indicative of the probability that the signal received is periodic. No specific algorithm is applied in the preferred embodiment, as the values are primarily empirical corrections to the periodicity detector 80.

The outputs of each of the decision processors 76, 78 and 81 are soft decisions indicative of the probability that a voiced segment or a tone (pitch) has been detected. In order to enhance the flexibility of the resultant decision, there is associated with each soft decision a weighting coefficient 82, 84 and 86 which respectively weights the value of the soft decisions by multiplication through multipliers 88, 90 and 92 of the respective outputs. The respective outputs are summed at the summer 94 and supplied to the comparator 96 whose threshold is preferably set to zero. Thus, if the result is positive, the indication is the presence of pitch in the signal.

The final first level decision stage is the decision combiner 98. It combines the pitch decision with the audio/no audio decision of the signal presence detector 74. If there is no audio present, then the output of the first level decision means 50 is UNS (no voice or tone) no matter what the total output of the summer 94 is. However, the VT/UNS decision as well as the pitch estimate are passed to the second level decision processor 52.

Referring again to FIG. 3, there are shown the principal elements of the second level decision means 52. The median smoother 56 looks at a given odd number of previous first level decisions and determines which of the two states is in the majority. It provides as its output a state which represents the state of the majority of the previous given odd number of the first level decisions. Thus, it is operative to eliminate noise-induced short term transitions. A median smoother of this type is in accordance with that described by L.R. Rabiner and R.W. Schafer, Digital Processing of Speech Signals, pp. 158-161, 1978, (Prentice-Hall, Inc, Englewood Cliffs, NJ 07632).

The pitch estimate is supplied to the tone detector 58 or more precisely to a pitch consistency detector 58 having as parametric inputs the consistency tolerance and the window width. If the pitch estimate is within the consistency tolerance for a duration longer than a fixed minimum tone duration, then a tone presence decision T is issued to the decision combiner 60.

The decision combiner 60 of the second level decision means 52 combines the smoothed output of the median smoother 56 and the Tone decision T of the tone detector 58 to generate a signal indicating that the signal is a voiced signal V or unvoiced, noise or silence (UNS), suppressing specifically frames containing tones. The V/UNS decision is provided to the decision storage means 62 of the third level decision means where speech-segment-level decisions are rendered.

Referring to FIG. 5, there is shown a portion of the third level decision means 54 comprising the decision storage means 62 and the decision extender/modifier 64. As previously explained, all frame decisions are captured and stored for a period of time in the decision storage means 62. Several speech-segment-level decision processes are performed on the accumulated data. First a short voice segment tester 100 is provided for deleting or changing to a UNS decision all V segments whose duration is shorter than a preselected minimum kV.

An initial backward extension 102 and a final backward extension 104 are provided for testing the backward extension in time of all voice decisions V. The purpose is to include with voiced speech segments any related unvoiced speech segments which may precede and should be passed with the speech decision. A typical extension is 5 to 10 frames. (Since the sum of the initial backward extension time and the final backward extension time have a direct impact on the time delay, care must be taken to avoid long times if a short VOX hang is desirable.)

An initial forward extension 106 and a final forward extension 108 are provided for testing the forward extension in time of all voice segments V. The purpose is to include with speech segments the any related unvoiced speech segments which may trail and should be passed with the speech decision, as well as to provide a limited amount of hang between words and sentences. The initial forward extension parameter is typically 5 frames. (Forward extensions have no impact on VOX time delay.)

A short silence interval tester 110 is also provided to convert silence intervals shorter than a preselected length kS to voiced decisions V.

The final backward extension is set typically in the range of zero to up to 15 frames. The parameter is selected on the basis of the allowable overall time delay.

The final forward extension is set to a minimum of ten frames to ensure the inclusion of unvoiced speech following detected voiced speech. The maximum is limited only by the available memory. Values of 500 ms to up to three seconds are considered sufficient for contemplated applications.

In order to augment the understanding of the invention, an appendix is provided containing schematic flow charts of the processes involved together with a step by step explanation of the processes of a specific embodiment of the invention.

The invention has now been explained with reference to specific embodiments. Other embodiments, including realizations in hardware and realizations in other preprogrammed or software forms, will be apparent to those of ordinary skill in this art. It is therefore not intended that this invention be limited except as indicated by the appended claims. ##SPC1##

* * * * *