The present invention recites a method and computer program product for identifying and correcting systematic noise in a pattern recognition classifier. A plurality of input patterns that have been determined not to be associated with any of a set of at least one represented output class are rejected by the pattern recognition classifier. A subset of pattern samples is selected from the rejected input patterns based upon the similarity of each pattern to one of the represented output classes. The selected pattern samples are subjected to an independent review to determine if they were correctly rejected. The classifier is retrained based upon this independent review of the selected pattern samples.
