A method of operating a computer system to store and retrieve information in a database uses a signature file of the database that is divided into subsets. A word signature is mapped to a particular subset during creation of the file and the same mapping information is used to retrieve the information in response to a query word. Each word signature is a logical word signature and has two components a physical word signature and a subset designation field. In this way, when information is retrieved from the database, only that subset containing the relevant word signature is scanned. The signature file is automatically created by the system as the database is stored on the data storage modules. During retrieval, the control reviews information received from the data storage means and if a match occurs between a physical word signature for a query word and a particular physical word signature arriving from the input section, the control sends the physical word signature to the FIFO buffer in memory together with the document identifier located subsequent to the matched physical word signature. The control then moves on to process the next physical word signature received from the data storage means. If there is no match, the control ignores the physical word signature and moves on to process the next physical word signature received from the data storage means. The control is effectively capable of processing several query words in parallel.
A neighboring plural-character occurrence bitmap of a practical capacity capable of eliminating noises by hashing is realized, and a high speed full text search is realized equivalently, by greatly reducing the number of documents to be searched even if a search term constituted by a combination of English characters and words is used. Text data is segmented into words, and n-character strings at every (m+l)-th character positions are extracted from each word. A neighboring plural-character occurrence bitmap is created which stores data representing a presence of each neighboring plural-character string at a certain entry thereof. N-character strings at every (m+l)-th character positions are extracted from a search term and the neighboring plural-character occurrence bitmap is searched by using a search control program. Since the neighboring plural-character occurrence bitmap is searched prior to searching condensed texts, documents not relevant to the search term can be discarded and a high speed full text search can be realized.
A search method selecting unit included in a pattern search apparatus can automatically select a search method with the highest search speed from a plurality of search methods according to the number of search patterns, form, etc., in order to find a search pattern in a search target. An optimum search or replacement can be realized by finding or replacing the search pattern using the selected method.
A system and method for allocating the blocks of index file to the postings for words found in documents of a database is disclosed. The index file is provided with blocks that is partitioned into successively decreasing levels of blocks in size. The blocks in each successive level have same size and the sum of sizes of blocks in each successive level equals the size of initial block. An information retrieval interface allocates to the postings for a word a free block in the level the size of which is closest to the size of postings for the word in the index file.
This invention encodes information (such as the field values of a database record, or the words of a text document) so that the original information may be efficiently searched by a computer. An information object is encoded into a small "signature" or codeword using a method. A base or "leaf" signature S1 34 is computed by a known technique such as hashing. The logical intersection (AND) of each possible combination of pairs of bits of the base signature is computed, and the result is stored as one bit of a longer combinatorial signature CS1 42. The bit-wise logical union (bit-OR) of the combinatorial signatures of a group of records produces a second-level combinatorial signature CS2 52 representing particular field values present among those records. Higher-level combinatorial signatures CS3 60, CS4, etc. are computed similarly. These combinatorial signatures avoid a "saturation" problem which occurs when signatures are grouped together, and a "combinatorial error" problem which falsely indicates the existence of nonexistent records, thereby significantly improving the ability to reject data not relevant to a given query. When the combinatorial signatures are stored in a hierarchical data structure, such as a B-tree index of a database management system, they provide means for more efficiently searching database records or document text by eliminating large amounts of nonmatching data from further consideration.
A parallel query manager accepts a list of file extents to be searched and produces a number of search lists, one for each disk to be searched. The query manager first uses a mapper to find out how the database spaces are stored on disk. It then matches the search extent list with the mapping information to determine which parts of which disks are to be searched. It then initiates several searches in parallel so that all the affected disks can be kept busy at the same time. The query manager then checks for return data on each stream, and merges the results.
