Method and system for producing from document image a form display with blank fields and a program to input data to the blank fields

We claim:

1. A method of producing on a screen a form display including at least one blank field in which characters are to be written, the method comprising the steps of:

recognizing from an original form image a physical layout of a plurality of character patterns and a plurality of line patterns constituting a form;

converting the layout of the form recognized in said recognizing step into a second layout of the form including a blank field modified to correspond to an integral multiple of a basic pitch of a cursor movement responsive to input operations of characters on a display screen; and

presenting on the display screen a form display having the second layout obtained in said converting step;

wherein a character string is displayed in said blank field of the form display, said character string inputted from an input device, with a predetermined character size at a position associated with the cursor pitch.

2. A method according to claim 1, further including the steps of:

presenting on the display screen a form display comprising the character and line patterns in the physical layout recognized in said recognizing step; and

correcting in response to a user's input operation a portion of the character or line pattern included in the form display presented on the display screen;

said converting step conducting the layout conversion on the character and line patterns corrected in said correcting step.

3. A method according to claim 1, further comprising a step of converting a form document into image data by use of an image input means,

said recognizing step being executed on the image data as said original form image.

4. A method according to claim 3, further including the steps of:

detecting an inclination of the form document based on the image data supplied from the image input means; and

correcting the inclination, said recognizing step being executed on the image data undergone the inclination correction as said original form image.

5. A method according to claim 1, wherein said recognizing step comprises:

extracting, from a plurality of character patterns included in said original form image, at least one character string constituted by a plurality of adjacent characters; and

comparing a group of standard words or terms stored beforehand in a memory with the extracted character string to automatically correct a part of the extracted character string so as to match the character string with one of said standard words or terms,

said converting step executing the layout conversion on the character and line patterns thus corrected.

6. A method or producing a form document, the method comprising the steps of:

recognizing from an original document image, a layout of character and line patterns constituting a document, and storing a recognition result as physical structure data of the document;

converting the layout of the character and line patterns recognized in said recognizing step into a second layout of a form including at least one blank field modified to correspond to an integral multiple of a basic pitch of a cursor movement responsive to an input operation of character or numeric data on a display screen and storing a conversion result as normalized document data;

presenting on the display screen a form display having the second layout created from the normalized document data;

displaying characters or numerals inputted by the user in said modified blank field in the form display; and

combining the data of characters or numerals inputted by the user in said displaying step with the physical structure data of the document and outputting a combined result to a printer, to thereby produce a form document filled with data.

7. A method according to claim 6, further including the steps of:

presenting on the display screen a form display comprising the character and line patterns corresponding to said physical structure data obtained in said recognizing step;

correcting in response to a user's key operation a portion of the physical structure data and the character or line pattern included in the form display presented on the display screen; and

said converting step conducting the layout conversion on the character and line patterns represented by said physical structure data thus corrected.

8. A method according to claim 6, further comprising a step of converting a form document into image data by use of an image input means,

said recognizing step being executed on the image data as the original document image.

9. A method according to claim 8, further including a step of detecting an inclination of the form document based on the image data supplied from the image input means and correcting the inclination,

said recognizing step being executed on the image data which was subject to the inclination correction as the original document data.

10. A method according to claim 6, further including the steps of:

extracting, from a plurality of character patterns included in said original document image, at least one character string comprising a plurality of adjacent characters;

comparing a group of standard words or terms stored beforehand in a memory with the extracted character string to automatically correct a character pattern constituting part of the character string so as to match the character string with one of said standard words or terms;

said converting step executing the layout conversion on the layout of character and line patterns thus corrected.

11. A method of inputting data by use of a form display having a plurality of blank input/output fields in which data items are to be set, the method comprising the steps of:

recognizing a layout of character and line patterns constituting a form from an original form image to store in a memory a recognition result as physical structure data of the form;

analyzing the physical structure data to detect data input/output fields included in the form and item names associated with the respective data input/output fields;

referencing, based on the detected item names, a knowledge base to determine for each said input/output field a field attribution indicating an output source or a generation rule of data to be set thereto;

presenting on a display screen a form display created based on the physical structure data; and

achieving, for each said data input/output field included in the form display presented on the display screen, data processing associated with the field attribution thereof to set in the data input/output field a data manually supplied from the user or a data automatically generated.

12. A method according to claim 11, wherein said referencing sep determines whether the field attribution of an input/output field indicates that input data from a keyboard is set to the field, that data read from a data file is set to the field, or that data obtained through a calculation using data set at least to one of said input/output fields is set to the field.

13. A method of automatically generating a program for setting data in a form display having a plurality of blank input/output fields to be loaded with data, the method comprising the steps of:

recognizing from an image of a form document having a plurality of input/output fields, physical structure data denoting a layout of character and line patterns constituting a form document;

analyzing the physical structure data to detect a plurality of input/output fields for use in data input/output operations and at least one fixed field including therein a character string indicating an item name;

examining a positional relation between the fixed field and the input/output fields to determine for each said input/output field an item name of data to be set thereto;

determining for each input/output field, based on the item name of data and a knowledge base, a field attribution indicating an input source or a generation rule of data to be set in the input/output field; and

automatically generating a program for the user to input data to blank input/output fields of the form display presented on the display screen based on the field attributions and the physical structure data.

14. A method according to claim 13, further including the steps of:

producing from the physical structure data normalized form data having a layout of the character and line patterns converted to be adaptive to an integral multiple of a basic pitch of a cursor movement responsive to an input operation of characters on the display screen,

said analyzing step being conducted such that, in place of the physical structure data, the normalized form data is analyzed for the field recognition.

15. A method according to claim 13, wherein said analyzing step further includes the steps of:

extracting a plurality of cell regions each including a rectangular area defined by at least three side liens formed with line patterns;

determining a rectangular area in each of said cell regions as an input/output field or as a fixed field by detecting the presence or absence of any character string in each said cell region and/or by analyzing a content of the detected character string; and

detecting a particular word outside the cell region to identify a blank area adjacent to the word as an input/output field.

16. A method according to claim 14, wherein said analyzing step further includes the steps of:

extracting a plurality of cell regions each including a rectangular area defined by at least three side lines formed with line patterns;

determining a rectangular area in each of said cell regions as an input/output field or as a fixed field by detecting the presence or absence of any character string in each of said cell regions and/or by analyzing a content of the detected character string; and

detecting a particular word outside the cell regions to identify a blank area adjacent to the word as an input/output field.

17. A data processing system for inputting data to a form display presented in a display screen comprising:

first memory means for storing therein image data of a form having a plurality of blank fields in which character or numeric data are to be placed;

display means provided with a display screen for presenting the form display;

input means for inputting character and/or numeric data; and

control means for achieving a control operation such that data supplied from said input means is inputted to a blank field indicated by a cursor in the form display presented in the display screen;

said control means including:

means for recognizing based on the form image stored in said first memory means a layout of a plurality of character and line patterns constituting a form and for producing physical structure data of the form; and

means for producing normalized form data from the physical structure data of the form by modifying the layout of the character and line patterns to correspond to an integral multiple of the basic pitch of the cursor movement responsive to an input operation of character or numeric data on the display screen,

the normalized form data defining a form display to be presented on the display screen for a data input operation.

18. A system according to claim 17, further including a printer apparatus wherein:

said control means combines data inputted to the blank fields of the form display with a format defined by the physical structure data to supply the combined data to the printer apparatus.

19. A system according to claim 17, further including image input means for converting an image of a form to image data, wherein

the image data supplied from said image input means is stored in said first memory means.

20. A data processing system for inputting data to a form display presented in a display screen comprising:

first memory means for storing therein image data of a form document having a plurality of blank fields for setting therein character or numeric data;

display means provided with a display screen for presenting a form display;

input means for inputting character and/or numeric data;

second memory means for storing therein a plurality of data records;

knowledge base means loaded, for each word or term, with knowledge information of a plurality of items;

control means for achieving a control operation of data inputs to blank fields in the form display presented in the display screen;

said control means including:

first means for recognizing based on the form document image stored in said first memory means a layout of a plurality of the character and line patterns constituting a form document and for producing physical structure data of the form document;

second means for analyzing the physical structure data to detect data input fields included in the form document and item names associated with said data input fields;

third means for referencing based on the item names the knowledge base and for respectively detecting for said input field attributions indicating input sources or generation rules of data to be set to the input fields; and

fourth means for automatically producing based on the field attributions a program for inputting data to blank data input fields of the form display defined by the physical structure data and presented on the display screen, wherein

when the program is executed, data inputted from said input means by the operator, data of either one of the items included in a data record read out from said second memory means, and data obtained through a calculation are selectively set to the blank input fields of the form display.

21. A system according to claim 20, wherein:

said control means further includes fifth means for generating normalized data from the physical structure data of the form document by automatically correcting the layout of the character and line patterns so as to be adaptive to an integral multiple of the basic pitch of the cursor movement responsive to an input operation of character or numeric data on the display screen;

said second means analyzes the normalized data in place of the physical structure data; and

said fourth means automatically generates a program for inputting data to blank data input fields of the form display defined by the physical structure data and presented on the display screen.

22. A system according to claim 21, further including image input means for receiving as an input thereto an image of a form document, wherein

the form document image supplied from said image input means is stored in said first memory means.

23. A system according to claim 21, further including a printer means for producing a form document, wherein:

said control means for editing data inputted to the blank fields of the form display with a document format defined by the physical structure data, thereby supplying the edited data to the printer means.

24. A method of automatically generating a program for setting data in a form display having a plurality of blank fields to be filled with data, the method comprising the steps of:

recognizing from an image of a form document having a plurality of fields, physical structure data denoting a layout of the form document;

analyzing the physical structure data by referring to a knowledge base to obtain logical structure data denoting attributions of respective fields in said document and relations between said fields; and

automatically generating based on said logical structure data and said physical structure data a program for presenting a form display on a display screen and for setting data in blank fields of said form display in response to an user's data input operations.

25. A method according to claim 24, the method further comprising the steps of:

presenting on the display screen the recognized data input fields with a style visually distinctive from the other portions of the form display, thereby to amend the form display by user operations if an error exists in a result of automatic recognition for the logical structure data.

26. A method according to claim 24, the method further comprising the steps of:

normalizing the layout of the form document such that each of blank fields has a size adaptive to an integral multiple of a basic pitch of a cursor movement responsive to input operations of characters or numerals on the display screen, wherein said program is generated so that data input operations by the user are carried out on the display screen presenting a form display having the normalized layout.

27. A method according to claim 24, the method further comprising the steps of:

producing a hard copy of a document including a layout of document generated from the physical structure data and having a plurality of fields filled with data inputted by the user operation on the display screen.

28. A method according to claim 24, the method further comprising the steps of:

presenting on the display screen a layout of the form display generated from said physical structure data obtained in said recognizing step; and

correcting the physical structure data in response to user operations to change a part of the form display presented in the display screen,

wherein said analyzing step and generating step are executed on the physical structure data thus corrected.

29. A method according to claim 24, the method further comprising the steps of:

normalizing the layout of the form document such that each of blank fields has a size adaptive to an integral multiple of a basic pitch of a cursor movement responsive to input operations of characters on the display screen, wherein said program is generated such that data input operations by the user are carried out on the display screen presenting a form display having the normalized layout.

30. A method according to claim 24, the method further comprising the steps of:

producing a hard copy of a document including a layout of document generated from the physical structure data and having a plurality of fields filled with data inputted by the user operation on the display screen.

Description Submit all comments and votes

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application relates to the Japanese patent application Ser. No. 59-180517 (Aug. 31, 1984) as the U.S. patent application Ser. No. 485986 (Feb. 27, 1990) which is continuation application of the U.S. patent application Ser. No. 399411 (Aug. 25, 1989) which is continuation application of the U.S. patent application Ser. No. 178768 (Mar. 29, 1988) which is continuation application of the U.S. patent application Ser. No. 766943 (Aug. 19, 1985) and the Japanese patent application Ser. No. 63-209975 (Aug. 24, 1988) as the U.S. patent application Ser. No. 397117 (Aug. 21, 1989).

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a document data processing apparatus employing image data, and in particular, to an apparatus for and a method of processing data for producing, for example, in general business processing fields, a document of a predetermined format or form having blank or empty cells for character data input and/or output operations. Cells are provided, to input data from a keyboard to a data base or to output data from the data base or the keyboard.

2. Description of Related Art

There exists a case where a business processing apparatus used in an office such as a work station, an office processor, a personal computer, or a word processor is adopted to issue documents (forms) having predetermined formats, for example, application blanks (or forms), data input slips, chits, vounchers, and other forms for various items such as accounting, statistic, and detailed items and/or to input data to a data base in accordance with formats preset on the document forms displayed. In such a situation, document formats or forms having blank fields or cells are required to be defined on a display screen of the apparatus for the data input/output operations. In a typical format or form definition method of conventional technology described, for example, in Hitachi's Manual entitled "ETOILE/OP", a cursor or the like is moved on a display screen to define ruled lines and characters. Thereafter, a program which issues the defined items is produced and is edited by use of a program editor or other programming tools.

However, in accordance with the conventional technology, in order to issue a new document in which the format is unknown to the apparatus, a long period of time is required by an expert is necessary knowledge and with respect to the program. Such a new format of document appears not only at an initial installation of the business equipment in a company but also in a rather routine manner. For example, a firm related to the pertinent company may request a specified slip or form to be used. Consequently, it has been desired to simplify the format definition job or processing and to increase the speed of the processing.

In this regard, the applicant of the present invention has been proposed apparatus in the Japanese patent application Ser. No. 63-209975 (U.S. patent application Ser. No. 397117). In this reference, a document processing system is provided where a form document having blanks or blank fields in which data items are to be described is transformed into image data such that based on the document image, a structure of a table form included in the document is recognized to automatically create ruled line vector data items, which are employed to draw a table. Moreover, the applicant of the present invention has been proposed in the Japanese patent application Ser. No. 59-180517 (U.S. patent application Ser. No. 766943) where a document analysis system in which form data for a character recognition by means of an optical character recognition (OCR) equipment is automatically generated from document image data.

In the document analysis system, the document image data is processed to recognize a physical structure of a table form constituted with line patterns included in the document. Thereafter, the system analyzes kinds, attributes or attributions, and relations of dependence with respect to rectangular cell regions (which are called frames in the preceding application) constituting the table form, thereby recognizing a logical structure of the document. However, in the conventional technology, the logical structure is recognized to apply a recognition result to the OCR. Namely, the analysis is limited to the cell types (indicating a cell for setting therein an item name or a cell to be loaded with a data item), the cell attributions (kinds and allowance ranges of characters loaded in cells), and relations of dependence between cells.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a method of and a system for processing form document data in which the operator references a form display presented on a screen to easily attain document form information suitably associated with data key-in operation to supply data to the data processing system.

Another object of the present invention is to provide a method of and a system for processing form document data suitable for an operation in which the data supplied from the operator and/or data obtained from a file are filled in predetermined blanks of a form of a document, thereby printing or displaying the document on a print form or a screen.

Still another object of the present invention is to provide a method of and a system for automatically generating a program associated with a form document processing suitable for creating a data processing program through a simple operation of the operator. The program is executed when the operator inputs data to a data processing system while referencing a form display of a predetermined format having blanks (data input/output fields) for the data input/output operations.

In order to achieve the above objects, there is provided a document processing system or apparatus in accordance with the present invention comprising an image input means for inputting an image of a table form document having a table form or format with input/output field cells in which variable data items are to be written. Fixed field cells are provided in which fixed data or item names have been written. A key-in means (which may be replaced with a pointing device such as a mouse) for the operator to input commands and data items. Further a data processor is provided for executing the data processing operation according to the programs. In this system, the data processor recognizes, based on the image data of the table form document supplied via the image input means, a physical structure and a logical structure of the input document. In the physical structure recognition, the processor recognizes such items included in the document as line segments constituting the table form and characters in the inside and outside regions of the table, thereby converting the image data into code data. The logical structure recognition accomplishes a field identification for identifying an input/output field in which variable data is to be inputted and a fixed field loaded with fixed data or a item name; a field relation recognition for establishing a correspondence between an input/output field or a fixed field and a field name, and a field attribution recognition for referencing a knowledge base to recognize a data supply source containing data to be inputted to an input/output field. In accordance with the present invention, for each input/output field, a source from which data is supplied thereto is recognized through a logical structure of the input document. Consequently, when a form document image having blanks for items such as "Item code", "Item name", "Unit price", "Quantity", and "Amount" is processed and then the operator inputs only an item code and a quantity value, the other blanks can be automatically loaded with data items attained from a data file or calculation results, thereby implementing a data input and/or document output processing system. In addition, by using the results of the physical and logical structure recognition of the document, a program can be automatically generated for the data input and/or document output processing.

Another feature of the present invention resides in that as a logical structure recognition of a document, based on data items representing results of the physical structure recognition on the document, a size of a table form along a row direction and a column size of each input/output field are corrected to be adaptive to a basic pitch or a character pitch of a cursor on a display screen so as to develop a normalized form display. In addition to the document structure data, the normalized form display suitable for the data input on the screen is created. This facilitates the operator's key-in operation for a data input and a screen output of data items obtained from a keyboard or through a program processing.

In this connection, sizes of characters to be written in the respective input/output fields of the form document, numbers of the respective characters, and character pitches associated with the characters can be determined in association with a physical size of each input/output field and a type of data set to the input/output field (decided by a field name assigned to the input/output field). Information items respectively designating the character sizes, character counts, and character pitches are stored as a portion of field attributes or attributions. The normalization processing of the row and column above is achieved by referencing the field attributions.

Character data set to an input/output field in the form display is matched or aligned with a document format defined by physical structure recognition data of an input document in association with the character size and the character pitch defined as the field attributions. This in return allows printing of the data items on a print form or sheet by use of a printer. As a result, the data items are printed on a print form identical to the input document form, which is different in the character size and the form display presented on the screen at the data input operation, with the blanks filled with characters of sizes associated with the field sizes.

In accordance with the present invention, the document data items attained through the physical structure recognition are employed to produce on a display screen a form display in which ruled lines can be moved or deleted and/or characters altered, added, or deleted through key operations. Consequently, the operator can perform an operation to correct a portion of the form display on the display screen, thereby causing the logical structure recognition and the automatic program generation to be accomplished.

The foregoing and other objects, advantages, manner of operation and novel features of the present invention will be understood from the following detailed description when read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system configuration diagram schematically showing an overall constitution of a document processing system in accordance with the present invention;

FIG. 2 is a flowchart showing the main operation steps of the document processing program executed in the document processing system;

FIG. 3 is a schematic diagram showing an example of a document inputted as an image to the system;

FIG. 4 is a diagram showing an example of a document produced from the system;

FIG. 5 is a flowchart showing a practical example of the document processing program;

FIG. 6 is a flowchart of a subroutine executed in an image input step 1;

FIGS. 7A and 7B are diagrams each for explaining relationships between an input image and an input image memory;

FIG. 8 is a flowchart showing a first embodiment of a subroutine executed in a document form or format recognition step 2;

FIGS. 9a-b are a flowchart showing details of a subroutine executed in a physical structure recognition step 21;

FIG. 10 is a flowchart showing in detail a block division step 211 of FIG. 9;

FIGS. 11 A and 11B are diagrams respectively showing configurations of tables T1 and T2 employed in the block division step 211;

FIG. 12 is a schematic diagram showing an example of an image for specifically explaining the block division;

FIG. 13 is a flowchart showing details of a region division step 214;

FIGS. 14A to 14C are explanatory diagrams useful for concretely explaining an execution process of the physical structure recognition step 21;

FIG. 15 is a flowchart showing in detail the region kind recognition step of FIG. 9;

FIGS. 16a-b are a flowchart showing details of a left side line recognition step 216-1 of FIG. 15;

FIG. 17 is a diagram schematically showing configuration of a table T3 used in the left side line recognition step 216-1;

FIGS. 18A and 18B are diagrams showing concrete examples respectively of an original image and a recognition result processed in the left side line recognition step 216-1;

FIG. 19 is a schematic diagram showing a table TBI1 to store therein recognition results of line segments;

FIGS. 20A to 20F are diagrams respectively showing typical patterns of non-cell elements;

FIG. 21 is a schematic diagram useful to explain processing in which a circular arc pattern is detected from an input image;

FIG. 22 is a configuration diagram showing the constitution of a table TBL2 for storing therein oblique line information;

FIG. 23 is a configuration diagram showing the constitution of a table TBL3 for storing therein circular arc information;

FIG. 24 is a detailed flowchart showing the operation of a construction element recognition step 218 of FIG. 9;

FIG. 25 is a diagram for explaining a type judgement conducted on construction elements;

FIG. 26 is a detailed flowchart showing the operation of a straight line recognition step 2181 of FIG. 24;

FIG. 27 is a schematic diagram for explaining conditions related to construction elements of a broken line.

FIG. 28 is a flowchart showing in detail a broken line recognition step 2182 of FIG. 24;

FIG. 29 is a diagram illustratively showing the constitution of a broken line table TBL4;

FIG. 30 is a flowchart showing details of a character recognition step 2183 of FIG. 24;

FIG. 31 is a schematic diagram for explaining conditions associated with constituent elements of a character string;

FIGS. 32A to 32C are diagrams for explaining processing steps of the character recognition step 2183.

FIG. 33 is a configuration diagram of a table TBL5 for storing therein character recognition results;

FIG. 34 is a flowchart showing another embodiment of the document form or format recognition step 2;

FIG. 35 is a flowchart showing details of a preprocessing step 20 of FIG. 34;

FIGS. 36A and 36B are schematic diagrams showing an example of an input image processed in the preprocessing step 20;

FIG. 37 is a detailed flowchart of a post-processing step 29 of FIG. 34;

FIG. 38 is a detailed flowchart of a line correction processing step 291 of FIG. 37;

FIG. 39 is a detailed flowchart of a character correction processing step 292 of FIG. 37;

FIG. 40 is a diagram showing the construction of table TBL6 for storing therein word information;

FIG. 41 is a flowchart showing another embodiment of the document form recognition step 2;

FIG. 42 is a flowchart showing a first embodiment of a logical structure recognition step 22 of FIG. 41;

FIG. 43 is a detailed flowchart showing a row and column normalization step 221 of FIG. 42;

FIG. 44 is a diagram showing the configuration of a table TBL7 employed in the row and column normalization step 221;

FIGS. 45A and 45B are diagrams useful to explain base points of patterns associated with characters and the like;

FIGS. 46A and 46B are flowcharts respectively showing alternative embodiments of the logical structure recognition step 22 of FIG. 41;

FIGS. 47a-b are a flowchart showing in detail a field position recognition step 222 of FIG. 46;

FIGS. 48A and 48B are diagrams for explaining a table format or form normalization processing 2221 executed to facilitate the field position recognition;

FIG. 49 is a diagram showing the construction of a field table TBL8;

FIGS. 50A and 50B are diagrams each useful to explain an example of an execution result of the field position recognition;

FIGS. 51A and 51B are flowcharts respectively showing alternative embodiments of the logical structure recognition step 22 of FIG. 41;

FIG. 52 is a flowchart showing details of a field relation recognition step 223 of FIG. 51;

FIG. 53 is a diagram showing the structure of a table T4 for storing therein unit information;

FIG. 54 is a diagram illustratively showing an example of field relation recognition results;

FIGS. 55a-b are a flowchart showing in detail of a field attribution recognition step 224 of FIG. 51;

FIGS. 56 to 59 are diagrams respectively showing various tables KNW1 to KNW4 referenced in a name matching step 2242 of FIG. 55;

FIGS. 60 to 65 are diagrams respectively showing various attribution information setting tables TBL9 to TBL14 referenced in the field attribution recognition step 224;

FIGS. 66a-b are a diagram showing a concrete example of contents of tables TBL9 to TBL14 generated through the logical structure recognition of the document;

FIG. 67 is a flowchart showing a third embodiment of the document form recognition step 2;

FIG. 68 is a flowchart showing in detail a document construction program generation step 23 of FIG. 67;

FIG. 69 is a diagram schematically showing an example of a program created in the generation step 23;

FIG. 70 is a detailed flowchart of an input/output field processing generation step 235 of FIG. 68;

FIG. 71 is a flowchart showing a first embodiment of the document construction or generation step 3 of FIG. 2;

FIG. 72 is a detailed flowchart of a print or output form data generation step 39 of FIG. 71;

FIGS. 73A and 73B are diagrams showing details of the operation to generate print or output from data in the bit mat format;

FIGS. 74A and 74B are diagrams useful to explain the operation to generate print or output form data in the command sequence format;

FIG. 75 is a flowchart showing another embodiment of the document construction step 3;