System and method for automatic data file format translation and transmission having advanced features

Claims

What is claimed:

1. In a distributed computer system including a data provider section, a main processing section, and a data subscriber section, each said section selectably couplable to another section by a communications link including communications control means, a method for translating and transferring data from a provider source format to a subscriber destination format using a priori knowledge of said source and destination formats between said data provider section and a said data subscriber section, said method comprising the steps of:

(A) transmitting, by said provider section, a particular provider data file from said provider section to said main processing section via said communications link, said transmitting including the steps of:

monitoring communications from said provider section;

recognizing receipt of a valid communications initiation prompt generated by said provider section communications control means according to predetermined communications initiation rules, said rules providing safeguards for ensuring the quality of the data delivered to valid data subscribers;

initiating connection of provider section with main processing section in response to receipt of said valid communications initiation prompt;

connecting said provider section with said main processing section and upon being validated as a valid provider section, receiving clearance from said main processing section to transmit a copy of said provider data file to said main processing section;

reading said provider data file;

transmitting said provider data files over said communications link to said main processing section; and

determining whether said provider data file has been transmitted successfully and signaling successful data transmission to said main processing section; and

(B) receiving, by said main processing section, said provider data file from said provider section and storing said received data files in said main processing section;

(C) translating, by said main processing section, said stored provider data file by: (i) copying said stored provider data file to a work area file within said main processor section, and (ii) converting said work area file from the native format of said provider data file to a midformat file format;

(D) forming a communications link, by said main processing section, with a particular one of said subscriber sections and transmitting said midformat file to said subscriber section;

(E) receiving, by subscriber section, said midformat file from said main processing section and storing said received file in storage within said subscriber section; and

(F) translating using said selected second-to-third file translation procedure, by said subscriber section, said midformat file into a subscriber destination format file compatible with said preselected subscriber application software routine.

2. The method in claim 1, wherein said step (A) of transmitting, further includes, prior to said step of monitoring, the steps of:

appending a subscriber destination address to said provider data file uniquely representative of a specific subscriber that the data in said particular data file is intended to ultimately reach; and

loading said provider data file with said appended destination address into an outgoing data box associated with said provider within said provider section.

3. The method in claim 2, wherein said step (A) of transmitting, further includes, after said step of determining whether said provider data file has been transmitted successfully and signaling successful data transmission to said main processing section, the steps of:

in response to receipt of said successful data transmission signal, moving the original provider data file from said outgoing data box to a history data box within storage means of said provider section; and

in response to an indication that said data transmission to said main processing section was not successful, writing an error indication in a provider section error log.

4. The method in claim 1, wherein said provider data file format is a digital image data bit stream encoding a printed document.

5. The method in claim 4, wherein said data provider section comprises a facsimile machine including an optical scanner for generating a graphical image format file of a printed document, and wherein said provider file is communicated to said main processing section via facsimile transmission of said graphical image file.

6. The method in claim 4, wherein said digital image data bit stream is generated directly from information stored in memory in a digital computer.

7. The method in claim 4, wherein said data provider section and said data subscriber section are the same entity, and wherein said entity facsimile transmits provider data in the form of a scanned printed statement to said main processing section and receives a subscriber format file in return from said main processing section.

8. The method in claim 4, further comprising the steps of:

processing said graphic data file using a form processing procedure to generate a non-graphical digital data file including information extracted from said scanned graphical data.

9. The method in claim 4, wherein said form processing procedure includes the steps of:

accepting an input file in the form of a graphical image of a document; identifying the type of document represented from among a plurality of known possible document types; aligning the graphical image with known document type fields; locating each field of information; extracting each field; segmenting between the characters in each said field; performing character recognition; and producing an interpretation of the document's image including alphanumeric components in digital form.

10. The method in claim 1, wherein said communication channel between said provider section and said main processing section comprises physical transportation of the data containing media from a location of a provider to a location of said main processing section; conversion of said data containing media into electronic form; and input of said electronic form to said main processing section.

11. The method in claim 10, wherein said data containing media is selected from the group consisting of a paper statement, a data-containing diskette, a data-containing tape, and combinations thereof.

12. The method in claim 11, wherein said physical communications is selected from the group consisting of: mail, courier, and combinations thereof.

13. In a distributed computer system including a data provider section, a main processing section., and a data subscriber section, each said section selectably couplable to another section by a communications link including communications control means, a method for translating and transferring data from a provider source format to a subscriber destination format using a priori knowledge of said source and destination formats between said data provider section and a said data subscriber section, said method comprising the steps of:

(A) transmitting, by said provider section, a particular provider data file from said provider section to said main processing section via said communications link;

(B) receiving, by said main processing section, said provider data file from said provider section and storing said received data files in said main processing section, said step of receiving including the steps of:

monitoring a communication link port for messages from a calling party requesting a communication connection according to predetermined monitoring rules and determining whether a provider section is calling main processing section requesting a communication connection;

determining whether to allow said calling party to connect to said main processing section based upon predetermined security information stored in said computer system and security access rules;

disconnecting from an invalid calling party not satisfying said security access rules; and

maintaining said communication connection between a valid provider section and a valid main processing section satisfying said security access rules and granting access between said valid provider section and said valid main processing section; and

(C) translating, by said main processing section, said stored provider data file by: (i) copying said stored provider data file to a work area file within said main processor section, and (ii) converting said work area file from the native format of said provider data file to a midformat file format;

(D) forming a communications link, by said main processing section, with a particular one of said subscriber sections and transmitting said midformat file to said subscriber section;

(E) receiving, by subscriber section, said midformat file from said main processing section and storing said received file in storage within said subscriber section; and

(F) translating using said selected second-to-third file translation procedure, by said subscriber section, said midformat file into a subscriber destination format file compatible with said preselected subscriber application software routine.

14. The method in claim 13, wherein said step of determining whether to allow said calling party to connect within step (B), comprises the steps of:

activating a billing and administrative system database storing information regarding all valid providers including each provider's respective provider section, and corresponding passwords and security levels; and

accesses said billing and administrative system database to lookup and determine whether the calling party is a valid provider section.

15. The method in claim 14, wherein said step (B) of receiving said provider data file from said provider section further comprises, after said step of maintaining said communication connection, the steps of:

responding to a provider section request that said main processing section receive a particular provider data file;

determining which provider incoming data box should be used based on predetermined provider criteria and storing said particular provider data file to said incoming data box; and

sending a data transmission receipt to said provider section informing said provider section that said particular provider data file transmitted successfully.

16. In a distributed computer system including a data provider section, a main processing section, and a data subscriber section, each said section selectably couplable to another section by a communications link including communications control means, a method for translating and transferring data from a provider source format to a subscriber destination format using a priori knowledge of said source and destination formats between said data provider section and a said data subscriber section, said method comprising the steps of:

(A) transmitting, by said provider section, a particular provider data file from said provider section to said main processing section via said communications link;

(B) receiving, by said main processing section, said provider data file from said provider section and storing said received data files in said main processing section;

(C) translating, by said main processing section, said stored provider data file by: (i) copying said stored provider data file to a work area file within said main processor section, and (ii) converting said work area file from the native format of said provider data file to a midformat file format;

(D) forming a communications link, by said main processing section, with a particular one of said subscriber sections and transmitting said midformat file to said subscriber section, said step of forming a communication link further comprising the steps of:

scanning a communication link port for a calling party attempting to communicate with said main processor section according to predetermined rules;

determining whether to allow calling party to communicate with said main processing section based upon security information and security access rules;

disconnecting from a calling party determined to be invalid based on said security information and security access rules; and

maintaining said communication connection between a valid subscriber section and a valid main processing section and granting subscriber section access between said valid subscriber section and said valid main processing section; and

accepting a request for data from said subscriber section and transmitting said requested data in said midformat file in response thereto; and

(E) receiving, by subscriber section, said midformat file from said main processing section and storing said received file in storage within said subscriber section; and

(F) translating using said selected second-to-third file translation procedure, by said subscriber section, said midformat file into a subscriber destination format file compatible with said preselected subscriber application software routine.

17. The method in claim 16, wherein said, step (D) of forming said communications link further comprises, after disconnecting from said invalid calling party, the step of recording information concerning the invalid communication attempt including a date-stamp and time-stamp in a system database.

18. In a distributed computer system including a data provider section, a main processing section, and a data subscriber section, each said section selectably couplable to another section by a communications link including communications control means, a method for translating and transferring data from a provider source format to a subscriber destination format using a priori knowledge of said source and destination formats between said data provider section and a said data subscriber section, said method comprising the steps of:

(A) transmitting, by said provider section, a particular provider data file from said provider section to said main processing section via said communications link;

(B) receiving, by said main processing section, said provider data file from said provider section and storing said received data files in said main processing section;

(C) translating, by said main processing section, said stored provider data file by: (i) copying said stored provider data file to a work area file within said main processor section, and (ii) converting said work area file from the native format of said provider data file to a midformat file format;

(D) forming a communications link, by said main processing section, with a particular one of said subscriber sections and transmitting said midformat file to said subscriber section;

(E) receiving, by subscriber section, said midformat file from said main processing section and storing said received file in storage within said subscriber section; and

(F) translating using said selected second-to-third file translation procedure, by said subscriber section, said midformat file into a subscriber destination format file compatible with said preselected subscriber application software routine, said step (F) of translating further including the steps of:

selecting a particular post-processor based on predetermined post-processor selection rules including the identity of said requesting subscriber section and the identity of said preselected application software routine;

searching for a particular midformat file from among available midformat files based on predetermined search criteria;

reading said midformat file data into a memory of said subscriber section;

reformatting the data from said midformat file into the destination format required by subscriber's application software according to predetermined data manipulation routines; and

inserting said data into said subscriber application software.

19. The method in claim 18, further comprising, after the step of reformatting said data from said midformat file, the steps of:

testing said reformatted data in the translated destination format to validate the data; and if said testing indicates that an error has occurred during said reformatting, then issuing an error condition message to the user to indicate an error condition.

20. In a computer system including a data provider section, a main processing section, and a data subscriber section, each said section selectably couplable to another section by a communications channel including communications control means, a method for translating and transferring data from a native provider source format to a subscriber destination format using a priori knowledge of said source and destination file formats between said data provider section and a said data subscriber section, said method comprising the steps of:

(A) transmitting a provider data file, formatted in a particular provider data format from among a plurality of possible provider data formats, from said provider section to said main processing section via said communications channel;

(B) receiving said provider data file from said provider section by said main processing section;

(C) selecting one particular provider-to-midformat file format translation procedure, from among a plurality of possible provider-to-midformat translation procedures, for translating said particular provider source format into a particular midformat based only on a priori knowledge of the identity of said provider and a file type characteristic of said particular provider file;

(D) translating said received provider data file from said provider source format to said midformat file using said selected one particular provider-to-midformat translation procedure without iterative trial fitting of said provider data into said midformat file;

(E) communicating said file in said midformat file from said main processing section to said subscriber section;

(F) selecting a particular midformat-to-subscriber format file translation procedure from among a plurality of possible midformat-to-subscriber format file translation procedures for translating said particular midformat file into a particular subscriber format based only on a priori knowledge of the identity of said subscriber destination and a type characteristics of said subscriber file; and

(G) translating, using said selected midformat-to-subscriber format file translation procedure, said midformat file into said subscriber destination format file compatible with said preselected subscriber application software routine.

21. The method in claim 20, wherein said step (A) of transmitting provider data files from said provider section to said main processing section via said communications channel further comprises the steps of:

monitoring communications from said provider section;

recognizing receipt of a valid communications initiation prompt generated by said provider section communications control means according to predetermined communications initiation rules, said rules providing safeguards for ensuring the quality of the data delivered to valid data subscribers;

initiating connection of provider section with main processing section in response to receipt of said valid communications initiation prompt;

connecting said provider section with said main processing section and upon being validated as a valid provider section, receiving clearance from said main processing section to transmit a copy of said provider data file to said main processing section;

reading said provider data files;

transmitting said provider data files over said communications link to said main processing section; and

signaling successful data transmission to said main processing section.

22. The method in claim 20, wherein said computer system is a distributed computer system.

23. The method in claim 20, wherein said provider data file format is a digital image data bit stream encoding a printed document.

24. The method in claim 23, wherein said data provider section comprises a facsimile machine including an optical scanner for generating a graphical image format file of a printed document, and wherein said provider file is communicated to said main processing section via facsimile transmission of said graphical image file.

25. The method in claim 23, wherein said data provider section and said data subscriber section are the same entity, and wherein said entity facsimile transmits provider data in the form of a scanned printed statement to said main processing section and receives a subscriber format file in return from said main processing section.

26. The method in claim 23, wherein said communication channel between said provider section and said main processing section comprises physical transportation of the data containing media from a location of a provider to a location of said main processing section; conversion of said data containing media into electronic form; and input of said electronic form to said main processing section.

27. The method in claim 23, further comprising the steps of:

processing said graphic data file using a form processing procedure to generate a non-graphical digital data file including information extracted from said scanned graphical data.

28. The method in claim 27, wherein said form processing procedure includes the steps of:

accepting an input file in the form of a graphical image of a document; identifying the type of document represented from among a plurality of known possible document types; aligning the graphical image with known document type fields; locating each field of information; extracting each field; segmenting between the characters in each said field; performing character recognition; and producing an interpretation of the document's image including alphanumeric components in digital form.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the present invention relates to automated data transfer and translation. The invention provides a method and structure for automatically transferring data, such as accounting data, from a variety of remote sources, in a variety of formats to a plurality of remote sites, where the data is entered into computer applications or storage, the transfer including format translation for compatibility. The invention serves as means to eliminate time-consuming and error-producing manual data re-entry.

2. Background

Computerized data, or electronically stored information, must frequently be moved from one computer to another. In the case of accounting data, for example, banks generally keep computerized records of all of transactions affecting their client's accounts--information which the clients often also wish to keep track of on their own computerized accounting systems. In response to this, a variety of sophisticated computer accounting programs have recently become available to users of small and medium powered computers. However, due to a variety of reasons discussed below, such clients usually have to re-enter data manually from paper printouts obtained from the data provider, for example, from statements from a bank. Manual re-entry of data is not only time-consuming, and hence expensive to undertake in terms of man-hours, but it also is likely to introduce errors into the data set. It has been estimated that manual data re-entry, verification, and validation costs several dollars per transaction.

Methods of automated data transfer are known in the art. The "Intellicharge" system for Quicken (Intuit Inc., Menlo Park, Calif.) downloads credit card transaction information to a Quicken user's computer. The data used, however, comes from a uniform source--a single bank. Hence, Intellicharge does not employ a multiple-format data translation scheme, nor a multiple-source data transmission scheme. Similarly, in the United States Internal Revenue Service "Electronic Filing" program, data is entered and transmitted in a single, specific pre-prescribed format, to a single recipient.

Methods of data translation are also known in the art. A software application entitled "Data Junction" (Tools and Techniques, Inc., Austin, Tex.) translates multiple formats of data. However, the package depends on manual operation to designate the files to be translated, and the formats of the source and destination files. Furthermore, this software does not perform any data transfer, verification, validation, exception reporting, or journal entry correction.

Conventional electronic data exchange (EDI) systems involve two or more companies that have agreed to interact with one another according to a pre-designated standard dictated by the industry in which the transaction is taking place. In order for such a system to work for a given industry, there must be an agreed-upon standard that is used-much like in the case of the IRS system described above. Those industries that do not have such a standard cannot participate. Data analysis, such as exception reporting or statistical analysis, are not features of such systems. Obviously, such systems lack flexibility and versatility. Additionally, the computer systems that support conventional EDI are expensive to operate and maintain because they are specialized to serve specific industry segments, and hence cannot achieve the efficiency and low cost that economies of scale might permit in a more widely applicable system.

In summary, conventional methods of automated data transfer, and of data translation, are quite limited, due primarily to limited network transfer capabilities, and the lack of universal data format standards. Hence, anyone wishing to automatically transfer data from a variety of computer systems to a variety of others must contend with a plethora of incompatible formats, and a lack of reliable transfer and error detection means. For these reasons, existing data transfer systems have been tailored to work with only one, or very few types of data sources and recipients, and these data translation methods rely heavily on manual intervention. Data transfer technologies and data translation technologies have not, in the prior art, been efficiently integrated.

OBJECTS AND SUMMARY OF THE INVENTION

As can be appreciated from the foregoing discussion, it would be desirable to have a flexible transfer and translation system that would operate on a wide variety of formats, not hampered by the lack of data format standards--a system that takes data in any format, transfers it to a remote location, and inserts it into the software application in which the data is needed. It would be desirable for a system to automatically understand what specific format translations are needed for a specific data recipient, then to be able to automatically carry out such a translation regardless of what format the data was originally in. It would also be desirable for such a system to operate with minimal user interaction, making it faster, cheaper, and more reliable than manual or semiautomatic performance of such tasks. It would also be desirable to enable companies that want to make their computerized data available to outside agencies and individuals to be able to export this data without the time and expense of developing a custom export system. It would also be desirable to have such a system that is modular and scalable on hardware and software levels, so as to provide reliability, serviceability, and adjustability of the system's performance. The present invention fills these aforementioned needs.

The present invention, in one embodiment, comprises a system capable of automatically receiving, at an intermediate processing location, data from a wide variety of remote sources, identifying the format of the data, translating the data to a common format, sending the data to a recipient in an intermediate format, then translating the data to the specific format needed by the particular recipient. A unique system for automatically selecting and implementing specific translation modules may be used. Error checking features ensure that the transferred data matches the original data, and receipts are sent to each section of the system that sends data, and logical, statistical and mathematical operations may be performed on the data. The system utilizes internal databases which allows it to know what format data will arrive in, what format to translate it to, and how many transactions to bill a data-receiving subscriber for. The system performs data translation and transfer, and performs validation, exception reporting, data analysis, and generates and sends receipts. In alternative embodiments, some or all such data processing is performed at one or both of the data source and the data destination, without an intermediate processing location.

In an alternative embodiment, a provider data file is prepared from a scanner, such as a fax transmission, which provides image data which are processed in order to derive the data file.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the physical apparatus of one embodiment of the invention.

FIGS. 2A, 2B, and 2C illustrate the major aspects of the physical apparatus and spacial layout of some alternative embodiments.

FIG. 3 shows an overview of an embodiment of the system architecture from the standpoint of process. Each block within this figure corresponds with one or more detailed separate figures.

FIG. 4 shows one embodiment of the major components and processes which occur at block 100 of FIG. 3.

FIG. 5 shows one embodiment of the major components and processes which occur at block 200 of FIG. 3.

FIGS. 6A, 6B, and 6C show some embodiments of the major components and processes which occur at block 300 of FIG. 3. FIG. 6A focuses on the role of the PrepServer Software. FIG. 6B focuses on the role of a selected PreProcessor. FIG. 6C illustrates some of the components utilized in FIG. 6B.

FIG. 7 shows one embodiment of the major components and processes which occur at block 400 of FIG. 3.

FIG. 8 shows one embodiment of the major components and processes which occur at block 500 of FIG. 3.

FIG. 9 shows one embodiment of the major components and processes which occur at block 600 of FIG. 3.

FIG. 10 shows one embodiment of this invention in which a scanner, such as a fax machine, is used to provide data from a printed document.

BRIEF DESCRIPTION OF THE TABLES

Table 1 shows one embodiment of the Subscriber Database Key to the Master Customer Definition Table.

Table 2 shows one embodiment of the Subscriber Database Key to the Master Mailbox Definition Table.

Table 3 shows one embodiment of the Subscriber Database Key to the Transactions Table for Tracking and Accounting.

Table 4 shows one embodiment of the Subscriber Database Key to the System Log File.

Table 5 shows one embodiment of the Subscriber Database Key to the Incrementing Batch ID Number.

Table 6 shows sample PreProcessor Command-Line Arguments.

Table 7 shows an example of the layout of a payroll file received from a Data Provider.

Table 8 shows an example of the layout of a bank account file received from a Data Provider.

Table 9 shows an example of the layout of a Midformat payroll file.

Table 10 shows an example of the layout of a Midformat bank account file.

Table 11 shows an example of the layout of a Destination Format payroll file.

Table 12 shows an example of the layout of a Destination Format bank account file.

DETAILED DESCRIPTION

Definitions and Terminology

ComServer Host--A machine on which the ComServer Software is operating, allowing the machine to control the telecommunication operations of a networked, distributed-task version of the Main Processing Section.

ComServer Software--Telecommunications software, in some embodiments residing on a ComServer Host computer, which receives remotely transmitted data and stores it. It may also transmit processed and unprocessed data to remote sites. Such a system is similar to a computerized mail box system as commonly seen in on-line services such as America Online (Vienna, Va.). ComServer Software functions may be carried out by commercially available software packages such as WildCat! BBS (Mustang Software, Bakersfield, Calif.)

Data Provider File Management Program--A system which enables a user of a Provider Section to connect to the Main Processing Section, and to transmit Provider Data Files, store copies of data files that hav