Passive data collection system for market research data

Claims

What is claimed is:

1. A non-obtrusive data collection apparatus for assembling selected data transactions representing the choices made by a panel of consumers from among a multiplicity of data transactions on a communications loop connecting a plurality of point-of-sale terminals to a store controller, said data transactions being communicated by a particular loop protocol and in a particular loop format of message units, said apparatus comprising:

coupling means for passively coupling to said communications loop at a location where all data transactions of the loop can be monitored and in such a manner as to not interfere with said data transactions;

converting means, connected to said coupling means, for converting said data transactions from said loop protocol into a standard digital protocol of data characters;

a transaction processor; and

means, connected to said converting means, for communicating said converted data transactions to said transaction processor;

said transaction processor including:

means for assembling characters of said data transactions into said message units;

means for selectively sorting said message units into consumer transactions;

means for selectively sorting said consumer transactions into panelist transactions involving choices made by said panel;

means for storing said panelist transactions; and

means, in response to an inquiry from an external device, for communicating said stored panelist transactions to said external device.

2. A data collection method for the market research of a plurality of individual stores for the purchases of predetermined consumers forming a buying panel, wherein each store has a plurality of point-of-sale terminals which communicate data transactions in an SDLC format over a network loop to a central store computer, said method comprising:

(a) passively monitoring data transactions on said network;

(b) assembling from said monitored data transactions data frames corresponding to respective data transactions;

(c) selecting from all of said data frames, data frames of at least one particular type;

(d) sorting said selected frames based upon an information field which contains information indicating the beginning and the end of a respective purchasing transaction;

(e) temporarily storing said selected data frames corresponding to a respective purchasing transaction;

(f) searching said temporarily stored selected data frames for panelist identification; and

(g) storing said searched selected data frames corresponding to a respective purchasing transaction when any of them include said panelist identification.

3. A data collection method as set forth in claim 2 which further includes:

repeating steps (a)-(g) of claim 2 for each store, and each network loop of each store; and

communicating said selected data frames pertaining to panelist transactions to a central processing point.

4. A data collection method as set forth in claim 2 wherein:

said steps (a)-(g) of claim 2 are performed on a real time basis.

5. A data collection method as set forth in claim 4 wherein said step of assembling frames includes:

receiving and storing each character of a data frame on an interrupt basis as it is monitored on said network loop; and

transferring a completed frame to a frame buffer upon the detection of an end of frame character.

6. A data collection method as set forth in claim 5 wherein said step of selecting frames includes:

determining the number of frames in said frame buffer;

sorting the frames stored in said frame buffer until all stored frames have been sorted when the number is in excess of a predetermined value.

7. A data collection method as set forth in claim 6 wherein said step of selecting frames further includes:

sorting all information frames into frame blocks having the same point of sale terminal address.

8. A data collection method as set forth in claim 7 wherein said step of sorting said selected frames includes sorting said frame blocks into respective purchasing transactions.

9. A data collection method as set forth in claim 8 wherein said steps of searching and storing include:

searching purchasing transactions for panelist identification; and

storing purchasing transactions which contain said panelist identification.

10. A data collection method as set forth in claim 9 which includes the further step of:

inserting panelist identification in said data transactions.

11. A data collection method as set forth in claim 10 wherein said step of inserting includes:

inserting said panelist identification as a universal product code which is not assigned a product.

12. In a system for market research wherein data are gathered at a central station from a plurality of individual stores in respect to transactions made by respective selected shoppers, said selected shoppers having respective identification indicia, and each of said stores having a data processing controller connected by a communications network to a plurality of transaction terminals at which input transaction data in respect to transactions with respective shoppers are entered, including transaction data corresponding to universal product codes for respective items bought and the respective identification indicia, such input transaction data being communicated to the respective controller over said communications network, and controller data from said respective controller being communicated to the respective terminals over said communications system, said controller data including controller transaction data related to the same transactions as respective input transaction data: a data acquisition and transfer system comprising means for passively coupling to a respective said communications network in a respective store to receive respective input and controller transaction data without introducing any signals into said communications network, storage means, means responsive to received transaction data corresponding to said identification indicia for storing in said storage means said received input and controller transaction data and instructions relating to each transaction made by a selected shopper to the exclusion of transaction data and instructions relating to transactions made by shoppers other than selected shoppers, means for reading out said stored transaction data from said storage means, and means for transferring said read out stored transaction data to said central station.

13. A method for electronically collecting market research information from a plurality of sales locations wherein the information is collected from each of the sales locations and transmitted to a host processor for subsequent evaluation, and wherein each of said sales locations has at least one automatic checkout system including a plurality of point-of-sale terminals communicating with a store controller over a communications network, said method comprising:

passively monitoring communications between the store controller and each of said point-of-sale terminals;

separating relevant market research data from said communications;

forming market research file structures compatible with the host processor;

storing said market research file structures in nonvolatile memory; and

periodically transferring said market research file structures to the host processor.

14. A method for electronically collecting market research information from a plurality of sales locations wherein the information is collected from each of the sales locations and transmitted to a host processor for subsequent evaluation, wherein each of said sales locations has at least one automatic checkout system including a plurality of point-of-sale terminals communicating with a store controller over a communications network, and wherein the communications protocol of the automatic checkout system is phase encoded, said method comprising:

passively monitoring communications between the store controller and each of said point-of-sale terminals, said step of passively monitoring including converting the phase encoded protocol into a standard binary protocol;

separating relevant market research data from said communications;

forming market research file structures compatible with the host processor;

storing said market research file structures in nonvolatile memory; and

periodically transferring said market research file structures to the host processor.

15. A method for electronically collecting market research information from a plurality of sales locations wherein the information is collected from each of the sales locations and transmitted to a host processor for subsequent evaluation, wherein each of said sales locations has at least one automatic checkout system including a plurality of point-of-sale terminals communicating with a store controller over a communications network, and wherein communications on the network are configured for two wire half duplex operation in a loop, said method comprising:

passively monitoring communications between the store controller and each of said point-of-sale terminals, said step of passively monitoring including connecting a receive data tap in parallel between the receive port of the store controller and the last point-of-sale terminal on the loop;

separating relevant market research data from said communications;

forming market research file structures compatible with the host processor;

storing said market research file structures in nonvolatile memory; and

periodically transferring said market research file structures to the host processor.

16. A method for electronically collecting market research information from a plurality of sales locations wherein the information is collected from each of the sales locations and transmitted to a host processor for subsequent evaluation, wherein each of said sales locations has at least one automatic checkout system including a plurality of point-of-sale terminals communicating with a store controller over a communications network, and wherein communications on the network are carried on in SDLC format, said method comprising:

passively monitoring communications between the store controller and each of said point-of-sale terminals;

separating relevant market research data from said communications, said step of separating relevant market research data including the steps of separating the communications on the network into SDLC frames and other communication frames thereby separating information in the SDLC frames from the totality of communications; separating said SDLC frames into information format frames and other SDLC frames, thereby separating information contained in the information format frames from the totality of SDLC frames; and separating said information format frames into market research frames and other information format frames thereby separating the market research information from the totality of information contained in the information format frames;

forming market research file structures compatible with the host processor;

storing said market research file structures in nonvolatile memory; and

periodically transferring said market research file structures to the host processor.

17. A method as set forth in claim 16 wherein said step of forming market research file structures includes the step of:

parsing information contained in said market research frames into said market research file structures.

18. A method as set forth in claim 17 wherein said step of parsing further includes the step of:

parsing each transaction file by separate market research file structures.

19. Passive data collection apparatus for assembling selected data transactions corresponding to purchases made by respective customers from among a multiplicity of data transactions on a communications network connecting a plurality of point-of-sale terminals in a store to a store controller in a store, said data transactions being communicated by a predetermined network protocol and in a predetermined network format of message units, said apparatus comprising:

coupling means for passively coupling to said communications network to receive all data transactions on the network between said store controller and said terminals in such a manner as not to add message units to said network or delete or otherwise modify message units on said network or control or modify the operation of the network, store controller or terminals;

converting means connected to said coupling means for converting said received data transactions from said network protocol into a digital protocol of data characters;

a transaction processor;

communications means connected to said converting means for communicating said converted data transactions to said transaction processor;

said transaction processor including:

means for assembling characters of said converted data transactions into message units;

sorting means for selectively sorting said assembled message units according to predetermined market research criteria;

storing means for storing data corresponding to said sorted message units; and

means for communicating said stored data to an external device remote from the store.

20. Apparatus according to claim 19 wherein:

said data transactions on said communications network include identification of respective selected customers, said sorting means includes means for sorting said assembled message units into customer transactions, and means for selectively sorting said customer transactions according to selected customers identified by respective said identification, said storing means stores said selectively sorted customer transactions, and said means for communicating to said external device provides identification of the selected customers involved in respective customer transactions.

21. Apparatus according to claim 19 wherein:

said sorting means includes means for sorting said assembled message units according to products purchased at respective prices, and said transaction processor further includes means responsive to said sorted message units for counting the number of respective products bought at respective prices.

22. Apparatus according to claim 20 wherein:

said sorting means includes means for sorting said assembled message units according to products purchased at respective prices, and said transaction processor further includes means responsive to said sorted message units for counting the number of respective products bought at respective prices.

23. A data collection method for market research in respect to each of a plurality of individual stores for the purchases of selected customers forming a buying panel, each of said selected customers having a respective panelist identification, wherein each store has a plurality of point-of-sale terminals which communicate data transactions including respective panelist identification over a communications network to a central store controller said method comprising:

(a) passively monitoring data transactions on a respective said network;

(b) assembling from said monitored data transactions data frames corresponding to respective data transactions;

(c) selecting from all of said data frames, data frames of at least one particular type;

(d) sorting said selected frames based upon an information field which contains information indicating the beginning and the end of a respective purchasing transaction;

(e) temporarily storing said selected data frames corresponding to a respective purchasing transaction;

(f) searching said temporarily stored selected data frames for a said panelist identification; and

(g) storing said searched selected data frames corresponding to a respective purchasing transaction when any of them include said panelist identification;

searching all purchase transactions of said frame blocks for a panelist identification; and

storing those purchasing transactions of said frame blocks if they contain said panelist identifications.

24. A data collection method as set forth in claim 23 which further includes:

communicating said selected data frames pertaining to panelist transactions from each of said stores to a central processing point.

25. A data collection method as set forth in claim 23 wherein:

said steps (a)-(g) are performed on a real time basis.

26. A data collection method as set forth in claim 25 wherein said step of assembling frames includes:

receiving and storing each character of a data frame on an interrupt basis as it is monitored on said network; and

transferring a completed frame to a frame buffer upon the detection of an end of frame character.

27. A data collection method as set forth in claim 26 wherein said step of selecting frames includes:

determining the number of frames in said frame buffer;

sorting the frames stored in said frame buffer until all stored frames have been sorted when the number is in excess of a predetermined value.

28. A data collection method as set forth in claim 27 wherein said step of selecting frames further includes:

sorting all information frames into frame blocks having the same point-of-sale terminal address.

29. A data collection method as set forth in claim 28 wherein said steps of sorting temporarily storing selected frames include:

sorting said frame blocks into purchase transactions.

30. A data collection method as set forth in claim 29 wherein said steps of searching and storing include:

searching all purchase transactions of said frame blocks for a panelist identification; and

storing those purchasing transactions of said frame blocks if they contain said panelist identifications.

31. Data collection apparatus for market research in respect to each of a plurality of individual stores for the purchases of selected customers forming a buying panel, each of said selected customers having a respective panelist identification, wherein each store has a plurality of point-of-sale terminals which communicate data transactions including respective panelist identification over a communications network to a central store controller said apparatus comprising:

(a) means for passively monitoring data transactions on a respective said network;

(b) means for assembling from said monitored data transactions data frames corresponding to respective data transactions;

(c) means for selecting from all of said data frames, data frames of at least one particular type;

(d) means for sorting said selected frames based upon an information field which contains information indicating the beginning and the end of a respective purchasing transaction;

(e) means for temporarily storing said selected data frames corresponding to a respective purchasing transaction;

(f) means for searching said temporarily stored selected data frames for a said panelist identification; and

(g) means for storing said selected data frames corresponding to a respective purchasing transaction when any of them include said panelist identification.

32. In a system for market research wherein data are gathered at a central station from a plurality of individual stores in respect to transactions made by respective selected shoppers, said selected shoppers having respective identification indicia: a market research data monitor including

a buffer;

nonvolatile storage means for storing purchase transaction records;

programmed means for executing a communications program at a particular time of day to transfer stored purchase transaction records from said nonvolatile storage means to a central station;

programmed means for executing a real time monitor program on an interrupt basis to store in said buffer purchase transaction records of all purchases made by selected shoppers as they occur; and

programmed means for executing a data storage program when said communications program is not executing to transfer the nonvolatile storage of said purchase transaction records on a periodic basis from said buffer to said nonvolatile storage means.

The invention pertains generally to the collection of market research data and is more particularly directed to the passive monitoring and collection of retail sales data transmitted over a communications network between a store controller and a plurality of point-of-sale terminals.

BACKGROUND OF THE INVENTION

Market research into the behavior of consumers, such as their response to particular advertisements, is useful for the marketing of products which are sold in retail markets. In one such type of market research, a plurality of consumers or perspective customers are selected as a panel forming a representative subgroup of a particular demography or population. The buying habits of the selected panelists are recorded and analyzed. The entire population under study is then exposed to media oriented information messages (advertisements) which it is believed will influence their buying habits. These panelists may shop in different stores of a chain or may shop in other establishments in a particular geographic area. A panel is limited in its geographic extent and limited in the establishments where the respective panelists can purchase items so that a market researcher may tabulate the purchases of respective panelists to determine which advertisements each particular panelist may have watched, what purchases he made in response thereto, and where.

In a more controlled environment, the panelists can be provided with targetable television which shows specific advertisements to households which are taking part in the study. The panelist then makes purchases from particular stores in the geographic area, and those purchase responses are recorded. The recordation of the purchase response can be automatic, such as by way of product code scanning in respective stores, or manual, such as by the panelists marking their purchases on paper. Preferably, such market research is made substantially transparent to the panelists such that the purchases are recorded automatically so as not to influence the behavior of the panelist or be influenced thereby as might be the case where the panelists have to record their own purchases. In one successful system the advertisements used for the market research are substituted for regular advertisements in an unobtrusive manner so that a panelist does not know which advertisements the market researcher is studying his response to.

One such advantageous system for collecting panelist data utilizing targetable television is illustrated in U.S. Pat. No. 4,331,973 issued to Eskin, et al., and U.S. Pat. No. 4,404,589, issued to W. Andrew Wright, Jr.

Another system for collecting market survey data is illustrated in U.S. Pat. No. 4,355,372, issued to Johnson, et al.

A system for coupon distribution which utilizes Universal Product Code (UPC) store scanners is illustrated in U.S. Pat. No. 4,723,212 issued to Mindrum, et al.

In some of these prior art systems, data collection means for market research data takes advantage of a store scanning system which includes a plurality of point-of-sale terminals, a communications network, and a central processor, sometimes referred to as a store controller. Each of the point-of-sale terminals has a cash register for entering transaction data and an optical scanner which reads the Universal Product Code (UPC) bar codes on each product. Messages from a point-of-sale terminal are transferred over the communications network to the store controller for inquiry. The store controller generally responds to such inquiry by a look up of the UPC or item code and provides the point-of-sale terminal with price, quantity or other data useful in making a transaction record on a register tape for the customer. The store controller may also record the transaction for accounting or inventory purposes.

In this regard, item movement data are also useful to a market researcher because they allow an evaluation of consumer purchase behavior, as, for example, behavior variation with price change. Item movement data comprise the recordation of the sales quantity of a particular item, at a particular price and for a particular time. The day by day recording of the quantity of a particular item, its total sales, and the correlation of these sales to price change permits another factor in the complex purchasing behavior of consumers to be evaluated. It is difficult to obtain item movement data which are market research oriented from a store controller because the item data the controller collects are inventory oriented. Those data are based on the amount of a particular item which is "on hand" such that it pertains not only to product on the shelf but also that stored in the inventory area. Sales are subtracted from this total amount and thus the difference between inventory numbers will be sales for a particular day. However, if an inventory is taken and the total reset or new product is added to inventory, the sales data for a particular day can be erased before a market researcher can acquire them. Further, such inventory data may be unavailable on a day-by-day basis and price changes may or may not be included.

The communications on the store communications networks include all the transaction data for the market researcher that are necessary to evaluate the impact of advertisements, to collect item movement data, or to conduct other market research. However, in many cases it is impossible to obtain this information because access to the store controller may not be available to the market researcher. Further, even if the market researcher has access to the store controller, it is very difficult to integrate market research collection software into the store controller system without disturbing the main purpose of the store controller, which is to control and communicate with the point-of-sale terminals. Moreover, the store controller may already have too many point-of-sale terminals or communications networks such that it is overloaded and additional data processing overhead for market research is not possible or practical.

What is needed in these situations is a passive data monitoring system which will collect market research data in a transparent manner without disturbing communications between the store processor and the point-of-sale terminals. The passive data monitor needs to operate unobtrusively to read all transaction data on the communications network and selectively process the desired market research data therefrom without disturbing the normal transactions on the communications network.

While there are a number of reasons why a passive data monitor would be advantageous for collecting market research data from the scanning networks of an automated checkout system, there are also some associated problems with accomplishing this task. Initially, there is the problem of determining where on the communications network that a passive connection can be made so as not to disturb the communications on the network, while still obtaining all the data which is transacted on the network. Another problem is that the data must be collected in real time asynchronously to the processor of the passive monitor. This is because it is unknown when a panelist transaction, or other desired transaction, will take place. This problem is made more difficult because many times the protocol and data format of the communications on the store network are different from the one most optimum for the processor of a passive monitor.

Moreover, the passive data monitor must sift through all the traffic on the store network to find the market research data that are desired. Much of the data traffic on the communications network is overhead and control protocol which is of no use to the market researcher. This overhead data must be discarded but not until after they are examined, and it is determined such cannot be used. Thereafter, the passive data monitor must quickly assemble the useful data into conveniently sized records and protect them by placing them in nonvolatile storage. This operation should take place in real time so that none of the useful data are lost. Finally, when it is convenient the passive data monitor should be able to transfer the recorded data to a central processing center without interfering with the real time monitoring of the store network.

All of the these tasks increase in difficulty when monitoring a store scanning system which has multiple scanning networks. The traffic which must be monitored and the data that must be checked increase dramatically with every network which is added. Additionally, in this situation, a passive data monitor for multiple loops has the disadvantage of being not only asynchronous to a single communications network while it is monitoring, but also to all of the multiple networks. Such monitoring of multiple communications networks must be coordinated carefully because a passive monitor cannot request a transaction to be repeated and, once a transaction is missed, the data will be lost in respect to the monitor.

SUMMARY OF THE INVENTION

In one embodiment, the invention provides a non-obtrusive data monitor and collection system for the sales transaction data of an automated checkout system of a retail establishment. Such automated checkout system comprises a central processor means connected by a communications network to a plurality of point-of-sale terminals which communicate to detail the sales transactions of the establishment. The data monitor and collection system reads all the communications on the network and records selective parts of such as monitored data. After the monitored data are collected, they are parsed into particular categories which are useful for market research. In the preferred embodiment, the collected transaction data are parsed by processing means into panelist files relating to the advertisement influenced buying reactions of selected persons among a demographic sample and into item movement files relating to the reactions of all consumers of a particular retail establishment, as to price modifications of a product. The panelist files and the item movement files are transferred periodically by the data monitor and collection system to a remote central processing system where they can be evaluated.

The data files gathered from a plurality of such monitoring and collection systems can be used to produce market research conclusions for a wide geographic area. A market research data collection system utilizing such passive data monitors at stores over a wide region includes means for storing the data files at each monitor and transferring them to a host processor. In a multi-store system when the market research data are panelist data, the panelists will shop at more than one store. Item movement data gathered from more than one store becomes more relevant because they are taken from a larger sample and small inconsistencies are filtered out.

An implementation of the passive data monitor and collection system in a preferred form includes means for passively connecting to the communications network of an automated checkout system of a retail establishment. An implementation of such passive connection for a store operating a SDLC (synchronous data link control) loop checkout system is provided by a high impedance parallel connection at the receive port of each store loop. All transactions on each store loop are received by the store processor at its receive port, and thus, any data necessary for selection in the market research are available in the communications system at that point on the network. In this manner, the passive data monitor will not interrupt or slow the automated checkout system of the store, and will appear completely transparent to other communications on the network.

The passive data monitor further includes a processing means having means for assembling the transaction data from character strings collected from the communications network into frames which can then be parsed into market research data, in the preferred embodiment, panelist files and item movement files. The means for assembling includes conversion means for converting the transaction data from the protocol of the communications loop into a standard digital protocol that can be read and stored by a microcomputer system.

After the transaction data are assembled into frames, sorting means are used to select only those frames which contain information data. In this manner much of the standard traffic frames of the communications network are ignored, and the informational essence of the messages extracted for further parsing. The information containing frames are then selectively sorted into transaction blocks for each point-of-sale terminal. Such transaction blocks contain all the particular data pertaining to sales of each product by each terminal and are arranged by transactions. Transactions from each terminal are then appended to a transaction list before further processing.

The assembly of the transaction data into frames, information frames, transaction blocks, and a transaction list is accomplished on a real time basis by an interrupt driven monitor routine. The monitor routine is memory resident in the processor means and builds the transaction list as those data are monitored from each of the store loops. The monitoring routine operates in the foreground of an executive routine which also calls a data logging and formatting routine in background to read the transaction list and assemble the panelist files and item movement files therefrom.

The panelist files are written to a nonvolatile memory, preferably a hard disk, as they are assembled. The executive routine on a regular basis, for example, daily, calls a first communications routine to transfer the panelist files from the disk to a remote host central processing system for evaluation. The item movement files are used to update an item movement data base in the memory which is moved periodically to nonvolatile memory. The executive routine on a regular basis, for example, weekly, calls a second communications routine to move the item movement data base to the host central processor where it can be evaluated.

Separating the task of forming a transaction list from the assembling of the panelist files, item movement files and host communications produces an advantageous monitoring and collection system where the sifting of large amounts of data is accomplished rapidly. The formation of a transaction list permits the exclusion of unwanted communications in the first instance thereby substantially reducing the amount of information which the logging and formatting routine must handle. Further, the information is placed in a logical format, by separate transactions with a customer, which solves the problem of dealing with multiple asynchronous networks where several transactions may be happening simultaneously. The transaction list is formed by a minimum number of sorts which are processed in such a way that none of the communications on any of the store networks are missed. The transaction list is then further processed in background when the system can conveniently provide such service.

Other objects, features, aspects, and advantages of the present invention, together with the or