Method and apparatus for providing implicit computer-implemented assistance

What is claimed is:

1. A computer system having assistance capabilities, comprising:

(a) computation means for performing assistance functions,

(b) memory means for maintaining a data base of assistance-pertinent events, said memory means being coupled to said computation means,

(c) means for providing a smart field responsive to information of a predefined type.

(d) means for noticing assistance-pertinent event occurrences subject to potential assist action performance, the means for noticing assistance-pertinent event occurrences including:

means for determining whether the information of a predefined type has been entered into the smart field; and

search means responsive to the entry of the information of a predefined type into the smart field, the search means operable for searching a portion of the data base of assistance-pertinent events associated with the type of information of the smart field to identify and compile a list of any alternatives matching or partially matching the specific information entered in the smart field,

(d) means for determining whether implicit assistance responsive to the event is indicated, and

(e) means for providing the implicit assistance indicated.

2. A computer system as recited in claim 1 wherein said means for providing the implicit assistance indicated includes means for automatically displaying an icon adjacent to the smart field, the icon indicating the existence of the compiled list.

3. A computer system as recited in claim 1 wherein said means for providing the implicit assistance indicated further includes:

means for determining when the icon has been selected; and

means for displaying the compiled list when the icon has been selected.

4. A computer system as recited in claim 1 wherein said means for providing the implicit assistance indicated includes means for automatically displaying the compiled list.

5. A computer system as recited in claim 1 wherein said means for providing the implicit assistance indicated includes:

means for displaying the compiled list; and

means for enabling a user of the computer system to select an alternative from the compiled list.

6. A computer system as recited in claim 5 further including means for updating the database to contain information regarding the selected alternative.

7. A computer system as recited in claim 1 wherein the nature of the smart field is indicated by text, graphics, or a combination of text and graphics.

8. A computer system as recited in claim 1 wherein the smart field is responsive to information selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes.

9. A computer system as recited in claim 8 wherein the database is comprised of templates, each template comprised of linked fields, each linked field being selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes.

10. A method of providing implicit assistance on a computer system, the method comprising the steps of:

maintaining a database of assistance-pertinent events on a memory of the computer system;

displaying on a display screen of the computer system a smart field responsive to information of a predefined type;

determining whether information of a predefined type has been entered into the smart field by a user of the computer system;

searching a portion of the data base of assistance-pertinent events associated with the type of information of the smart field to identify and compile a list of any alternatives matching or partially matching the specific information entered into the smart field;

determining whether implicit assistance responsive to the specific information entered in the smart field is indicated; and

providing the indicated implicit assistance.

11. A method as recited in claim 10 wherein the step of providing the indicated implicit assistance includes the substeps of:

automatically displaying an icon adjacent to the smart field, the icon indicating the existence of the compiled list;

determining whether the icon has been selected; and

displaying the compiled list when the icon has been selected.

12. A method as recited in claim 10 wherein the step of providing the indicated implicit assistance includes the substep of automatically displaying the compiled list.

13. A method as recited in claim 12 wherein the step of providing the indicated implicit assistance further includes the substeps of:

receiving a user selection from the compiled list; and

updating the database to contain information regarding the selected alternative.

14. A method as recited in claim 10 further including the step of displaying text, graphics or a combination of text and graphics in order to indicate the nature of the smart field.

15. A method as recited in claim 10 wherein the smart field is responsive to information selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes.

16. A method as recited in claim 15 wherein the database is comprised of templates, each template comprised of linked fields, each linked field being selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes.

17. A computer program stored on a computer readable medium, the computer program comprising computer executable instructions for:

maintaining a database of assistance-pertinent events on a memory of a computer system executing the computer program;

displaying on a display screen of the computer system a smart field responsive to information of a predefined type;

determining whether information of a predefined type has been entered into the smart field by a user of the computer system;

searching a portion of the data base of assistance-pertinent events associated with the type of information of the smart field to identify and compile a list of any alternatives matching or partially matching the specific information entered into the smart field;

determining whether implicit assistance responsive to the specific information entered in the smart field is indicated; and

providing the indicated implicit assistance.

18. A computer program as recited in claim 17 wherein the computer executable instruction for providing the indicated implicit assistance includes subinstructions for:

automatically displaying an icon adjacent to the smart field, the icon indicating the existence of the compiled list;

determining whether the icon has been selected; and

displaying the compiled list when the icon has been selected.

19. A computer program as recited in claim 18 wherein the computer executable instruction for providing the indicated implicit assistance further includes the subinstructions for:

receiving a user selection from the compiled list; and

updating the database to contain information regarding the selected alternative.

20. A computer program as recited in claim 17 wherein the smart field is responsive to information selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes, and the database is comprised of templates, each template comprised of linked fields, each linked field being selected from the group consisting of persons, telephone numbers, dates, document names, account numbers, addresses, and access codes.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

The present invention relates generally to computer systems, and more particularly to computer-implemented assistance methods and apparatus.

Computerized personal organizers are becoming increasingly popular with a large segment of the population. Computerized personal organizers tend to be small, lightweight, and relatively inexpensive, and can perform such functions as keeping a calendar, an address book, a to-do list, etc. While many of these functions can also be provided in conventional computer systems, personal organizers are very well suited to the personal organization task due to their small size and portability. Personal organizers are available from such companies as Sharp and Casio of Japan.

A relatively new form of computer, the pen-based computer system, holds forth the promise of a marriage of the power of a general purpose computer with the functionality and small size of a personal organizer. A pen-based computer system is typically a small, hand-held computer where the primary method for inputting data includes a "pen" or stylus. A pen-based computer system is commonly housed in a generally rectangular enclosure, and has a dual-function display assembly providing a viewing screen along one of the planar sides of the enclosure. The dual-function display assembly serves as both an input device and an output device. When operating as an input device, the display assembly senses the position of the tip of a stylus on the viewing screen and provides this positional information to the computer's central processing unit (CPU). Some display assemblies can also sense the pressure of the stylus on the screen to provide further information to the CPU. When operating as an output device, the display assembly presents computer-generated images on the screen.

The dual-function display assemblies of pen-based computer systems permit users to operate the computer as a computerized notepad. For example, graphical images can be input into the pen-based computer by merely moving the stylus on the surface of the screen. As the CPU senses the position and movement of the stylus, it generates a corresponding image on the screen to create the illusion that the stylus is drawing the image directly upon the screen, i.e. that the stylus is "inking" an image on the screen. With suitable recognition software, text and numeric information can also be entered into the pen-based computer system in a similar fashion.

One approach to computerized assist operations is to provide assistance automatically when a situation in which assistance could be provided is recognized. However, such an approach may provide unsatisfactory results when the user is provided with assistance that is unwanted or disproportionate.

Simply stated, concerns have arisen about assist functions being undertaken by the computer without adequate user control and interaction. When the assist function has been undertaken without adequate user control, assistance would be provided awkwardly and at times when the assistance was not necessarily desired.

It is essential that the control, the timing, and the application of the assistance is considered appropriate and well-tuned. The confidence of the user is undermined when the assistance provided only obliquely addresses particularized user needs without precisely providing the particular results objectively and subjectively required.

SUMMARY OF THE INVENTION

According to the invention, a method and apparatus has been developed for providing computer-assisted implicit and explicit assistance for a variety of user-supportive information functions. If no implicit assist actions are desired or indicated, then a logical process is initiated to determine whether explicit assistance should be undertaken. If implicit assistance is indicated, a list of action alternatives is displayed for the user. Alternatively, a process can be undertaken to automatically select a best action alternative of several identified alternatives.

If explicit assistance is desired by the user, particular object(s) from which the assistance may be inferred are entered into an assistance operation. An attempt is then made to recognize possible intents expressed by the objects entered into the assistance process. If no user intent is, in fact, recognized, the assist operation is usually terminated. If a possible intent is recognized, the actual intent is hypothesized. A check is further undertaken, to determine whether a hypothesis is in fact available. If no hypothesis is found, the process permits the user to supply a proposed action. If no hypothesis is found and no user action is proposed, assistance efforts terminate. However, if a hypothesis is available, preparations for execution are undertaken. A final inquiry is made as to whether to undertake the hypothesized assist. If the response to an inquiry whether to assist as hypothesized is affirmative, execution of the hypothesized action is undertaken.

Accordingly, it is an intent of the invention to provide user explicit assist functions which are provided under oversight and with the interaction of the user, and implicit assist functions in certain structured instances. The involvement of the user in the implementation of implicit assist operations is a key aspect of the invention. User intent is not merely deduced, but specific user queries are made to ensure controlled application of assist operations.

These and other advantages of the present invention will become apparent upon reading the following detailed descriptions and studying the various figures of the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computer system in accordance with the present invention.

FIG. 2 is a top plan view of the screen, case, and keypad of the computer system of FIG. 1.

FIG. 3 is a flow diagram of a process according to the invention for providing controlled computer-assisted user assistance.

FIG. 4a shows a process for determining whether or not implicit assistance is desired by the user, in connection with providing computer assisted support to the user.

FIG. 4b shows an example of an implicit assist operation with a phone slip window having a smart name field evoked, for example, by either highlighting the verb "call" or by writing it on the note field before evoking window.

FIG. 4c shows the phone slip window of FIG. 4b with the name formal font form. ISAAC having been recognized and established in the name field in

FIG. 5 shows an example of a generic <PERSON> type frame along with a particular set of specific frames of the <PERSON> type.

FIGS. 6a-6c show respective assist windows in successive stages of an assist process, including first a window containing a first informational level directed at the name ISAAC alone, a second window with a pop-up menu offering a user choice among several known ISAACs, and a third window showing the selection of a particular ISAAC, that is ISAAC ASIMOV, having been accomplished.

FIG. 7 illustrates a brief flow diagram illustrating a heuristic process for the selection of a particular choice among alternatives when the user is not queried for selection of alternatives.

FIGS. 7a-7c show selected examples of heuristic rules of thumb which are effective in making automatic choices between alternative ISAACs, respectively directed toward selection schemes such as selecting the last used ISAAC, selecting the last in order of ISAACs according to particular position within a selected table, and finally selecting a particular ISAAC based upon prior frequency of choice of that particular ISAAC.

FIG. 8a is a flow diagram illustrating the updating process for data base information in linked smart fields.

FIG. 8b is a call slip illustration of an updated smart field window in which the phone number field information has been updated.

FIG. 9a is a flow diagram of the process according to the invention in which a query is made as to whether a specific selection has been made as to a particular object.

FIG. 9b-9c indicate graphically the performance of the selection query operation as expressed in FIG. 9A.

FIG. 9d illustrates the transferal of the highlighted objects of FIG. 9c transferred to a selected window.

FIG. 10a illustrates the input of a handwritten object into a smart field in a window.

FIG. 10b illustrates the recognition of the handwritten object of FIG. 10a arid its conversion into formal font form.

FIG. 11a is a flow diagram illustrating the recognition of objects process.

FIG. 11b illustrates an object combination under operation.

FIG. 11c shows a template for organizing in preset form a variety of object combinations.

FIG. 12a illustrates a process for hypothesizing user intent as to particular activities.

FIG. 12b is a flow diagram setting forth a process for determining whether the user wishes to provide or supply a particular action.

FIG. 12c shows a window with a menu partially overlapping its topside in order to provide the user with an array of activity choices.

FIG. 12d shows the window of FIG. 12c with the activity of calling having been selected, establishing the combined objects CALL and ISAAC in the activity field of the window.

FIG. 13 illustrates the process for preparing for execution of a particular activity.

FIG. 14a is a short flow diagram of a process for determining whether to proceed with execution of a particular selected assist activity.

FIG. 14b illustrates an example of a confirmation of action slip that could be produced upon completion of a particular activity, in this case completion of the process of faxing information to another party.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is well suited for pointer based computer systems such as the pen-based, pen-aware and mouse controlled systems that are currently popular. For the purposes of illustration, the invention will be described in connection with a pen-based system.

As shown in FIG. 1, a pen-based computer system 10 in accordance with the present invention includes a central processing unit (CPU) 12, read only memory (ROM) 14, random access memory (RAM) 16, input/output (I/O) circuitry 18, and a display assembly 20. The pen-based computer system 10 may also optionally include a mass storage unit 22, a keypad (or keyboard) 24, a serial port 26, an infrared (I/R) port 28, and a clock 30.

The CPU 12 is preferably a commercially available, single chip microprocessor. While CPU 12 can be a complex instruction set computer (CISC) chip, it is preferable that CPU 12 be one of the commercially available, reduced instruction set computer (RISC) chips which are known to be of generally higher performance than CISC chips. CPU 12 is coupled to ROM 14 by a unidirectional data bus 32. ROM 14 contains the basic operating system for the pen-based computer system 10. CPU 12 is connected to RAM 16 by a hi-directional data bus 34 to permit the use of RAM 16 as scratch pad memory. ROM 14 and RAM 16 are also coupled to CPU 12 by appropriate control and address busses, as is well known to those skilled in the art. CPU 12 is also coupled to the I/O circuitry 18 by bi-directional data bus 36 to permit data transfers with peripheral devices.

I/O circuitry 18 typically includes a number of latches, registers and direct memory access (DMA) controllers. The purpose of I/O circuitry 18 is to provide an interface between CPU 12 and such peripheral devices as display assembly 20, mass storage 22, keypad 24, serial port 26, and I/R port 28.

Display assembly 20 of pen-based computer system 10 is both an input and an output device. Accordingly, it is coupled to I/O from a variety of vendors. The input device of display assembly 20 is preferably a thin, clear membrane which covers the LCD display and which is sensitive to the position of a stylus 38 on its surface. With such a structure, the display assembly 20 can serve as an input "tablet." These position sensitive membranes are also readily available on the commercial market. Alternatively, other types of tablets can be used, such as inductively coupled tablets. Combination display assemblies such as display assembly 20 which include both the LCD and the input membrane are commercially available from such vendors as Scriptel Corporation of Columbus, Ohio.

Some type of mass storage 22 is generally considered desirable. Mass storage 22 can be coupled to I/O circuitry 18 by a bi-directional data bus 40. However, the mass storage 22 can be eliminated by providing a sufficient amount of RAM 16 to store user application programs and data. In that case, the RAM 16 can be provided with a backup battery to prevent the loss of data even when the pen-based computer system 10 is turned off. However, it is generally desirable to have some type of long term mass storage 22 such as a commercially available miniature hard disk drive, nonvolatile memory such as flash memory, battery backed RAM, a Personal Computers Memory Card International Association (PCMCIA) card, or the like.

The keypad 24 can comprise an array of mechanical buttons or switches coupled to I/O circuitry 18 by a data bus 42. Alternatively, keypad 24 can comprise an entire, standard QWERTY keyboard. In the present embodiment, a separate keypad 24 is not used in favor of a "pseudo" keypad 24'. This "pseudo" keypad 24' comprises "button" areas which are associated with a bottom edge of the tablet membrane that extends beyond the lower edge of the LCD display. These button areas are defined by a printed or silk-screened icons which can be seen through the transparent membrane of the input tablet. When the "buttons" are selected by engaging the stylus 38 with the membrane over these printed icons, the membrane senses the pressure and communicates that fact to the CPU 12 via data bus 38 and I/O 18. An example of pseudo keypad 24' is shown in FIG. 2.

Other types of pointing devices can also be used in conjunction with the present invention, for example, an interrupt port of the CPU 12 which can count the clock pulses to provide the time function. However, this alternative clock embodiment tends to be wasteful of CPU processing power. Clock 30 is coupled to CPU 12 by a data bus 48.

In operation, information is input into the pen-based computer system 10 by "writing" on the screen of display assembly 20 with the stylus 38. Information concerning the location of the stylus 38 on the screen of the display assembly 20 is input into the CPU 12 via data bus 38 and I/O circuitry 18. Typically, this information comprises the Cartesian (i.e..times.& y) coordinates of a pixel of the screen of display assembly 20 over which the tip of the stylus is positioned. Commercially available combination display assemblies such as the aforementioned assemblies available from Scriptel Corporation include appropriate circuitry to provide the stylus location information as digitally encoded data to the I/O circuitry of the present invention. The CPU 12 then processes the data under control of an operating system. While the method of the present invention is described in the context of a pen-based system, other pointing devices such as a computer mouse, a track ball, or a tablet can be used to manipulate a pointer on a screen of a general purpose computer. Therefore, as used herein, the terms "pointer", "pointing device", "pointing means", and the like will refer to any mechanism or device for pointing to a particular location on a screen of a computer display.

Serial port 26 is coupled to I/O circuitry by a bi-directional bus 44. The serial port 26 can be used to couple the CPU to external devices and networks.

Infrared (I/R) port 28 is coupled to I/O circuitry by a bi-directional bus 46. The I/R port can be used for outgoing information (e.g. to control a printer or some other external device, or to communicate with other computer systems) or for incoming information from other computers or devices.

Clock 30 preferably comprises a real-time clock to provide real-time information to the system 10. Alternatively, clock 30 can simply provide regular clock pulses td possibly an application program stored in ROM 14, RAM 16, or mass storage 22. The CPU 12 next produces data which is transferred to the display assembly 20 via I/O circuitry 18 and data bus 38 to produce appropriate images on the screen portion of the display assembly 20.

In FIG. 2, the pen-based computer system 10 of FIG. 1 is shown housed within a generally rectangular enclosure 50. The CPU 12, ROM 14, RAM 16, I/O circuitry 18, and clock 26 are preferably fully enclosed within the enclosure 50. The display assembly 20 (FIG. 1) is mostly enclosed within the enclosure 50, but a viewing screen 52 of the display assembly is exposed to the user. As used herein, the term "screen" will refer to the portion of the display assembly 20 which can display an image that can be viewed by a user. Also accessible to the user is the pseudo keypad 24' that was described with reference to FIG. 1.

Upon power-up, pen based computer system 10 displays on screen 52 an initial "note" area 54a including a header bar 56a and a number of guidelines 58. The header bar 56a preferably includes the date of creation of the note area 54a and a number of icons and "soft" buttons, not particularly germane to the discussion of the present invention. For this reason, the header bar 56a will not be discussed in detail herein. The optional guidelines 58 aid a user in entering text, graphics, and data into the pen-based computer system 10. A graphic object G in the form of a triangle is shown entered within note area 54a.

Additional note areas, such as a note area 54b, can be formed by the user by drawing a substantially horizontal line across the screen 52 with the stylus 38. The substantially horizontal line is recognized by the system 10 and is converted into a second header bar 56b. Additional text, graphical, and other data can then be entered into this second note area 54b. For example, the text object T comprising "ISAAC" has been entered into second note area 54b.

In this preferred embodiment, the keypad 24', as explained previously, comprises a printed or silk-screened member 60 provided beneath a lower edge of a thin, clear, stylus-sensitive membrane 62 of the input "tablet." Alternatively, a keypad could comprise a mechanical keypad (or keyboard) 24, or a keypad could comprise "soft buttons" i.e. images generated at convenient locations on the screen 52, in which case a "button" would be activated by touching the stylus to the screen over the image of the button. The keypad 24' preferably includes a number of dedicated function buttons 64 and a pair of scroll buttons 66a and 66b. The operation of the note areas 54a, 54b, etc., scroll buttons 66a and 66b, and other aspects of computer system 10 are discussed in greater detail in co-pending U.S. patent application 07/868,013, filed Apr. 13, 1992 on behalf of Tchao et al., assigned to the assignee of the present invention and incorporated herein by reference in its entirety.

The screen illustrated in FIG. 2 is referred to as the "notepad", and is preferably an application program running under the operating system of the pen based computer system 10. In this preferred embodiment, the notepad is a special or "base" application which is always available beneath higher level applications. The notepad application, like other applications, runs within a window, which in this instance comprises the entire viewing screen 52. Therefore, as used herein, a "window" is the entire screen or any portion of an entire screen which is dedicated to a particular application program.

A status bar 68 is provided at the bottom of the notepad application. The status bar 68 is provided with a number of active and display areas, which again are not particularly germane to the present invention and will therefore not be discussed in detail herein. U.S. patent application Ser. No. 07/976,970 filed Nov. 16, 1992 on behalf of Foster et. al, entitled "Status Bar for Application Windows" and assigned to the assignee o