Method for manipulating notes on a computer display

What is claimed is:

1. A method for manipulating notes on a screen of a computer display comprising:

generating an initial note area on the screen of the computer display;

dividing said initial note area into a plurality of note areas in response to at least one division gesture implemented by moving a pointer across the width of the screen such that a left edge of the division gesture is within a first defined distance of a left side of said computer display and a right edge of the division gesture is within a second defined distance of a right side of said computer display, wherein the division gesture is made in a horizontal motion having a slope of less than a predefined slope value, and wherein each division gesture creates divider indicia in the form of a header bar on said screen which visually separates two adjacent note areas; and

scrolling in response to one or more scrolling commands such that for note areas having heights of less than the screen's length, scrolling to a new note area for each new scroll command, and, for note areas having heights of greater than the screen's length, scrolling to the next successive portion of the note area for each new scroll command until the end of the note area is reached.

2. A method as recited in claim 1 wherein said step of generating an initial note area on the computer screen includes creating a first data structure including a note number and a note height dimension.

3. A method as recited in claim 1 wherein said step of dividing said initial note area comprises:

detecting a theoretical line drawn on said display by said pointer;

determining whether said theoretical line is a division gesture; and

generating a header bar on said computer display for a new note area if said theoretical line is determined to be a division gesture.

4. A method as recited in claim 3 wherein said step of detecting a theoretical line comprises:

collecting a plurality of sample points corresponding to a movement of said pointer across said display; and

forming said theoretical line from at least two of said plurality of sample points.

5. A method as recited in claim 3 wherein said step of determining whether said theoretical line is a division gesture includes one or more of the following steps:

determining whether there is a sample point farther from said theoretical line than a predetermined maximum distance value;

determining whether the absolute value of a sum of the signed distances of said plurality of sample points from said theoretical line is greater than a predetermined maximum sum value;

determining whether the absolute value of a slope of said theoretical line differs from a predetermined header bar slope by more than a predetermined maximum slope value; and

determining whether either end of said theoretical line is separated from an edge of said screen by more than a predetermined maximum margin value,

wherein when one of the determining steps fails, said theoretical line is not a division gesture.

6. A method for moving divider indicia in the form of a header bar displayed on a computer display as a linear region which separates two adjacent note areas on a screen of the computer display, the header bar containing information corresponding to at least one of the adjacent areas, said method comprising:

detecting the selection of the header bar by a pointer;

detecting a sizing gesture made with said pointer on said header bar;

moving said header bar as indicated by said sizing gesture; and

scrolling in a scroll direction through said adjacent note areas in response to one or more scrolling commands such that when an end of a note area is reached, the header bar of the next successive note area is displayed at the top of the computer screen in response to a scrolling command, said scrolling including

scrolling, in response to said scrolling command, in the scroll direction to display a remaining undisplayed area of a displayed one of the note areas as well as at least a portion of an adjacent note area when the remaining undisplayed area of the displayed one of the note areas in the scroll direction has an undisplayed length that is greater than zero but less than the height of the computer screen,

scrolling, in response to said scrolling command, in the scroll direction to display a next successive portion of a remaining undisplayed area of a displayed one of the note areas as well as to not display any portion of an adjacent note area when the remaining undisplayed area of the displayed one of the note areas in the scroll direction has an undisplayed length that is greater than the height of the computer screen, and

scrolling, in response to said scrolling command, in the scroll direction to display a least a portion of an adjacent note area with the header bar associated therewith displayed at the top of the computer screen when the end of the displayed one of the note areas and a portion of the adjacent note area are displayed.

7. A method as recited in claim 6 wherein said pointer comprises stylus means contacting a screen of a pen-based computer system.

8. A method as recited in claim 7 wherein the header bar includes a sizing button, the sizing button being operative to adjust a length of the note area associated with the header bar, and

wherein said sizing gesture is made to the sizing button, and said sizing gesture comprises contacting said stylus with said screen over the sizing button of said header bar and moving said stylus without lifting said stylus from said screen.

9. A method as recited in claim 6 wherein said step of moving said header bar comprises:

changing a height dimension of a note associated with said header bar; and

re-drawing at least said note on the computer screen.

10. A method for scrolling a computer generated image including a plurality of note areas, where adjacent note areas have an immediately preceding note area and an immediately subsequent note area and are separated by divider indicia in the form of a header bar, the header bar containing information corresponding to the immediately subsequent note area, at least one of the note areas has a length greater than the height of a computer screen on which one or more of the note areas and the respective header bars are displayed, said method comprising:

detecting a scrolling command; and

scrolling through said adjacent note areas in a scroll direction indicated by said scrolling command using discrete jumps, said scrolling including

scrolling, in response to said scrolling command, in a discrete jump in the scroll direction to display a remaining undisplayed area of a displayed one of the note areas as well as at least a portion of an adjacent note area when the remaining undisplayed area of the displayed one of the note areas in the scroll direction has an undisplayed length that is greater than zero but less than the height of the computer screen,

scrolling, in response to said scrolling command, in a discrete jump in the scroll direction to display a next successive portion of a remaining undisplayed area of a displayed one of the note areas as well as to not display any portion of an adjacent note area when the remaining undisplayed area of the displayed one of the note areas in the scroll direction has an undisplayed length that is greater than the height of the computer screen, and

scrolling, in response to said scrolling command, in a discrete jump in the scroll direction to display a least a portion of an adjacent note area with the header bar associated therewith displayed at the top of the computer screen when the displayed one of the note areas and a portion of the adjacent note area are displayed and the displayed one of the note areas does not have any remaining undisplayed area in the scroll direction.

11. A method as recited in claim 10 wherein said detecting step detects an up-scroll command, and wherein said step of scrolling comprises:

decrementing a current note number counter when an end of one of said note areas is reached; and

drawing a new note corresponding to said decremented current note number counter such that the new note's header bar is displayed at the top of the computer screen.

12. A method as recited in claim 10 wherein said detecting step detects a down-scroll command, and wherein said step of scrolling comprises:

incrementing a current note number counter when an end of one of said note areas is reached; and

drawing a new note corresponding to said incremented current note number counter such that the new note's header bar is displayed at the top of the computer screen.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

The present invention relates generally to the manipulation of images on a computer screen, and more particularly to methods for manipulating images on the screen of a pen-based computer system.

A pen-based computer system is a small, often hand-held, computer system where the primary method for inputting data includes a "pen" or stylus. A typical pen-based computer system is housed in a generally rectangular enclosure, and has a dual-function display assembly providing a viewing screen along one of the planar sides of the enclosure. The dual-function display assembly serves as both an input device and an output device. When operating as an input device, the display assembly senses the position of the tip of the stylus on the viewing screen and provides this positional information to the computer's central processing unit (CPU). Some display assemblies can also sense the pressure of the stylus on the screen to provide further information to the CPU. When operating as an output device, the display assembly presents computer-generated images on the screen.

The dual-function display assemblies of pen-based computer systems permit users to operate the computer as a computerized notepad. For example, graphical images can be input into the pen-based computer by merely moving the stylus on the surface of the screen. As the CPU senses the position and movement of the stylus, it generates a corresponding image on the screen to create the illusion that the stylus is drawing the image directly upon the screen. With suitable recognition software, text and numeric information can also be entered into the pen-based computer system in a similar fashion.

Users often want to input more than one screen-full of information into their computer systems. To accomplish this, computer systems of the prior art have adopted two different input and display techniques. With a first technique, the screen images are treated as "pages" of a notepad. Users can then either sequentially access pages by "flipping" back or forth through the notepad, or they can jump to a particular page by page number. A second technique is to consider the screen of the display assembly to be a "window" on a long, continuous scroll of paper. The "scroll" is moved past the imaginary window (or the window is moved along the scroll) to provide a partial display of the contents of the scroll on the computer screen.

Both of these techniques have their advantages and disadvantages. The paging technique has the disadvantage of having a fixed page size which is usually equal to the size of the computer's screen. In consequence, if an image is too big to fit on one page, it must be divided to fit on two or more separate pages. While this is not particularly limiting for text, it makes the handling of large graphical images difficult. On the positive side, "paging" tends to be an intuitive way for users to access multiple screens of information. Users are familiar with the use of conventional books, notebooks, and notepads, all of which are essentially page based artifacts. An example of the intuitive nature of paging involves visually locating an image which was created on a particular page of memory. If, for example, a user knows that he drew a particular image in the lower right-hand corner of a page, he can quickly "flip" through the multiple pages while fixing his eyes on the lower right-hand corner of the screen to quickly spot the appropriate image.

The advantages and disadvantages of the scrolling technique are almost precisely the reverse of the advantages and disadvantages of the paging technique. An advantage of the scrolling technique is that images of virtually any length can be created. A disadvantage of the scrolling technique is that it is less intuitive than the paging technique. Using the previous example, finding a particular image by scrolling tends to be more difficult than finding the image by paging. This is due, in part, to the fact that when scrolling through the images stored in the computer, a particular desired image can be located at any vertical location on the screen, requiring a user to visually search a much larger image area. Also, with the scrolling technique it is more difficult for a user to know his or her relative location in a document. For example, with the paging technique a user might intuitively know that a desired image is about on page twelve, or is about two thirds of the way through the document. This type of intuitive knowledge is more difficult to achieve with the scrolling technique.

A further disadvantage of the scrolling technique is that it is inherently slow since images on the screen must not be moved so fast that they cannot be viewed. This can make the viewing of large amounts of data by scrolling techniques a time consuming process.

Yet another disadvantage of scrolling techniques is that there is no clear division between adjacent but unrelated images. For example, if a user first writes a letter and then makes a sketch, it would be desirable to make a clear division between these two unrelated items. This disadvantage also applies to a lesser extent to paging techniques when two or more unrelated items are placed on a single page.

SUMMARY OF THE INVENTION

In the present invention, images are grouped into note areas which form part of a continuous scroll. These notes are manipulated by: (a) generating an initial note area on the screen of a computer display; (b) dividing the initial note area into a number of contiguous note areas in response to one or more division gestures implemented by a pointing device, where each division gesture creates a header bar on the screen which visually separates two adjacent note areas; (c) modifying the size of a selected note area in response to a sizing gesture made to a header bar associated with the selected note area; and (d) scrolling within the note areas in response to a scrolling command.

The initial note area is provided with a header bar which preferably includes the date of creation, the note number, and other indicia. This initial note area can be considered to be of indeterminate or infinite height. Graphical, text, and data objects can be created within this initial note area.

When a user desires to make a new note, a division gesture is made on the computer display with a pointing device. For example, in a pen-based computer system a stylus can be moved substantially horizontally across the screen to indicate a division gesture. Once a division gesture is detected, the height of the preceding note is determined, and the height of the new note can be considered to be indefinite or infinite. Preferably, the division gesture creates a new header bar for the new note indicating the date of creation, the note number, and/or other pertinent information.

Preferably, each header bar also includes a "sizing" button. By making a sizing gesture to the sizing button, the height of the associated note can be modified to make the note longer or shorter.

The notes on the display are preferably scrolled in a fashion which is a hybrid between traditional paging and scrolling techniques. The scrolling technique of the present invention can be considered to be a "quantized" scroll where objects displayed on the screen tend to be located in the same area of the screen in which they were created. This is accomplished by scrolling in quantized jumps such that the header bar of a desired note jumps either to the top of the screen or to about its original creation location on the screen.

The note areas and quantized scroll of the present invention overcome many of the aforementioned problems of the prior art. Related objects can be grouped together in a single note, and notes longer than a screen length can be easily generated. The height of individual notes can be modified by the sizing gesture, and the quantized scrolling of the present invention allows for the quick, intuitive scan of a large number of notes.

These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following specification of the invention and a study of the several figures of the drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a pen-based computer system in accordance with the present invention;

FIG. 2 is a pictorial representation of the screen of a computer display assembly of the present invention;

FIG. 3 illustrates the screen of FIG. 2 after graphical, text and data objects have been added, and after the screen has been divided into two note areas;

FIG. 4 graphically illustrates a number of note areas arranged in a scroll and a "window" representing the screen of the computer display;

FIGS. 5a-5f illustrate six views of the scroll as seen through the window of FIG. 4;

FIG. 6 illustrates the viewing of a note which has a height greater than the height of the viewing window;

FIG. 7 is a flow diagram illustrating a method for manipulating notes on a computer display in accordance with the present invention;

FIG. 8 is a flow diagram of a method for detecting a division gesture on the screen of a computer display assembly;

FIGS. 8A and 8B illustrate two of many potential gestures which can be made with a stylus, the first of which will be recognized as a division gesture and the second of which will not be recognized as a division gesture;

FIG. 9 illustrates the data structure of a note in accordance with the present invention;

FIG. 10 is a flow diagram illustrating a method for processing the division gesture detected by the method illustrated in FIG. 8;

FIG. 11 is a flow diagram illustrating a method for detecting a sizing gesture of a selected note;

FIG. 12 is a flow diagram illustrating a method for processing the sizing gesture detected by the method illustrated in FIG. 11;

FIG. 13 is a flow diagram illustrating a method for drawing all visible images on the screen of a computer display assembly;

FIG. 14 is a flow diagram of a method for making a quantized up-scroll through notes on a computer screen; and

FIG. 15 is a flow diagram which illustrates a method for making a quantized down-scroll through notes on a computer screen.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In FIG. 1, a pen-based computer system 10 in accordance with the present invention includes a central processing unit (CPU) 12, read only memory (ROM) 14, random access memory (RAM) 16, input/output (I/O) circuitry 18, and a display assembly 20. The pen-based computer system 10 may also optionally include a mass storage unit 22 such as a disk drive unit or non-volatile memory such as flash memory, and an array of input buttons 23.

The CPU 12 is preferably a commercially-available, single chip microprocessor. While CPU 12 can be a complex instruction set computer (CISC) chip, it is preferable that CPU 12 be one of the commercially available, reduced instruction set computer (RISC) chips which are known to be of generally higher performance than CISC chips. CPU 12 is coupled to ROM 14 by a uni-directional data bus 24. ROM 14 contains the basic operating system for the pen-based computer system 10. CPU 12 is connected to RAM 16 by a bi-directional data bus 26 to permit the use of RAM 16 as scratch pad memory. ROM 14 and RAM 16 are also coupled to CPU 12 by appropriate control and address busses, as is well known to those skilled in the art. CPU 12 is further coupled to the I/O circuitry 18 by bi-directional data bus 28 to permit data transfers with peripheral devices.

I/O circuitry 18 typically includes a number of latches, registers and direct memory access (DMA) controllers. The purpose of I/O circuitry 18 is to provide an interface between CPU 12 and such peripheral devices as display assembly 20, mass storage 22, and the array of input buttons 23.

Display assembly 20 of pen-based computer system 10 is both an input and an output device. Accordingly, it is coupled to I/O circuitry 18 by a bi-directional data bus 30. When operating as an output device, the display assembly 20 receives data from I/O circuitry 18 via bus 30 and displays that data on a suitable screen. The screen for display assembly 20 is preferably a liquid crystal display (LCD) of the type commercially available from a variety of manufacturers. The input device of display assembly 20 is preferably a thin, clear membrane which covers the LCD display and which is sensitive to the position of a stylus 32 on its surface. These position-sensitive membranes are also readily available on the commercial market. Combination display assemblies such as display assembly 20 which include both the LCD and the input membrane are commercially available from such vendors as Scriptel Corporation of Columbus, Ohio.

Other types of pointing devices can also be used in conjunction with the present invention. While the method of the present invention is described in the context of a pen-based system, other pointing devices such as a computer mouse, a track ball, or a tablet can be used to manipulate a pointer on a screen. Therefore, as used herein, the terms "pointing device", "pointing means", and the like will refer to any mechanism or device for pointing to a particular location on a screen of a computer display.

Some type of mass storage 22 is generally considered desirable. However, the mass storage 22 can be eliminated by providing a sufficient amount of RAM 16 to store user application programs and data. In that instance, the RAM 16 could be provided with a back-up battery to prevent the loss of data even when the pen-based computer system 10 is turned off. However, it is generally desirable to have some type of long term storage 22 such as a commercially available miniature hard disk drive, or non-volatile memory such as flash memory or battery-backed RAM.

In operation, information is input into the pen-based computer system 10 by "writing" on the screen of display assembly 20 with the stylus 32. Information concerning the location of the stylus 32 on the screen of the display assembly 20 is input into the CPU via I/O circuitry 18. The CPU 12 then processes the data under control of an operating system and possibly an application program stored in ROM 14 and/or RAM 16. The CPU 12 then produces data which is output to the display assembly 20 to produce appropriate images on its screen.

In FIG. 2, the pen-based computer system 10 is shown housed within a generally rectangular enclosure 36. The CPU 12, ROM 14, RAM 16, I/O circuitry 18, and mass storage 22 are preferably fully enclosed within the enclosure 36. The display assembly 20 is mostly enclosed within the enclosure 36, but a viewing screen 38 of the display assembly is exposed to the user. As used herein, the term "screen" will refer to the portion of the display assembly 20 which can display an image that can be viewed by a user. Also accessible to the user are the array of input buttons 23.

Upon power-up, pen-based computer system 10 displays on screen 38 an initial note area N(1) including a header bar B(1) and a number of guidelines 42. The header bar B(1) preferably includes the date of creation 44 of the note N(1), a note number 46, and a sizing "button" 48. The optional guidelines 42 aid a user in entering text, graphics, and data into the pen-based computer system 10.

In this preferred embodiment, the array of input buttons 23 are not part of the screen 38 but, rather, are permanent, hard-wired input buttons coupled to the CPU 12 by I/O circuitry 18. Alternatively, the array of input buttons 23 could be "soft" buttons generated at a convenient location on the screen 38, in which case a button would be activated by touching the stylus to the screen over the image of the button. The array of input buttons 23 preferably include a number of dedicated function buttons 50 and a pair of scroll buttons 52A and 52B. The function of the scroll buttons 52A and 52B will be discussed in greater detail subsequently.

In FIG. 3, several types of images or objects have been entered into the computer system 10 by the stylus 32. More particularly, in note N(1), a text object 54 describing a house at 123 Maple Street is entered near the top of the screen 38, a sketch of the layout for the house is entered as a graphic object 56, and calculations of the square footage have been entered as a data object 58. Furthermore, a new header bar B(2) has been added to create an additional note area N(2), and to separate this additional note area N(2) from the initial note area N(1).

A conceptual representation of the images seen on screen 38 will be discussed with reference to FIG. 4. In FIG. 4, the screen images can be conceptualized as being printed on a long scroll 62 of paper, where only a portion of the scroll can be viewed at a time through a window 64 (corresponding to the screen 38 of the display assembly 20). The width w of screen 38 is preferably equal to the width W of the scroll 62. If, however, the width w of the screen 38 is less than the width W of the scroll 62, the entire width W of the scroll 62 can be viewed by a lateral scroll, as is well known to those skilled in the art.

Also seen in FIG. 4, the scroll 62 includes the initial note area N(1) and can also include one or more additional note areas N(2), N(3), etc. All of the note areas have an associated header bar B(1), B(2), B(3), etc. along their upper edge.

As mentioned previously, portions of the scroll 62 can be viewed through the screen 38 of window 64. To view other portions of the scroll 62, the images are "scrolled" up or down past the screen 38. As used herein, an up-scroll will permit lower numbered note areas to be seen, and a down-scroll will allow higher numbered note areas to be seen. Therefore, an up-scroll can be visualized as moving the window 64 upwardly along the scroll 62, or by moving the scroll 62 downwardly past window 64. Similarly, a down-scroll can be visualized as moving the window 64 downwardly along the scroll 62, or by moving the scroll 62 upwardly past window 64.

Preferably, each of the note areas has the same width W. However, each of the note areas will have its own height depending upon where the header bar is drawn. For example, the height of the initial note N(1) is H(1), the height of the second note N(2) is H(2), the height of the third note N(3) is H(3), etc. The height of the last note of the scroll 62 (in this case H(5)) is indeterminate and can be considered infinite. Once a new header bar has been added to the bottom of note N(5), its height H(5) will become determinate, and the height of the new last note N(6) can be considered to be indeterminate or infinite.

In FIGS. 5a-5f, a "quantized" down-scroll in accordance with the present invention will be described. In FIG. 5a, the window 64 is positioned at the top of scroll 62 to view the initial note N(1). The header bar B(1) of the initial note N(1) is at the top of the screen 38, and the header bar B(2) of additional note N(2) is positioned in the bottom third of screen 38. Upon the sensing of a down-scroll command by a user pressing button 52b, the header bar B(2) jumps to the top of screen 38 and the header bar B(3) moves onto the bottom portion of the screen 38. With another down-scroll command sensed as the button 52b is pressed, the header bar B(3) jumps to the top of screen 38 as shown in FIG. 5c. Since the height H(3) of note N(3) is greater than the length L of screen 38, only a portion of the note N(3) will be seen on the screen. In FIG. 5d, another down-scroll command permits the viewing of the middle of note N(3) without any header bars showing on the screen 38. Yet another down-scroll command will show the bottom portion of note N(3) along with the header bar B(4) of note N(4), as illustrated in FIG. 5e. Finally, in FIG. 5f, another down-scroll command will cause the header bar B(4) to jump to the top of screen 38 and the header bar B(5) will appear near the middle of the screen.

It should be apparent from the preceding description that the "quantized" scrolling technique of the present invention is a hybrid between prior art paging and scrolling techniques. In this invention, the images on the screen 38 can be viewed as if they were formed in a continuous scroll 62, but the scrolling action comprises discrete, quantized jumps rather than the continuous scrolling action of the prior art. In this way, various text, graphical and data objects will appear in approximately the same location on the screen 38 as they were when they were created, allowing a user to quickly jump through the images on scroll 62 to locate a desired object. For example, if a user knows that he drew a sketch near the lower left-hand corner of the screen 38, he can jump through the notes quickly, fixating his eye on the lower left-hand corner of the screen to find the appropriate image. The up-scroll technique operates in a similar fashion.

FIG. 6 is used to illustrate the viewing of a note, such as note N(3), having a height greater than the length L of screen 38. Here, the window 64 is positioned near the middle of note N(3), corresponding to the image