 |
Description  |
|
|
BACKGROUND OF THE INVENTION
This invention relates generally to computer systems, and more particularly
to graphical user interfaces for computer systems.
Graphical user interfaces or GUI are becoming increasingly popular with
computer users. It is generally accepted that computers having graphical
user interfaces are easier to use, and that it is quicker to learn an
application program in a GUI environment than in a non-GUI environment.
A relatively new type of computer which is well suited for graphical user
environments is the pen-based computer system. A pen-based computer system
is typically a small, hand-held computer where the primary method for
inputting data includes a "pen" or stylus. A pen-based computer system is
often housed in a relatively flat enclosure, and has a dual-function
display assembly which serves as both an input device and an output
device. When operating as an input device, the display assembly senses the
position of the tip of a stylus on the viewing screen and provides this
positional information to the computer's central processing unit (CPU).
Some display assemblies can also sense the pressure of the stylus on the
screen to provide further information to the CPU. When operating as an
output device, the display assembly presents computer-generated images on
the screen.
The dual-function display assemblies of pen-based computer systems permit
users to operate the computer as a computerized notepad. For example,
graphical images can be input into the pen-based computer by merely moving
the stylus across the surface of the screen. As the CPU senses the
position and movement of the stylus, it generates a corresponding image on
the screen to create the illusion that the stylus is drawing the image
directly upon the screen, i.e. that the stylus is "inking" an image on the
screen. With suitable recognition software, text and numeric information
can also be entered into the pen-based computer system in a similar
fashion. Besides serving as a notepad, pen-based computers can provide a
number of useful functions, such as serving as an address book, an
appointment calendar, a to-do list, etc.
It is often desirable to be able to move or copy objects (i.e. associated
images) on a computer screen. These objects can be moved within a single
application program (e.g. a notepad application program) or between two
different application programs (e.g. between a notepad application program
and a calendar application program). The term "object" as used herein
means any selected text, graphic, or other image on the computer screen
which is to be moved or copied as a unit.
In the prior art, a move or copy is often accomplished with a conceptual
device known as a "clipboard." For example, to copy an object, the object
is first selected on a computer screen by means of some type of pointing
device such as a mouse, trackball, etc., and the selected object is then
copied into an area of memory used by the clipboard. The pointing device
is then used to indicate where the content of the clipboard is to be
inserted, and then the content of the clipboard is "pasted" into the
application program at the indicated location. A move is accomplished in
substantially the same manner as a copy, except the selected object
removed from the original place on the screen. These processes are
sometimes referred to as "copy-and-paste" and "cut-and-paste",
respectively.
A problem with prior art clipboards is that they are invisible to the user.
Users can therefore forget that they have something stored in the
clipboard, or they may forget what is in the clipboard. This can result in
a variety of errors. For example, if a user is interrupted during a move
process and forgets to paste the object back into an application program.
The contents of the clipboard may be permanently lost. Also, if the user
forgets what the content of the clipboard is, they may inadvertently paste
incorrect objects into an application program. Furthermore, since the
clipboard is invisible, it is difficult to provide multiple clipboards
storing a variety of objects.
SUMMARY OF THE INVENTION
The present invention provides one or more visible clipboards which can be
used in a cut-and-paste (move) and copy-and-paste (copy) operations.
Preferably, the clipboards include indicia representative of the content
of the clipboard. Since the clipboards are visible, a user is constantly
reminded of the presence of the stored objects. Furthermore, a visual
clipboard facilitates the use of multiple clipboards, thereby greatly
increasing the effectiveness of the clipboard function.
A visible clipboard in accordance with the present invention is preferably
implemented on a pen-based computer system including a touch sensitive
screen and a stylus. The stylus is used to select an image displayed on a
screen, and the image is dragged by the stylus across the screen until a
boundary of the screen is impinged. The boundary of the screen can be one
of the edges of the screen, or it can be any boundary created within the
screen itself, e.g. an edge of a window, palette, status bar, anchor
point, etc. Once the boundary has been impinged, the selected image is
removed from the screen and a small icon representing a clipboard is
displayed at or near the impinged boundary. Preferably, the icon includes
indicia indicating the contents of the clipboard.
A method for creating a visual clipboard in accordance with the present
invention includes the steps of selecting an object on a computer screen,
dragging the object with a pointing device, and replacing the object with
an icon when a boundary on the computer screen is impinged. The step of
replacing the object with the icon preferably occurs when the pointing
device (e.g. stylus) contacts or is within a predetermined distance of the
boundary. Alternatively, a step of replacing the object can occur when the
object contacts or is within a predetermined distance of the boundary.
A method for moving the visual clipboard includes the steps of selecting a
visual clipboard icon on a computer screen, dragging the icon on the
screen with a pointing device, and depositing the icon when a boundary on
the computer screen is impinged. The icon can be deposited when the
pointing device is at or near the boundary, or when the icon itself is at
or near the boundary. The process permits a visual clipboard to be moved
around a computer screen for organizational purposes.
Finally, a method for moving an object in accordance with the present
invention includes the steps of selecting an object at a first location on
a computer screen, converting the object to an icon by dragging the object
to a second location on the screen, dragging the icon to a third location
on the screen, and converting the icon back to the object at the third
location. Again, the process is preferably implemented on a pen-based
computer system where the steps of selecting an object and dragging an
object are accomplished with a stylus.
While the preferred embodiments of the present invention comprise a
cut-and-paste (move) operation, a copy-and-paste (copy) operation can be
implemented by retaining an image of the dragged object at its original
position.
The visible clipboard of the present invention has several distinct
advantages over the invisible clipboards of the prior art. For one, a user
is constantly reminded of the presence of a clipboard and is provided with
visual feedback of its contents. Another important advantage is that
multiple clipboards can be used and manipulated. Finally, the visible
clipboard provides a friendlier and more intuitive method for moving or
copying objects in a graphical user interface environment.
These and other advantages of the present invention will become apparent
upon a reading of the following descriptions and a study of the various
figures of the drawing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a computer system in accordance with the
present invention;
FIG. 2 is a top plan view of the screen, case, and keypad of the computer
system of FIG. 1;
FIG. 3 is a view of the screen displaying a graphical object and a text
object;
FIG. 4a illustrates the selection and dragging of the graphical object of
FIG. 3;
FIG. 4b illustrates the continued dragging of the graphical object of FIG.
4a;
FIG. 4c illustrates the impingement of the stylus point on a boundary of
the computer screen;
FIG. 4d illustrates the conversion of the graphical object into a clipboard
icon which is attached to the boundary of the screen;
FIG. 4e illustrates the selection and dragging of the text object of FIG.
3;
FIG. 4f illustrates the conversion of the text object into a clipboard icon
which is attached to a boundary of the screen;
FIG. 4g illustrates the process for moving the clipboard icons to other
boundaries of the screen;
FIG. 4h illustrates the attachment of a clipboard icon to a boundary
provided in a central location of the screen;
FIG. 5a illustrates the selection and dragging of a clipboard icon in
preparation of pasting the contents of the icon into an application
program;
FIG. 5b illustrates the graphical object which has been pasted into the
application program;
FIG. 5c illustrates the selection of a clipboard icon, its conversion into
a text object, and the dragging and final disposition of the text object;
FIG. 5d illustrates the computer screen after the contents of the
clipboards have been pasted into an application program;
FIG. 6 is a flow diagram of the process steps of the present invention for
creating a visible clipboard;
FIG. 7 illustrates the relationship between the clipboard icon and the
object that it represents;
FIGS. 8a-8c illustrate three possible indicia for indicating the contents
of the visible clipboards;
FIGS. 9a-9c illustrate three alternative indicia for indicating the
contents of the visible clipboards;
FIG. 10a is a flow diagram illustrating the process steps for moving the
visual clipboard and for pasting the contents of the visual clipboard into
an application program; and
FIG. 10b is a flow diagram of an alternative process for moving a visible
clipboard and for pasting the content of the clipboard into an application
program.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present invention is well suited for pointer based computer systems
such as the pen-based, stylus-based and mouse-based systems that are
currently popular. For the purposes of illustration, the invention will be
described in connection with a pen-based system.
As shown in FIG. 1, a pen-based computer system 10 in accordance with the
present invention includes a central processing unit (CPU) 12, read only
memory (ROM) 14, random access memory (RAM) 16, input/output (I/O)
circuitry 18, and a display assembly 20. The pen-based computer system 10
may also optionally include a mass storage unit 22 such as a disk drive
unit or nonvolatile memory such as flash memory, a keypad 24, and a clock
26.
The CPU 12 is preferably a commercially available, single chip
microprocessor. While CPU 12 can be a complex instruction set computer
(CISC) chip, it is preferable that CPU 12 be one of the commercially
available, reduced instruction set computer (RISC) chips which are known
to be of generally higher performance than CISC chips. CPU 12 is coupled
to ROM 14 by a unidirectional data bus 28. ROM 14 contains the basic
operating system for the pen-based computer system 10. CPU 12 is connected
to RAM 16 by a bi-directional data bus 30 to permit the use of RAM 16 as
scratch pad memory. ROM 14 and RAM 16 are also coupled to CPU 12 by
appropriate control and address busses, as is well known to those skilled
in the art. CPU 12 is also coupled to the I/O circuitry 18 by
bi-directional data bus 32 to permit data transfers with peripheral
devices.
I/O circuitry 18 typically includes a number of latches, registers and
direct memory access (DMA) controllers. The purpose of I/O circuitry 18 is
to provide an interface between CPU 12 and such peripheral devices as
display assembly 20, mass storage 22, and the keypad 24.
Clock 26 provides a series of clock pulses and is typically coupled to an
interrupt port of CPU 12 by a data line 34. The clock pulses are used to
time various functions and events relating to the computer system 10. The
clock 26 can be eliminated and the clock function replace by a software
clock running on CPU 12, but this tends to be a wasteful use of CPU
processing power. In the present invention, clock 26 provides clock pulses
at 60 hertz (Hz).
Display assembly 20 of pen-based computer system 10 is both an input and an
output device. Accordingly, it is coupled to I/O circuitry 18 by a
bi-directional data bus 36. When operating as an output device, the
display assembly 20 receives data from I/O circuitry 18 via bus 36 and
displays that data on a suitable screen. The screen for display assembly
20 is preferably a liquid crystal display (LCD) of the type commercially
available from a variety of manufacturers. The input device of display
assembly 20 is preferably a thin, clear membrane which covers the LCD
display and which is sensitive to the position of a stylus 38 on its
surface. These position sensitive membranes are also readily available on
the commercial market. Combination display assemblies such as display
assembly 20 which include both the LCD and the input membrane are
available from such vendors as Scriptel Corporation of Columbus, Ohio.
The keypad 24 can comprise an array of switches. In the present embodiment,
the keypad 24 comprises mechanical buttons which overlie the bottom edge
of the membrane which covers the LCD display. When the buttons are
depressed, the membrane senses the pressure and communicates that fact to
the CPU 12 via I/O 18.
Other types of pointing devices can also be used in conjunction with the
present invention. While the method of the present invention is described
in the context of a pen-based system, other pointing devices such as a
computer mouse, a track ball, or a tablet can be used to manipulate a
pointer on a screen of a general purpose computer. Therefore, as used
herein, the terms "pointer", "pointing device", "pointing means", and the
like will refer to any mechanism or device for pointing to a particular
location on a screen of a computer display.
Some type of mass storage 22 is generally considered desirable. However,
the mass storage 22 can be eliminated by providing a sufficient amount of
RAM 16 to store user application programs and data. In that case, the RAM
16 could be provided with a backup battery to prevent the loss of data
even when the pen-based computer system 10 is turned off. However, it is
generally desirable to have some type of long term storage 22 such as a
commercially available miniature hard disk drive, nonvolatile memory such
as flash memory, battery backed RAM, PC-data cards, or the like.
In operation, information is input into the pen-based computer system 10 by
"writing" on the screen of display assembly 20 with the stylus 38.
Information concerning the location of the stylus 38 on the screen of the
display assembly 20 is input into the CPU 12 via I/O circuitry 18.
Typically, this information comprises the Cartesian (i.e. x & y)
coordinates of a pixel of the screen of display assembly 20 over which the
tip of the stylus is positioned. Commercially available combination
display assemblies such as the aforementioned assemblies available from
Scriptel Corporation include appropriate circuitry to provide the stylus
location information as digitally encoded data to the I/O circuitry of the
present invention. The CPU 12 then processes the data under control of an
operating system and possibly an application program stored in ROM 14
and/or RAM 16. The CPU 12 next produces data which is output to the
display assembly 20 to produce appropriate images on its screen.
In FIG. 2, the pen-based computer system 10 is shown housed within a
generally rectangular enclosure 40. The CPU 12, ROM 14, RAM 16, I/O
circuitry 18, mass storage 22, and clock 26 are preferably fully enclosed
within the enclosure 40. The display assembly 20 is mostly enclosed within
the enclosure 40, but a viewing screen 42 of the display assembly is
exposed to the user. As used herein, the term "screen" will refer to the
portion of the display assembly 20 which can display an image that can be
viewed by a user. Also accessible to the user is the keypad 24.
Upon power-up, pen based computer system 10 displays on screen 42 an
initial note area N including a breaker bar B and a number of guidelines
44. The breaker bar B preferably includes the day and date of creation 46
of the note N, a note number 48, and a "router" button 50 which allows
notes to be dispatched to a printer, facsimile, the trash, etc. The
optional guidelines 44 aid a user in entering text, graphics, and data
into the pen-based computer system 10.
In this preferred embodiment, the keypad 24 is not a part of the viewing
screen 42 but rather, is a permanent array of input buttons coupled to the
CPU 12 by I/O circuitry 18. Alternatively, the keypad 24 could comprise
"soft buttons" generated at a convenient location on the screen 42, in
which case a "button" would be activated by touching the stylus to the
screen over the image of the button. The keypad 24 preferably includes a
number of dedicated function buttons 52 and a pair of scroll buttons 54A
and 54B. The operation of the scroll buttons 54A and 54B, and other
aspects of computer system 10 are discussed in greater detail in
co-pending U.S. patent application Ser. No. 07/868,013, filed Apr. 13,
1992 on behalf of Tchao et al. and entitled "Method for Manipulating Notes
on a Computer Display" still pending. That application is assigned to the
assignee of the present application and its disclosure is hereby
incorporated by reference in its entirety.
The screen illustrated in FIG. 2 is referred to as the "notepad", and is an
application program running under the operating system of the pen based
computer system 10. In this preferred embodiment, the notepad is a special
or "base" application which is always available beneath higher level
applications. The notepad application, like other applications, run within
a window, which in this instance comprises the entire viewing screen 42.
Therefore, as used herein, a "window" is the entire screen or any portion
of an entire screen which is dedicated to a particular application
program.
A status bar 56 is provided at the bottom of the notepad application. The
status bar 56 is provided with a number of active areas including a real
time clock 58, a view button 60, a font button 62, a formulas button 64, a
text button 66, a graphics button 68, and a nib button 70. Co-pending
patent application Ser. No. 07/976,970, filed Nov. 16, 1992 on behalf of
Foster et al., entitled "Status Bar for Application Windows" still pending
and assigned to the assignee of the present invention describes the
operation of the status bar, and is hereby incorporated herein by
reference in its entirety.
FIG. 3 illustrates a screen 72 provided with the header bar B, status bar
56, a graphical object G and a text object T. In this instance, the
graphical object G is an equilateral triangle, and the text object T is
the sentence "Call Ted today." The graphical object G and the text object
T are preferably provided on the screen 72 by means of the stylus 38,
although they could also be entered through other means such as the serial
port. The graphical object G and text object T can be stored internally to
the computer system 10 in either recognized or unrecognized (i.e. as ink)
form.
In FIG. 4a, the graphical object G has been selected by placing the tip of
the stylus 38 at a point 74 within a "bounding box" 76 of the graphical
object G. As will be discussed in greater detail subsequently, the
graphical object G is stored with a "frame system" which provides
information concerning the object G, including the coordinates of its
bounding box 76. Essentially, the bounding box 76 is the minimum size box
which fully encloses an object with a little buffer space. In other words,
the lower left corner of the bounding box is substantially defined by the
minimum x coordinate (x.sub.min) and the minimum y coordinate (y.sub.min)
for an object, and the upper right-hand corner of the bounding box 76 is
substantially defined by the maximum x coordinate (x.sub.max) and the
maximum y coordinate (y.sub.max) of the object. It is therefore a
relatively straightforward task to determine whether the point 74 of the
stylus 38 is within the bounding box 76 by determining whether its x
coordinate lies between x.sub.min and x.sub.max and whether its y
coordinate lies between y.sub.min and y.sub.max.
Once the graphical object G has been selected, it is preferably to provide
some form of user feedback. In this instance, the graphical object G has
been "ballooned" into an exaggerated version of itself, and a visible
bounding box 76 has been provided. Other forms of selection feedback can
also be provided, such as showing the graphical object G in reverse video.
With the stylus tip still on the screen 76, the graphical object G is
dragged as indicated by the arrow 78. Techniques for dragging objects are
well known to those skilled in the art. For example, it is a simple
process to detect a .increment.x position and a .increment.y position of
the tip of stylus 38 within a period of time .increment.t. Each point on
the object is then also translated by the coordinates .increment.x &
.increment.y. If the period of time .increment.t is sufficiently small,
e.g. 1/10th of a second, the stylus 38 appears to be physically "dragging"
the object across the screen.
In FIG. 4b, the graphical object G has been dragged to a boundary B1 of the
screen 72. In this instance, the boundary B1 is the left-hand edge of the
screen 72. As will be discussed in more detail subsequently, any of the
four edges of the screen 72 may be used as a boundary, or artificial
boundary lines, anchor points, or other perimeters can be provided within
the viewing area of the screen 72. It should be noted that as the
graphical object G is dragged towards the boundary B1, portions of the
graphical object G area are obscured by the edge of the screen. This
phenomenon can be likened to imagining the screen 72 as a "window" on a
larger surface which includes the graphical object G. Portions of the
graphical object G that are not within the "window" of screen 72 can be
conceptualized as existing on this larger surface but obscured by the edge
of the window.
In FIG. 4c the graphical object G has been dragged toward boundary B1 until
the point 74 of the stylus 38 is at or near (e.g. within a few pixels of)
the boundary B1. At this point, the stylus 38 is lifted from the screen as
indicated by the circle 80 or, in the alternative, simply moves off of the
screen past boundary B1. In this preferred embodiment, this "impingement"
of the point 74 of the stylus 38 with the boundary B1 indicates to the
computer system 10 that the object G is to be converted into a visible
clipboard attached to the boundary B1.
FIG. 4d illustrates a visible clipboard 82 attached to the boundary B1 of
screen 72. In this instance, visible clipboard 82 includes indicia 84
indicating that the clipboard includes a graphic object. The indicia 84 is
arbitrary, and in this case includes a circle partially obscured by a
triangle. However any other indicia 84 (such as a square or the letter G)
could also have been used to indicate that the visible clipboard 82
includes a graphical object.
The visible clipboard 82 is "attached" to the boundary B1 of screen 72.
Even if a user scrolls within the application program displayed on the
screen 72 or opens a new application, the visible clipboard 82 will always
remain stationery and visible on the screen 82. This permits a user to
move through an application and among various applications within losing
sight of the visible clipboard and without forgetting the presence and
contents of the visible clipboard.
In FIG. 4e, text object T is selected by a point 86 created by the stylus
38. Again, the text object T has information including the coordinates of
its bounding box 88, so that it is a straightforward matter to determine
whether the point 86 of the stylus 38 is within the bounding box 88 and
thus selecting the text object T. The text object T is dragged toward the
boundary B1 by moving the point 86 along the screen 72 as indicated by the
arrow 90.
When the point 86 "impinges" upon the boundary B1, a visible clipboard icon
92 is formed and is attached to the boundary B1. Again, by "impinge upon"
it is meant that the point 86 of stylus 38 contacts or is near to the
boundary B1.
In an alternate embodiment of the present invention, the visible clipboard
icons (such as icons 82 and 92), can be formed when the graphical object
or text object, respectively, impinge upon a boundary such as boundary B1.
In this alternative embodiment, impingement could be determined when the
bounding box for the object is sufficiently close to a boundary, or when a
designated percentage (e.g. 20%) of the bounding box of the object
overlaps the desired boundary. It should therefore be apparent that
"impingement" can comprise any move or gesture which reasonably
unambiguously indicates to the computer system 10 that a dragged object is
to be converted into a visible clipboard icon and attached to a boundary
of the computer screen.
In FIG. 4g, a method for repositioning the visible clipboard icons is
illustrated. The original position of visible clipboard icon 82 is shown
in solid lines, and a final position of the visible clipboard 82 is shown
at 82' in phantom. The icon is moved from the position at 82 to 82' by
contacting a point 94 within the bounding box of icon 82 with a tip of the
stylus 38 and dragging the icon 82 as indicated by the arrow 94. When the
point at 94' is sufficiently close or contacts a second boundary B2 of
screen 72, the icon 82' attaches to the boundary B2, and the icon 82 has
been successfully moved from a first boundary B1 to a second boundary B2
of the screen 72.
Still referring to FIG. 4g, the original position of visible clipboard icon
92 is shown in solid lines attached to boundary B1 of screen 72. A final
position for icon 92 is shown on 92' in phantom. To move the icon from the
position at 92 to the position at 92', a point 98 of the stylus 38 is
placed within the boundary box of icon 92 and the icon 92 is dragged
toward its final position as indicated by the arrow 100. When the point 98
reaches the location 98', the icon 92 attaches itself to the boundary B3
of the screen 72. It should be noted that in this instance the boundary B3
is an edge of the status bar 56, and not the boundary B4, which is the
bottom edge of the screen 72. The icon 92 could be attached to the
boundary B4 by continuing to move the point 98 through the status bar 56
to the boundary B4.
It should once again be noted that parameters of the icons themselves
rather than the point of the stylus could determine whether the icon had
"impinged" upon a boundary of the screen 72. For example, if one of the
edges of the icons 82 or 92 contacts or is sufficiently close to a
boundary of the screen 72, this gesture could be used to indicated that
the icon is to be attached to that boundary. Alternatively, an overlap of
a certain percentage of the visible clipboard icon with a boundary (e.g.
20% of the icon) could indicate that the icon is to be attached to that
boundary. Therefore, it is once again intended that the "impingement" of a
visible clipboard icon upon a boundary be interpreted as any gesture or
movement with a stylus or other pointing mechanism which indicates that an
icon is to be attached to a boundary of the screen.
In FIG. 4h, the visible clipboard icon 92 remains attached to the boundary
B1 of screen 72, but the visible clipboard icon 82 has been moved (as
described previously) and attached to a boundary B5 of a palette 102.
Palettes such as palette 102 are "popped" onto the screen 72 to perform a
variety of functions. In this case, the palette 102 allows the user to
select bold, italic, or normal fonts. The palette 102 remains stationary
on the screen until removed by the activation of a "close box" 104 with
stylus 38. As such, the palette 102 provides a useful boundary B5 to which
a visible clipboard 82 can be attached.
In FIG. 5a, the visible clipboard icon 82 is shown attached to the boundary
B5 of palette 102 and the visible clipboard icon 92 is shown attached to
the boundary B1 of the screen 72. Icon 82 is selected by placing a point
106 of the stylus 38 within its bounding box, and is dragged to a new
position 82' as indicated by the arrow 108. Once at the position 82', the
point 106' of the stylus is lifted from the screen as indicated by the
circle 110. Since the point 106' is not near any boundary of the screen
72, i.e. is not near any boundary such as B1-B5, the computer system 10
determines that this is a gesture indicating that the contents of the
visible clipboard 82 are to be pasted to the applica | | |
