 |
Claims  |
|
|
We claim:
1. A first data processing system for transmission and reception of
realtime freehand drawing to and from a second data processing system,
each system having a processor coupled to a memory, a display and an input
device, each of said system having an operating system which runs a
plurality of application programs each displayed in a respective one of a
plurality of windows on the display, the first data processing system
comprising:
means for presenting a first set of points in a first window on the display
in said first data processing system, the first set of points representing
freehand drawing generated by the input device at the first data
processing system;
means for sending the first set of points to the second data processing
system;
means for receiving and presenting a second set of points in the first
window, the second set of points representing freehand drawing generated
at the second data processing system;
means for presenting a first set of image data from a first image handling
application in a second window in said first data processing window;
means for importing said first image data from said second window to said
first window;
said sending means, sending said first image data to said second data
processing system from said first window;
wherein the input device is a touch sensitive overlay disposed over the
viewing surface of a display which generates the first set of points as
the overlay is contacted on a surface facing opposite to the viewing
surface of the display.
2. A first data processing system for transmission and reception of
realtime freehand drawings to and from a second data processing system,
each system having a processor coupled to a memory, a display and an input
device, each of said system having an operating system which runs a
plurality of application programs each displayed in a respective one of a
plurality of windows on the display, the first data processing system
comprising:
means for presenting a first set of points in a first window on the display
in said first data processing system, the first set of points representing
freehand drawing generated by the input device at the first data
processing system;
means for sending the first set of points to the second data processing
system;
means for receiving and presenting a second set of points in the first
window, the second set of points representing freehand drawing generated
at the second data processing system;
means for presenting a first set of image data from a first image handling
application in a second window in said first data processing system;
means for importing said first image data from said second window to said
first window;
said sending means, sending said first image data to said second data
processing system from said first window;
wherein the means for importing and presenting image data is a screen
capture module which can capture image data from the first image handling
application even if the first image handling application cannot interact
with a utility of the operating system which transfers data between
applications.
3. A first data processing system for transmission and reception of
realtime freehand drawing to and from a second data processing system,
each system having a processor coupled to a memory, a display and an input
device, each of said system having an operating system which runs a
plurality of application programs each displayed in a respective one of a
plurality of windows on the display, the first data processing system
comprising:
means for presenting a first set of points in a first window on the display
in said first data processing system, the first set of points representing
freehand drawing generated by the input device at the first data
processing system;
means for sending the first set of points to the second data processing
system;
means for receiving and presenting a second set of points in the first
window, the second set of points representing freehand drawing generated
at the second data processing system;
means for presenting a first set of image data from a first image handling
application in a second window in said first data processing system;
means for importing said first image data from said second window to said
first window;
said sending means, sending said first image data to said second data
processing system from said first window;
means for presenting a second window on the display the second window
presenting an overall image which is larger than and includes the image
presented in the first window and a rectangle indicating which portion of
the overall image is presented in the first window.
4. A first data processing system for transmission and reception of
realtime freehand drawing to and from a second data processing system,
each system having a processor coupled to a memory, a display and an input
device, each of said system having an operating system which runs a
plurality of application programs each displayed in a respective one of a
plurality of windows on the display, the first data processing system
comprising:
means for presenting a first set of points in a first window on the display
in said first data processing system, the first set of points representing
freehand drawing generated by the input device at the first data
processing system;
means for sending the first set of points to the second data processing
system;
means for receiving and presenting a second set of points in the first
window, the second set of points representing freehand drawing generated
at the second data processing system;
means for presenting a first set of image data from a first image handling
application in a second window in said first data processing system;
means for importing said first image data from said second window to said
first window;
said sending means, sending said first image data to said second data
processing system from said first window;
wherein, the first data processing system is in communication with a
plurality of data processing systems each equipped similarly to the first
data processing system, the second window presenting the overall image and
a plurality of rectangles indicating which portions of the overall image
are presented in a respective first window of each of the plurality of
data processing systems.
5. A method for transmission and reception of realtime freehand drawing
between a first data processing system and a second data processing
system, each of the data processing systems having a processor, a memory,
an input device and a display and an operating system which runs a
plurality of application programs each displayed in a respective one of a
plurality of windows on the display, the method comprising the steps of:
presenting a first set of image data from a first image handling
application in a first window on the display of the first data processing
system;
importing said first image data from said first window to a second window
on the display of said first data processing system;
presenting a first set of points representing freehand drawing in the
second window, the first set of points generated by the input device at
the first data processing system;
sending the first image data and the first set of points from said second
window to the second data processing system;
receiving and presenting a second set of points representing freehand
drawing in the second window, the second set of points generated at the
second data processing system; and
presenting a third window on the display, the third window presenting an
overall image which is larger than and includes the image presented in the
second window and a rectangle to indicate which portion of the overall
image is presented in the second window.
6. The method as recited in claim 5 wherein the system is in communication
with a plurality of data processing systems each equipped similarly to the
first data processing system, the third window presenting the overall
image and a plurality of rectangles to indicate which portions of the
overall image are presented in a respective second window of each of the
plurality of data processing systems.
7. The method as recited in claim 6 which further comprises the step of
sending the image presented in the second window of the first data
processing system to the plurality of data processing systems to assure
that the same image is shared by the plurality of data processing system
for the portion of the overall image presented by the second window of the
first data processing system. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
DESCRIPTION
Field of the Invention
The present invention relates generally to electronic data processing
systems which allows real time conferencing of image data. More
particularly, it relates to a method and apparatus for providing a
realtime conference facility which transmits freehand drawings, graphical
data, image data and screen capture bitmap data between remote data
processing systems connected by a variety of communications media.
BACKGROUND OF THE INVENTION
Communication activities such as face to face meetings are a large part of
the typical businessman's daily activities. It is recognize that a meeting
with all the necessary parties, which allows a spontaneous exchange of
information is often the spark which leads to important decisions.
However, there is a great desire to avoid the cost, dislocation delays and
inconveniences associated with travel. While voice telephone communication
is readily available for long distance communication, it is limited in the
types of information which can be transmitted. While there have been many
efforts in the past to provide automated communication facilities which
integrate voice and image data, relative few of these system are
commercially available. The systems are expensive and typically require
high speed phone lines as well as special video equipment.
Many conferencing systems have been proposed in the prior art. All of these
system, while providing image data in addition to voice data, require
dedicated proprietary hardware. All parties must have the same equipment
and are often medium specific. A given prior art system may operate
effectively on telephone lines, yet have no provision for local area
network or other data line communication. The necessity for specialized
hardware has limited the appeal and acceptance of the prior art
conferencing systems. Other deficiencies in the prior art include the
limitation of the sources of image data transmitted to the remote data
processing systems. A particular conferencing system will contain
provisions for input from hand drawn messages from a digitizing tablet,
but will contain no provision for image input from scanners or other
nonvolatile storage of images. Other prior art systems may contain
provisions for facsimile or video data, but contain no provision for hand
writing data. These deficiencies might be overcome if it were possible to
link several of the prior art systems together. However, the prior art
methods typically operate in closed environments, i.e., they do not
interact with other software or hardware, thus preventing the importation
of facilities from other transmission devices or software.
In view of the deficiencies of the known prior art it would be advantageous
to provide a method and generalized apparatus which allows multiple users
to carry on long distance conferencing activities independent of special
hardware. Optimally, the apparatus and method would allow input from a
variety of graphical and image sources, as well as providing an open
architecture to take advantage of feature not originally programmed or
provided and to keep up with future advances in image devices and
hardware.
SUMMARY OF THE INVENTION
It is therefore an object of the invention to allow multiple users to
interact in a realtime conferencing facility supporting freehand drawing,
graphical function and image data.
It is another object of the invention to provide a realtime conferencing
facility which does not require specialized hardware or a particular
communications medium.
It is still another object of the invention to allow a user to select from
a plurality of image data derived from a variety of input devices namely a
touch sensor, a scanner, a television camera and previously stored image
data from disk storage.
It is yet another object of the invention to accommodate input from other
application software which may be running on the apparatus of the present
invention.
These objects and others are accomplished by an advanced user interface
which operates in an integrated operating environment. The integrated
operating environment and its associated operating system is capable of
running a plurality of application programs running on a standard
standalone processor, such as a personal computer. In the preferred
embodiment, a processor is coupled to a system bus which is also coupled
to a memory. The memory stores a set of program instructions as code
modules. Some of these code modules contain instructions for a variety of
input devices which are also coupled to the system bus. Other code modules
contain instructions which control printer and display functions of the
data processing system. Further, the memory also contains a plurality of
applications which operate within an integrated operating environment and
pass data in a compatible manner. One of the applications provides a
realtime conferencing facility supporting freehand drawings, graphical and
image data.
The Telesketch Application an application which uses a touch interface to
provide a "real time conference facility" using freehand drawings and
images. Freehand figures can be drawn in the Telesketch window using a
touch pointing device or a mouse pointing device. The image seen in the
local Telesketch window is also seen on another window of the Telesketch
application running on a remote machine. The computer systems may be
connected via asynchronous (modem/dialup) lines, or via NETBIOS sessions
across a Token-Ring LAN. System users at each end of the connection may
draw on the shared underlying image, while speaking with each other over
the telephone. In addition to freehand drawing, Telesketch provides a
number of simple image primitives to easily produce lines, rectangles,
filled rectangles, circles, filled circles, and erased rectangles with the
touch stylus or finger. Text may also be entered into the image using the
keyboard. Images may also be scanned from documents using a scanner
compatible with the operating system running on the local machine.
Combining the Telesketch scanning and communications features makes it
possible to use Telesketch like a facsimile machine. Images may be
"captured" from the currently available other desktop application windows
in the operating environment and placed within the current Telesketch
image. In the preferred embodiment, images may also be pasted from the
Presentation Manager.TM. clipboard into the Telesketch image. The intent
of Telesketch is to focus on the remote conference capabilities provided
rather than image editing functions. It is assumed that a full-fledged
image editing program of some sort will be available and used if complex
image editing is desired. The invention allows multiple people to
participate in a Telesketch conference at the same time.
BRIEF DESCRIPTION OF THE DRAWINGS
The above objects and other objects features and advantages of the present
invention will be more fully appreciated with reference to the following
drawings.
FIGS. 1A and 1B are block diagrams of two computer systems in realtime
communication using freehand drawing and image data according to the
present invention.
FIG. 2 is a block diagram of components of the Telesketch computer module
which controls communication between the two systems depicted in FIGS. 1A
and 1B.
FIG. 3 is a pictorial representation of a display window according to the
present invention in which geometric functions and freehand drawings have
been utilized.
FIG. 4 is a flow diagram of a method according to the present invention
which can generate the pictorial representation of FIG. 3.
FIG. 5 is a pictorial representation of a display window according to the
present invention in which the user has annotated scanner input.
FIG. 6 is a flow diagram depicting a method by which the pictorial
representation of FIG. 5 may be generated.
FIG. 7 is pictorial representation of a window according to the present
invention in which a bitmap from disk storage has been read in to the
application and has been annotated by the user.
FIG. 8 is the flow diagram depicting a method of generating the pictorial
representation in FIG. 7.
FIG. 9 is a pictorial representation of the overview window feature of the
present invention.
FIG. 10 is a pictorial representation of the overview window depicted in
FIG. 9 after the user has scrolled his individual workspace.
DESCRIPTION OF PREFERRED EMBODIMENT
The invention is preferably implemented in two or more stand alone
processors such as a personal computers, i.e., the IBM PS/2.TM. computer.
These computers can be connected by any number of communication media
including a local area network (LAN) or other data link lines, ISDN or
regular phone lines, cellular phones or satellite communications networks.
Further, the invention could be implemented in a distributed data
processor such as an IBM 3090 mainframe attached to plurality of
individual workstations. In general, the invention can be implemented on
any hardware configuration which includes the components described in the
following illustrative embodiment.
The preferred embodiment of the invention utilizes two or more data
processing systems as depicted in FIGS. 1A and 1B. For clarity of
discussion, only the interconnections of the data processing system in
FIG. 1A are discussed. The data processing system in FIG. 1B in this
example is set up in an identical manner. A processor 11A is connected via
system bus 12A to a memory 13A. The memory 13A stores a set of program
instructions in the form of code modules which the processor 11A utilizes
to control the elements of the data processing systems. General operating
system functions are handled by the code in module 15A. The integrated
operating environment code module 17A allows applications stored in memory
13A to interact with each other. In the preferred embodiment discussed in
the following pages, the operating system 15A is OS/2.TM.. Also, in the
preferred embodiment, the integrated operating environment 17A is or
Presentation Manager.TM..
The memory 13A also contains code modules which control specific components
of the data processing system. The display code 19A is used to control
presentation of information to the user on the display 20A. Printer code
21A is used to control output by the printer 22A. FIG. IA also depicts a
number of input devices by which the user can input data into the data
processing system. Scanner code 23A interprets the electrical signals sent
by scanner 24A when a document is scanned and converted to a bitmap for
further processing. The keyboard driver 25A interprets user inputs from
the keyboard 26A, as the mouse driver 27A interprets electrical signals
generated by the mouse 28A. The preferred embodiment is particularly
adapted to operate with the touch driver 29 and touch sensor 30A and
stylus 31A described in commonly assigned U.S. Pat. No. 4,868,332 to E.
Greanias et al., entitled "Combined Finger Touch and Stylus Detection
System For Use On the Viewing Surface Of A Visual Display Device", filed
Jun. 26, 1986 which is hereby incorporated by reference. However, any
touch sensor device which supports handwriting or freehand drawing could
be utilized in the present invention. Also, included in the data
processing system of FIG. 1A is a read only memory 32A which contains
fixed instructions executed by processor 11A to carry out elementary
operations for the computer system. Disk storage 33A can permanently store
code modules when they are not in use in memory 13A. The asynchronous
driver code 35A and the NETBIOS driver code 37A control communications
between the data processing system via the I/O devices 38A. The
asynchronous driver is used for communication to the remote computer
system in FIG. 1B via a modem and telephone lines 39A, where as the
NETBIOS striver 37A is used for communications over a local area network
(LAN) 39A to the remote computer system in FIG. 1B. Although only two
communication media are depicted in the preferred embodiment, the present
invention can utilize any suitable transmission medium. Finally, the
memory 13A also stores several application programs 40A, 42A, 44A one of
which is Telesketch 40A. It is envisioned that one of the other
applications running on the system would be a full fledged editing
application program.
As depicted in FIG. 2, Telesketch 40 contains a number of separate program
modules. These modules work together as described below. The main window
procedure 43 is a standard Presentation Manager (PM) application window.
It handles all of the regular PM messages which result from user input
such as mouse pointer movements, button presses, and key strokes as well
as input from the touch sensor 30, scanner 24 and bitmaps from the disk
storage 33. The application 40 also handles bitmaps from a screen capture
program 41 which may optionally be incorporated in the code package with
Telesketch 40. The main window procedure 43 is responsible for updating
the common underlying image and scrolling the locally viewed image, and
all of the typical actions caused by local user input at the Telesketch
window. Further, this module 43 controls the various communication states
and connections between the other modules of the application.
Outgoing drawing and protocol messages are sent from the main window
procedure 43 to the communication router 45 if a remote connection has
been established. The communications router 45 performs preliminary
compaction 51 (compression) of outgoing messages from the main window
procedure 43 and calls the appropriate send function depending upon the
current connections to the remote data processing system. Messages
destined to be sent to an asynchronous connection such as a telephone line
are placed in a circular buffer which is used by the asynchronous send
module 47. Messages destined for a local area network (LAN) are sent
directly to the NETBIOS send module 49. Other LAN protocols could be
supported in alternate embodiments of the invention. In OS/2.TM.,
individual subprograms may be assigned individual "threads" to aid
processing by the computer system. This multithreaded technique is
analogous to multitasking except that separate portions of a single
application are divided on to separate threads, rather than separate
applications as is the case with multitasking. Because asynchronous
communication is relatively slow, Telesketch 40 has been designed so that
the asynchronous send module 47 is on a separate thread. However, because
LAN communication is very rapid, as compared to asynchronous
communication, no separate thread is needed for the NETBIOS send module
49.
Next, the Telesketch message is sent from either the async send module 47
or NETBIOS send module 49 to the message compact module 51. This module 51
compacts (compresses) the Telesketch message into a smaller format for
quicker transmission if appropriate. As a local area network has a high
band width, a great deal of information can be sent very quickly. It is
rarely necessary to compress or compact a message when the communication
with the remote data processing system is carried out over a local area
network. However, when an asynchronous phone line is used it is almost
always necessary to compress the data. The outgoing message is sent to the
async driver 35A or a NETBIOS driver 37A as appropriate. In the preferred
embodiment, these drivers are not part of the Telesketch application 40,
but are part of the operating system 15A. The async driver 35A handles all
asynchronous communication, whereas the NETBIOS driver 37A handles all
communication over the LAN.
Incoming messages from remote computer systems are handled either by the
async driver 35A or the NETBIOS driver 37A. The incoming messages are sent
to the async receive module 53 if phoneline is used as a communication
medium or the NETBIOS received module 55 if the local area network is
used. The async receive module 53 is delegated a separate thread, and
continually reads the async device driver 35 for incoming messages. When
the message comes in, it is sent to the uncompact function 57 to be
uncompacted or decompressed. A separate thread is also used for the
NETBIOS receive module 55 which continually reads the NETBIOS driver 57A
for incoming data messages. When a LAN message comes, it is sent to the
message uncompact function 57 where the message may or may not be
decompressed. The message uncompact function 57 is used to expand
Telesketch messages from their compressed format into a standard 10 byte
format if necessary. Decompression is generally not be necessary for
messages handled via over the LAN.
The message director module 59A handles incoming Telesketch messages from
the remote data system. Some processing functions are included in a
message director 59A. In some cases, however, the messages are simply
posted to the main window procedure 43A for processing.
The overview window procedure 61 handles a reduced version of the
Telesketch image displayed by the main window procedure 43 on the display
of the data processing system. It depicts the local and remote users'
screen position within the overall image. Further discussion of the
overview window in connection with FIGS. 9 and 10 is found below.
FIG. 3 is a pictorial representation of the Telesketch window according to
the present invention in which the geometric functions of Telesketch and
the freehand drawing capability of the touch sensor 30A have been utilized
to create the window image. In Presentation Manager(.TM.) 17A, a set of
rules called Common User Access (CUA) have been developed to make the
graphical user interface of applications which run on Presentation
Manager(.TM.) 17A consistent. Telesketch 40A follows the CUA rules in
presenting the information. The Telesketch window 70 has an action bar 41
from which the user can select of the Telesketch functions. The client
area 72 is used to display the shared image, common to both computer
systems engaged in electronic conference. The client area 72 is also the
primary workspace in which the user enters data into the Telesketch
application 40A.
As Telesketch 40A is a Presentation Manager(.TM.) compatible application it
can import data from other Presentation Manager(.TM.) applications. This
can be accomplished either by the screen capture function 41 or by
"cutting and pasting" output from the other application programs 42A, 44A
to the Presentation Manager(.TM.) clipboard. With the Presentation
Manager(.TM.) clipboard the other program needs to know how to interact
with the clipboard with the screen capture function 41, a screen can be
built where the application does not understand how to interact with
different PM applications. The screen capture program 41 allows the user
to build images using another application program i.e., a page maker could
be used for sizing a figure or a paint program could be used for
sophisticated pictures or a graphics program could be used to rotate a
geometric figure or perform other geometric functions.
In the Telesketch window 70 in FIG. 3, the user has assembled an image
using the geometric function of Telesketch and by freehand drawing on the
touch sensor. First, the user moves the mouse pointer or the stylus to the
draw action in the action bar 71. When the appropriate user input is made,
e.g., a double click with a mouse button, a menu pulldown appears on the
menu to allow the user to begin geometric and/or freehand drawing in the
window 70. As illustrated in FIG. 3, the user first has drawn a series of
filled circles 73 which are then connected with four lines 75, and finally
has enclosed the circles 73 and lines 75 with a rectangle outline 77.
Next, the user selects the freehand subaction from the pulldown to add the
note 79 to the client area 72 of the window 70. FIG. 4 is a flow diagram
depicting a steps which would generate the pictorial representation of
FIG. 3. In step 80 the user invokes the Telesketch application and opens
window 70. Next, the user moves the pointer to the draw action in action
bar 71, step 81. In step 82, the user selects the primitive geometric
functions to draw filled circles 73 lines 75 and a rectangle outline 77 in
the client area 72 of the window 70. Next, the user selects the freehand
function from the draw pulldown to write note 79 in the client area 72. As
the user is drawing on the touch sensor 30 that is creating the pictorial
representation depicted in the client area 72 of the window 70, the
information is being routed to the user of the computer system depicted in
FIG. 1B. In the case of freehand drawing such as the note 79 the image is
being transmitted realtime through the communication lines 39A and 39B. In
the case of the geometric FIGS. 73, 75 and 77, the image is not
transmitted until the user has finished drawing the figure as indicated by
the stylus 31 leaving the surface of the touch sensor 30. The messages are
first routed to the communication router in step 84. If the communications
between the computer system is carried out by phone line the message is
routed to the async send module 47 in step 85. Because of the slow speed
of a synchronous communication the message is compacted in step 86 by the
message compact module 51. The Telesketch message then leaves the
Telesketch application 40 and is sent to the asynchronous driver 35A of
the computer system depicted in FIG. 1A. The Telesketch message is then
routed through the I/O device 38A to phone lines 39A and 39B to the I/O
device 38B of the data processing system in FIG. B. It is then routed to
the asynchronous driver 35B of the remote data processing system in step
88. The asynchronous receive module 33 in the remote system is monitoring
the asynchronous driver 35B for incoming messages. After receipt, in step
90, the async receive module sends the compressed message to the message
uncompact module 57 to decompress the message step 90. After
decompression, the Telesketch message is sent to the message director 59
in step 91 which hands it off to the main window procedure 43 to be
displayed to the remote user in step 92.
FIG. 5 is a pictorial representation of the Telesketch window 100 in which
a newspaper article has been scanned by scanner 24 and has been reduced to
a bitmap for presentation in client area 102 by the user of the computer
system in FIG. 1A. The scanned image was then sent to the user of the
computer system in FIG. 1B who added the rectangle outline 104 and a
freehand drawing annotation 106. To scan the newspaper article into
Telesketch 40, the first user moves a pointer to the edit action in action
bar 101, to activate the scan subaction found in the pulldown of edit.
Similar to the embodiment depicted in FIG. 3, both the graphic rectangle
outline and the freehand annotation 106 are accomplished by the second
user through the use of the draw action in action bar 101.
FIG. 6 depicts a flow diagram of the steps used to generate the pictorial
representation of the Telesketch 100 in FIG. 5. In step 100, user A, the
user of the computer system in FIG. 1A, selects the edit action from
action bar 101 after the pulldown is displayed, step 111, the user selects
scan. Thereupon the user scans the newspaper article into the scanner 24
to generate the bitmap representation of the newspaper article to be
displayed in the local and remote Telesketch windows 100. The bitmap image
is then routed through the communications router 45 in the first computer
system step 113. Further, the document image is sent through to the
NETBIOS send 49 to communicate between the two computer systems beyond
local area network. As a local area network has a sufficient band width to
eliminate the need for compression, no compression of the bit image is
preformed in the message compact module 51. Instead, the next step 115 is
to route the document to the NETBIOS driver 37A in the computer system of
FIG. IA. The bitmap image is then sent via LAN 39 to the NETBIOS driver
37B of the remote computer system depicted in FIG. 1B. The bitmap image is
then routed through the NETBIOS receive 55B step 117, a message director
59B, step 118, and is displayed by the main window procedure 43B in the
Telesketch application 40B of the remote computer system, step 119. After
seeing the scanned image, the user of the remote computer system selects
the draw action from action bar 101 in step 120. The user then selects the
rectangle outline 104 in step 121 and the freehand drawing option in step
122. The Telesketch messages associated with a freehand note 106 and the
rectangle outline 104 are then routed back to the computer in FIG. 1A
steps 123 through 128. First, the Telesketch message goes through the
communication router 45 in step 123, then to the NETBIOS send module 49 in
step 124 and finally to the NETBIOS driver 37B of the remote computer
system. The Telesketch message is transmitted over the LAN 39 and received
by the NETBIOS driver 37A of the computer depicted in FIG. 1A. The
Telesketch message is then routed through the NETBIOS received module 55
step 127. Further, it is taken by the message director 59A in step 128 to
be presented to the first user by the main window procedure 43 as the
display is repainted in step 129.
FIG. 7 is a pictorial representation of a window according to the present
invention in which a bitmap from disk storage 33 has been read into the
client area 132 of the Telesketch window 130. The image from disk store 33
is copied into the window by selecting the file action in action bar 131.
When the file action is selected the pulldown displays an open subaction
which acts as a standard open dialog box in standard CUA applications. The
dialog box allows a user to select the bitmap file from disk storage 33 to
be loaded into the Telesketch window 130. As indicated in previous
examples, the freehand drawing 134 can be entered into the Telesketch
window 130 by selecting the draw action from the action bar 131.
FIG. 8 shows a flow diagram for generating the contents of a Telesketch
window 130 depicted in the pictorial representation of FIG. 7. No
messaging function to the remote computer system is depicted in FIG. 8. In
step 140, the user selects a file action from action bar 131. Next, the
user selects the open subaction in the pulldown step 141 and imports the
bitmap from disk storage 33 in step 142. After the Telesketch window 130
has been filled by the bitmap image the user selects the draw action from
the action bar 131 in step 143. Freehand in the preferred embodiment is a
default action 144. The user then annotates the bitmap image in step 145.
INTERACTING WITH THE PRIMARY WINDOW
To draw into the Telesketch primary window, the user touches down with the
stylus or finger, and moves the pointer around the window. An "ink trail"
is created, just like drawing with a pencil and paper.
The start-up window is sized by the shell using FCF.sub.-- SHELLPOSITION.
An underlying bitmap image is first created with an actual size the same
as the full screen size. The image displayed in the primary window can be
scrolled as desired, using the standard scroll bar controls on the frame.
If the frame window is sized, the underlying image remains in the same
position relative to the physical screen. It will not necessarily remain
in the same offset relative to the image origin. This has proven to be
more intuitive to the user when scrolling and sizing images.
Quick window repainting and scrolling is accomplished by using a bitmap
presentation space for the underlying image. All drawing done by the user
is drawn into the Telesketch window and also into hpsWork. When a window
refresh is needed, i.e, when a WM.sub.-- PAINT message occurs, the client
window is redrawn by simply copying the underlying image from the bitmap
presentation space into the client area. Scrolling offsets (controlled by
the scroll bars) control which portion of the bitmap is copied into the
visible Telesketch window.
If a new image is loaded (using the File+Open, Scan, or Send/Receive image
operations), the underlying Telesketch image changes size, in PU.sub.--
PELS units, not actual size, to match that of the loaded image.
If the user is currently connected to a remote Telesketch system, pointer
movements and drawing on either side will also appear on the other end of
the communication, if one user is viewing and drawing into a portion of
the Telesketch image while the remote person is viewing another portion
(the image windows are scrolled differently), the updates will occur
relative to the underlying Telesketch image. They will be viewable again
if the user scrolls to the corresponding portion of the image.
ACTION BAR AND PULL-DOWNS
The Telesketch windows depicted in FIG. 3, 5, and 7 provide many more
features than described above. A more detailed description of the action
bar action and pull-down menu subactions follows. The action bar main menu
has the following pull-down structure:
File
1. New (disabled, never used)
2. Open . . .
3. Save (disabled, never used)
4. Save As . . .
5. Print
Edit
1. Mark
2. Copy
3. Paste
4. Print Mark
5. Scan . . .
a. Quick Scan
b. Full Page . . .
c. Using Frame . . .
d. Using Coordinates . . .
6. Erase
7. Capture Screen
8. Send Image
Connection
1. Call . . .
2. Hang Up . . .
3. Beep
Setup
1. Directory . . .
2. Local Name . . .
3. Overview Window
4. Send on Paste
5. Send on Capture Screen
6. Send on Scan
7. Send on Open
Draw
1. Freehand (checked at start-up, default drawing mode)
2. Line
3. Circle
4. Rectangle
5. Erase Rectangle
6. Fill
7. Text
Help
1. Help for help . . .
2. Extended help . . .
3. Keys help . . .
4. Help index . . .
5. Tutorial . . . (disabled, never used)
6. About . . .
These options function as described below. Some of the pull-down choices
described in the following are provided in Telesketch for CUA conformance,
but are never available. Such options are displayed with "disabled"
attributes.
FILE
The File option is used to save and load image files from disk.
New: (This option is disabled and never used.)
Open: This option displays a standard Open dialog box, like those in
standard CUA advanced interfaces. It lets a user to select which bitmap
file is to be loaded as the current image. The loaded image is initially
scrolled to the upper left of the Telesketch window.
Save: (This option is disabled and never used.)
Save As: This option displays a standard Save As dialog box, like that
shown in CUA Advanced Interfaces. It lets a user specify the name of a
file in which to store the current bitmap image. Images are stored only as
BMP (bitmap) files.
Print: This option sends the current image to the default PM print spooler.
EDIT
This menu option is for manipulating the current Telesketch image in some
way.
Mark: This option works the same way as the Mark feature of the OS/2
Windowed Command Prompt. It is used to mark an area displayed on the
current image for subsequent Copying to the Clipboard. Selecting Mark
toggles on and off the Mark mode, as indicated by a checkmark next to the
pull-down item.: When in non-Mark mode, the mouse pointer or touch stylus
is used to draw into the current image in the primary window.
When in Mark mode the pointer changes to a NSEW 4-arrowed symbol. When a
touchdown (WM.sub.-- BUTTON1DOWN) occurs, it defines a corner of a "rubber
band" rectangle for a custom tracking routine. The user moves the pointer
around the window to drag the opposite corner of the rectangle and stretch
the rectangle; the rectangle is redrawn each time to indicate which area
is being marked. Unlike WinTrackRect, this routine will allow the user to
track the rectangle in any direction.
The marking rectangle is "fixed" into place by lifting off the window or
releasing a mouse button). The marked area is defined as the area included
inside the rectangle, including the edges of the rectangle. The marked
area is relative to the area being viewed in the client window, rather
than the underlying image. The user may change the marked area by touching
down again. At this point, the previous mark rectangle outline is erased,
and the new tracking operation starts. The tracking rectangle cannot be
stretched outside the primary Telesketch client area window. The area to
be marked must be completely viewable in the Telesketch primary window. If
the image is scrolled, the marked are remains constant with respect to the
new portion of the image being viewed. The Mark function remains active
until explicitly toggled off again by the user, or when a "load image"
function occurs (Open, Send/Receive, or Scan).
Copy: Copy is only available if an area has been marked with the Mark
pull-down.
When this is selected, the current marked area in the client window is
copied as a bitmap image to the PM Clipboard. It is copied in the same
bits-per-pel format as it exists in the underlying Telesketch image which
may not necessarily match the full capabilities of the display device.
Paste: This option is only selectable if the PM Clipboard currently
contains a bitmap image.
When selected, the pointer goes into tracking mode with a rectangle size
equal to the image size currently found in the Clipboard so the image can
be placed. The user moves the tracking rectangle around the Telesketch
client area as a normal tracking operation.
Tracking is started so that no portion of the tracking rectan | | |
