 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates generally to the area of image display, and more
particularly to a method and system for displaying recursively images on a
display screen of a mobile device, wherein the images are of larger
dimensions than that of the display screen and the mobile device may
include a cellular telephone, a two-way pager, and a palm-sized computing
device:
2. Description of the Related Art
The Internet is a rapidly growing communication network of interconnected
computers and computer networks around the world. Together, these millions
of connected computers form a vast repository of multimedia information
that is readily accessible by any of the connected computers from anywhere
at any time. To navigate the Internet, the connected computers, such as
workstations and desktop computers, typically operate what is commonly
called "browser", an application (client) program that generally uses the
Hypertext Transfer Protocol (HTTP) to make requests to the multimedia
information throughout the Internet. These computers capable of operating
the browser using HTTP are generally powerful, having sufficient computing
resources, such as processing power, memories, display capabilities and
user interfaces. When the multimedia information is static image data,
only constrained to the local memory, the image data can be downloaded to
the computers and then manipulated and displayed. To provide mobility and
portability of the Internet, interactive two-way communication mobile
devices are introduced and capable of communicating, via wireless data
networks, with the Internet. The interactive two-way communication mobile
devices, including two-way pagers, cellular phones, palm-sized computing
devices and personal digital assistant (PDA) apparatuses are among the
fastest emerging communication devices introduced recently that enable
users to receive, collect, analyze, review and disseminate information as
they travel or move about. Contrary to the computers coupled to the
Internet, the mobile devices are characterized by thin designs in terms of
power consumption and cost constraints that may include less memory, Lower
processing power, etc. Consequently, displaying images on the mobile
devices has been a difficult task. Further, the input interface provided
is often limited to a keypad or soft keys that have far fewer available
keys than a PC keyboard does. Furthermore, the mobile devices, such as the
cellular phones, are often lack of a pointing mechanism as opposed to a
mouse coming with nearly every desktop computer. Hence interacting with a
displayed image becomes even a more difficult task.
There is, therefore, a great need for a solution that allows an image to be
displayed on the screen of a mobile device without taking up the local
memory therein and further permits a user to interact with the image when
being displayed.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above described
problems and needs and has particular applications to the navigation of
Internet web pages by two-way interactive communication mobile devices.
The design constraints commonly seen in the mobile devices includes a
relatively slow processor, less memory and limited graphics make it
economically and technically impractical for the mobile devices to operate
a regular browser so as to display images created for those personal
computers. The principles of this invention, nevertheless, make it now
possible for the thin designed mobile devices to effectively interact with
the Internet to display recursively those images.
According to one aspect of the present invention, the image requested by a
mobile device is first processed in a server device. In accordance with a
set of parameters about the screen of the mobile device, the requested
image is transformed to a reduced version that fits well into the screen.
In addition, the reduced version is inherently divided into a number of
subareas, each embedded a link to a detailed version thereof and
associated with one of the keys in the mobile device. When the reduced
version is displayed on the mobile device, a user decides to view an area
that falls into one of the subareas, a corresponding key is activated. A
new request including the link is sent to the server device that
consequently returns the detailed version. Similary, the detailed version
is inherently divided into the same number of subareas. As a result, all
parts in the original image can be recursively viewed.
The advantages of the invention are numerous. Different embodiments or
implementations may yield one or more of the following advantages. The
original image does not have to be downloaded into a mobile device, which
aleviates a very high bandwidth requirement on a wireless data network and
reduces traffics therein. A user will not experience any significant delay
when requesting an image from a web site as now only a fixed number of
image data are transmitted every time a request is made. Further the
images could be created independently from the screen sizes of mobile
devices that are often varying in sizes and types.
Accordingly, one of the objects of this invention is to provide a generic
solution to two-way communication mobile devices that can effectively
interact with a data network, such as the Internet, for images.
Other objects, together with the foregoing are attained in the exercise of
the invention in the following description and resulting in the embodiment
illustrated in the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will be readily understood by the following detailed
description in conjunction with the accompanying drawings, wherein like
reference numerals designate like structural elements, and in which:
FIG. 1 illustrates a schematic configuration in which the present invention
may be practiced;
FIG. 2 depicts a typical digital cellular phone that can be used as one of
the mobile devices in the arrangement of FIG. 1 to practice the present
invention;
FIGS. 3A and 3B show, respectively, a functional block diagram of a link
server device and a mobile device according to an embodiment of the
present invention;
FIG. 4 shows an exemplary structure of possible user accounts managed by an
account manager in a server device;
FIG. 5A shows an exemplary image that may be fetched from a resource (e.g.
service server) on the Internet;
FIGS. 5B, 5C, and 5D, respectively, shows a reduced image being displayed
on a screen of a mobile device;
FIGS. 6A and 6B illustrate, respectively, a process flowchart of the image
navigation process in a mobile device and a link server according to one
embodiment of the present invention; and
FIG. 7 shows an exemplary image hierarchy.
DETAILED DESCRIPTION OF THE INVENTION
According to the principles of this invention, an image being requested by
a mobile device is first preprocessed in a server that reduces the
dimensions of the image to proper dimensions for display on the screen of
the mobile device. The server further provides respective hyperlinks to
subareas of the reduced image divided by a grid so that a detailed version
can be provided when one of the subareas is activated. Similarly, the
detailed version is divided by the grid and hyperlinks are respectively
provided to the subareas in the detailed version. As a result, a user is
able to see recursively all the details that the original image provides.
It will be appreciated that the present invention disclosed herein can be
advantageously used on portable devices with small screens to access
images originally created for regular personal computers. In the following
description of the present invention, numerous specific details are set
forth in order to provide a thorough understanding of the present
invention. However, it will become obvious to those skilled in the art
that the present invention may be practiced without these specific
details. The description and representation herein are the common means
used by those experienced or skilled in the art to most effectively convey
the substance of their work to others skilled in the art. In other
instances, well known methods, procedures, components, and circuitry will
not be described in detail to avoid unnecessarily obscuring aspects of the
present invention.
Referring now to the drawings, in which like numerals refer to like parts
throughout the several views. FIG. 1 illustrates a schematic configuration
in which the present invention may be practiced. Landnet 100 is a landline
network that may be the Internet, the Intranet and a data network of other
private networks. Coupled to landnet 100 are a personal computer (PC) 110
and a network server 104. Personal computer 110 may be a Pentium-based
desktop personal computer. Preferably, personal computer 110 runs a
HyperText Markup Language (HTML) browser, such as Netscape Navigator, via
landnet 100 using HyperText Transfer Protocol (HTTP) to access information
stored in network server 104 that may be a workstation from Sun
Microsystems, Inc. The information stored in network server 104 may be
hypermedia information including image data created and accessible by
personal computer 110 for display thereon.
There are n mobile devices 106 serviced by airnet 102. Mobile devices 106
herein are considered interactive two-way communication devices that
include, but are not limited to, mobile computing devices, cellular
phones, palm-sized computing devices with PDA (Personal Data Assistants)
functionality and Internet-capable appliance remote controllers, and
capable of communicating wirelessly with antenna 108 via airnet 102. For
simplicity, antenna 108 also represents a wireless carrier infrastructure
that generally comprises a base station and an operations and maintenance
center. The base station controls radio or telecommunication links with
mobile devices 106. The operations and maintenance center comprises a
mobile switching center performing the switching of calls between the
mobile devices and other fixed or mobile network users. Further the
operations and maintenance center manages mobile account services, such as
authentication, and oversees the-proper operation and setup of the
wireless network. Each of the hardware components and processes in carrier
infrastructure 108 are known to those skilled in the art and not to be
described herein to avoid unnecessarily obscuring aspects of the present
invention.
Between landnet 100 and airnet 102 there is a link server device 114
functioning as a bridge between the two networks 100 and 102. Link server
device 114, also referred to as proxy server or wireless data server or
network gateway server, may be a workstation or a personal computer. Link
server 114, loaded with many processes including compiled and linked
version of one embodiment implementing the present invention, couples
airnet 102 to landnet 100 and performs many functions to be described in
detail below. Those skilled in the art understand that any server devices
coupled to landnet 100 may be used to provide functions carried out by
link server device 114.
To facilitate the description of the present invention, FIG. 2 depicts a
typical GSM digital cellular phone 200 that can be used as one of the
mobile devices 106 in the arrangement of FIG. 1 to practice the present
invention. Cellular phone 200 includes a small screen 202 and an extended
phone keypad 204. Screen 202 is typically a LCD display capable of
displaying perhaps four lines high by twenty or more characters and the
graphics capabilities thereof are limited. Extended phone keypad 204
comprises, preferably, a regular phone keypad 206, a pair of generic keys
208 and 210 and positioning key 212. Generic keys 208 and 210, if there
are any, are used to activate soft keys displayed in screen 202 and
positioning key 212 is to reposition an element indicator or a cursor to
activate, for example, one of the hyperlinks displayed in screen 202. It
should be understood; however, generic keys 208 and 210 and positioning
key 212 are not necessary in practicing the present invention, they can be
replaced by a set of designated keys in regular phone keypad 206 but
provide preferred convenient means for a user to interact efficiently with
the phone 200. It should be noted that some of the mobile devices
sometimes have no physical keys at all, such as those palm-size computing
devices that, however, use soft keys or icons for users to activate them
by using a finger or a pseudo-pen. In the following, unless otherwise
specifically described, keys or buttons are generally referred to as
either the physical keys or soft keys.
Each of hardware components in digital cellular phone 200 is known to those
skilled in the art, so the hardware components are not described in detail
herein. According to one embodiment, compiled and linked processes of the
present invention are stored in random access memory (ROM) as a client
module and a support module. Upon activation of a predetermined key or key
sequences utilizing keypad 204, a physical layer processor in digital
cellular phone 200 causes the client module, via a radio transceiver
therein, to communicate with link server 114 of FIG. 1. It is generally
understood that a computing device equipped with an HTML browser using
HTTP can access image data in a network server and download the image data
for local display. However, this paradigm is not desirable when the
computing device is a thin client, such as a cellular phone 200 of FIG. 2.
Current HTTP requires considerable computing power and network bandwidth
resources. For example, a request from the computing device to establish a
communication session with a network server may require an exchange of a
number of data packets. In addition to the resources required to implement
HTTP, significant resources must be supported in the computing device to
request, format, process and display image information. This is not a
significant disadvantage in many situations because the computing device,
including personal computers and workstations coupled to a network
generally has sufficient computing power, memory and display capabilities.
Cellular phone 200 or mobile devices 106 of FIG. 1, however, do not
currently have the computing resources to implement HTTP to run an HTML
browser. To keep the portability and usability of a mobile device, there
are many design constraints, as opposed to those of a desktop computer,
including limited maneuvering mechanism, low memory and display
capabilities. For example, a typical color image for display on a desktop
computer is 640.times.480 that have megabyte data. It would be a forbidden
task to download any data of that magnitude in a wireless network.
From the perspective of displaying on a mobile device a color image
generated for a desktop computer, it may be understood to those skilled in
the art that the mobile device is not a mere replacement of a desktop
computing device or a combination of a wireless communication module with
a personal computer. Further, it may be appreciated that making a mobile
device, such as cellular phone 200, capable of displaying such images is a
significant departure from prior art systems.
Referring now to FIGS. 3A and 3B, there are respectively shown a functional
block diagram of a link server device and a mobile device according to an
embodiment of the present invention. Link server device, or simply link
server 300, that may represent link server 102 of FIG. 1, is typically a
server computer and mobile device 350 may, for example, correspond to one
of mobile devices 106 of FIG. 1 or cellular phone 200 of FIG. 2. To avoid
obscuring aspect of the present invention, well known methods, procedures,
components and circuitry in link server 300 and mobile device 350 are not
described in detail.
Link server 300 comprises a landnet communication protocol (LCP) interface
302 that couples to landnet 304, a wireless communication protocol (WCP)
interface 306 that couples to a wireless network 308 via a carrier's
infrastructure (not shown in the figure). LCP interface 302 implements a
communication protocol operated in landnet 304. Generally, LCP interface
302 is an HTTP interface when landnet 304 is the Internet. Similarly,
airnet 308 may support a wireless communication protocol suitable for the
characteristics of a particular wireless network such as Cellular Digital
Packet Data (CDPD), Global System for Mobile Communications (GSM), Code
Division Multiple Access (CDMA) and Time Division Multiple Access (TDMA)
to name a few. One of the common wireless communication protocols is
Handheld Device Transport Protocol (HDTP) (formerly known as Secure Uplink
Gateway Protocol (SUGP)), which runs on User Datagram Protocol (UDP). In
this embodiment, WCP interface 306 is implemented with a UDP or HDTP
interface. Other possible protocols for airnet 308 may include HTTP,
Wireless Session Protocol (WSP) and other emerging protocols that
facilitate the transport of data in a wireless data network.
To facilitate the description of the present invention, the wireless
communication protocol is HDTP according to one embodiment. It can be
appreciated by those skilled in the art that this particularity does not
imply any limitation of the present invention to this exemplary
communication protocol. As described above, the wireless communication
protocol may be HTTP in which case both landnet 100 and airnet 102 are
supporting the same protocol and there is no need to perform data mapping
between two protocols. This is a typical case when link server 300 is
implemented in one of the server devices on the Internet.Link server 300
further comprises a server module 310 coupled between LCP interface 302
and WCP interface 306. Server module 310, typically loaded in a memory,
performs traditional server processing as well as protocol conversion
processing from one communication protocol to another communication
protocol. In particular, the protocol conversion processing includes
protocol conversion between HDTP/UDP and HTTP/TCP according to the
embodiment.
In server module 310, account manager 312 manages through account interface
314 a plurality of user accounts for all the mobile devices serviced by
link server 300. Each of the mobile devices, such as 350, is assigned with
a device identifier or identification (ID). Device ID may be a phone
number of the device or an IP address or a combination of an IP address
and a port number, for example: 204.163.165.132:01905 where
204.163.165.132 is the IP address and 01905 is the port number. The device
ID is further associated with a subscriber ID created and administrated by
a carrier administrating link server 300 as part of the procedures to
activate a subscriber account for mobile device 350. The subscriber ID may
take the form of, for example, 861234567-10900_pn .mobile.att.net by AT&T
Wireless Service, and is a unique identification to a mobile device. In
other words, each of mobile devices 106 serviced by link server 114 in
FIG. 1 has a unique device ID that corresponds to a respective user
account managed in link server 114.
Additionally, account manager 312 is responsible for creating a user
account for a mobile device that anonymously communicates with link server
114. In this case, account manager 312 ensures proper (limited) access of
the anonymous mobile device to services provided by link server 114. FIG.
4 shows an exemplary structure 400 of possible user accounts managed by
account manager 312. It should be noted that the user accounts may not be
physically located in link server 300 and in reality can be remotely
located in one of the computing devices coupled to landnet 104. Through
account interface 314 that has proper and secure access to the user
accounts, account manager 312 can conduct the duties of account management
as discussed more below. Device ID column 402 is filled with the device
IDs of mobile devices that further respectively correspond to subscriber
IDs in subscriber ID column 404. Credential information column 406 lists
respective credential information needed to access each associated
account. User info 408 may include the account configuration information,
for example, device ID "6508171453" is a mobile phone that is
pre-configured to work in a GSM network and, probably, may be provided
with an option to switch to a CDMA network if the user moves into an area
that has only CDMA network available. Further entries in user info column
408 may include pointers or linkages 410 to other account related
information, such as parameters of the mobile device associated with the
account. The device parameters may include the type of the mobile device,
the user input mechanism thereof, the screen sizes (i.e. width and height)
and etc.
Returning now to FIGS. 3A and 3B, maintaining a database of user accounts
permits account manager 312 to conduct authentication and verification
processes of the subscribed mobile devices and control access to provided
services by all mobile devices (subscribed or anonymous devices) in
wireless data network 308. More importantly in the present invention,
account manager 312 is responsible for providing device information to
facilitate the interactive display of an image fetched from the Internet
104.
Referring to FIG. 5A, there is shown an exemplary image 500 that may be
fetched from a service server on the Internet. Image 500 is go Am
generated by the service server, for example, MapQuest.com, Inc., to be
displayed on a desktop computer with a large screen display. With a mobile
device, image 500 would not be properly displayed on a screen of less than
3 inches commonly seen in the mobile device. According to one embodiment
of the present invention, image 500 is first fetched into link server 300
when image 500 is requested by mobile device 350 through link server 300.
Image 500 is then preprocessed with the parameters of mobile device 350
provided in the associated account. The parameters used may include the
screen size and the type. The purpose of the preprocessing is to ensure
that a reformatted version of image 500 can be properly displayed. Image
500, for example, has a size of 640 by 480 pixels and a screen of mobile
device 350 can display an image of 70 by 60 pixels. One aspect of the
preprocessing is to reduce or decimate image 500 to the size of 70 by 60
pixels. A linear interpolation approach, known to those skilled in the
art, is used to reduce image 500 to the appropriate size although there
are other methods that are available to achieve essentially the same.
FIG. 5B shows a reduced image 504 being displayed on screen 502 of mobile
device 350. With the absence of a pointing mechanism, which is commonly
seen in a cellular phone, one of the important features in the present
invention is to provide navigation in a chain of areas of interest. The
numeric keys in a phone keypad are arranged in a 3 by 3 matrix and each of
the keys is designated to activate a sub-region in the displayed image
when being pressed or activated. Reduced image 504 is geometrically and
inherently divided into 9 hot or sensitive areas, each corresponding to
one of the keys. As shown in FIG. 5B, the top three areas corresponding
respectively to the keys "1", "2", and "3", the middle three areas
corresponding respectively to the keys "4", "5", and "6", and the bottom
three areas corresponding respectively to the keys "7", "8", and "9". Each
of the sensitive areas, referred to as "1", "2", "3", "4", "5", "6", "7",
"8", and "9" area, respectively, is provided by a hyperlink to another
image.
To be specific, a user is interested in the Bay area in California which is
too small to be viewed clearly together with all the states of the country
being displayed in the screen 502. Since the state of California appears
located in the "4" area of the middle row, the key "4" is then activated
to request a detailed version of the "4" area. FIG. 5C shows the detailed
version of the "4" area is displayed. If the user further presses the key
"4", the details of the Bay area starts to reveal in the screen as shown
in FIG. 5D. Depending on the original image 500, the details can be
requested until the original image 500 could provide.
From the image resolution perspective, the original image 500, regardless
of the dimensions thereof, can be displayed, block by block and
hierarchically, on the small screen of a mobile device.
FIGS. 6A and 6B illustrate, respectively, a process flowchart of the image
navigation process in mobile device 350 and link server 300 and should be
understood in conjunction with the previous figures and FIG. 7. The
flowchart in FIG. 6A is for the process in the mobile device. At 600, the
mobile device checks if a communication session is established or valid
with the link sever. If the communication session is not established or
expired, the mobile device at 602 sends a message to the link server to
establish a new communication session. Otherwise at 604, the mobile device
sends a request to fetch an image of interest. Typically, a request
comprises a device identification identifying the device and a first
address identifier (e.g.. a domain address) identifying the link server
and a second address identifier identifying a resource from which the
image is fetched. The resource may be another server device coupled on the
landnet or the Internet and typically provides hypermedia information
including image data for others to access.
After the request is sent, the mobile device awaits a response from the
link server at 606. Generally, there is a time limit, namely after a
predefined period if no response has been received, the mobile device
aborts the request and sends another request. At 608, an image is
received. It must be pointed out, the received image, referred to as a
reduced image, is not the image originating from the resource. The reduced
image, as the name suggests, is a transformed image from the original
image and preferably fits perfectly in the screen of the mobile device.
The reduced image is inherently divided into a number of subareas, each of
the subareas embedded with a hyperlink to a detailed version thereof. The
number of the subareas is typically the number of keys that are designated
or available for navigation. For example, there are at least 10 numerical
keys in a typical phone keypad, here the reduced image is preferably
divided equally into 3 b | | |
