 |
Description  |
|
|
FIELD OF THE INVENTION
This invention relates to a user interface or shell of an operating system, and more particularly relates to the incorporation of rich and dynamic multimedia content into such an interface.
BACKGROUND AND SUMMARY OF THE INVENTION
It is now common for operating systems to have a shell which provides a graphical user interface (GUI). The shell is a piece of software (either a separate program or component part of the operating system) that provides direct communication
between the user and the operating system. The graphical user interface typically provides a graphical icon-oriented and/or menu driven environment for the user to interact with the operating system.
The graphical user interface of many operating system shells is based on a desktop metaphor. More specifically, the graphical user interface is intended to create a graphical environment which simulates working at a desk. These graphical user
interfaces typically employ a windowing environment with a desktop.
The windowing environment presents the user with specially delineated areas of the screen called windows, each of which is dedicated to a particular application program, file or document. Each window can act independently, as if it were a
virtual display device under control of its particular application program. Windows can typically be resized, moved around the display, and stacked so as to overlay another. In some windowing environments, windows can be minimized to an icon or
increased to a full-screen display. Usually, the windows have a top to bottom order in which they are displayed, with top windows at a particular location on the screen overlaying any other window at that same location. The top-most window has the
"focus" and accepts the user's input. The user can switch other windows to the top by clicking with a mouse or other pointer device, or by inputting certain key combinations. This allows the user to work with multiple application programs, files and
documents in a similar manner as physically working with multiple paper documents and items which can be arbitrarily stacked or arranged on an actual desk.
The desktop of the graphical user interface is a screen display containing icons representing programs, files and resources available to the user. As such, the desktop acts as a launching point for running application programs, opening documents
or files, and initiating operating system
services. In accordance with the desktop metaphor, the desktop simulates the top of an actual desk on which various work items are made available to the office worker. The desktop in some graphical user interfaces thus includes icons
representing resources found on a real desk, such as a file cabinet, telephone, wastebasket, and scratchpad, which are used to access their computer equivalents. In typical graphical user interfaces, the desktop always remains as a full-screen
background display relative to the windowing environment. In other words, the desktop cannot be moved or resized, and all visible windows of the windowing environment overlay the desktop as would paper documents and other items on top of an actual desk.
A drawback to many prior desktops is their limited capacity to present multi-media content enhancements. For example, the shell of the Microsoft Windows.RTM. 95 operating system provides a graphical user interface with a windowing environment
and a desktop. As a default, this desktop includes a "my computer" icon, a "network neighborhood" icon, and a "recycle bin" icon against a solid color background, and also includes a task bar along a bottom edge of the screen with a "start" button for
menu-driven interaction. It is possible to add additional icons onto this desktop to represent other application programs, documents, files, and resources. The start menu also can be customized to include additional items, such as for launching
particular applications, and opening documents or files. Further, a graphic image can be selected as a background of the desktop (called "wallpaper") in place of the default solid color background.
Other aspects of the Windows.RTM. 95 shell also are limited in their capacity to present multi-media content enhancements. In particular, the shell provides windowed folder views accessed through the my computer and network neighborhood icons
that represent the contents (i.e., files and sub-folders) of a directory or folder in the file system of the computer or a connected local area network (LAN). The folder views have four display modes: large icon, small icon, list and detail. In the
icon modes, the folder view displays icons representing the files and sub-folders in a window against a white background. The icons used to represent the files and sub-folders in the folder view are dependent on the type or format of the file, e.g.,
documents having a Microsoft Word format are represented with an icon indicative of that application program. In the list and detail modes, the folder view displays a text listing of the files and sub-folders in its window also against a white
background.
An add-on product for the Microsoft Windows.RTM. 95 operating system, called the Microsoft Windows.RTM. 95 Plus Pack, includes packaged enhancements called "themes" for the graphical user interface of the Windows.RTM. 95 operating system.
Each theme includes a group of resources which alter the appearance and feel of the graphical user interface. These resources include substitute icons, mouse pointer graphics and animations, sounds, a wallpaper, and a screen saver. A particular theme
can be selected and applied to the graphical user interface using a themes applet which the plus pack installs into the Windows.RTM. 95 control panel (an application program group which includes small application programs or applets that control various
aspects of the operating system). Again however, the themes provide only limited multi-media content enhancements to the desktop. The applied theme can change the desktop's wallpaper, and the graphics of the my computer, network neighborhood and
recycle bin icons.
In contrast to the limited capacity for multi-media enhancement on the desktop, multi-media content commonly appears in a windowed application program in the graphical user interfaces of Windows.RTM. 95 and like operating system shells.
Application programs which present multi-media content include desktop publishing, video games, multi-media encyclopedias and like references, Internet browsers, and many others. Since the windowed application programs are separate and independent of
the desktop, the multi-media content presented in the application program windows cannot effectively enhance the presentation on the desktop itself.
Further, multi-media content is made available in numerous formats. Still images are available in JPEG (Joint Photographic Experts Group), GIF, BMP (Windows.RTM. bitmap), and other file formats. Sounds are distributed in WAV (wave), MIDI and
other file formats. Video is distributed in MPEG (Motion Picture Experts Group), AVI and other file formats. The hypertext markup language (HTML) format is widely used to distribute documents or pages including text, images, video and sound on the
World-Wide Web of the Internet. Three dimensional environments are now being developed in virtual reality markup language (VRML) and other formats. These various multi-media formats provide a facility for expressing multi-media content, but do not of
themselves provide a facility for providing desktop functionality with multi-media enhancement.
The present invention provides multi-media content enhancements to the desktop of an operating system's graphical user interface. In one system according to the invention, an operating system shell synthesizes the display for the desktop into a
hypertext multimedia document format (the HTML format, for example). The synthesized document includes the graphical icon oriented and menu driven user interface elements of the desktop, and also can include multi-media enhancements, such as text,
graphics, sounds, animations, video, hypertext links, etc. These enhancements can add informative or explanatory content to the desktop, or otherwise customize the appearance and/or behavior of the desktop. The shell also acts as a hypertext multimedia
document viewing software to display the synthesized document as the desktop in a graphical user interface, preferably as a full-screen background display to a windowing environment.
According to one aspect of the invention, the shell synthesizes the hypertext multimedia document from a template which contains the multi-media enhancements or references to the enhancements. The shell includes a pre-processor which processes
the template and produces the synthesized document which is to be displayed as the desktop. This pre-processor converts soft or variable parameters into data in the hypertext multimedia document's format which is output in the synthesized document for
display with the multi-media enhancements. The templates also contain document data for output in the synthesized document to cause embedding of a software object or objects in the displayed view of the synthesized document. These software objects
implement the functionality of the desktop's graphical icon-oriented and/or menu driven user interface elements which control operating system and/or file system services. When the synthesized document is then displayed as the desktop in the graphical
user interface, the object(s) provide the user interface features and functionality (e.g., for launching application programs, opening documents and files, drag and drop functionality, etc.) of the desktop. By embedding the objects in the synthesized
document, these user interface features are displayed in combination with the multi-media enhancements by the shell.
According to a further aspect of the invention, the shell also synthesizes hypertext multimedia documents for display as the folder views and other displays in the shell's graphical user interface. The hypertext multimedia documents for the
various displays are synthesized from templates which are identified in a configuration or ".ini" file. When the user navigates to one of the displays, the shell looks up the appropriate template to use for the display. The shell then processes the
template into a hypertext multimedia document with embedded objects to provide the user interface elements required for the display (e.g., the graphical icons and drag and drop functionality in a folder view). The synthesized document is then displayed
by the shell. The shell is thus able to provide multi-media content enhancements to these additional shell displays.
Synthesizing the desktop and other displays from templates further allows the multi-media enhancements to be easily and flexibly added and changed. The templates are in the format of the hypertext multimedia documents, and additionally contain
directives to replace soft parameters by the pre-processor. The enhancements provided by the templates can be altered by editing the templates using hypertext document editing software or even a text editor. Alternatively, a new template or complete
set of templates can be swapped in by changing entries in the configuration file. Accordingly, the overall appearance and behavior of the shell can be immediately changed by substituting the set of templates identified in the configuration files.
In an embodiment of the invention illustrated herein, the synthesized documents for the various displays are displayed in a single designated area or frame in the graphical user interface, such as the full-screen background display area of the
windowing environment. As the user navigates from one display (e.g., the desktop) to another (e.g., a folder view), the shell switches the hypertext document displayed in the designated area in a fashion similar to navigating a hyperlink between
hypertext documents.
The shell in the illustrated embodiment further operates as viewer or browser of hypertext documents. In addition to the synthesized hypertext documents for shell displays, the shell also displays and navigates between other hypertext documents,
such as those available from the Internet. The shell thus extends hyperlink navigation and the rich multi-media content of hypertext documents to the shell's graphical user interface.
Additional features and advantages of the invention will be made apparent from the following detailed description of an illustrated embodiment which proceeds with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a computer system that may be used to implement a method and apparatus embodying the invention for incorporating multi-media enhancements to an operating system shell.
FIG. 2 is a data flow diagram showing the synthesis of a hypertext multimedia document by an operating system shell for a desktop display in a graphical user interface.
FIG. 3 is a view of a navigation bar incorporated by the shell of FIG. 2 in the folder display of FIG. 7.
FIG. 4 is a block diagram of typical data structures for an object in the computer system of FIG. 1.
FIG. 5 is an object framework in the shell of FIG. 2 which supports presenting a hypertext document incorporating graphical user interface functionality as a desktop display.
FIG. 6 is a view of a desktop display presented by the shell of FIG. 2.
FIG. 7 is a view of a folder display presented by the shell of FIG. 2.
DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS
1. Computer Overview
Referring to FIG. 1, an operating environment for an illustrated embodiment of the present invention is a computer system 20 with a computer 22 that comprises at least one high speed processing unit (CPU) 24, in conjunction with a memory system
26, an input device 28, and an output device 30. These elements are interconnected by at least one bus structure 32.
The illustrated CPU 24 is of familiar design and includes an ALU 34 for performing computations, a collection of registers 36 for temporary storage of data and instructions, and a control unit 38 for controlling operation of the system 20. The
CPU 24 may having any of a variety of architectures including Alpha from Digital, MIPS from MIPS Technology, NEC, IDT, Siemens, and others, x86 from Intel and others, including Cyrix, AMD, and Nexgen, and the PowerPc from IBM and Motorola.
The memory system 26 generally includes high-speed main memory 40 in the form of a medium such as random access memory (RAM) and read only memory (ROM) semiconductor devices, and secondary storage 42 in the form of long term storage mediums such
as floppy disks, hard disks, tape, CD-ROM, flash memory, etc. and other devices that store data using electrical, magnetic, optical or other recording media. The main memory 40 also can include video display memory for displaying images through a
display device. Those skilled in the art will recognize that the memory 26 can comprise a variety of alternative components having a variety of storage capacities.
The input and output devices 28, 30 also are familiar. The input device 28 can comprise a keyboard, a mouse, a physical transducer (e.g., a microphone), etc. The output device 30 can comprise a display, a printer, a transducer (e.g., a speaker),
etc. Some devices, such as a network interface or a modem, can be used as input and/or output devices.
As is familiar to those skilled in the art, the computer system 20 further includes an operating system and at least one application program. The operating system is the set of software which controls the computer system's operation and the
allocation of resources. The application program is the set of software that performs a task desired by the user, using computer resources made available through the operating system. Both are resident in the illustrated memory system 26.
In accordance with the practices of persons skilled in the art of computer programming, the present invention is described below with reference to acts and symbolic representations of operations that are performed by computer system 20, unless
indicated otherwise. Such acts and operations are sometimes referred to as being computer-executed. It will be appreciated that the acts and symbolically represented operations include the manipulation by the CPU 24 of electrical signals representing
data bits which causes a resulting transformation or reduction of the electrical signal representation, and the maintenance of data bits at memory locations in memory system 26 to thereby reconfigure or otherwise alter the computer system's operation, as
well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, or optical properties corresponding to the data bits.
2. Shell Overview
With reference to FIG. 2, a shell 50 for an operating system of the computer 20 (FIG. 1) provides a graphical user interface for a user of the computer to interact with the operating system. The graphical user interface includes a desktop
display 52 presented on a video screen of the computer's output device 30 (FIG. 1). The desktop display 52 preferably is presented by the shell in a windowing environment as a full-screen background display. Specifically, any visible windows that are
not minimized to an icon are displayed overlaying the desktop display 52 in the windowing environment.
In the illustrated shell 50, the shell presents a variety of views 54 (listed in table 1 below) for different aspects of the graphical user interface on the desktop display 52. These views 54 are similar to displays presented in the graphical
user interface of the Microsoft Windows.RTM. 95 operating system, but additionally include multi-media enhancements incorporated in the views in accordance with the invention. More particularly, the views 54 include the desktop and desktop container
displays of the Microsoft Windows.RTM. 95 operating system. The desktop container displays are sub-displays of the Microsoft Window.RTM. 95 desktop, that contain a group of related graphical icons for file management or for launching application
program and operating system services (e.g., My Computer, Network Neighborhood, Control Panel, etc.).
For each of the views 54 presented on the desktop display 52, the shell 50 synthesizes the view as a hypertext page 56. The illustrated hypertext page 56 is in a hypertext markup language (HTML) format. The HTML format is a standard format for
multi-media hypertext documents which is used on the Word-Wide Web portion of the Internet. (The Internet is a global network of cooperatively interconnected computer networks, consisting of millions of individual computers. A portion of the Internet
referred to as the "World-Wide Web" consists of computers, also referred to as "sites," which make multi-media documents in HTML format generally available for downloading or retrieval by individuals having a computer with Internet access.) HTML format
documents are ASCII encoded files which employ tags to designate text formatting, hyperlinks, and multi-media content to be incorporated from other resources (e.g., files) into the document. Further details of the HTML format of the illustrated
hypertext page 56 are described in the HTML Reference, attached hereto as Appendix Q.
A hypertext document is a document that contains hyperlinks. Hyperlinks are
references to other documents which are generally indicated in a displayed view of the document as a graphic, underlined text, or highlighted text, and which can be activated by user input to the viewing software to cause the viewing software to
navigate to the referenced document. A multimedia document is a document which contains or incorporates multiple forms and/or formats of information content such as text, images, sounds, software objects, video, animations, etc.
The shell 50 obtains the hypertext page 56 from which a view in the graphical user interface is synthesized from processing a stored hypertext template, or alternatively directly from a stored hypertext page. In the former case, the shell 50
includes a pre-processor 60 which synthesizes the hypertext page 56 from one or more of a set of templates 62 and one or more desktop interface controls 64. The templates 62 are files which contain data in the HTML format which is to be incorporated
into the hypertext page 56, and additionally include pre-processor directives. The directives are instructions to the pre-processor for converting soft parameters into html formatted data in the hypertext page 56. The templates for each of the views in
the illustrated shell are shown in the following Table 1.
TABLE 1 ______________________________________ Hypertext View Templates Friendly Name File Name ______________________________________ Briefcase brfcase.htm Control Panel control.htm Default folder folder.htm Desktop desktop.htm Favorites
folder favorite.htm File-system folder directry.htm My Computer mycomp.htm Network Neighborhood nethood.htm Printers printer.htm Recycle Bin recycle.htm Start Menu and subfolders startmnu.htm Workgroups domain.htm Vendor-specific workgroups
msdomain.htm Vendor-specific networks msnet.htm Vendor-specific servers msserver.htm Servers server.htm Audio CD audiocd.htm Dial-Up Networking dialupnt.htm Entire Network fullnet.htm Fonts folder fonts.htm History history.htm My Documents
mydocs.htm Network Workgroup workgrp.htm Program Files folder progfile.htm Root of data CD datacd.htm Root of floppy disk drive floppy.htm Root of hard disk harddrv.htm Windows folder windows.htm ______________________________________
In general, the templates listed in Table 1 are related one-to-one with folders and are used to synthesize a hypertext page for a display associated with the related folders. Some of the folders correspond to actual directories in a file system
of the computer's memory system 26 (FIG. 1). For example, each of the "windows folder," "root of hard disk," and "my documents" folders correspond to actual file system directories. The displays associated with these folder generally represent (at
least in part) the contents of the corresponding directory, and are called "folder views." Others of the folders (termed "virtual folders") do not correspond to any file system directory. Accordingly, the displays associated with these folders generally
do not represent the contents of a file system directory. For example, the "my computer" folder is a virtual folder.
The templates listed in Table 1 are for producing displays associated with a set of standard folders in the Windows.RTM. operating system. In addition to these standard folder templates listed in table 1, the templates 62 also can include
additional templates for non-standard folders (herein called "custom templates"), such as folders corresponding to file system directories created by a user or added by a software installation program. For example, an installation program of an
application software product (such as a productivity software, computer game, or utility software) that creates a new folder in which to install the application software's files also can add a template associated with the folder to the set of templates
62. When the newly created folder is viewed in the graphical user interface, the shell 52 uses this added custom template to produce a folder view display representing the newly created folder's contents. These added custom templates can contain
multimedia content enhancements specific to the new folder, such as graphic images, text, hyperlinks, or software objects relating to the application software product or its vendor.
The template (or alternatively stored hypertext page) to be used by the shell in synthesizing the hypertext page view in each display in the graphical user interface is identified in one or more configuration files 66. The configuration files 66
can include both local and global configuration files. More particularly, folders that are actual file system directories can contain a hidden local configuration file (named "desktop.ini" in the illustrated computer 20). (Hidden files are files having
a flag or attribute which is set to indicate that the file normally is not displayed by a file management tool, e.g., the Windows Explorer in the Windows.RTM. 95 operating system.) This "desktop.ini" configuration files stores data identifying the
template (e.g., by path and file name in the computer's file system) to be used in producing a folder view display of the folder. A listing of a representative desktop.ini file is shown in the following table 2.
TABLE 2 ______________________________________ Representative Desktop.Ini File Listing. ______________________________________ [ExtShellFolderViews] Default={FB7E5040-1F6D-11D0-89A9-00A0C9054129}
{FB7E5040-1F6D-11D0-89A9-00A0C9054129}={FB7E5040-1F6D- 11D0-89A9-00A0C9054129 {00000001-0001-0002-0003-000000000001}={25336920-03F9-11CF- 8FD0-00AA00686F13 } {00000002-0001-0002-0003-000000000002}={00020900-0000-0000- C000-000000000046 }
{00000003-0001-0002-0003-000000000003}={00020810-0000-0000- C000-000000000046 } [{00000001-0001-0002-0003-000000000001}] PersistFile="pageone.html" MenuName="friendly-name-for-view-1" ToolTipText="Html View" HelpText="This shows a HTML document"
[{00000002-0001-0002-0003-000000000002}] PersistFile="word.doc" MenuName="friendly-name-for-view-2" ToolTipText="Word Document View" HelpText="This shows a Word document" [{00000003-0001-0002-0003-000000000003}] PersistFile="Excel.xls"
MenuName="friendly-name-for-view-N" ToolTipText="Excel Spreadsheet view" HelpText="This shows an Excel spreadsheet" [{FB7E5040-1F6D-11D0-89A9-00A0C9054129}] IconArea.sub.-- Image="c:.backslash.win95B.backslash.bubbles.bmp" IconArea.sub.-- Pos=1
______________________________________
The above representative desktop.ini file begins with a section having the heading "[ExtShellFolderViews]." This section lists globally unique identifiers ("GUIDs") associated with software objects that implement views of the folder in which the
desktop. ini file is stored. A line beginning "default=" specifies a default view of the folder. The four lines below this specify alternative custom views of the folder in the format, <GUID>=<GUID>. The left GUID on each line identifies
the software object that implements the view. If there is an entry with this left GUID in the system registry for the view object (i.e., the GUID is a CLSID registered in the system registry), then the line modifies some of the view's attributes. In
such case, the right GUID is a CLSID that identifies the pre-processor 60 to be used for processing the template for that view, or alternatively identifies the viewer object (e.g., the HTML viewer 70 of FIG. 2 or other document object) which displays the
hypertext page in the view (i.e., for use when the hypertext page itself is stored rather than a template from which the hypertext page is produced). If there is no entry with the left GUID in the system registry, then the line indicates a custom view
and is unique only with the respective desktop.ini file.
More specifically, in the above representative desktop.ini file, the first or default line of the "[ExtShellFolderViews]" section identifies an object that implements a default view for the folder. The second line overrides some attributes of
one of the folder's views. A separate section at the bottom of the desktop.ini file has lines which change the "IconArea.sub.-- Image" and the "IconArea.sub.-- Pos" attributes of that view, specifically the background bitmap of the view and its position
(e.g., whether centered or tiled).
The third, fourth and fifth lines of the "[ExtShellFolderViews]" section specify custom views for the folder. The left hand identifiers are not CLSIDs of views registered in the system registry, and are unique only within the desktop.ini file.
The right hand GUIDs on these lines are CLSIDs of objects registered in the system registry. Specifically, the right hand GUIDs are CLSIDs of the HTML viewer 70, a Microsoft.RTM. Word document object, and a Microsoft.RTM. Excel document object,
respectively. (Document objects are described below.) The desktop.ini file contains a section for each of the custom views which have the line "PersistFile= . . . " When one of these views is selected for display, the respective document object of the
view is instantiated with the hypertext page, Word document, or Excel document, respectively, identified on the "PersistFile" line.
The template (or alternatively stored hypertext page) associated with a folder (whether an actual file system directory or virtual folder) also can be identified by entries in a global configuration file, which in the illustrated shell 52 is the
system registry. In the Microsoft.RTM. Windows.RTM. operating system, the system registry is a database which stores configuration information for the operating system, including information to enumerate and track applications, device drivers, and
operating system control parameters. For a detailed discussion of the registry, see Win32 Programmers Reference, Vol. 2, published by Microsoft Press, Redmond, Wash. (1993). Representative entries in the system registry for the illustrated shell 52
are listed in the following Table 3.
TABLE 3 ______________________________________ Representative System Registry Entries for Folder Views Folder Type Registry Entry ______________________________________ Virtual Folder HKCR.backslash.CLSID.backslash.{guid of virtual
folder}.backslash.shellex.backslash.ExtShellFolderViews.backslash .{guid of view} PersistFile | | |
