Network management system having virtual catalog overview of files distributively stored across network domain

What is claimed is:

1. A network system comprising:

(a) a network-linking backbone;

(b) a plurality of file-servers operatively coupled to the backbone for providing file-serving services over the backbone, each file server having a nonvolatile data storage device storing a plurality of data files, the respective data storage device of each file server further having a local catalog stored within said respective data storage device for identifying each file of the respective data storage device by name and storage location; and

(c) a domain administrating server (DAS) operatively coupled to the backbone,

wherein the DAS has a domain-wide virtual catalog containing copies of the file identifying information currently stored in the local catalogs of said plurality of file-servers,

wherein the DAS has oversight means for overseeing and managing domain-wide activities including a transfer of file data from a first of the file servers to a second of the file servers, and

wherein the oversight means consults the domain-wide virtual catalog to identify the location of a source file in said first file server from which said to-be-transferred file data is to be obtained.

2. The network system of claim 1 wherein the oversight means consults the domain-wide virtual catalog to identify the name and location within the second file server of a destination directory into which said to-be-transferred file data is to be sent.

3. The network system of claim 1 wherein the DAS includes:

(c.1) historical database means for storing, in addition to the copies of the file identifying information currently stored in the local catalogs which copies define the current domain-wide virtual catalog, copies of previous domain-wide virtual catalogs, said current and previous domain-wide virtual catalogs defining a searchable, historical record of domain-wide virtual catalog snapshots,

wherein the historical database means includes searching means for searching the historical record of domain-wide virtual catalog snapshots for files according to one or more primary and secondary search fields selected from the group consisting of:

(c.1a) chronological file attributes,

(c.1b) file storage location,

(c.1c) file name, and

(c.1d) file access attributes.

4. The network system of claim 1 further comprising:

(d) a plurality of workstations operatively coupled to the network-linking backbone, wherein each workstation has a same user interface by which a user can access the domain-wide virtual catalog held in the domain administrating server (DAS).

5. The network system of claim 4 wherein:

the user interface of each workstation includes a tree listing means for displaying at the respective workstation a multi-leveled system tree having at least a Domain item and a Server item as expandable items on respective first and second levels of the multi-leveled system tree;

expansion of the Domain item produces a displayed listing of a plurality of N servers within a pre-designated current domain, each of the N servers being identified by a predefined ServerName displayed in the listing, any one of which servers can be designated as a currently-selected server; and

expansion of the Server item produces a displayed listing of a plurality of M volumes within a pre-designated current server, each of the M volumes being identified by a predefined VolumeName displayed in the listing, any one of which volumes can be designated as a currently-selected volume.

6. The network system of claim 5 wherein:

expansion of the Domain item further produces in the displayed listing of said plurality of N servers additional information regarding the location and status of each server;

said domain administrating server (DAS) includes a searchable DAS database containing said domain-wide virtual catalog and further containing said information regarding the location of each server; and

the information in said displayed listing is obtained from said DAS database.

7. The network system of claim 6 wherein:

the tree listing means generates said multi-leveled system tree to further have a Volume item and a Directory item as expandable items on respective third and fourth levels of the multi-leveled system tree;

expansion of the Volume item produces a displayed listing of a plurality of K directories within a pre-designated currently-selected volume, each of the K directories being identified by a predefined DirectoryName displayed in the listing, any one of which directories can be designated as a currently-selected directory; and

expansion of the Directory item produces a displayed listing of a plurality of J files within a pre-designated currently-selected directory, each of the J files being identified by a predefined FileName displayed in the listing, any one of which files can be designated as a currently-selected file.

8. The network system of claim 7 wherein:

the user interface of each workstation includes a file manipulating means for moving or otherwise manipulating a file designated as a currently-selected file by said tree listing means;

the domain administrating server (DAS) includes a task scheduler for scheduling domain-wide data transfers; and

the file manipulating means of each workstation submits file transfer requests to the task scheduler in order to carry out a user-defined file transfer.

9. A network system comprising:

(a) a network-linking backbone;

(b) a plurality of file-servers operatively coupled to the backbone for providing file-serving services over the backbone,

wherein each file server has a nonvolatile data storage device for storing and retrieving a plurality of data files,

wherein each file server further has an operations supporting infrastructure for supporting file storage and retrieval operations of the file server,

wherein each file server additionally has a local infrastructure monitoring and reporting agent for monitoring the operations supporting infrastructure of the file server and for issuing an alert report onto the network-linking backbone in the event that a problem develops in the corresponding operations supporting infrastructure; and

(c) a domain administrating server (DAS) operatively coupled to the backbone,

wherein the DAS has a backbone monitoring means for monitoring communications along the network-linking backbone, detecting alert reports issued by any of the infrastructure monitoring and reporting agents, collecting the alert reports and storing the alert reports for immediate or later analysis.

10. The network system of claim 9 wherein:

the backbone monitoring means includes means for detecting alert reports that are predefined as needing immediate response and for flagging such reports as immediate-response reports; and

the DAS has immediate alert forwarding means for forwarding immediate-response reports to either a communications device of human administrator or to a rule-base driven artificial administrator.

11. A network system according to claim 9 wherein said operations supporting infrastructure of each file-server includes

power supply means for supplying operational power to the local data storage device of the respective file-server.

12. A network system according to claim 9 wherein said operations supporting infrastructure of each file-server includes

local temperature control means for controlling the temperature of the respective file-server.

13. A network system according to claim 9 wherein said operations supporting infrastructure of each file-server includes

local component security means for assuring physical security of one or more local components within the respective file-server.

14. A network system according to claim 9 wherein said operations supporting infrastructure of each file-server includes

local data path integrity checking means for assuring proper interconnections between two or more local components within the respective file-server.

15. A centralized file management system for managing files stored in plural data storage devices of a network domain, wherein the plural data storage devices of the domain are interconnected by a domain-linking backbone and the files of said data storage devices are accessed by way of the domain-linking backbone, wherein each storage device stores a local catalog that identifies a name, location and/or other attributes of each local file and/or directory contained within the respective storage device, said system comprising:

(a) scan means, coupled to domain-linking backbone, for periodically scanning the network domain and interrogating the local catalog of each data storage device in the network domain.

16. The file management system of claim 15 further comprising:

task scheduler means, operatively coupled to the network-linking backbone, for detecting traffic patterns on the backbone and scheduling the timing of data transfer operations that use the network-linking backbone so as to minimize traffic congestion;

wherein the scan means is responsive to the task scheduler means and performs said scanning of the network domain during time periods which would otherwise have substantially minimal traffic congestion.

17. The file management system of claim 15 wherein the scan means takes periodic snapshots of the network domain and the catalog integrating means responsively integrates the periodically collected file identifying information so as to form a historical plurality of domain-wide virtual catalog snapshots.

18. A centralized file management method for managing files stored in plural data storage devices of a network domain, wherein the plural data storage devices of the domain are interconnected by a domain-linking backbone and each storage device stores a local catalog that identifies a name, location and/or other attributes of each local file and/or directory contained therein, said method comprising the steps of:

(a) interrogating the local catalog of each data storage device in the network domain for file identifying information stored within said local catalog; and

(b) integrating the file identifying information collected by said interrogating step from each local catalog into a domain-wide virtual catalog so that each file of the network domain can be identified by name, location or another attribute by consulting the domain-wide virtual catalog.

19. A file access method comprising the steps of:

(a) interrogating a local catalog of each data storage device in a network composed of plural data storage devices linked to one another by a network-linking backbone;

(b) retrieving from each interrogated local catalog, file identifying information identifying a name, a storage location and/or other attributes of each file stored in the interrogated device; and

(c) integrating the retrieved file identifying information collected from each local catalog into a domain-wide virtual catalog so that each file stored on the network can be identified by name, location and/or another attribute by consulting the domain-wide virtual catalog.

20. A network system comprising:

(a) a network-linking backbone;

(b) a plurality of file-servers operatively coupled to the backbone for providing file-serving services over the backbone, each file server having a data storage device for storing a plurality of data files, the respective data storage device of each file server further having a local catalog for identifying each file currently-stored in the respective data storage device by name and storage location; and

(c) a domain administrating server (DAS) operatively coupled to the backbone, wherein the DAS has a first virtual catalog containing copies of the file identifying information currently stored in the local catalogs of said plurality of file-servers.

21. A network system according to claim 20 wherein the DAS further includes

a second virtual catalog containing copies of file identifying information previously stored in the local catalogs of said plurality of file-servers at a first time substantially earlier than that of the currently-stored files.

22. A network system according to claim 21 wherein the DAS further includes

a third virtual catalog containing copies of file identifying information previously stored in the local catalogs of said plurality of file-servers at a second time substantially earlier than that of the currently-stored files.

23. A network system according to claim 22 wherein the first through third virtual catalogs define a relational database and wherein the DAS further includes

historical database means for searching through the first through third virtual catalogs in accordance with a supplied relational query.

Description Submit all comments and votes

BACKGROUND

1. Field of the Invention

The invention relates generally to the field of computerized networks. The invention relates more specifically to the problem of managing a system having a variety of file storage and file serving units interconnected by a network.

2. Cross Reference to Related Applications

The following copending U.S. patent application(s) is/are assigned to the assignee of the present application, is/are related to the present application and its/their disclosures is/are incorporated herein by reference:

(A) Ser. No. 08/151,525 [Attorney Docket No. CONN8675] filed Nov. 12, 1993 by Guy A. Carbonneau et al and entitled, SCSI-COUPLED MODULE FOR MONITORING AND CONTROLLING SCSI-COUPLEDHAID BANK AND BANK ENVIRONMENT;

3. Description of the Related Art

Not too long ago, mainframe computers were the primary means used for maintaining large databases. More recently, database storage strategies have begun to shift away from having one large mainframe computer coupled to an array of a few, large disk units or a few, bulk tape units, and have instead shifted in favor of having many desktop or mini- or micro-computers intercoupled by a network to one another and to many small, inexpensive and modularly interchangeable data storage devices (e.g., to an array of small, inexpensive, magnetic storage disk and tape drives).

One of the reasons behind this trend is a growing desire in the industry to maintain at least partial system functionality even in the event of a failure in a particular system component. If one of the numerous mini/micro-computers fails, the others can continue to function. If one of the numerous data storage devices fails, the others can continue to provide data access. Also increases in data storage capacity can be economically provided in small increments as the need for increased capacity develops.

A common configuration includes a so-called "client/server computer" that is provided at a local network site and has one end coupled to a local area network (LAN) or a wide area network (WAN) and a second end coupled to a local bank of data storage devices (e.g., magnetic or optical, disk or tape drives). Local and remote users (clients) send requests over the network (LAN/WAN) to the client/server computer for read and/or write access to various data files contained in the local bank of storage devices. The client/server computer services each request on a time shared basis.

In addition to performing its client servicing tasks, the client/server computer also typically attends to mundane storage-management tasks such as keeping track of the amount of memory space that is used or free in each of its local storage devices, maintaining a local directory in each local storage device that allows quick access to the files stored in that local storage device, minimizing file fragmentation across various tracks of local disk drives in order to minimize seek time, monitoring the operational status of each local storage device, and taking corrective action, or at least activating an alarm, when a problem develops at its local network site.

Networked storage systems tend to grow like wild vines, spreading their tentacles from site to site as opportunities present themselves. After a while, a complex mesh develops, with all sorts of different configurations of client/server computers and local data storage banks evolving at each network site. The administration of such a complex mesh becomes a problem.

In the early years of network management, a human administrator was appointed for each site to oversee the local configuration of the on-site client/server computer or computers and of the on-site data storage devices.

In particular, the human administrator was responsible for developing directory view-and-search software for viewing the directory or catalog of each on-site data storage device and for assisting users in searches for data contained in on-site files.

The human administrator was also responsible for maintaining backup copies of each user's files and of system-shared files on a day-to-day basis.

Also, as primary storage capacity filled up with old files, the human administrator was asked to review file utilization history and to migrate files that had not been accessed for some time (e.g., in the last 3 months) to secondary storage. Typically, this meant moving files that had not been accessed for some time, from a set of relatively-costly high-speed magnetic disk drives to a set of less-costly slower-speed disk drives or to even slower, but more cost-efficient sequential-access tape drives. Very old files that lay unused for very long time periods (e.g., more than a year) on a "mounted" tape (which tape is one that is currently installed in a tape drive) were transferred to unmounted tapes or floppy disks and these were held nearby for remounting only when actually needed.

When physical on-site space filled to capacity for demounted tapes and disks, the lesser-used ones of these were "archived" by moving them to more distant physical storage sites. The human administrator was responsible for keeping track of where in the migration path each file was located. Time to access the data of a particular file depended on how well organized the human administrator was in keeping track of the location of each file and how far down the chain from primary storage to archived storage, each file had moved.

The human administrator at each network site was also responsible for maintaining the physical infrastructure and integrity of the system. This task included: making sure power supplies were operating properly, equipment rooms were properly ventilated, cables were tightly connected, and so forth.

The human administrator was additionally responsible for local asset management. This task included: keeping track of the numbers and performance capabilities of each client/server computer and its corresponding set of data storage devices, keeping track of how full each data storage device was, adding more primary, secondary or backup/archive storage capacity to the local site as warranted by system needs, keeping track of problems developing in each device, and fixing or replacing problematic equipment before problems became too severe.

With time, many of the manual tasks performed by each on-site human administrator came to be replaced, one at a time on a task-specific basis, by on-site software programs. A first set of one or more, on-site software programs would take care of directory view-and-search problems for files stored in the local primary storage. A second, independent set of one or more, on-site software programs would take care of directory view-and-search problems for files stored in the local secondary or backup storage. Another set of one or more, on-site software programs would take care of making routine backup copies and/or routinely migrating older files down the local storage migration hierarchy (from primary storage down to archived storage). Yet another set of on-site software programs would assist in locating files that have been archived. Still another set of independent, on-site software programs would oversee the task of maintaining the physical infrastructure and integrity of the on-site system. And a further set of independent, on-site software programs would oversee the task of local asset management.

The term "task-segregation" is used herein to refer to the way in which each of the manual tasks described above has been replaced, one at a time by a task-specific software program.

At the same time that manual tasks were being replaced with task-segregated software programs, another trend evolved in the industry where the burden of system administration was slowly shifted from a loose scattering of many local-site, human administrators ----one for each site----to a more centralized form where one or a few human administrators oversee a large portion if not the entirety of the network from a remote site.

This evolutionary movement from local to centralized administration, and from task-segregated manual operation to task-segregated automated operation is disadvantageous when viewed from the vantage point of network-wide administration. The term "network-wide administration" is used here to refer to administrative tasks which a human administrator located at a central control site may wish to carry out for one or more client/server data storage systems located at remote sites of a large network.

A first major problem arises from the inconsistency among user interfaces that develops across the network. In the past, each local-site administrator had a tendency to develop a unique style for carrying out man-to-machine interactions. As a result, one site might have its administrative programs set up to run through a graphical-user interface based on, for example the Microsoft Windows.TM. operating environment, while another site might have its administrative programs running through a command-line style interface based on, for example the Microsoft DOS 6.0.TM. operating system or the AT&T UNIX.TM. operating system. A network-wide administrator has to become familiar with the user interface at each site and has to remember which is being used at each particular site in order to be able to effectively communicate with the local system administrating software programs. Inconsistencies among the interfaces of multiple network sites makes this a difficult task.

Another problem comes about from the task-segregated manner in which local administrative programs have developed over the years. A remote human administrator (or other user) has to become familiar with the local topology of each network site when searching for desired files. In other words, he or she has to know what kinds of primary, secondary, backup and archive storage mechanism are used at each site, how they are connected, how data files migrate through them, and which "file manager" program is to be used to view the files of each type of storage mechanism.

More specifically, if a file cannot be found in the directory of a primary storage device located at a particular network site, the administrator has to switch from the primary storage viewing program to a separate, migration-tracking program to see if perhaps the missing file has been migrated to secondary or archive storage at that site. The administrator may have to switch to a separate, backup-tracking program to see if a file that is missing from primary and secondary storage might be salvaged out of backup storage at the same or perhaps a different site. Sometimes, the administrator may wish to see a historical profile of a file in which revisions have been made to the file over a specified time period. A separate file-history tracking program at the site might have to be consulted, if it exists at all, to view such a historical profile.

If a file cannot be found at a first site then perhaps a copy might be stored at another site. To find out if this is the case, the administrator has to log out of the first site, log-in to the system at a next site and repeat the above process until the sought after data is located or the search is terminated.

Each switch from one site to a next, and from one independent file-managing program to another disadvantageously consumes time and also introduces the problem of inconsistent user interfaces.

A similar set of problems is encountered in the overseeing of lower-level infrastructure support operations of a networked data storage system. Included in this category are the scheduling and initiation of routine file backup and file migration operations at each site, the tracking of problems at each site and so forth.

A method and system for integrating all the various facets of system administration on a network-wide basis is needed.

SUMMARY OF THE INVENTION

The invention overcomes the above-mentioned problems by providing a network management system having virtual catalog overview function for viewing of files distributively stored across a network domain.

A network management system in accordance with the invention comprises: (a) a domain administrating server (DAS) coupled to a network-linking backbone of a network domain for scanning the network domain to retrieve or broadcast domain-wide information, where the domain administrating server (DAS) has means for storing and maintaining a domain-wide virtual catalog and for overseeing other domain-wide activities, and where the domain-wide virtual catalog contains file identifying information for plural files distributively stored in two or more file servers of the network domain; and (b) one or more workstations, coupled by way of the network-linking backbone to the domain administrating server for accessing the domain-wide information retrieved by the domain administrating server.

A method in accordance with the invention comprises the steps of: (a) interrogating the local catalog of each data storage device in a network composed of plural data storage devices linked to one another by a network-linking backbone, (b) retrieving from each interrogated local catalog, file identifying information identifying a name, a storage location and/or other attributes of each file stored in the interrogated device; and (c) integrating the retrieved file identifying information collected from each local catalog into a domain-wide virtual catalog so that each file stored on the network can be identified by name, location an/or another attribute by consulting the domain-wide virtual catalog.

BRIEF DESCRIPTION OF THE DRAWINGS

The below detailed description makes reference to the accompanying drawings, in which:

FIG. 1 is a block diagram showing a centralized domain management system in accordance with the invention;

FIG. 2 is a perspective view of a multi-dimensional viewing window for visualizing domain-wide activities spatially, temporally and by file attributes;

FIGS. 3A-3B show a set of trend analysis graphs that may be developed from the domain-wide, virtual catalog snapshots obtained by the system of FIG. 1;

FIGS. 4A-4B show side-by-side examples of pie charts showing used-versus-free storage space on respective storage drives DRIVE-A and DRIVE-B within the domain of FIG. 1;

FIG. 5 a job scheduling chart for minimizing traffic congestion on the network-linking backbone; and

FIG. 6 shows a logical flow map between various data and control mechanisms distributed amongst the domain administrating server (DAS), an administrative workstation, and a given server computer.

DETA