 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the invention
The present invention relates generally to a Space Management System which
forms a part of a Data Access System of a File Access Processor. A File
Access Processor services requests from a set of Application Support
Processors, which can exist in a global network, with each Application
Support Processor sharing access to data in files stored by the File
Access Processor. Application Support Processors may operate
asynchronously, sharing the set of file resources managed by one or more
File Access Processors. A File Access Processor manages access to a set of
data files and information about files held in file directories therein.
File directories allow for managing file collections, can relate to each
other hierarchically, and may be shared. The File Access Processor also
maintains therein a set of catalogs, which are repositories of information
in the File Access Processor for its own internal use, and which are not
available and accessible to Application Support Processors, as are the
data files and directories. Each Application Support Processor also
maintains therein an internal cache of file information to improve
performance by reducing communications required with the File Access
Processor for information about files. The File Access Processor provides
the Application Support Processors with information for updating and
maintenance of local caches of directory and file description information
through a centralized accumulation and distribution of cache change
notifications.
More particularly, the present invention relates to a Space Management
System which manages file space assigned to each Application Support
Processor and supports the space management functions of adding space,
deleting space, setting thresholds for limit warnings, querying space,
charging space used, warning notifications, crediting space freed, and
permitting file spaces to be shared by more than one Application Support
Processor.
2. Discussion of the Prior Art
For the discussion herein, the features of the present Data Access System
are compared with features of two previous IBM systems, a SQL/DS system,
and a VM Minidisk File System.
The SQL/DS system is based on a substantially different system design
providing compiled access routines which are compiled and executed, as
opposed to the present system which provides interpretive request routines
which are specifically designed to accomplish particular functions.
The SQL/DS system does not provide for an internal cache of file
information similar to the present invention.
Moreover, in the SQL/DS system changes by an updater become available to
other users before the changes are committed. In the present system
changes are not available to other Application Support Processors until
the work unit is committed successfully. Each Application Support
Processor retains the version of the file obtained when the file was
initially accessed.
The SQL/DS system also does not provide for on- line data space management,
and provides no warning as the SQL/DS equivalent of file space thresholds
are approached. Dynamic changes to space limits are not provided, and the
system can only provide for migration to new data base space. The SQL/DS
system basically represents an entirely different approach to recognition
of space limits based on different requirements. The SQL/DS system
provides an immediate recognition when a consumption limit is reached,
whereas DAS Space Management permits temporarily exceeding limits (until
time of commit), such that only permanent space consumption is subject to
enforced limits.
The VM Minidisk File System does not provide for sharing of files or for a
hierarchical directory support. There is no provision for globally
maintained cache information, and no provision for work units (requires
keeping temporary copies of old versions of files for example). The VM
Minidisk File System does not provide for sharing of data, and for support
of separate caches.
The VM Minidisk File System also has no provision for space management
similar to the present invention, and only provides for mini-disks
(virtual disks implemented as contiguous segments of real disks). The
system does not provide a capability for on-line administration of space
(changes to available space, limits, etc.) Since space is directly
associated with a contiguous storage assignment, an expansion requires a
reassignment to another (larger) minidisk, and physical transfer of data
files leading to delays, fragmentation of space, administrative burdens,
etc. Moreover, the sharing of minidisks is haphazard and awkward. No
concurrency controls are provided for synchronizing multiple writers
(unpredictable results). After updates, readers must re-access the
minidisk or risk unpredictable results (loss of data integrity). Moreover,
no space thresholds are provided.
In the VM Minidisk File System access authorization is controlled at the
minidisk level, whereas the present system controls access at the file
level which provides greater granularity-finer control.
SUMMARY OF THE INVENTION
Accordingly, it is a primary object of the present invention to provide a
Space Management System for a Data Access System of a File Access
Processor as described hereinabove which manages file space assigned to
each application support processor and supports the space management
functions of adding space, deleting space, setting thresholds for limit
warning, querying space, charging space used, warning notifications,
crediting space freed, and permitting file spaces to be shared by more
than one application support processor.
In accordance with the teachings herein, the present invention provides a
Space Management System which is provided with access to a Space Catalog
which records therein a record for each Application Support Processor
representing logical file space that is assigned thereto. The Data Access
System includes a Global Control structure which is maintained permanently
for all activations of the Data Access System, and which anchors and
provides a basis for locating File Space Control Blocks which maintain
file space information in temporary storage, with each File Space Control
Block being initially created from a Space Catalog record when a work unit
is initiated for a particular Application Support Processor. Moreover, the
Data Access System updates the record in the Space Catalog for the
particular Application Support Processor when the work unit is finished
and committed.
In accordance with further details herein, multiple Application Support
Processors can consume space in a file space concurrently, and the Data
Access System and Space Management System enforce and support a common set
of space consumption limits and thresholds. Moreover, in the system, one
Application Support Processor can authorize another Application Support
Processor to share its file space. When an Application Support Processor
initiates file space consumption in its own file space, the File Space
Control Block created during activation initiation is utilized for space
management. When file space consumption is in a file space owned by
another Application Support Processor, the initiating Application Support
Processor utilizes the file Space Control Block created during the
activation of the owning Application Support Processor. Moreover, if the
owner is not active, a File Space Control Block is created by the first
non-owner that utilizes the file space.
It is an object of the Space Management System of the present invention to
minimize the periods of non-concurrency for multiple Application Support
Processors participating in updates to separate files in a file space. In
this system write locks are utilized to prevent concurrent changes to the
same file. However, concurrent changes to different files in a file space
should be allowed except during the period of Space Catalog update and
commit of a work unit. The Space Management System is designed to allow
Space Catalog change coordination between contending Application Support
Processors in such a manner as to maximize concurrency.
The Space Management System accomplishes these system objections by the
sequence of operations at commit time. Initially, all catalogs are updated
except the Space Catalog. Write locks exist on individual files, and are
held until the commit is completed. A commit latch on an FSCB serializes
commits of the Space Catalog changes. The FSCB latch is only held while
doing the commit call to the Space Access System, thereby minimizing the
period of non-concurrency for writers to a particular file space.
The basic sequence of operations at commit time is:
a. Update all catalogs except the Space Catalog.
b. Update the Space Catalog from FSCBs, latching the FSCBs.
c. Invoke the Space Access System to commit the catalog and file changes.
d. Release the FSCB latch.
This results in minimizing the period of non-concurrency for multiple
Application Support Processors participating in updates to separate files
in a file space.
In accordance with the details of a disclosed embodiment herein, the File
Space Control block for an Application Support Processor includes fields
for, SPACE LIMIT which is the total space permitted for files, SPACE
THRESHOLD which is a percentage of the space limit, SPACE CURRENTLY
CONSUMED which is updated for each file request that affects space
consumption and is utilized to determine if the space threshold has been
reached, SPACE COMMITED which is updated when the work unit is committed,
and File Space Control Block LATCH which is used to synchronize changes to
the space catalog.
Additionally, for controlling the release of a File Space Control Block, an
activity counter is maintained in each File Space Control Block which is
incremented by one for each activity initiated that involves a change in
space consumption and decremented by one for each committed or terminated
activity of that File Space Control Block. The File Space Control Block is
finally released only when its activity counter goes to zero, which
retains the File Space Control Block while it is expected to be reused or
currently active.
In accordance with further details of the disclosed embodiment, the Data
Access System also builds Local Control structures, one for each
activation of the Data Access System by a particular Application Support
Processor. The Local Control structure provides a basis for locating a
File Control Block which is used to retain information about a particular
file while it is being accessed by a local activation of an Application
Support processor. The File Control Block has particular fields for space
management, including a locator for the File Space Control Block that is
set when the file access is initiated, and used for rapid reference to the
File Space Control Block thereafter. In greater particularity, the File
Control Block also includes fields for SPACE CONSUMED-CURRENT REQUEST
which is the change in space consumption caused by the completion of the
most recent request against the file represented by this File Control
Block, and SPACE CONSUMED-CURRENT WORK UNIT which is the change in space
consumption caused by all changes to the file represented by this File
Control Block in the current work unit.
The global control structure also includes a File Space Control Block hash
table by which a File Space Control Block of a particular Application
Support Processor can be located.
Additionally, when a particular file access is completed successfully, a
Work Request Block is built containing information required for updating
File Space Control Blocks and the Space Catalog records. Information is
transferred from the File Control Block to the Work Request Block, and
then the File Control Block is released. The Work Request Blocks are used
to retain information that is needed to update the Space Catalog for
changes in space management information when the work unit is successfully
completed and committed. A separate Work Request Block is provided for
each record in each catalog that must be updated.
In greater detail, the Work Request Block includes fields for, SPACE
CONSUMED-CURRENT WORK UNIT which is the change in space consumption caused
by all changes to the file represented by this Work Request Block in the
current work unit, REQUEST TYPE which is an encoding of the type of
request that caused the Work Request Block generation, PARAMETER VALUE
which is used to hold the new space limit or space threshold for
add-space, delete-space, and change-threshold requests so that it is
available for modifying the File Space Control Block and Space Catalog at
commit time, and SPACE CATALOG RECORD which is a copy of the Space Catalog
record read into this area in preparation for selectively updating it.
Moreover, the local control structure also provides a basis for locating
UNDO Lists which are used to save the status of File Space Control Blocks
at the beginning of the current work unit so that the File Space Control
Blocks can be restored in the event of a failure in updating the Space
Catalog during commit processing.
In accordance with the teachings herein, the Space Management System
supports a technical approach which presents a clear separation of
functions to permit high performance, allowing easy maintenance and
extensibility, while supporting the required functions.
The control structures of the Data Access System support the following
particular features of the design.
1. DAS Global Control anchor for all global control structures, one for all
activations of the DAS.
2. DAS Local Control anchor for all DAS local control structures, one per
activation of the DAS by an ASP.
3. Work Request Blocks (WRBs) support deferred updates of catalogs, space
consumption values, and cache updates.
4. File Space Control Blocks (FSCBs) support dynamic threshold warnings and
space consumption limits shared by all users of a file space.
5. Cache structures in support of the central cache maintenance capability.
The design of the Data Access System provides structures for the following.
1. Maintenance of local caches of directory and file description
information through centralized accumulation and distribution of cache
change notifications.
2. Permanent storage in the form of catalogs for retaining shared
information about files, directories, and control information. A separate
access method and storage facility is utilized for these catalogs from
that utilized for file data, permitting flexibility in access to catalog
information (access required only occasionally), while retaining high
performance in the access to file data.
3. Permanent storage and retrieval of file data blocks.
4. Fast path processing for each request type through individual and
separate request processing routines, deferred updates to catalogs, and
judicious balancing of in-line with centralized services.
5. Services covering routing, response formulation and buffer management,
catalog access, storage management, concurrency controls, etc. that are
available for request handling routines, with minimal environmental
switching (in-line or fast path).
6. Grouping requests into work units that can be subject to reversal
(back-out of changes) in the event of a failure and control when a second
writer to a common file can gain access.
7. Concurrent activations, each servicing a logical grouping of file
requests (supports sharing).
8. Previewing of catalog changes whereby the originating Application
Support Processor sees changes before they are committed (end of work
unit), while other Application Support Processors do not have access to
them until after the commit. This is made possible through deferred
updates of catalogs and in-flight search of WRBs before catalog reads.
9. On-line file space management with consumption limits and thresholds
whereby more than one ASP can concurrently consume space in the same file
space and experience the same limits/thresholds. Also supports on-line
administration (changes in limits, thresholds, new file space etc.).
The DAS Cache Management component is provided for maintaining local caches
in each Application Support Processor which contain the current status of
file information for cached directories. Cache Management utilizes a
centralized collection of cache change information, along with facilities
for recording and distributing the changes to the appropriate Access
Support Processors, and includes:
1. A set of control structures for recording the information.
2. A set of functions for performing the required work.
3. Control flow descriptions, relating Cache Management with other
components of DAS.
The DAS cache management design provides for data sharing and high
performance, and more specifically:
1. Permits each Application Support Processor to see directory changes,
even those made by others on a timely basis.
2. Concurrency controls required for the global control structures that are
required to support the function.
3. Piggyback cache change notifications with normal responses.
4. Non-replicated centralized change records, Directory Change Refresh
Blocks (DCRBs) minimize storage.
5. Deferred recording of changes until the successful end of a work unit.
The DAS Space Management component provides for support of on-line control
of file space consumption limits in an environment where multiple ASPs can
consume space in a repository (file space) concurrently, while enforcing
and supporting a common set of space consumption limits and thresholds,
and includes.
1. A set of control structures for recording the limits, thresholds, and
other control information for each active file space.
2. A set of functions for performing the required work.
3. Control flow descriptions, relating Space Management with other
components of DAS.
4. On-line setting and changes to consumption limits for individual file
space.
5. On-line setting and changes to threshold levels for warnings.
6. Dynamic recording of space consumption (at commit time).
7. Dynamic reporting of threshold levels reached for each file write (hence
the particular need for FSCBs).
8. Coordinated commit of Space Catalog changes with other work unit updates
to catalogs etc. in such a way as to maximize concurrency.
9. Capability for backing out changes in a consistent manner in the event
of a failure.
BRIEF DESCRIPTION OF THE DRAWINGS
The foregoing objects and advantages of the present invention for a Space
Management System for a Data Access System of a File Access Processor may
be more readily understood by one skilled in the art with reference to the
following detailed description of a preferred embodiment thereof, taken in
conjunction with the accompanying drawings wherein like elements are
designated by identical reference numerals throughout the several views,
and in which:
FIG. 1 illustrates the relationships between the major subcomponents of the
Data Access System, as well as the relationships between it and the other
two components of the File Access Processor, the Service System and the
Storage Access System;
FIG. 2 shows the major control structures of the Data Access System;
FIG. 3 illustrates the set of control structures which are relevant to the
Space Management Subcomponent;
FIG. 4 shows pertinent fields for Space Management in the FSCB, FCB, WRB,
and UNDO List;
FIG. 5 illustrates the flow of control and data which occurs at the
completion of a work unit (commit or rollback);
FIGS. 6, 7 and 8, when placed together with FIG. 6 on top, FIG. 7 in the
middle, and FIG. 8 on the bottom, illustrate a logic flow diagram of the
COMMIT operation supported by Space Management;
FIG. 9 illustrates a logic flow diagram of the ROLLBACK operation supported
by Space Management;
FIG. 10 illustrates a logic flow diagram of the UNDO operation supported by
Space Management; and
FIG. 11 illustrates a logic flow diagram of the UNLATCH operation supported
by Space Management.
DETAILED DESCRIPTION OF THE DRAWINGS
Referring to the drawings in detail, in FIG. 1 a Data Access System 14 is
illustrated in terms of the functions of its subcomponents, the
relationships between its subcomponents, and its relationships with other
components of a File Access Processor 12.
The Data Access System 14 is described and claimed in U.S. patent
application (attorney's docket 6776). The Data Access System 14 is one
component of a File Access Processor 12, which also includes two other
components, a Service System 16 and a Storage Access System 18, the
functions of which are generally known in existing computer systems, and
are not described in detail herein, but their basic functions are
described briefly to establish perspective.
A File Access Processor 12 services requests from a set of Application
Support Processors 10 which can exist in a global network, with each
Application Support Processor 10 sharing access to data in files stored by
the File Access Processor 12. Application Support Processors 10 may
operate asynchronously, sharing the set of file resources managed by one
or more File Access Processors 12. Each Application Support Processor 10
also maintains therein an internal cache of file information to improve
performance by reducing communications with the File Access Processor 12
for information about files. The File Access Processor 12 provides the
Application Support Processors 10 with information for updating this
cache, and also responds to normal file access requests.
A File Access Processor 12 manages access to a set of data files and
information about files held in file directories. File directories allow
for managing file collections, relate to each other hierarchically, and
may be shared. The File Access Processor also maintains therein a set of
catalogs, which are repositories of internal information in the File
Access Processor for its own internal use, and which are not available and
accessible to Application Support Processors, as are the data files and
directories. Some of the requests supported by the File Access Processor
12 are as follows:
OPEN, READ, WRITE and CLOSE for file access,
DELETE files,
COPY files,
GRANT and REVOKE authorization to files
CREATE and DELETE directories,
OPEN, READ and CLOSE for directory access,
RENAME files and directories,
RELOCATE files from one directory to another, and
ADMINISTER file storage for files.
The primary functions of the File Access Processor 12 components are as
follows.
The Service System 16 provides environmentally dependent services for the
processor 12, including the initial receipt of requests, forwarding
requests for file access, and dispatching and activation of the Data
Access System 14 of the present invention. These activations persist
through a set of file requests that represent a recoverable unit of work
(defined as a work unit herein), which is explicitly concluded by a commit
or rollback request (back-out) or certain other implicit terminations.
The Data Access System 14 processes individual requests, and maintains a
set of catalogs that contain control and descriptive information
concerning the set of files managed by the File Access Processor 12. This
information includes data file control, security, integrity, concurrency,
inter-relationship, recovery, status, and control information. The Data
Access System 14 also accumulates cache update information, representing
changes to file descriptions and characteristics, that are forwarded to
maintain current the data in the caches of the Application Support
Processors.
The Storage Access System 18 manages the data blocks that comprise the data
files--mapping them to external storage devices, the set of records that
comprise the catalogs, the set of locks that permit concurrency, and the
grouping of work items that represent recovery units. It maps access to
physical data with minimal sensitivity to logical grouping or content. The
Storage Access System 18 runs in the same activation as the Data Access
System 14 that invokes it.
The Data Access System 14 Subcomponents have the following functions.
The Session Management Subcomponent 20 is at the hierarchical top of Data
Access System 14 request processing. It receives requests through the
Service System 16. It is the focal point of control and includes a set of
fundamental services for response formulation. It determines which Request
Management Subcomponent 32 routine should be invoked to service the
request and passes control to it. At the end of request processing,
control is passed back to the Session Management Subcomponent 20 to finish
request processing and forward the requested response.
The Session Management Subcomponent provides the primary routing and
control functions for the Data Access System. In addition, it manages work
units, supports deferred commit of work, supports response buffer
management and ordering, as well as initialization and termination of the
Data Access System. A number of routines support the operation of Session
Management. Some of these routines are also utilized by other
subcomponents of the Data Access System as common services.
Following are some of the service routines included in the Session
Management Subcomponent 20.
Start-up routine 22--to initialize Data Access System 14 control
structures.
Storage Pool routine 24--to maintain and dispense working storage for
control structures. The Pool Routine is a general Data Access System
service for allocating and de-allocating control structures from large
blocks of working storage. The routine supports allocating or deallocating
single or multiple control structures of the same type (changed) with a
single invocation.
Response routine 26--to assist Request Management Subcomponent 32 routines
in the building of response messages, and to assemble the response and
cache update information into a properly formatted response message. The
Response routine has two aspects.
1. It supplies a Getbuffer Service used throughout the Data Access System
for providing buffers that hold response information. For this service, it
not only supplies the buffers, but also implicitly accounts for them,
accumulating lists of buffers acquired in preparation for assembling the
response for the current request.
2. It builds the response to the current request from several sources:
a. It invokes the Cache Management RETRIEVE routine for finding all of the
Cache Notification Records (CNRs) and placing them in buffers supplied by
the Response Routine.
b. It builds a locator list of buffers consisting of the CNR buffers,
buffers allocated by the Getbuffer Service, and standard request
diagnostic information. This locator list is sent to the Service System
for return of the response to the requesting Application Support
Processor.
Work routine 28--to coordinate and initiate processing required when a
logical group of requests are "committed" or "rolled back", i.e. manage
activities at designated points of recovery (termination of a work unit).
The Work routine 28 supports the commit and rollback explicit requests as
well as implicit commit and rollback conditions. For explicit requests,
the Work routine 28 is invoked through the Request Management Subcomponent
32. To maintain consistency and integrity, actual catalog updates are
deferred until the successful completion of the work unit. The Work
routine 28 invokes:
a. the Catalog Management Subcomponent 36 to accomplish catalog updates
based upon WRBs (Work Request Blocks) built by Request Management
Subcomponent 32 routines.
b. the Space Managment Subcomponent 34 to coordinate the update of the
Space Catalog based upon WRBs built by Request Management 32 routines.
c. the Storage Access System 18 to commit changes to file blocks and
catalogs.
d. the Cache Management Subcomponent 38 to store cache notification
information (also contained in Work Request Blocks built by the Request
Management Subcomponent 32 routines).
Terminate routine 30--to clean up control structures at the conclusion of
an activation of the Data Access System 14. The Terminate Routine manages
the clean-up of storage and control structures when service to a
particular Application Support Processor is ended. Terminations occur from
implicit failures (communications between an Application Support Processor
and a File Access Processor, deadlock on resources, etc.) and explicit
actions (system operator forced, Application Support Processor ended,
etc.). Terminations are discovered upon initial entry into Session
Management or at any other point when an activation of the Data Access
System regains control after an invocation | | |
