Sysplex shared data coherency method

Having thus described our invention, what we claim as new and desire to secure by Letters Patent is:

1. A method of controlling data coherence for a computer multi-system complex, comprising the steps of

structuring the computer multi-system complex as a sysplex which includes a plurality of central processor complexes (CPCs), sysplex direct access storage devices (DASDs) connected to the CPCs for storing data elements permanently in the sysplex, and a shared electronic storage (SES) attached to the CPCs, command issuers which execute in the CPCs to issue commands to the SES, said SES containing a SES cache with a SES cache directory for storing names of the data elements, the same data element name being used throughout the sysplex to identify all copies of the same data element stored anywhere in the sysplex whether or not any data element copy is changed relative to the data element or any copy of the data element,

communicating a registration command to the SES by a command issuer in a CPC for providing a data element name and associated CPC location information for locating in the CPC a validation indication associated with a data element copy stored in a local LCB in the CPC, the registration command controlling SES to register the data element copy for the SES cache,

registering the associated CPC location information in a local cache register (LCR) associated with a SES cache directory entry storing the data element name in the SES cache directory in response to the communicating step, and

accumulating all CPC location information in the LCR for all registration commands received for the same SES cache entry containing the data element name for locating all validation indicators for all copies of the data element in all connected CPCs in the sysplex for enabling coherence control for the copies of the data element among the CPCs in the sysplex.

2. A method of controlling data coherence For a computer multi-system complex as defined in claim 1, further comprising the step of

performing in the SES cache a name replacement operation which is communicating a data element name and associated CPC location information for a copy of a data element to be deregistered,

finding any SES directory entry having the data element name,

matching the communicated CPC location information with CPC location information in the LCR associated with the directory entry, and

deleting any CPC location information found in the LCR that matches the communicated CPC location information to deregister the copy of the data element.

3. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, further comprising the steps of:

performing in the SES cache a re-registration operation which is communicating a data element name and associated CPC location information to be registered,

finding a SES directory entry having the data element name, and

replacing the currently registered associated CPC location information with the communicated CPC location information.

4. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, further comprising the steps of

issuing by a CPC to SES a read command for requesting registration of a data element in the SES cache, and

requesting reading of a data element sent from the CPC to SES with a data element name.

5. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, the registering step further comprising

registering with the SES cache directory entry in the associated LCR the CPC location information for the invalidation request bit associated with the LCB containing the copy of the data item which is subject to a registration command.

6. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, the registering step further comprising

assigning into the SES cache directory entry of a data element name for a copy of a data element located in an LCB in response to a request from a CPC when the data element name is not found in any entry in the SES cache directory.

7. A method of controlling data coherence for a computer multi-system complex as defined in claim 5, further comprising the steps of

issuing by a CPC to SES a command providing a name of a data to request invalidation of the content of all LCB(s) in the CPCs in the sysplex currently having a copy of the data element and

SES responding to the CPC issuing the command with a completion signal when all invalidation signalling by SES to the CPCs is completed for the command.

8. A method of controlling data coherence for a computer multi-system complex as defined in claim 7, further comprising the steps of

issuing by a command issuer in a CPC to SES a command providing a name of a data element to request invalidation of the content of all LCB(s) in the sysplex currently having the name in a SES cache directory entry found to be associated with the name except not requesting invalidation of the copy of the data element in the LCB associated with the command issuer.

9. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, further comprising the steps of

designating for each local cache (LC) in each CPC a set of invalidation request bits, each LC containing one or more LCBs, each invalidation request bit being associated with a respective LCB,

locatinq the LCR associated with the SES cache directory entry containing the data element name

readinq the associated CPC location information in the LCR for determining the locations of all invalidation request bits for all conflicting copies of the data element in all LCBs in all CPCs in the sysplex in response to a command issuer sending to SES a command that requires a determination of conflicting copies of the data element,

determining each CPC indicated by the associated CPC location information read from the LCR to locate an invalidation request bit for each conflicting copy of the data element,

signalling by the SES the associated CPC location information Of the LCR to each CPC determined by the determining step to locate each invalidation request bit for each conflicting copy of the data element in any LCB in any CPC in the sysplex, and

requesting by the signalling step each determined CPC to set to an invalidation state each invalidation request bit located by the signalled CPC location information received by the CPC.

10. A method of controlling data coherence for a computer multi-system complex as defined in claim 9, further comprising the steps of

controlling by a program in a CPC the setting and testing of states of the invalidation request bits associated with LCBs of that CPC, and

invalidating by the program of a copy of a data element in an LCB for which the program tests an invalidation state for an associated invalidation request bit.

11. A method of controlling data coherence for a computer multi-system complex as defined in claim 10, further comprising the steps of

executing in a CPC an instruction to determine a current state for an invalidation request bit assigned to a specified LCB.

12. A method of controlling data coherence for a computer multi-system complex as defined in claim 10, further comprising the steps of

executing an instruction by CPC operating software to set to a valid state an invalidation request bit assigned to a specified LCB value for a specified LC,

transmitting by the CPC to SES a registration command, and

setting in the CPC the invalidation request bit to an invalid state if the registration command does not perform a registration in SES.

13. A method of controlling data coherence for a computer multi-system complex as defined in claim 1, further comprising the steps of

issuing by a CPC to SES a write command for requesting registration of a copy of a data element in the SES cache, and

requesting writing of the copy of the data element sent from the CPC to SES with a data element name.

14. A method of controlling data coherence for a computer multi-system complex as defined in claim 13, further comprising the steps of

issuing by a CPC to SES a write command which sends a data element to SES with a data element name, and

performing a write operation by writing the data element in a SES cache data area only if the data element name is found in a directory entry in the SES cache, and CPC location information for the copy in the CPC is found in the LCR associated with the directory entry.

15. A method of controlling data coherence for a computer multi-system complex as defined in claim 13, further comprising the steps of

sending with the write command a change indication to be stored in the directory entry, where a first value for the change indication indicates the sent copy of the data element is not changed from a version of the same data element stored in a sysplex DASD or SES, and where a second value for the change indication indicates the sent copy of the data element is changed from a version of the same data element stored in a sysplex DASD or SES.

16. A method of controlling data coherence for a computer multi-system complex as defined in claim 15, further comprising the steps of

responding by SES to the CPC issuing the command with SES refusing to write the data element in an unchanged state when the SES directory entry indicates a changed state for a SES-stored data element.

17. A method of controlling data coherence for a computer multi-system complex as defined in claim 15, further comprising the steps of

writing the data element in a SES data area associated with the SES cache entry containing the data element name, and p1 signalling an invalidation request by SES to each CPC having an LCB(s) with a complement copy of the data element as identified by CPC location information associated with the SES cache directory entry which indicates a changed data element is stored in SES.

Description Submit all comments and votes

CROSS REFERENCE TO RELATED APPLICATIONS

This application contains subject matter which is related to the subject matter of the following applications/patents which are assigned to the same assignee as this application:

"Configurable, Recoverable Parallel Bus" by N. G. Bartow et al, U.S. Pat. No. 5,357,608, Filed: Feb. 20, 1992; "Communications System Having A Plurality of Originator and Corresponding Recipient Buffers With Each Buffer Having Three Different Logical Areas For Transmitting Messages in Single Transfer" by N. G. Bartow et al, U.S. Pat. No. 5,412,803, Filed: Feb. 20, 1992; and "Frame-Group Transmission And Reception For Parallel/Serial Buses", by N. G. Bartow et al, U.S. Pat. No. 5,267,240, Filed: Feb. 20, 1992.

Specifications Incorporated by Reference

The entire specifications of the following listed applications are completely incorporated by reference as part of the subject application. Each of the following listed applications is owned by the same assignee as the subject application, is filed on the same day as the subject application, and has the same priority date as the subject application. They are:

"Communicating Messages Between Processors And A Coupling Facility" by D. A. Elko et al, U.S application Ser. No. 08/420,893; "Method and Apparatus For Distributed Locking of Shared Data, Employing A Central Coupling Facility" by D. A. Elko et al, U.S. Pat. No. 5,339,427 ;"Command Quiesce Function" by D. A. Elko et al, U.S. Pat. No. 5,339,405; "Software Cache Management Of A Shared Electronic Store In A Sysplex" by D. A. Elko et al, U.S. Pat. No. 5,457,793; "Management Of Data Movement From A SES Cache To DASD" by D. A. Elko et al, U.S. application Ser. No. 07/860,806; "Command Execution System For Using First and Second Commands To Reserve and Store Second Command Related Status Information In Memory Portion Respectively" by D. A. Elko et al, U.S. Pat. No. 5,392,397; "Integrity Of Data Objects Used To Maintain State Information For Shared Data At A Local Complex" by D. A. Elko et al, U.S. Pat. No. 5,331,673; "Management Of Data Objects Used To Maintain State Information For Shared Data At A Local Complex" by J. A. Frey et al, U.S. Pat. No. 5,388,266; "Recovery Of Data Objects Used To Maintain State Information For Shared Data At A Local Complex" by J A Frey et al, U.S. Pat. No. 5,394,542; "Message Path Mechanism For Managing Connections Between Processors And A Coupling Facility" by D. A. Elko et al, U.S. application Ser. No. 08/324,447; "Data Processing System and Method For Providing Notification In A Central Processor of State Changes For Shared Data Structure On External Storage" by J. A. Frey et al, U.S. Pat. No. 5,390,328; "Method And Apparatus For Performing Conditional Operations on Externally Shared Data" by D. A. Elko et al, U.S. application Ser. No. 07/860/655; "Apparatus And Method For List Management In A Coupled Data Processing System" by J. A. Frey et al, U.S. Pat. No. 5,410,695; "Interdicting I/O And Messaging Operations From Sending Central Processing Complex To Other Central Processing Complexes And To I/O Device In A Multi-System Complex" by D. A. Elko et al, U.S. Pat. No. 5,394,554; and "Method And Apparatus For Coupling Data Processing Systems" by D. A. Elko et U.S. Pat. No. 5,317,739.

INTRODUCTION

The invention deals with maintaining data coherence in a system complex (sysplex) which comprises a plurality of central processing complexes (CPCs) having local buffers connected to a shared electronic storage (SES) device which contains electronic random access storage for containing data shared by the CPCs.

The invention concerns caching data in a cache serving a multi-system data sharing complex. In particular, the invention concerns the caching of a data page by one database system in the shared cache in view of the possibility that another system could be trying to cache a later copy of the same page. This invention concerns the technique to detect such a condition and to bar entry of the earlier copy of the page into the shared cache.

BACKGROUND

Typically, prior hardware caches in a processor comprise a storage array having fixed-size data blocks. Such storage is viewed by the hardware as a linear array of blocks. The blocks of data in the cache are managed by a cache directory, which invalidates or casts data blocks in or out of such cache based on LRU (least recently used) storage references. Well-known types of hardware caches are used in central processing units (CPUs), and in DASDs (direct access storage devices). The main storage (MS) of many computer systems is partitioned into data lines accessed wholly or partly as data blocks for a CPU cache using real or virtual addresses. Data in the cache is accessed by store, fetch, and update requests for data within the data blocks. Competing data requests to the cache directory are arbitrated by prioritizing the requests to the cache.

A DASD cache is usually partitioned in blocks equal to the DASD tracks. Data requests to the cache directory are read or write I/O operations. Competing requests are arbitrated by storage controls in the DASD control unit.

A DASD type of cache in a CPC hierarchy is described in an article "Cache-DASD Storage Design for Improving System Performance" by C. P. Grossman in the IBM Systems Journal, Vol 24. Nos. 3/4, 1985.

Multi-processors with and without caches have long had coherence controls to maintain the integrity of data in a system. The cross-interrogate and cross-invalidate (XI) operations used therein are different from those used by the subject invention herein.

The general solution of the data coherence problem is well-known to require the invalidation of all copies but the currently changed copy. A number of adverse side affects commonly occur during such invalidation operations including significant performance degradation to system operation. More specific causes of performance degradation include:

1. Significant delays to the transactions changing the records, due to waiting for the other copies to be checked and invalidated before the change process can complete;

2. Lengthy overhead processing by processors performing invalidation of other copies; and

3. Performing invalidations beyond the transaction changing the record to disturb other transactions and their resources not involved with any copy which should have been invalidated.

In a database system wherein a plurality of independently-operating computer systems share data, global locking is required to maintain coherency of data in the different systems. A.J. van de Goor, in COMPUTER ARCHITECTURE AND DESIGN, Addison Wesley, 1989, discusses the data coherency problem as one in which sharing data among a proliferation of processors raises the possibility that multiple, inconsistent copies of data may exist because of multiple paths to the data and because of opportunities to locally modify the data.

Solutions to the data coherency problem have been proposed. All are based essentially on the existence of a global lock on data retrieved from a central location. Assuming pagination of data, one computer system of a multi-computer system which shares data stored on a disk acquires a global lock on a page of data and obtains and updates the page. The lock signifies to the other computer systems that the page has been acquired for updating. Prior to releasing the lock on the page, the computer system holding the lock writes the page to the disk, after which it generates and sends a message to the other computer systems to invalidate any copies of the page which may be held in their local cache. The lock on the page is not released until acknowledgement is received from every other computer system having access to the page. This solution is described in detail in U.S. Pat. Nos. 4,399,504 and 4,965,719, which are assigned to the assignee of this patent application, and which are incorporated herein by reference. A commercial product available from the assignee of this application and which incorporates this solution is the IMS/VS system with the data sharing feature.

The prior art global locking system provides great advantage in maintaining data coherency. However, the overhead penalties inherent in it include the requirement for performing an I/O procedure when a page is updated and undertaking message exchange after the I/O procedure in order to notify the other systems and release the lock.

When used in a non-data-shared single system case, the global-locking prior art still incurs extra overhead in maintaining data coherency (consistency) between transactions by implementing a commit policy requiring each transaction which updates data to write the modified data, together with log records, to storage before the transaction is fully committed. This requires one I/O procedure per page for each modifying transaction, which increases overhead costs.

In contrast, the IBM DB2 in the single system non-data-sharing case follows a policy which does not require an I/O process to write an updated page back to storage in order to commit a transaction. If the protocol described above is used in a data-sharing situation where a plurality of computer systems access one or more data storage sites, the performance could degrade significantly because of the required write back to storage and message delay. In this regard, see C.J. Date's discussion of concurrency at pages 593-595 in Vol. I of AN INTRODUCTION TO DATABASE SYSTEMS, Addison-Wessley, 1986.

In a multi-computer, data-sharing system which includes multiple levels of storage, it is contemplated that a first level of storage would consist of one or more direct access storage devices (DASD's) which are shared by independently-operating computer systems. Typical nomenclature for hierarchally-arranged storage systems classify DASD and other such storage facilities as "secondary" storage. In this regard, secondary storage includes all facilities from which data must be moved to "primary" storage before it can be directly referenced by a CPU. See Detiel, OPERATING SYSTEMS, Second Edition, 1990, by Addison Wesley. It is further contemplated that caching techniques would be useful to provide a high-speed, frequently-accessed storage for shared data. For various reasons, data would be entered into the shared cache by the database systems after acquisition from DASD's. In this regard, a shared cache would be included in a primary level of storage for a multi-computer, data-sharing system.

In such a structure, a potential hazard would exist if one computer system obtained a block of data from DASD for the purpose of caching it after the same block of data had been obtained, modified by another computer system and cached, but not yet returned to DASD. In this situation, the outdated block obtained from DASD is referred to as a "down-level" version of the updated block in cache. The challenge is to prevent the overwriting of the updated block by the down-level version without incurring the expense of locking the DASD version.

Typically, global locking protocols are used by a database system to serialize access to the record of interest in the data sharing case. The inventors contemplate that serializing would still be used. Also typically, there would be global locking on a page to serialize updates to the page from different database systems. The avoidance of serialization described in this invention is for inserting a down level page from the secondary storage into the shared cache by different database systems.

A global locking protocol is described in an IBM TDB article published March 1986 on pages 4642 to 4650, entitled "Integrated Concurrency and Shared Buffer Coherency Control for Multi-System". This article provides a controller (processor) called a centralized concurrency & coherency controller, which maintains multiple states for each data item written in one or more system local buffers and/or the shared memory. The subject invention does not use a centralized controller for coherency control, but distributes the coherency control to the systems having the local buffers containing the data needing coherency control, with the distributed locations being listed in a cache directory entry accessible to all systems.

SUMMARY OF THE INVENTION

A primary object of this invention is to provide a method and structure in a shared data, multi-computer system which guarantees that any page of data in a shared cache (aka SES) will not be overwritten by an earlier version of that page obtained from any other shared data storage resource.

A significant advantage of the invention is that it tolerates the retention of known methods and structures for DASD access, while maintaining the integrity of data obtained from a DASD and cached in a memory (aka SES) shared by a plurality of computer systems.

An important object of this invention is to provide a method for operating a multi-system, data-sharing complex in which data is cached in a shared electronic storage (aka SES).

This invention involves a sysplex using a plurality of operating systems (OSs). Any CPC in the sysplex may have its resources logically partitioned among a plurality of different OSs to provide a multiplicity of independent OSs within a single CPC, such as by the IBM processor resource/system management (PR/SM) system. Thus a sysplex may have any mix of OSs running on its different CPCs, some CPCs each having one OS, and other CPCs each having a plurality of OSs running independently of each other. One or more subsystems may be running under any OS in any CPC, including the IBM DB2, DFP, IMS, VSAM, etc. subsystems.

Different copies of the same data base subsystem program may be running simultaneously and independently in the different CPCs. With this invention, these different programs may be accessing the same or different data elements or records in the data base, which may simultaneously be in MS/ES local caches (LCs) of the different CPCs.

It is a primary object of this invention to provide coherence information and invalidation request control for copies of data records residing in any CPC local cache in a sysplex, without significantly degradating the sysplex performance by:

1. avoiding coherence control delay to a transaction accessing data in a CPC local cache;

2. using little processing in the CPC(s) for coherence control;

3. restricting the scope of coherence control to only those CPC local cache buffers in a sysplex which may require coherence control;

4. allowing any CPC to perform a coherence control operation requested by SES when a reference to a local cache buffer is requested. This avoids any interruption in CPC processing while doing work unrelated to the requested coherence operation.

In a multi-system, data-sharing complex, a database system executing on a first computer system could be caching an updated page in a shared cache while another database system could be trying to cache a copy of the same page obtained from a DASD. The invention detects such a condition and, without a serialization mechanism such as locking, bars entry of the copy obtained from the DASD.

Local Cache Structure

The executing program in any CPC uses one or more allocated local cache buffers (LCBs) in the local caches (107A, 107B, and 107C) of the CPC's MS/ES to contain the data elements and records in recent use that have been generated, backed-up, retrieved and/or stored-in by the CPC. A CPC in the sysplex may have more than one type of program (for example DB2 and IMS) currently executing on the central processors and I/O processors of that CPC. Then, these plural executing programs in a CPC may have their own LCBs allocated in the CPC's MS/ES which contain their most recently accessed data elements and records. Hence, a complexity of different LCBs may be used by different types of programs, and they may simultaneously exist in the MS/ES of any one or more CPCs in a sysplex. The term "local cache" (LC) is used herein to refer to a collection of LCBs set up and used by a CPC programming subsystem.

The allocation and size of each LCB is dependent on the respective program being used in the CPC. The LCBs may have different sizes and different numbers in the different LCs. Any local cache buffer may be changed to a different size during program execution.

A local cache attached to a SES is identified in the SES by a local cache identifier (LCID) which the operating system assigns uniquely to the LCs when they are logically attached to the SES. The LCID is saved in local cache controls (105) at the SES cache. Local cache controls are used by SES to maintain information regarding each attached local cache.

The invention provides invalidation signalling controls in SES to request the coherence of data shared among the MS/ES buffers in the different CPCs, so that each CPC may determine the validity of local cache buffers when access to the data contained in a buffer is requested by programming. During initialization of a local cache, operating system services are invoked to authorize access of the program to the SES cache. These operating system services assign a local cache identifier to be used to uniquely identify the local cache and its attachment to the SES cache.

The operating system services also invoke a CPU instruction (define vector DV) to cause a bit vector (termed a coherence vector (V)) to be created (108A, 108B, and 108C) in the Hardware System Area (HSA) in FIG 1. Completion of the instruction to define this coherency bit vector returns a local cache token (LCT). A new entry for each new coherence vector bit is placed in an HSA table T (209) (shown as table T in HSA 205 in FIG. 2) to represent the locations of the new coherence vector bits in the HSA associated with a particular LC in the CPC. The LCT and LCID are provided to SES by the operating system when a local cache is attached to the SES cache and stored in the local cache controls (105).

The CPC can change the size of any vector in its HSA to any required size, and this is done whenever the CPC changes the size of the associated LC in the CPC's programmable memory.

A local cache is managed in its CPC by a local cache directory (LCD) comprised of a plurality of entries which contain pointers (addresses) to respective LCBs accessed through the respective LCD entries. Any CPC is able to store a data element in each LCB and assign a unique name to the data element; the name is put into the LCD entry associated with the LCB. This data element name is used to communicate with SES as the address of the data element when it is registered in SES, regardless of the size of the data element. The operating system service invoked to interface with the SES uses a CPU instruction (set vector entry SVE) to set the coherence vector bit when the name of a data item is successfully registered at the SES cache.

When a local cache buffer contains a data element shared among the CPCs in the sysplex, programming associates a bit in the coherency vector with the LCB. The entry within the coherency vector is termed the local cache entry number (LCEN). Each LCD entry has a respective local cache entry number (LCEN) which is used by the CPC and SES to distinguish between the LCBs of a local cache. Each LCD entry indicates the valid/invalid state of its LCB contents.

The CPC is physically connected to a SES through a storage controller (not shown) to its MS 204 and HSA 205. This storage controller has microcode addressability to HSA 205 for accessing a vector bit position in HSA by LCT and LCEN values being signalled by the channel (208 in FIG. 2 and 106-1 thru 106-M in FIG. 1) from the attached SES. A table (T in FIG. 2 at 209) in the HSA 205 translates the LCT/LCEN values received from the channel into a corresponding coherence vector bit position in HSA.

Local Cache Coherency

The invention prevents multiple-version contamination of sysplex data when plural copies of a data element concurrently exist in different LCBs in the CPCs. Multiple copies of a record are allowed in different LCBs as long as all CPCs are only reading the records. Contamination occurs when a data element is changed in any LCB when a second copy is allowed to concurrently exist unchanged in any other LCB, since then the two different versions exist for the same data element. Then, any changes made to the second copy will not be cumulative with the changes made to the first copy, making all copies incorrect, including the copy in the common DASD which is not updated until one of the new versions is committed by being stored back to DASD.

This multiple-version data contamination can be avoided if all LCB copies are invalidated except for one LCB copy which is changed. Then, the one remaining copy receives all changes and represents the latest version of the data element existing in any CPC, since only it can receive any changes and only it can be stored back to the common DASD to represent the latest copy of the record in the system.

The avoidance of multiple-version contamination is herein called "maintaining data coherence". Invalidation of all outstanding copies of a record except the copy being changed is the generally accepted solution for preventing multiple-version contamination in a record.

To prevent multiple-version contamination, the invention requires that any CPC, wanting to access (read or write) a record in the sysplex common DASD, must first register the record in a SES directory, and preferably read the record from the SES cache if it exists there.

The program in each CPC operates to maintain the coherency of data handled solely within the CPC, in any manner designed into the program, which may have been previously provided in the program independent of this invention. Hence, the data coherence controls provided by this invention in a sysplex are in addition to data coherence controls which may have previously existed in each CPC. Hence, the invention operates additional data coherence controls.

An important feature in each CPC, insofar as this invention is concerned, is the relationship between its local caches (LC) and its coherency vectors (V). In FIG. 1, CPC-1 is shown with local caches LC-1 and LC-2, which have associated coherency vectors V1 and V2, respectively. Note that LC-1 has N number of buffer entries (i.e. LCB-0 through LCB-N), which are respectively associated with coherency vector V1 bits 0-N. And that LC-2 has K number of entries (i.e. LCB-0 through LCB-K), which are associated with V2 bits 0-K. CPC-M is shown with a single LC(1), having LCBs 0-L associated with V(1) coherency bits 0-L.

Different versions may exist of the same record among these multiple copies in their different locations. The latest version exists in SES (if a copy is written in SES), and in the CPC making the last change in the record. Generally, the DASD copy is the most out-of-date version, because this copy is the last to be updated due to the slow nature of accessing the electro-mechanical DASD.

SES can only set these vector bits to an invalid state through a communication channel between SES and the CPC. When SES sets any coherence vector bit to the invalid state in a CPC's HSA, the CPC programming is not disturbed by such setting when it happens. The CPC programming continues without interruption due to the SES setting of any coherence vector bit. It is up to the CPC to determine when it will test the state of the vector bits to determine if any invalidation has been indicated. This manner of operation even gives the CPC the option of not using the sysplex coherence controls if the CPC has a situation in which coherence is not needed.

When programming determines that data in a local cache buffer is to be used, the CPC uses a "test vector entry" (TVE) CPU instruction to test the current state of the data coherence bit for that LCB in the CPC. The TVE instruction operates to invoke microcode which based on the specified LCT and LCEN locates the bit in the HSA m