Communications system having plurality of originator and corresponding recipient buffers with each buffer having three different logical areas

Claims

Having thus described our invention, what we claim as new and desire to secure by Letters Patent is as follows:

1. High performance communications channels for exchanging messages with low latency between elements of data processing systems, each of said communication channels comprising:

an originator buffer in a message originator element and a recipient buffer in a message recipient element, said message originator element containing a plurality of originator buffers and said message recipient element containing a plurality of recipient buffers, an originator buffer of said plurality of originator buffers being paired with a recipient buffer of said plurality of recipient buffers,

each of said originator buffer and said recipient buffer being composed of three logical areas designated as a request area, a response area and a data area, respectively, said request area for storing request messages, said response area for storing response messages and said data area for storing data, each of said three logical areas being separate from one another, each of said originator buffers and said recipient buffers being hardware communications buffers;

a transmission path connecting said originator buffer and said recipient buffer;

said message originator element managing both said originator buffer and said recipient buffer and transferring a message request from the request area of said originator buffer to the request area of said recipient buffer without requiring a correct request signal and a correct grant signal from said originator buffer and said recipient buffer, respectively, and, selectively transferring message dam from the data area of said originator buffer to the data area of said connected recipient buffer; and

said message recipient element responding by transferring a message response from the response area of said recipient buffer to the response area of said originator buffer and, selectively transferring message data from the data area of the recipient buffer to the data area of the originator buffer,

wherein a message transmission is performed in a single transfer without a handshaking request signal and a handshaking grant signal and wherein said channel supports a plurality of concurrent message operations.

2. The high performance channels recited in claim 1 wherein

each one of said plurality of originator buffers is paired with one and only one of said plurality of recipient buffers and

each of said pairs of originator and recipient buffers supports a single message operation.

3. The high performance channels recited in claim 2 wherein said data areas and response areas of said plurality of originator buffers and the data areas and the request areas of said plurality of recipient buffers comprise physical buffers.

4. The high performance channels recited in claim 3 wherein said request areas of said plurality of originator buffers are virtual addresses in memory of said message originator element and said response areas of said plurality of recipient buffers are virtual addresses in memory of said message recipient element.

5. A data processing system having at least first and second elements interconnected by high performance channels for synchronously exchanging messages with low latency between said first and second elements, each of said high performance channels supporting at least one message and comprising:

an originator buffer in said first element and a dedicated recipient buffer in said second element, each of said originator buffer and said recipient buffer being composed of three logical areas designated as a request area, a response area and a data area, respectively, said first element containing a plurality of originator buffers and a plurality of recipient buffers and said second element containing a plurality of originator buffers and a plurality of recipient buffers each of said originator buffers and said recipient buffers being hardware communications buffers, said request area for storing request messages, said response area for storing response messages and said data area for storing data, each of said three logical areas being separate from one another;

first and second transmission paths of a high performance link connecting channels;

said first element managing both a connected originator buffer and a dedicated recipient buffer and transferring only a single request message over said first transmission path without requiring a correct request signal and a correct grant signal from said originator buffer and said recipient buffer, respectively, said single request message comprising a message request from the request area of said originator buffer to the request area of said recipient buffer and, selectively transferring message data from the data area of said originator buffer to the data area of said recipient buffer; and

said second element responding by transferring only a single response message over said second transmission path, said single response message comprising a message response from said response area of said recipient buffer to said response area of said originator buffer and, selectively transferring message data from the data area of said recipient buffer to the data area of said originator buffer,

wherein a message transmission is performed in a single transfer without a handshaking request signal and a handshaking grant signal and said channel supports a plurality of concurrent message operations.

6. The data processing system recited in claim 5 further comprising:

an originator buffer in said second element and a connected dedicated recipient buffer in said first element, said second element controlling both the originator buffer in said second element and said connected dedicated recipient buffer in said first element; and

said high performance channels supporting true peer-to-peer communications between said first and second elements of said data processing system.

7. The data processing system recited in claim 5 wherein

each one of said plurality of originator buffers is paired with one and only one of said plurality of recipient buffers; and

each of said pairs of originator and recipient buffers supports a single message operation.

8. The data processing system recited in claim 7 wherein said first and second elements each comprise central processing complexes.

9. The data processing system recited in claim 7 wherein said first element comprises a central processing complex and said second element comprises an electronic element.

10. The data processing system recited in claim 9 wherein said electronic element comprises an electronic storage device which receives in the request area of said recipient buffer a read command in said request message and returns data from said electronic storage device to said data area of said originator buffer with said response message and receives in the request area of said recipient buffer a write command and in the data area of said recipient buffer data in a single request message and returns a response message to the response area of said originator buffer confirming that the data has been written to said electronic storage device.

11. The data processing system recited in claim 9 wherein said electronic element comprises a co-processor which receives data in the data area of said recipient buffer frown said central processing complex in a single request message and returns data to the data area of said originator buffer in a single response message.

12. The data processing system recited in claim 7 further comprising a plurality of elements interconnected by said high performance channels in a network forming said data processing system, each of said plurality of elements being connected to at least one other element with a high performance channel having an originator buffer and a connected dedicated recipient buffer.

13. The data processing system recited in claim 12 wherein said network comprises one of a local area network and a wide area network and said at least one of said plurality of elements comprises a server and others of said plurality of elements comprise workstations.

14. The data processing system recited in claim 12 wherein said network comprises a tightly coupled multiprocessor system and said at least one of said plurality of elements comprises an electronic storage device and others of said plurality of elements comprise central processing complexes,

said electronic storage device receiving a read command in the request area of a recipient buffer in a request message from an originator central processing complex and returning data from said electronic storage device to the data area of an originator buffer in said originator central processing complex with a response message and receives a write command in the request area and data in the data area of a recipient buffer in a request message from an originator central processing complex and returns a response to the response area of an originating buffer in said originator central processing complex confirming that the data has been written to said electronic storage device, said electronic storage device issuing cross-invalidate messages from originator buffers in said electronic storage device to connected dedicated recipient buffers at other central processing complexes in said network,

each of said other central processing complexes responding by issuing a response message from the response area of the connected dedicated recipient buffers to the response areas of the originator buffers in said electronic storage device.

15. The data processing system recited in claim 5 further comprising timing means in said first element for timing an expected period for response to a request message, said first element waiting for said response message from said second element after sending said request message.

16. The data processing system recited in claim 15 further comprising means responsive to a timeout signal from said timing means for recovering from an individual operation in a multiple operation environment.

17. In a data processing system including high performance channels for exchanging messages with low latency between elements of the data processing system, each of said high performance channels supporting multiple messages and comprising an originator buffer in a first element and a recipient buffer in a second element, each of said originator buffer and said recipient buffer being composed of three logical areas designated as a request area, a response area and a data area, respectively,

said originator buffer and said recipient buffer being connected by a pair of transmission paths of a high performance link having a plurality of transmission paths, said first element controlling both a connected originator buffer and recipient buffer and transferring only one message, without requiring a correct request signal and a correct grant signal from said originator buffer and said recipient buffer, respectively, said message comprising a message request from the request area of said originator buffer to the request area of said recipient buffer and, said first element selectively transferring message data from the data area of said originator buffer to the dam area of said recipient buffer, and said second element responding by transferring only one message comprising a message response from said response area of said recipient buffer to said response area of said originator buffer and, selectively transferring message data from the data area of said recipient buffer to the data area of said originator buffer,

a method for recovery of individual operations in a multiple operation environment comprising the steps of:

sending a request from said originator buffer to said recipient buffer and setting a timer for a first expected response time period;

processing all requests at said recipient buffer in an order in which received;

checking said timer and if said first expected response time period expires, sending a first cancel operation request from said originator buffer to said recipient buffer and setting said timer for a second expected response time period; and

checking said timer and if a cancel request response is received from the recipient buffer by the originator buffer within said second expected response time period, terminating said request, the step of terminating said request having no effect on other messages processing operations being simultaneously in a same high performance channel.

18. In a data processing system including high performance channels for exchanging messages with low latency between elements of the data processing system, each of said high performance channels supporting multiple messages and comprising an originator buffer in a first element and a recipient buffer in a second element, each of said originator buffer and said recipient buffer being composed of three logical areas designated as a request area, a response area and a data area, respectively,

said originator buffer and said recipient buffer being connected by a pair of transmission paths of a high performance link having a plurality of transmission paths, said first element controlling both a connected originator buffer and recipient buffer and transferring only one message comprising a message request from the request area of said originator buffer to the request area of said recipient buffer and, selectively transferring message data from the data area of said originator buffer to tile data area of said recipient buffer, and said second element responding by transferring only one message comprising a message response from said response area of said recipient buffer to said response area of said originator buffer and, selectively transferring message data from the data area of said recipient buffer to the data area of said originator buffer,

a method for recovery of individual operations in a multiple operation environment comprising tile steps of:

sending a request from said originator buffer to said recipient buffer and setting a timer for a first expected response time period;

processing all requests at said recipient buffer in an order in which received;

checking said timer and if said first expected response time period expires, sending a first cancel operation request from said originator buffer to said recipient buffer and setting said timer for a second expected response time period; and

checking said timer and if a cancel request response is received from the recipient buffer by the originator buffer within said second expected response time period, terminating said request, the method further including the step of establishing a system threshold for a number of cancel operation requests that may be transmitted for any original request message,

wherein if said second expected response time period expires, said method further comprises the steps of:

sending additional cancel operation requests from said originator buffer to said recipient buffer and, for each said additional cancel operation request sent, setting said timer for an additional expected response time period;

counting cancel operation requests sent to generate a number and comparing said number with said threshold; and

returning a status message from said high performance channel indicating a failure of the high performance channel if said number equals said threshold.

19. In a data processing system including high performance channels for exchanging messages with low latency between elements of the data processing system, each of said high performance channels supporting multiple messages and comprising an originator buffer in a first element and a recipient buffer in a second element, each of said originator buffer and said recipient buffer being composed of three logical areas designated as a request area, a response area and a data area,

said originator buffer and said recipient buffer being connected by a pair of transmission paths of a high performance link having a plurality of transmission paths, said first element controlling both a connected originator buffer and recipient buffer and transferring only one message comprising a message request from the request area of said originator buffer to the request area of said recipient buffer and, selectively transferring message data from the data area of said originator buffer to the data area of said recipient buffer, and said second element responding by transferring only one message comprising a message response from said response area of said recipient buffer to said response area of said originator buffer and, selectively transferring message data from the data area of said recipient buffer to the data area of said originator buffer,

a method for recovery of individual operations in a multiple operation environment comprising the steps of:

sending a request from said originator buffer to said recipient buffer and setting a timer for a first expected response time period;

processing all requests at said recipient buffer in an order in which received;

checking said timer and if said first expected response time period expires, sending a first cancel operation request from said originator buffer to said recipient buffer and setting said timer for a second expected response time period; and

checking said timer and if a cancel request response is received from the recipient buffer by the originator buffer within said second expected response time period, terminating said request,

wherein if said second expected response time period expires, said method further comprises the steps of:

sending a second cancel operation request from said originator buffer to said recipient buffer and setting said timer for said second expected response time period;

upon receiving a cancel request response after sending said second cancel operation request, sending a cancel complete request from said originator buffer to said recipient buffer; and

responding to a cancel complete response from said recipient buffer to said originator buffer by terminating said request.

20. The method for recovery of individual operations in a multiple operation environment recited in claim 19 further comprising the steps of:

setting said timer for said second expected response time period when said cancel complete request is sent; and

checking said timer and if a cancel complete response is not received from the recipient buffer by the originator buffer within said second expected response time period, sending a second cancel complete request from said originator buffer to said recipient buffer.

21. The method for recovery of individual operations in a multiple operation environment recited in claim 20 wherein if a cancel complete request is received, terminating said request message.

22. The method for recovery of individual operations in a multiple operation environment recited in claim 20 further comprising the steps of establishing a system threshold for a number of cancel complete requests which may be transmitted and setting said timer for said second expected response time period for each cancel complete request sent,

wherein if said second expected response time period expires after sending a cancel complete request from said originator buffer to said recipient buffer, said method further comprises the steps of:

sending additional cancel complete requests from said originator buffer to said recipient buffer;

counting said cancel complete requests sent to generate a number and comparing said number with said threshold; and

returning a status message from said high performance channel indicating a failure of the high performance channel if said number equals said threshold.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to channels for exchanging messages between central processing complexes and message processors, including input/output (I/O) devices and systems and other central processing complexes, and more particularly to high performance buffering which facilitates very low latency communications between elements of very high speed data processing systems. The invention allows true peer-to-peer communications between central processing complexes and implements a recovery procedure of individual operations in a multiple operation environment.

2. Description of the Prior Art

In a conventional data processing system, a central processing complex (CPC) having a channel path is connected to a control unit to which are attached one or more strings of direct access storage devices (DASDs), such as disk drives. The control unit includes a buffer for temporarily storing data sent by the central processing complex to be written to one or more of the DASDs. The central processing complex and the control unit operate in a master/slave relationship. The sending of data between the central processing complex channel and the control unit requires a number of handshaking messages to be sent between the two before data can start to be transmitted. The time it takes to do this handshaking slows down the performance of the data transfer significantly.

Various techniques have been employed to gain a marginal improvement in performance. One such technique involves a "shadow write" operation wherein the data transmitted by the central processing complex is buffered in the control unit but not written to DASD until later. Nevertheless, the central processing complex is notified by the control unit that the write operation has taken place, thereby eliminating the time delays normally associated in the electromechanical write operations.

Multiprocessor (MP) systems have been developed to increase throughput by performing in parallel those operations which can run concurrently on separate processors. Such high performance, MP data processing systems are characterized by a plurality of central processor units (CPUs) which operate independently and in parallel, but occasionally communicate with one another or with a main storage (MS) when data needs to be exchanged. In the type of MP system known as a tightly coupled multiprocessor system in which each of the CPUs have their own cache memory, there exist coherence problems at various levels of the system. A number of solutions to this problem are known in the art. One approach involves a cross-interrogate (XI)technique to insure that all CPUs access only the most current data.

Recently, there have been developed massive electronic storage devices which are replacing the slower, electromechanical DASDs used in older systems. These electronic storage devices, while representing a significant increase in the speed of MP systems, do not address the prior problem associated with the handshaking protocol of prior systems. Added to that is the cross-interrogate (XI) process that generally characterize memory management in a tightly coupled MP system.

There are also known data processing systems in which a plurality of central processing complexes (CPCs) are interconnected via a communication link. The CPCs run independently but must communicate with one another to transfer and/or process data. The CPCs may be large main frame computers which communicate via some message processor, or they may be a plurality of individual work stations communicating over a local area network (LAN) or wide area network (WAN) which might typically include a server. In either case, communication is typically handled in a master/slave relationship, even between large main frame computers. The master/slave designation changes depending on the flow of data, but the type of handshaking protocol described for I/O devices is typical resulting in delays in data transfer.

High speed data processing systems and elements are being developed wherein the communication process is, in many cases, the limiting factor in data throughput. The transmission capacities of the media, notably copper, has been a limiting factor in the performance of communications. New media, such as optical wave guides, e.g., fiber optic cables, have significantly higher transmission capabilities than was possible with previous media. There is considerable need for overall improvement in data communications and data throughput in high performance data processing systems.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide low latency channels for high performance data processing systems.

It is another object of the invention to provide true peer-to-peer communications between central processing complexes in a data processing network.

It is a further object of the invention to provide a protocol which supports recovery of individual operations in a multiple operation environment.

According to one aspect of the invention, buffers are provided in two elements between which data is to be transferred wherein both buffers are dedicated solely by the originator of the data transfer. In other words, the master/slave relationship communications of the prior art, and its attendant protocols, is eliminated by the subject invention. Only one transfer is required to transmit a message, and a second transfer acknowledges the completion of the function because message delivery to the receiver is guaranteed under the protocol implemented by the invention.

A message operation includes the exchange of up to four pieces of information. A request part is always passed from the originator of the message to the recipient. A response part is always passed from the recipient to the originator. Optional data parts may be passed from the originator to the recipient or from the recipient to the originator. Both, one or none of the data parts may be part of a given message.

The channels of the originator and the recipient are physically connected by a link. The link media, whether copper, fiber optic or other media, is presumed lossy. The loss of a portion of a message results in an error for the message. The corrective action of an error is to cancel the message, such that another message can be sent on another channel and there is no race condition between the original message that failed and the second message.

As a result, there are times when an operation must be canceled, as for example, when the originator does not receive an acknowledgment of the completion of the function. Therefore, when a request is sent by an originator, a message timer is started by the sender. When the normal response for the request is received, the timer is reset. A message timeout may span a number of message-level exchanges. An example is cross-interrogate (XI) messages generated as a result of execution of a write command to a message-processor that is a shared memory structure. The cross-interrogates are sent to the users of the shared memory structure.

If the message timer times out, whether due to no response or a response out of time, a recovery procedure is initiated. This recovery procedure involves canceling the original operation so as to clear the buffer at the recipient and assure the originator that it can issue another request without fear that it may be canceled by an earlier cancel operation request.

When an operation must be canceled, a cancel operation command is issued. However, when the cancel operation fails to get a response, one would like to be able to issue more cancel operation commands until the operation is canceled. There is a problem of determining to which cancel operation command the recipient is to respond; that is, is the recipient responding to the first, the last, or a command in between after several cancel operation commands have been issued.

According to a second aspect of the invention, if additional cancel operation commands are sent as the result of a failure of the first or subsequent cancel operation commands, a secondary command that is different from the cancel command is required to be issued and responded to prior to marking the operation as being canceled. Since the commands must be executed in the sequence in which they are received, a response to the secondary command ensures that there are no other cancel operation commands remaining in the receiver, allowing subsequent operations to start without danger of being canceled.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:

FIG. 1 is a high level block diagram of a conventional I/O system for a central processor complex;

FIG. 2 is a flow chart showing a typical handshaking protocol for the I/O system shown in FIG. 1;

FIG. 3 is a high level block diagram of a high performance channel according to the invention;

FIG. 4 is a flow chart showing the communications protocol implemented in the high performance channel shown in FIG. 3;

FIG. 5 is a block diagram of high performance channel message buffers for a message originator and a message recipient as used in the practice of the invention;

FIG. 6 is a block diagram illustrating high performance channels with multiple message buffers for peer-to-peer operations;

FIG. 7 is a block diagram showing a high performance link between a message facility and a message processor according to the invention;

FIG. 8 is a block diagram showing the frame structures according to the