Balancing data-processing work loads

What is claimed is:

1. The machine-implemented method of balancing loads between a first unit and a second unit,

the automatic machine executed steps of:

in each of said units separately measuring delays in current activity to be balanced for a plurality of respective work allocations and storing said measurements;

in each of said units summing and storing said measured delays for generating a measured stored total delay for each such unit, separately comparing all of the separately measured delays of each of said work allocations with a said total delays in the units, respectively, and for each of said units identifying a one of said work allocations as exhibiting a mean current activity for respective units; said means current activity being an activity which exhibits a separately measured delay having a middle value with respect to all of said separately measured delays in the respective unit;

in a first unit detecting and indicating when said stored total measurements in said first unit exceeds a predetermined threshold;

in response to said indicating, comparing the stored total delay measurement indicated in said first unit with the total stored delay measurement indicated in said second unit to obtain a difference signal, comparing said difference signal with a threshold difference signal, if said difference signal is less than said threshold difference signal, do nothing to balance loads; if said difference signal exceeds said threshold differece signal, then transfer said identified one work allocation of said first unit from said first unit to said second unit.

2. The machine-implemented method set forth in claim 1 further including the steps in each unit, during said separate comparison in each unit, of; of identifying a identifying a second work allocation exhibiting a minimal measured delay with respect to other work allocations,

separately and independently indicating a work capacity in each of said units; and

after each transfer of said one work allocation, comparing the number of work allocations of said second unit with said capacity indication, if said exceeds work allocations in said second unit exceed said capacity indication, transfer said second work allocation from said second unit to said first unit.

3. The machine-implemented method set forth in claim 1 wherein said first and second units are first and second control units, respectively, in a peripheral system connected to a host system and having a plurality of peripheral devices operable with either of said first or second control units and with data signal transfers proceeding through said control units with respect to said peripheral devices, said peripheral devices constituting said work allocations, respectively; and

transferring said devices for operation with said units from said first control unit to said second control unit to achieve said work allocation transfer.

4. The machine-implemented method set forth in claim 3 wherein each control unit can supply delay-indicating signals to a connected host indicating that data signal transfers between such host and control unit must be delayed due to operational status of a given one of said devices, the method further including the automatic steps of:

in each control unit, for each device operatively associated with such control unit, tallying the number of said delay-indicating signals as device tallies for constituting said measured delays, and tallying the total number of said delay-indicating signals sent by such control unit for all of said devices operationally associated therewith as control unit tallies for representing said measured delays; and

when the control unit tally of said first control unit exceeds a predetermined numerical threshold, subtracting the control unit tally of said second control unit from the control unit tally of said first control unit to create a difference tally corresponding to said difference signal.

5. The machine-implemented method set forth in claim 4 further including the automatic-machine steps of:

in said first control unit, dividing said difference tally by two to create a transfer tally,

separately comparing all of said device tallies in said first control unit to said transfer tally, and

selecting the device having a device tally closest to said transfer tally as a work allocation to be transferred from said first control unit to said second control unit.

6. The machine-implemented method set forth in claims 1 or 5 further including the automatic machine step of resetting all of said measured delays in both said first and second units whenever said stored total measurement in said first unit exceeds said predetermined threshold.

7. The machine-implemented method set forth in claims 1 or 5 wherein said first and second units each include a buffer memory having a first plurality of addressable registers, segments of said buffer memory including a second plurality less than said first plurality of said addressable registers, each such segment being allocatable to a one of said work allocations, further including the automatic machine steps of:

upon a transfer of said predetermined portion of said work allocations, deallocating a said segment in said first unit from said transferred work allocation and allocating a segment in said second unit to said transferred work allocation.

8. The machine-implemented method of balancing workloads between two parallel data processing paths, the automatic machine steps of:

segmenting the data processing in each path into work allocations;

separately tabulating undesired delays in data processing for each of said work allocations;

totaling said tabulations for each of said data processing paths;

independently comparing the total tabulations for each path with a first threshold of such undesired delay tabulations;

when one of said total tabulations for a given one of said data processing paths exceeds said first threshold, then determining the difference in the total delay tabulations between said paths; when the difference between the totals of delay tabulations exceeds a predetermined value, then transfer a one work allocation of said given one data processing path to a second data processing path; selecting said one work allocation to have an average number of all said separate undesired delay tabulations in the one data processing path with respect to said difference of said total delay tabulations; and

each time said first threshold is exceeded in either one of said paths, recalibrating said separate and total tabulations for all of said data processing paths and without affecting said determining step and said selecting step.

9. A machine-implemented method of operating a data processing system having a plurality of independent, program controlled, data processors for regulating the flow of tasks to said data processors from a plurality of sources to facilitate the processing of the tasks with a minimal delay, comprising the automatic machine steps of:

assigning said tasks to said data processors, each task being assigned to but one data processor, monitoring and totaling, individually for each task and collectively for each data processor for each data processor, the number of time delay occurrences in each of said data processors resulting from tasks then being performed by them, the totaling of said time delays being substantially concurrent with the occurrence of the respective time delays;

selecting a separate upper limit value for a total number of task delay occurrence for each said data processor;

diverting a task having a mean number of occurrences of said task delay occurrences of all task delay occurrences in a one of said data processors and assigned to one of said data processors to another of said data processors upon a determination that the difference between task delay occurrence totals of said data processors exceeds a given difference limit value; and

processing said diverted task in said another data processor.

10. The machine-implemented method set forth in claim 9 wherein said sources are input-output channels and said tasks include transferring data signals between said input-output channels and a plurality of peripheral devices and said data processors are control units interposed between said peripheral devices and said input-output channels and said step of diverting said coupled task includes disconnecting a peripheral device of said diverted coupled task and assigned to said one data processor and connecting it to said another data processor.

11. The machine-implemented method set forth in claim 9 or 10 further including the automatic machine steps in said another data processor after a one of said data processors has transferred a task to said another data processor, of:

establishing a return threshold and identifying a task therein as having a minimal number of time delay occurrences comparing a number of tasks assigned thereto with a said return threshold, and when said number of tasks exceeds said return threshold, diverting a task with said minimal number of task delay occurrences from said another data processor to said one data processor.

12. The machine-implemented method set forth in claims 9 or 10 further including the step of resetting all of said separate and total tabulations in all of said data processors each time any one of said collective total tabulations exceeds said upper limit value.

13. The machine-implemented method set forth in claim 12 wherein each of said assigned tasks includes multiple successive data operations, upon initiation of each data operation in any task in any of said data processors performing said monitoring and totaling step.

14. An automatic load-balancing data processing unit adapted to be connected to at least another data processing unit in a distributed processing network, a memory in each of said data processing units and each memory having allocatable segments for use with diverse data processing operations,

the improvement comprising:

means in each of said data processing units for indicating data processing delays in any of said data processing operations associated with said segments;

a work delay counter for each segment in each of said data processing units and respectively coupled to said indicating means to respond to each indicated delay for the respective segments for tallying a number of said indicated data processing delays respectively associated with data processing operations using each such segment;

a total delay counter in each of said data processing units and coupled to said indicating means, respectively, for tallying all of said delay indications independent of which data processing operation is involved in such delay;

a first threshold detector in each of said data processing units and coupled to said total delay counter, respectively, for detecting and indicating when a count in said total delay counter exceeds a predetermined value; and

transfer means in each of said data processing units and coupled to said work delay counters, respectively, in said data processing units, and responsive to said respective first threshold detector for separately comparing the number in each said work delay counters with the number in said total delay counter to indicate a one of said segments associated with a one of said data processing operations having a number of delays closest to a given mean number of delays with respect to all said number of work delays for said segments, in a respective one of said data processing units whereby such one data processing operation is indicated as eligible for transfer to said another data processing unit for initiating a load balancing operation between the two data processors.

15. The data processing unit set forth in claim 14 further including in combination:

message receiving means in a first given one of said data processing units for receiving, during a load balancing operation initiated by second given one of said data processing units connected to the first given one of said data processing units, identifications of data processing operations to be performed;

data processing operation capacity means in said first given one of said data processing units for analyzing all the data processing operations workload to be performed and having means for comparing said workload with a work threshold and to indicate when said work threshold is exceeded;

minimal work identifying means in said first given one of said data processing units responsive to said capacity means indication to identify a data processing operation exhibiting a minimal workload requirement and having means to actuate said transfer means to indicate another one of said segments associated with said identified minimal workload data processing operation as eligible for transfer to said another data processing unit; and

said message receiving means being coupled to said second given one of said data processing units, said capacity means being coupled to said receiving means, said identifying means being coupled to said capacity means and to said transfer means.

16. The data processing units set forth in claims 14 or 15 further including, in combination:

a difference threshold detector in each of said data processing units and coupled to said total delay counter for receiving its total delay count;

means in each of said data processing units coupled to another of said data processing units for receiving from said another data processing unit an indication of its total delay count and for supplying the receiving indication to said difference threshold detector in such each data processing unit;

reset means in each of said data processing units responsive to said first threshold detector of such each data processing unit indication of a count exceeding said predetermined value to reset all said counts in such each data processing unit;

said difference threshold detector in each data processing unit being interposed between said first threshold detector and said transfer means of such each data processing unit such that said difference threshold detector thereof responds to said first threshold detector indication thereof, independently of said reset means, to compare the total delay counts of said another data processing unit and said total delay counter of the each data processing unit and to actuate said transfer means thereof to indicate a one of said segments having a mean value of the difference in said total counts.

17. The data processing units set forth in claim 16 wherein each of said data processing units is a control unit for coupling a host to a plurality of peripheral devices, the improvement further including, in combination:

device-access indicating means in each control unit for indicating accessibility of said peripheral devices;

said data processing delay indicating means in each control unit being coupled and responsive to said device-access indicating means indicating no immediate access to a given device to indicate to a host a data processing delay; and

said allocatable segments in such each control unit being allocatable for exclusive use for respective ones of said peripheral devices such that said data processing operations are data transfers between said memory and said peripheral devices while said delays indicated to said host result from delays in said data transfers.

18. Data processing apparatus including a host processor system (12) connected to issue work instructions to a plurality of peripheral devices (13), each of said peripheral devices having an operational status and being allocatable for operation to either of two control units (11), each of which allocates work to its said allocated peripheral devices or stores such work, depending upon said operational status of the device concerned, and to initiate work transfer between control units each control unit including: means (41,42) to separately store the number of work delay occurrences for each of its said allocated peripheral devices means (45) to summate the the delay occurrences, means (46) to indicate when the summation of delay occurrences exceeds a threshold value, means (55), responsive to such indication, to compare the summation of delay occurrences in the two control units to obtain a difference, means (55) to compare such difference with a threshold difference value, means (58) responsive to initiation of work transfer to compare the separately stored number of work delay occurrences with said difference of summated delays and to select a one of said peripheral devices corresponding to a one of said stored number of work delay occurrences having a value closest of any of said stored numbers of work delay occurrences to a transfer value which is a predetermined proportion of said difference between the summated delay occurrences such that the transfer or non-transfer of any particular peripheral device depends on its own contribution to the overall work load of the pertinent control unit, and means (60), responsive to such difference exceeding the threshold difference value, to initiate transfer of said selected one peripheral device from a first one to a second one of said control units control unit.

19. Apparatus according to claim 18, in which each control unit is responsive to host processor commands to issue a signal (CCR) indicating a work delay occurrence, and tallying such signals in the current work delay store means (41,42).

20. Apparatus according to claim 19, in which each said control unit includes a memory (15) with segments (17) allocatable to said peripheral devices, and a separate counter for each of said segments in said summate means for tallying said separate work delay occurrences.

21. Apparatus according to claim 18, 19 or 20, including means to identify and select a work allocation involving a minimum number of delay occurrences for transfer said first one control unit.

22. Apparatus according to claim 21, in which the control-unit-receiving-work transfers other work to the control-unit-transferring-the-work when the summated delay occurrences of said control-unit-receiving-work exceeds the threshold value in the control-unit-receiving-work.

23. Apparatus according to claim 18, 19 or 20, including means (56) to halve the difference between the total delay occurrence counts in the control units, and in which the selection of work transferred is initially based on this halved value.

Description Submit all comments and votes

DOCUMENTS INCORPORATED BY REFERENCE

U.S. Pat. No. 3,688,274 shows a channel command retry (CCR).

U.S. Pat. No. 3,400,371 shows a central processor, input/output channel, and a peripheral subsystem.

Reference to Co-filed Commonly Assigned Patent Application

C. A. Milligan, et al., Ser. No. 241,274, filed Mar. 6, 1981, entitled "Buffered Peripheral Subsystem".

FIELD OF THE INVENTION

The present invention relates to multi-unit data processing systems, particularly to balancing workloads between various units within one system. While the invention is particularly useful for loosely-coupled data-processing systems, the principles of the invention can be applied to tightly-coupled systems, symmetrical systems, master-slave systems, and the like, with good advantage.

Balancing data processing loads between a plurality of units usually occurs at so-called task assignment time. That is, before data processing work ensues, a control mechanism determines which unit should do the task; once the task is assigned to a unit, that unit continues to operate even though it may be more heavily loaded than other units in the system. An example of such task assignment balancing is found in the IBM Technical Disclosure Bulletin, Vol. 20, No. 3, August 1977, pp. 937-938, in the article entitled "Load Balancing Control for a Multi-processor," by J. F. Baker and D. E. Holst. This article describes a loosely-coupled multi-processor control storage and memory system having load balance tables in various processors controlling the system. Load balance is achieved at assignment time based upon load balance tables which indicate a measurement of work Q depth. Load balance information is exchanged between the various processors of the memory system. The scheduling of timed processes is also described. Another example of load balancing at assignment time is found in a similar IBM Technical Disclosure Bulletin article, Vol. 14, No. 11, April 1972, in an article entitled "Input/Output Channel/Device Load Balancing Algorithm," by L. A. Jaikes, et al., wherein a peripheral subsystem has its work balanced at work assignment time.

Central processors or hosts in a multi-processing arrangement often provide for load balancing at task assignment time. An example is shown in U.S. Pat. No. 3,648,253 which shows tasks being assigned in a multi-processor arrangement by a programmed scheduler 15 based upon time to go on a present task. The balancing of work loads is by assignment of tasks before the tasks are initiated. U.S. Pat. No. 4,032,899 shows a data switching network which balances data traffic on an aggregate processing load. This balancing is achieved by scheduling output traffic to ports by individual processors on an exclusive assignment basis; i.e., load balancing again is achieved when the task to be performed is first assigned.

Load balancing also has been achieved upon detection of an error condition; for example, U.S. Pat. No. 3,787,816 shows a multi-processing system which may be reconfigured in a controlled manner to redesignate the functions assigned to particular similar units so as to provide continued data processing capabilities after a malfunction or error.

Activity monitors have been employed for balancing loads. U.S. Pat. No. 3,588,837 shows a system for measuring activity of all major data paths using a time interval utilization sampling technique. The samples are dynamically recorded to represent the ratio of number of samples for revealing the number of samples in one time INTERVAL compared with the number of samples taken during an earlier time interval whereby the activity of all potential queuing points within a dynamic environment are recorded to provide statistical data concerning utilization of data-processing and communication equipment. This patent shows a measurement system but not load balancing which could be driven by such a measurement system.

Not all work load balancing has been achieved at assignment time; for example, U.S. Pat. No. 4,099,235 shows a method of operating a data processing system having two real-time data processors wherein given tasks are performed in one of the data processors depending upon the character of the task and which processor has been selected to operate on such tasks. Each of the data processors is continuously monitored for the purpose of continually determining the utilization ratio for each processor. Each processor is assigned a predetermined upper limit value of such utilization ratio which lies below the processors overload limit. Whenever such upper limit is exceeded, the tasks being performed in the one data processor are diverted to another data processor such that the receiving or another processor performs the diverted task. This patent shows a utilization threshold as a method of instigating shifting of ongoing tasks between data processors. The method disclosed is preferably a ratio of the holding time resulting from tasks being performed to the free time for indicating utilization ratio. This patent also refers to U.S. Pat. No. 3,665,404 wherein peripheral devices are switched between data processors by means of electro-mechanical switch in connection with balancing the input/output load between processors. According to U.S. Pat. No. 4,099,235, many of the real-time operations are similar to batch processing; accordingly, tasks that are not currently being operated upon can be readily transferred between the processors. This means that only inactive tasks are transferred for purpose of a load balancing.

In a dynamic data processing system where activity can vary beyond a control of the controlling data processors, the load balancing between the various data processors/data processing paths should be such to fully accommodate subsequent unforeseen dynamic changes in activities such that load balancing activity is minimized for maximizing data processing throughput.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide load balancing between data processing units or paths that shift work exhibiting a predetermined portion of the total work load of the data processing unit/path to an alternate unit/path in a manner for minimizing load balancing activities and for maximizing data processing throughput.

In accordance with the present invention, a method of balancing work loads between parallel data processing paths, such as between plurality of input/output channels, plurality of control units in a peripheral subsystem, and a plurality of data processing units in a multi-processing unit environment, and includes segmenting the data processing in each path into work allocations. Further, the invention contemplates separately tabulating undesired delays in data processing for each of said work allocations in the respective parallel data processing paths. The separate tabulations are then totaled for each of the data processing paths. In each path the total tabulations are compared with a given threshold for such undesired delays. When one of the total tabulations for any one of the data-processing paths exceeds its threshold, a portion of the work allocation of its one data processing path can be transferred to a second data processing path, with the transfer selection being on a selected percentage of undesired delays for all work allocations. In one such data processing path, for example, a work allocation having a mean number of delays equal to one half the difference of the total tabulations of two paths can be transferred. It is preferred that no load balancing occur until a minimum difference in tabulations between paths is exceeded. In another version, a work allocation having an average number of such delays can be transferred. Upon any path tabulation exceeding the threshold, the total and work allocation tabulations are all recalibrated for instituting a new set of tabulations for load balancing.

In a preferred embodiment of the invention the data processing paths are control units in a peripheral subsystem. Work allocations are peripheral device assignments to respective control units with the undesired delays being channel command retrys (CCR's) sent to a host computer, such as described in U.S. Pat. No. 3,688,274.

When a receiving data processing unit/path has an excessive number of work allocations, such receiving units/path will reassign a work allocation to the sending units/path which work allocation has a least number of delays or can be a least used one of work allocations, i.e., the work allocation least contributing to the work load of the receiving units/path.

IN THE DRAWINGS

FIG. 1 is a diagram of a peripheral subsystem employing the present invention and in which the diagram accents certain aspects of the invention.

FIG. 2 is a logic diagram of a peripheral subsystem as would be constructed and which can incorporate teachings of the present invention.

FIG. 3 is a logic diagram of a control portion of the FIG. 2 illustrated peripheral subsystem.

FIG. 4 is a map of a control memory portion of the FIG. 3 illustrated control.

FIG. 5 is a generalized flow chart showing the work load balancing between the control units of the peripheral subsystem shown in FIGS. 1 and 2.

FIG. 6 is a logic diagram showing the flow of control in both control units for a load balancing operation.

FIG. 7 is a diagram of a control unit table used in connection with a preferred embodiment of the present invention.

FIG. 8 is a diagram of a logical device table which stores certain characteristic information and address information for a logical device in the FIGS. 1 and 2 illustrated peripheral subsystem.

FIG. 9 illustrates a command status table used in connection with the operation of the FIGS. 1 and 2 illustrated peripheral subsystem particularly as those operations relate to balancing loads between a plurality of control units.

FIG. 10 shows a buffer status table usable by the control of FIG. 3 in practicing certain aspects of the present invention when a control unit contains a buffer such as shown in FIGS. 1 and 2.

FIG. 11 is a logic diagram showing decoding input/output commands received by the control of FIG. 3.

FIG. 12 is a logic diagram showing incrementation of undesired delays which are termed logical device faults.

FIG. 13 is a logic diagram illustrating operation of a control unit for sending a logical device fault count to another control unit for implementing a load balancing operation.

FIG. 14 is a logic diagram illustrating transferring a device from one control unit to another control unit in a load balancing operation.

FIG. 15 shows a four byte set of registers constituting a sequence control table relating to sending devices between control units of the FIGS. 1 and 2 illustrated peripheral subsystem.

FIG. 16 is a logic diagram illustrating reallocation of a peripheral device to a receiving control unit.

FIG. 17 is a logic diagram showing deallocation of a buffer segment in connection with transferring a device from a control unit which has a segment of a buffer allocated to such device.

FIG. 18 is a logic diagram showing control unit activity in a load balancing operation which requires data processing functions to be performed before load balancing can be effected.

FIG. 19 is a logic diagram illustrating a control unit counting the number of logical devices currently active with respect to such control unit.

FIG. 20 is a logic diagram illustrating the transfer of a least active device from one control unit to another control unit.

FIG. 21 is a logic diagram relating to the allocation of a buffer segment to a device such as upon receipt of a device from a load balancing transferor or upon initialization of a logical device.

FIG. 22 is a logic diagram showing allocation of a single buffer segment to a device for constituting a logical device within a peripheral subsystem.

FIG. 23 is a logic diagram illustrating a simple scan technique used in connection with actuating the logic modules used in connection with the operation of the control units of FIGS. 1 and 2 illustrated peripheral subsystem.

FIG. 24 is a logic diagram showing adjustment of load balancing delay tabulations upon deactivation or other removal of a device from load balance operation of a peripheral subsystem.

FIG. 25 shows a sequence control table.

DETAILED DESCRIPTION

Referring now more particularly to the drawing, like numerals indicate like parts and structural features in the various diagrams. The invention is illustrated as being incorporated into storage subsystem 10 having a pair of control units 11, also denominated as CU-0 and CU-1. Storage subsystem 10 is connectable to a plurality of hosts 12 for receiving, storing and supplying data signals from and to the respective hosts under control of host operations as is practiced in the data processing art. Storage subsystem 10 stores data signals on behalf of the host in a plurality of data storage devices 13, such data storage devices include, without limitation, magnetic tape recorders, magnetic disk recorders, magnetic card recorders and unit record equipment. Communication between hosts 12 and storage subsystem 10 is via a plurality of input/output channels 14, constructed generally in accordance with the input/output channels set forth in Amdahl, et al., U.S. Pat. No. 3,400,371. For enhancing subsystem operations, each control unit 11 includes a data buffer 15, preferably constructed of a semiconductive random access memory element. Buffers 15 are the prime conduits between hosts 12 and devices 13; the arrangement is such that a host 12 can comunicate with a given device through either buffer 15 of either control unit 11. Communication from input/output channels 14 to buffer 15 is via channel adapter CXX 80 and bus 81 in CU-0 and via bus 96 to CU-1. It is understood that CU-1 is constructed identically to CU-0 with complementary connections (not shown) in CU-1. For example, bus 97 connects a channel adapter 14 (not shown) of CU-1 to buffer 15 of CU-0 via channel adapter circuits 16. Circuits 16 are known automatic data transfer (ADT) circuits commonly used in the data processing art. Since a plurality of devices 13 communicate through a single buffer 15 to a plurality of hosts 12, buffer 15 is dynamically managed as a plurality of segments indicated by dashed lines 17. That is, when a given device 13 is communicating with host 12, it is assigned or allocated a segment of buffer 15 for handling the data transfers. Devices that are not currently transferring data need not be assigned such segments; this allows buffer 15 to be relatively small, such as 256,000 bytes of storage. Buffer 15 allocations to devices are normally maintained between successive data transfers.

Communication between buffer 15 and devices 13 is also through an automatic data transfer mechanism 83 and also referred to as data flow circuits 83 in FIG. 2. Connections from automatic data transfer DX 83 to the devices 13 is via a pair of cables 90 and 94, as detailed in FIG. 2. In a similar manner, CU-1 is connected to the devices 13 by another pair of cables 93, 95.

The control of storage subsystem 10 resides jointly in control units 11, it being understood that the control circuits of CU-1 are identical to the illustrated control circuits of CU-0. The showing in FIG. 1 is simplified in that those control circuits not pertinent to an understanding of the present invention are not detailed. Each control unit 11 includes a control 33, preferably including a programmed digital computer or microprocessor 110. Miscellaneous controls are indicated by numeral 34. Since the control of storage subsystem 10 is shared between CU-0 and CU-1, interconnecting bus 109 provides communications between the two control units 11 for exchanging control data necessary to the logical control of the storage subsystem. Miscellaneous control 34 also controls the automatic data transfer circuits CX 16 and DX 83 and the operation of buffer 15. Connections 81, 35 and 36 as shown in FIG. 2 represent this control.

A first portion of the invention relates to measurements of activity and work allocations in CU-0 and CU-1 to ensure that the work is balanced between the two control units on a dynamic basis. Since the buffers 15 of CU-0 and CU-1 are the prime conduits for data transfers between hosts 12 and devices 13, the status of the two buffers 15 at the time of a request for data by a host 12 is used as an indication of the current ability of the respective control units 11 for satisfying host 12 requests. For example, whenever a host 12 requests data from storage subsystem 10 and that data is not in the buffer 15, then that event is used as an indication of control unit work load. In a similar manner, if a host 12 desires to record data on a device 13 and the buffer segment indicated by numeral 17 is full of data, then that status is an indication of control unit work load. The fact that a buffer segment is not allocated to a device when a host requests the use of that device requiring buffer operation is also an indication of control unit work load. Such indications are provided by control 34 responding to a host 12 supplied input/output channel commands and control 34 determination that it cannot be immediately satisfied by supplying a channel command retry (CCR) signal over line 40 through CXX 80 to the requesting host 12. Such channel command retries are fully explained in R. L. Cormier, et al., U.S. Pat. No. 3,688,274. According to the invention, each control unit 11 includes circuitry and controls for utilizing the CCR signal to maintain load balance between the control units 11.

First, for each allocated segment of a buffer 15, a separate tally "CCRK" is provided for the number of CCR's. Since tallies, CCRK, are held in registers 41, 42. The CCR signal on line 40 also goes to respective registers 41, 42 for increasing the respective CCRK count. Each CCR signal is associated with an address of device 13 (the device to which the segment represented by register 41, 42 is allocated); registers 41, 42 are addressed accordingly. In this manner, the busy state for each of the allocated buffer segments is maintained as a separate tally. These separate tallies indicate the relative responsiveness of the respective device 13 and associated buffer 15 segments to host 12 operations. The combination of an allocated buffer segment with a recorder or device 13 is termed a logical device, i.e., to hosts 12 the device 13 and allocated segment appear as a single unit. Each host 12 in its communication with storage subsystem 10 always addresses devices 13; accordingly, the buffer 15 segments are not explicitly addressed by hosts 12 but are implicit within a device 13 address and then only when allocated to a device 13.

When a control unit 11 determines that its load status is such that work load