An apparatus and method is disclosed to manage storage coherency in a symmetric multiprocessing environment having a plurality of nodes, each of which contain a multitude of processors, I/O adapters, main memory and a system controller comprising an integrated switch with a top level cache. The nodes are interconnected by a dual concentric ring topology. Local controllers on any given node initiate bus operations on behalf of said processors and I/O adapters on that node. Snoop requests are launched onto the ring topology simultaneously in both directions. As the messages traverse the nodes on the ring, they trigger remote controllers to perform coherent actions such as cache accesses or directory updates. Messages arriving on each node from both directions are combined with each other and with locally generated responses to form cumulative final responses. Additionally, controllers on the requesting node may perform local coherent actions based on the information conveyed by the returning final responses. Overall system coherency is maintained through the use of a dual token based scheme which provide coherency points to permit multiple non-contending requests for the same data unit. The cache coherency methods described herein further ensure the latest copy of data is always accessed or modified even when multiple copies are present throughout the multi-nodal system structure. Traditional cache management states are extended to include Intervention Master and Multiple Copy status which minimize overall bus utilization. A novel ring protocol is contemplated which efficiently packages coherency information into bus operational responses that also allow simultaneous data transfers in the direction of minimal latency.
