A method and an apparatus for performing just-in-time data prefetching within a data processing system comprising a processor, a cache or prefetch buffer, and at least one memory storage device. The apparatus comprises a prefetch engine having means for issuing a data prefetch request for prefetching a data cache line from the memory storage device for utilization by the processor. The apparatus further comprises logic/utility for dynamically adjusting a prefetch distance between issuance by the prefetch engine of the data prefetch request and issuance by the processor of a demand (load request) targeting the data/cache line being returned by the data prefetch request, so that a next data prefetch request for a subsequent cache line completes the return of the data/cache line at effectively the same time that a demand for that subsequent data/cache line is issued by the processor.

An apparatus in a microprocessor for selectively retiring a prefetched cache line is disclosed. The microprocessor includes a prefetch buffer that stores a cache line prefetched from a system memory coupled to the microprocessor. The microprocessor also includes a cache memory, comprising an array of storage elements for storing cache lines, indexed by an index input. One of the storage elements of the array indexed by an index portion of an address of the prefetched cache line stored in the prefetch buffer is storing a replacement candidate line for the prefetched cache line. The microprocessor also includes control logic that determines whether the replacement candidate line in the cache memory is invalid, and if so, replaces the replacement candidate line in the one of the storage elements with the prefetched cache line from the prefetch buffer.

A system for memory management including a tag controlled buffer in communication with a memory device. The memory device includes a plurality of pages divided into a plurality of individually addressable lines. The tag controlled buffer includes a prefetch buffer including at least one of the individually addressable lines from the memory device. The tag controlled buffer also includes a tag cache in communication with the prefetch buffer. The tag cache includes a plurality of tags, where each tag is associated with one of the pages in the memory device and each tag includes a pointer to at least one of the lines in the prefetch buffer. Access to the lines in the prefetch buffer is controlled by the tag cache.