Memory hierarchy: caches

The aim of this dossier is to study the hierarchy of caches located between a computer's processor(s) and main memory. There is an enormous difference in performance between the operating speeds of a processor and, more generally, the access times and transfer rates between memory elements located on a computer chip and the access times and transfer rates between different chips. Between a processor and its main memory, there's a hierarchy of caches, some on the processor chip, others on external chips, which act as throughput and access time adapters, since throughputs decrease and access times increase with distance from the processor. The other part of the memory hierarchy, between main memory and disks and other storage units, is the subject of a separate dossier.

This dossier presents the operating principles of caches and hardware techniques for improving performance, whether for low-end single-processor systems, systems with processors executing several instructions per cycle, or parallel systems using multi-core processors or clusters of multi-cores. The various techniques for ensuring cache coherence are presented, from basic centralized or decentralized protocols to protocols for hierarchical architectures.

Techniques for limiting the impact of caches and the relationship between cache operation and secondary memories, including address translations linked to virtual memory, are also covered.

While the dossier focuses primarily on hardware techniques for implementing cache hierarchies, the impact of caches on program execution times is highlighted by presenting classic software optimization techniques that take the existence of caches into account.