Enhancing Last-Level Cache Performance by Block Bypassing and Early Miss Determination

  • Haakon Dybdahl
  • Per Stenström
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4186)


While bypassing algorithms have been applied to the first-level cache, we study for the first time their effectiveness for the last-level caches for which miss penalties are significantly higher and where algorithm complexity is not constrained by the speed of the pipeline. Our algorithm monitors the reuse behavior of blocks that are touched by delinquent loads and re-classify them on-the-fly. Blocks classified as bypassed are only installed in the level-1 cache. We leverage the algorithm to early send out a miss request for loads expected to request blocks classified to be bypassed. Such requests are sent to memory directly without tag checks at intermediary levels in the cache hierarchy. Overall, we find that we can robustly reduce the miss rate by 23% and improve IPC with 14% on average for memory bound SPEC2000 applications without degrading performance of the other SPEC2000 applications.


Main Memory Cache Line Shadow Instruction Cache Block Reuse Distance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abraham, S.G., Sugumar, R.A., Windheiser, D., Rau, B.R., Gupta, R.: Predictability of load/store instruction latencies. In: MICRO, vol. 26 (1993)Google Scholar
  2. 2.
    Austin, T., Larson, E., Ernst, D.: SimpleScalar: an infrastructure for computer system modeling. IEEE Computer 35(2) (2002)Google Scholar
  3. 3.
    Belady, L.: A study of replacement algorithms for a virtual-storage computer. IBM Systems Journal 5(2), 78–101 (1966)CrossRefGoogle Scholar
  4. 4.
    Chi, C.-H., Dietz, H.: Improving cache performance by selective cache bypass. In: Annual Hawaii International Conference on System Sciences (1989)Google Scholar
  5. 5.
    Jalminger, J., Stenstrom, P.: A cache block reuse prediction scheme. Microprocessors and Microsystems V28, 373–385 (2004)CrossRefGoogle Scholar
  6. 6.
    John, L.K., Subramanian, A.: Design and performance evaluation of a cache assist to implement selective caching. In: Proc. of Intl. Conf. on Comp. Design (1997)Google Scholar
  7. 7.
    Johnson, T.L., Connors, D.A., Merten, M.C., Hwu, W.-M.W.: Run-time cache bypassing. IEEE Transactions on Computers 48(12), 1338–1354 (1999)CrossRefGoogle Scholar
  8. 8.
    Kampe, M., Stenström, P., Dubois, M.: Self-correcting LRU replacement policies. In: CF 2004: Proc. of the 1st conf. on Computing frontiers (2004)Google Scholar
  9. 9.
    Karlsson, M., Hagersten, E.: Timestamp-based selective cache allocation. In: High Performance Memory Systems. Springer, Heidelberg (2003)Google Scholar
  10. 10.
    McFarling, S.: Cache replacement with dynamic exclusion. In: ISCA 1992, pp. 191–200. ACM Press, New York (1992)CrossRefGoogle Scholar
  11. 11.
    Memik, G., Reinman, G., Mangione-Smith, W.H.: Just say no: Benefits of early cache miss determination. In: HPCA (2003)Google Scholar
  12. 12.
    Panait, V.-M., Sasturkar, A., Wong, W.-F.: Static identification of delinquent loads. In: Int. symp. on Code generation and optimization (2004)Google Scholar
  13. 13.
    Rivers, J.A., Tam, E.S., Tyson, G.S., Davidson, E.S., Farrens, M.: Utilizing reuse information in data cache management. In: ICS (1998)Google Scholar
  14. 14.
    Sugumar, R.A., Abraham, S.G.: Efficient simulation of caches under optimal replacement with applications to miss characterization. In: Joint International Conference on Measurement and modeling of computer systems (1993)Google Scholar
  15. 15.
    Tyson, G., Farrens, M., Matthews, J., Pleszkun, A.R.: A modified approach to cache management. In: MICRO (1995)Google Scholar
  16. 16.
    Wong, W.A., Baer, J.-L.: Modified LRU policies for improving second-level cache behavior. In: HPCA (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Haakon Dybdahl
    • 1
  • Per Stenström
    • 2
  1. 1.Dept. of Computer and Information ScienceNorwegian University of Science and TechnologyTrondheimNorway
  2. 2.Dept. of Computer Engineering, Dept. of Computer EngineeringChalmers University of TechnologyGoteborgSweden

Personalised recommendations