LeakBot: An Automated and Lightweight Tool for Diagnosing Memory Leaks in Large Java Applications

Abstract

Despite Java’s automatic reclamation of memory, memory leaks remain an important problem. For example, we frequently encounter memory leaks that cause production servers to crash. These servers represent an increasingly common class of Java applications: they are large scale and they make heavy use of frameworks. For these applications, existing tools require too much expertise, and, even for experts, require many hours interpreting low-level details. In addition, they are often too expensive to use in practice. We present an automated, adaptive, and scalable tool for diagnosing memory leaks, called LeakBot.

LeakBot incorporates three new techniques. First, it automatically ranks data structures by their likelihood of containing leaks. This process dramatically prunes the set of candidate structures, using object reference graph properties and knowledge of how leaks occur. Second, it uses Coevolving Regions to identify suspicious regions within a data structure and characterize their expected evolution. Third, it uses the first two methods to derive a lightweight way to track those regions’ actual evolution as the program runs. These techniques are mutually beneficial: we need only monitor what is highly ranked, and, because the tracking is so cheap, a region’s rank can be continually updated with information from production machines. Finally, this whole process can be done without user assistance.

We demonstrate LeakBot’s effectiveness on a number of large-scale applications that we have analyzed as part of the ongoing consulting practice our group maintains. We have found that the ranking analysis scales (e.g. written in Java, it analyzes 10⁶ objects in 30 seconds with a 300M heap), is selective (e.g. it prunes that set to three candidate leak roots), and is accurate (it discounts non-leaking roots). The CER generation completes in tens of seconds. The lightweight tracking refines the rankings, while lowering throughput by less than 5%