A Load Balancing Framework for Clustered Storage Systems

  • Daniel Kunkle
  • Jiri Schindler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5374)


The load balancing framework for high-performance clustered storage systems presented in this paper provides a general method for reconfiguring a system facing dynamic workload changes. It simultaneously balances load and minimizes the cost of reconfiguration. It can be used for automatic reconfiguration or to present an administrator with a range of (near) optimal reconfiguration options, allowing a tradeoff between load distribution and reconfiguration cost. The framework supports a wide range of measures for load imbalance and reconfiguration cost, as well as several optimization techniques. The effectiveness of this framework is demonstrated by balancing the workload on a NetApp Data ONTAP GX system, a commercial scale-out clustered NFS server implementation. The evaluation scenario considers consolidating two real world systems, with hundreds of users each: a six-node clustered storage system supporting engineering workloads and a legacy system supporting three email severs.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Eisler, M., Corbett, P., Kazar, M., Nydick, D., Wagner, C.: Data ONTAP GX: A scalable storage cluster. In: Proc. of the 5th Conf. on File and Storage Technologies, pp. 139–152. USENIX Association (2007)Google Scholar
  2. 2.
    Abd-El-Malek, M., et al.: Ursa Minor: versatile cluster-based storage. In: Proc. of the 4th Conf. on File and Storage Technologies, pp. 1–15. USENIX Association (2005)Google Scholar
  3. 3.
    Nagle, D., Serenyi, D., Matthews, A.: The Panasas ActiveScale storage cluster: Delivering scalable high bandwidth storage. In: Proc. of the ACM/IEEE Conf. on Supercomputing, Washington, DC, USA, p. 53. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  4. 4.
    Hitachi: Archivas: Single fixed-content repository for multiple applications (2007), http://www.archivas.com:8080/product_info/
  5. 5.
    Weil, S., Brandt, S., Miller, E., Long, D., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proc. of the 7th Symposium on Operating Systems Design and Implementation, pp. 22–34. USENIX Association (2006)Google Scholar
  6. 6.
    Saito, Y., Frølund, S., Veitch, A., Merchant, A., Spence, S.: FAB: building distributed enterprise disk arrays from commodity components. In: Proc. of ASPLOS, pp. 48–58 (2004)Google Scholar
  7. 7.
    Litwin, W.: Linear Hashing: A new tool for file and table addressing. In: Proc. of the 6th Int’l Conf. on Very Large Data Bases, pp. 212–223. IEEE Computer Society, Los Alamitos (1980)Google Scholar
  8. 8.
    IBM Corp.: TotalStorage productivity center with advanced provisioning (2007), http://www-03.ibm.com/systems/storage/software/center/provisioning/index.html
  9. 9.
  10. 10.
    Barham, P., Donnelly, A., Isaacs, R., Mortier, R.: Using Magpie for request extraction and workload modelling. In: OSDI, pp. 259–272 (2004)Google Scholar
  11. 11.
    Thereska, E., et al.: Stardust: tracking activity in a distributed storage system. SIGMETRICS Perform. Eval. Rev. 34(1), 3–14 (2006)CrossRefGoogle Scholar
  12. 12.
    Thereska, E., et al.: Informed data distribution selection in a self-predicting storage system. In: Proc. of ICAC, Dublin, Ireland (2006)Google Scholar
  13. 13.
    Anderson, E., Spence, S., Swaminathan, R., Kallahalla, M., Wang, Q.: Quickly finding near-optimal storage designs. ACM Trans. Comput. Syst. 23(4), 337–374 (2005)CrossRefGoogle Scholar
  14. 14.
    Weisstein, E.: Bin-packing and knapsack problem. MathWorld (2007), http://mathworld.wolfram.com/
  15. 15.
    Wilkes, J.: Traveling to Rome: QoS specifications for automated storage system management. In: Proc. of the Int’l. Workshop on Quality of Service, pp. 75–91. Springer, Heidelberg (2001)Google Scholar
  16. 16.
    Anderson, E., et al.: Hippodrome: Running circles around storage administration. In: Proc. of the 1st Conf. on File and Storage Technologies, pp. 1–13. USENIX Association (2002)Google Scholar
  17. 17.
    Keeton, K., Kelly, T., Merchant, A., Santos, C., Wiener, J., Zhu, X., Beyer, D.: Don’t settle for less than the best: use optimization to make decisions. In: Proc. of the 11th Workshop on Hot Topics in Operating Systems. USENIX Association (2007)Google Scholar
  18. 18.
    Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the strength pareto evolutionary algorithm for multiobjective optimization. In: Evolutionary Methods for Design, Optimisation and Control with Application to Industrial Problems (EUROGEN 2001), International Center for Numerical Methods in Engineering (CIMNE), pp. 95–100 (2002)Google Scholar
  19. 19.
    SPEC: SPEC sfs benchmark (1993)Google Scholar
  20. 20.
    Garvey, B.: Exchange server 2007 performance characteristics using NetApp iSCSI storage systems. Technical Report TR-3565, NetApp., Inc. (2007)Google Scholar
  21. 21.
    Lu, C., Alvarez, G., Wilkes, J.: Aqueduct: Online data migration with performance guarantees. In: Proc. of the 1st Conf. on File and Storage Technologies, p. 21. USENIX Association (2002)Google Scholar
  22. 22.
    Anderson, E.: Simple table-based modeling of storage devices. Technical Report HPL-SSP-2001-4 (2001)Google Scholar
  23. 23.
    Alvarez, G., et al.: Minerva: An automated resource provisioning tool for large-scale storage systems. ACM Transactions on Computer Systems 19(4), 483–518 (2001)CrossRefGoogle Scholar
  24. 24.
    Thereska, E., Narayanan, D., Ganger, G.: Towards self-predicting systems: What if you could ask what-if? In: Proc. of the 3rd Int’l. Workshop on Self-adaptive and Autonomic Computing Systems, Denmark (2005)Google Scholar
  25. 25.
    Mesnier, M., Wachs, M., Sambasivan, R., Zheng, A., Ganger, G.: Modeling the relative fitness of storage. In: Proc. of the Int’l. Conf. on Measurement and Modeling of Computer Systems. ACM Press, New York (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Daniel Kunkle
    • 1
  • Jiri Schindler
    • 1
  1. 1.Northeastern University and NetApp Inc.USA

Personalised recommendations