Journal of Grid Computing

, Volume 1, Issue 1, pp 75–93

Design, Implementation, and Evaluation of the Remos Network Monitoring System

  • Bruce Lowekamp
  • Nancy Miller
  • Roger Karrer
  • Thomas Gross
  • Peter Steenkiste
Article
  • 84 Downloads

Abstract

Remos provides resource information to distributed applications. Its design goals of scalability, flexibility, and portability are achieved through an architecture that allows components to be positioned across the network, each collecting information about its local network. To collect information from different types of networks, Remos provides several Collectors that use different technologies, including SNMP and benchmarking. By matching the Collector to the particular network environment and by providing an architecture for distributing the output of these collectors across all querying environments, Remos collects appropriately detailed information at each site and distributes this information where needed in a scalable manner. Remos has been implemented and tested in a variety of networks and is in use in a number of different environments.

network monitoring 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    S. Basu, A. Mukherjee and S. Klivansky, “Time Series Models for Internet Traffic,” Technical Report GIT-CC-95-27. College of Computing, Georgia Institute of Technology, 1995.Google Scholar
  2. 2.
    J. Bolliger and T. Gross, “Bandwidth Monitoring for Network-Aware Applications,” in Proc. 10 th IEEE Symp. High-Performance Distr. Comp., IEEE CS Press, San Francisco, CA, 2001, pp. 241–251.Google Scholar
  3. 3.
    Y. Chu, S. Rao, S. Seshan and H. Zhang, “Enabling Conferencing Applications on the Internet Using an Overlay Multicast Architecture,” in Proceedings of ACM SIGCOMM' 01, San Diego, CA, 2001, pp. 55–67.Google Scholar
  4. 4.
    Y. Chu, S. Rao and H. Zhang, “A Case for End System Multicast,” in ACM Sigmetrics 2000, Santa Clara, CA, 2000, pp. 1–12.Google Scholar
  5. 5.
    Cray Research Inc., “Nettest Networking Benchmark.” ftp://ftp.sgi.com/sgi/src/nettest.Google Scholar
  6. 6.
    K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman, “Grid Information Services for Distributed Resource Sharing,” in Proceedings of the 10 th IEEE International Symposium on High Performance Distributed Computing (HPDC 10), 2001.Google Scholar
  7. 7.
    M. den Burger, T. Kielmann and H.E. Bal, “TOPOMON: A Monitoring Tool for Grid Network Topology,” in International Conference on Computational Science (2), 2002, pp. 558–567.Google Scholar
  8. 8.
    T. De Witt, T. Gross, B. Lowekamp, N. Miller, P. Steenkiste, J. Subhlok and D. Sutherland, “ReMoS: A Resource Monitoring System for Network Aware Applications,” Technical Report CMU-CS-97-194, School of Computer Science, Carnegie Mellon University, 1997.Google Scholar
  9. 9.
    P. Dinda and B. Plale, “A Unified Relational Approach to Grid Information Services,” GWD-GIS-012-1, 2001. http://www.cs.northwestern.edu/~pdinda/relational-gis/.Google Scholar
  10. 10.
    P.A. Dinda and D.R. O'Hallaron, “An Extensible Toolkit for Resource Prediction in Distributed Systems,” Technical Report CMU-CS-99-138, School of Computer Science, Carnegie Mellon University, 1999.Google Scholar
  11. 11.
    P.A. Dinda and D.R. O'Hallaron, “Host Load Prediction Using Linear Models,” Cluster Computing, Vol. 3, No. 4, 2000. An earlier version appeared in HPDC' 99.Google Scholar
  12. 12.
    R. Govindan and H. Tangmunarunkit, “Heuristics for Internet Map Discovery,” in IEEE INFOCOM 2000, Tel Aviv, Israel, 2000.Google Scholar
  13. 13.
    N.C. Groschwitz and G.C. Polyzos, “A Time Series Model of Long-Term NSFNET Backbone Traffic,” in Proceedings of the IEEE International Conference on Communications (ICC'94), Vol. 3, 1994, pp. 1400–1404.Google Scholar
  14. 14.
    T. Gross and P. Steenkiste, “A Perspective on Network/Application Coupling,” in Proc. 8 th NOSSDAV Workshop (Network and Operating System Services for Digital Audio and Video). www.nossdav.org/1998/-tech. Short paper.Google Scholar
  15. 15.
    M. Hemy, P. Steenkiste and T. Gross, “Evaluation of Adaptive Filtering of MPEG System Streams in IP Networks,” in Proceedings of the IEEE International Conference on Multimedia and Expo 2000 (IDME 2000), New York, NY, 2000, pp. 1313–1317.Google Scholar
  16. 16.
    N. Hu and P. Steenkiste, “Estimating Available Bandwidth Using Packet Pair Probing,” Technical Report CMU-CS-02-166, Carnegie Mellon University, School of Computer Science, 2002.Google Scholar
  17. 17.
    J.M. Jaffe, “Bottleneck Flow Control,” IEEE Transactions on Communications, Vol. 29, No. 7, pp. 954–962, 1981.Google Scholar
  18. 18.
    S. Jamin, C. Jin, Y. Jin, D. Raz, Y. Shavitt and L. Zhang, “On the Placement of Internet Instrumentation,” in IEEE INFOCOM 2000, Tel Aviv, Israel, 2000.Google Scholar
  19. 19.
    J. Jannotti, D. Gifford, K. Johnson, M. Kaashoek and J. O'Toole, “Overcast: Reliable Multicasting with an Overlay Network,” in Proceedings of the 4 th Symposium on Operating System Design and Implementation (OSDI 2000), San Diego, CA, 2000, pp. 197–212.Google Scholar
  20. 20.
    R. Karrer and T. Gross, “Dynamic Handoff of Multimedia Streams,” in Proc. of NOSSDAV' 01, Port Jefferson, NY, 2001.Google Scholar
  21. 21.
    B. Lowekamp, N. Miller, D. Sutherland, T. Gross, P. Steenkiste and J. Subhlok, “A Resource Query Interface for Network-Aware Applications,” Cluster Computing, Vol. 2, No. 2, pp. 139–151, 1999.Google Scholar
  22. 22.
    B. Lowekamp, D. O'Hallaron and T. Gross, “Direct Network Queries for Discovering Network Resource Properties in a Distributed Environment,” in Proc. 8 th IEEE Symp. on High-Performance Distributed Computing (HPDC-8), Redondo Beach, CA, 1999, pp. 38–46.Google Scholar
  23. 23.
    B. Lowekamp, D.R. O'Hallaron and T. Gross, “Topology Discovery for Large Ethernet Networks,” in Proceedings of SIGCOMM 2001, 2001.Google Scholar
  24. 24.
    N. Miller and P. Steenkiste, “Collecting Network Status Information for Network-Aware Applications,” in IEEE INFOCOM 2000, Tel Aviv, Israel, 2000.Google Scholar
  25. 25.
    K. Obraczka and G. Gheorghiu, “The Performance of a Service for Network-Aware Applications,” in Proceedings of the ACM SIGMETRICS SPDT'98, 1997. Also available as USC CS Technical Report 97-660.Google Scholar
  26. 26.
    R. Ribler, J. Vetter, H. Simitci and D. Reed, “Autopilot: Adaptive Control of Distributed Applications,” in Proceedings of the 7 th IEEE International Symposium on High Performance Distributed Computing (HPDC 7), 1998, pp. 172–179.Google Scholar
  27. 27.
    M. Samadani and E. Kalthofen, “On Distributed Scheduling Using Load Prediction from Past Information,” Abstracts published in Proceedings of the 14 th Annual ACM Symposium on the Principles of Distributed Computing (PODC'95), 1996, p. 261 and in The 3rd Workshop on Languages, Compilers and Run-Time Systems for Scalable Computers (LCR'95), 1996, pp. 317–320.Google Scholar
  28. 28.
    S. Savage, T. Anderson, A. Aggarwal, D. Becker, N. Cardwell, A. Collins, E. Hoffman, J. Snell, A. Vahdat, G. Voelker and J. Zahorjan, “Detour: Informed Internet Routing and Transport,” IEEE Micro, Vol. 19, No. 1, pp. 50–59, 1999.Google Scholar
  29. 29.
    S. Savage, A. Collins, E. Hoffman, J. Snell and T. Anderson, “The End-to-End Effects of Internet Path Selection,” in Proceedings of ACM SIGCOMM'99, Boston, Massachusetts, 1999, pp. 289–299.Google Scholar
  30. 30.
    W. Stallings, SNMP, SNMPv2 and RMON, 2nd edn, Addison-Wesley, 1996.Google Scholar
  31. 31.
    M. Stemm, S. Seshan and R. Katz, “SPAND: Shared Passive Network Performance Discovery,” in USENIX Symposium on Internet Technologies and Systems, Monterey, CA, 1997, pp. 135–146.Google Scholar
  32. 32.
    W. Theilmann and K. Rothermel, “Dynamic Distance Maps of the Internet,” in IEEE INFOCOM 2000, Tel Aviv, Israel, 2000.Google Scholar
  33. 33.
    B. Tierney, R. Aydt, D. Gunter, W. Smith, M. Swany, V. Taylor and R. Wolski, “A Grid Monitoring Architecture,” Technical Report GFD-I.7, Global Grid Forum, 2002.Google Scholar
  34. 34.
    B. Tierney, B. Crowley, D. Gunter, M. Holding, J. Lee and M. Thompson, “A Monitoring Sensor Management System for Grid Environments,” in Proceedings of the 9 th IEEE International Symposium on High Performance Distributed Computing (HPDC 9), 2000, pp. 97–104.Google Scholar
  35. 35.
    B. Tierney, W. Johnston, B. Crowley, G. Hoo, C. Brooks and D. Gunter, “The NetLogger Methodology for High Performance Distributed Systems Performance Analysis,” in Proceedings of the 7 th IEEE International Symposium on High Performance Distributed Computing (HPDC 7), 1998, pp. 260–267.Google Scholar
  36. 36.
    R.Wolski, “Forecasting Network Performance to Support Dynamic Scheduling Using the Network Weather Service,” in Proceedings of the 6 th High-Performance Distributed Computing Conference (HPDC97), 1997, pp. 316–325. Extended version available as UCSD Technical Report TR-CS96-494.Google Scholar
  37. 37.
    R. Wolski, “Dynamically Forecasting Network Performance Using the Network Weather Service,” Cluster Computing, Vol. 1, No. 1, pp. 119–132, 1998.Google Scholar
  38. 38.
    R. Wolski, N. Spring and J. Hayes, “The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing,” J. Future Generation Computing Systems, Vol. 15, Nos. 5–6, pp. 757–768, 1998. Published also as UCSD Technical Report CS98-599.Google Scholar
  39. 39.
    R. Wolski, N. Spring and J. Hayes, “Predicting the CPU Availability of Time-Shared Unix Systems,” in Proceedings of the 8 th IEEE Symposium on High Performance Distributed Computing HPDC99, IEEE, pp. 105–112, 1999. Earlier version available as UCSD Technical Report CS98-602.Google Scholar
  40. 40.
    R. Wolski, N. Spring and C. Peterson, “Implementing a Performance Forecasting System for Metacomputing: The Network Weather Service,” in Supercomputing' 97, 1997.Google Scholar
  41. 41.
    B. Zhao, J. Kubiatowicz and A. Joseph, “Tapestry: An Infrastructure for Fault-Tolerant Wide-Area Location and Routing,” Technical Report UCB/CSD-01-1141, University of Berkeley, CA, 2001.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Bruce Lowekamp
    • 1
  • Nancy Miller
    • 2
  • Roger Karrer
    • 3
  • Thomas Gross
    • 4
    • 3
  • Peter Steenkiste
    • 2
  1. 1.Department of Computer ScienceCollege of William and MaryWilliamsburgUSA
  2. 2.School of Computer ScienceCarnegie Mellon UniversityPittsburghUSA
  3. 3.Computer Science DepartmentETH ZürichSwitzerland
  4. 4.School of Computer ScienceCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations