A Portal to Support Rigorous Experimental Methodology in Networking Research

  • Guillaume Jourjon
  • Thierry Rakotoarivelo
  • Max Ott
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 90)


Whilst dealing with topics that are more and more influenced by physical properties of the underlying media, the networking community still lacks a culture of rigorous result verification. Indeed, as opposed to most of the science and engineering fields there are very few benchmarks to test protocols against. Furthermore, in most publications the authors do not give the community access to the raw results or details of the performed experimental procedures. Therefore it is impossible to accurately reproduce their experiments. We propose to solve this problem by extending the state of the art experiment tool OMF with a public portal. This portal, while providing the experimenter with access to experimental resources, also provides the community with a system for comprehensive experiment description and result verification. The collection of both the measurement set and the experiment’s description is done in a transparent manner for the experimenter, who can decide to publish them via the portal once the research is mature enough.


scientific method data quality measurement testbeds OMF 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    git - The Fast Control System,
  2. 2.
    Networking Conferences Statistics,
  3. 3.
    NS3 Network Simulator,
  4. 4.
    OMNet++ Simulator,
  5. 5.
    PlanetLab Testbed,
  6. 6.
    The NITLab Scheduler em,
  7. 7.
    The OMF Testbed Control, Measurement and Management Framework,
  8. 8.
    The Orbit Measurment Library,
  9. 9.
    The R Project for Statistical Computing,
  10. 10.
    Albrecht, J., Tuttle, C., Snoeren, A.C., Vahdat, A.: PlanetLab Application Management Using Plush. ACM Operating Systems Review (2006)Google Scholar
  11. 11.
    Jourjon, G., Rakotoarivelo, T., Ott, M.: From Learning to Researching, Ease the Shift through Testbeds. In: Magedanz, T., Gavras, A., Thanh, N.H., Chase, J.S. (eds.) TridentCom 2010. LNICST, vol. 46, pp. 496–505. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Kohler, E., Handley, M., Floyd, S.: Designing DCCP: Congestion control without reliability. SIGCOMM Computer Communication Review 36(4), 27–38 (2006)CrossRefGoogle Scholar
  13. 13.
    Lacage, M., Ferrari, M., Hansen, M., Turletti, T.: NEPI: Using Independent Simulators, Emulators, and Testbeds for Easy Experimentation. In: Workshop on Real Overlays and Distributed Systems (ROADS) (2009)Google Scholar
  14. 14.
    Lewis, R.W.: Biology: A hypothetico-deductive science. The American Biology Teacher 50, 362–366 (1988)CrossRefGoogle Scholar
  15. 15.
    Mehani, O., Boreli, R., Ernst, T.: Analysis of TFRC in disconnected scenarios and performance improvements with Freeze-DCCP. In: Ott, J., Tan, K. (eds.) 4th International Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2009. ACM, New York (2009)Google Scholar
  16. 16.
    Popper, K.R.: The Logic of Scientific Discovery. Hutchinson London (1968)Google Scholar
  17. 17.
    Rakotoarivelo, T., Ott, M., Seskar, I., Jourjon, G.: OMF: a control and management framework for networking testbeds. In: SOSP Workshop on Real Overlays and Distributed Systems (ROADS 2009), Big Sky, USA, p. 6 (October 2009)Google Scholar
  18. 18.
    Raychaudhuri, D., et al.: Overview of the ORBIT Radio Grid Testbed for Evaluation of Next-Generation Wireless Network Protocols. In: Proc. IEEE Wireless Communications and Networking Conference, WCNC (2005)Google Scholar
  19. 19.
    Underwood, A.: Experiments in ecology and management: their logics, functions and interpretations. Australian Journal of Ecology 15, 365–389 (1990)CrossRefGoogle Scholar
  20. 20.
    White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., Joglekar, A.: An integrated experimental environment for distributed systems and networks. SIGOPS Oper. Syst. Rev. 36(SI), 255–270 (2002)CrossRefGoogle Scholar
  21. 21.
    White, J., Jourjon, G., Rakatoarivelo, T., Ott, M.: Measurement Architectures for Network Experiments with Disconnected Mobile Nodes. In: Magedanz, T., Gavras, A., Thanh, N.H., Chase, J.S. (eds.) TridentCom 2010. LNICST, vol. 46, pp. 315–330. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2012

Authors and Affiliations

  • Guillaume Jourjon
    • 1
  • Thierry Rakotoarivelo
    • 1
  • Max Ott
    • 1
  1. 1.NICTAEveleighAustralia

Personalised recommendations