Precision, Recall, and Sensitivity of Monitoring Partially Synchronous Distributed Systems
Runtime verification focuses on analyzing the execution of a given program by a monitor to determine if it is likely to violate its specifications. There is often an impedance mismatch between the assumptions/model of the monitor and that of the underlying program. This constitutes problems especially for distributed systems, where the concept of current time and state are inherently uncertain. A monitor designed with asynchronous system model assumptions may cause false-positives for a program executing in a partially synchronous system: the monitor may flag a global predicate that does not actually occur in the underlying system. A monitor designed with a partially synchronous system model assumption may cause false negatives as well as false positives for a program executing in an environment where the bounds on partial synchrony differ (albeit temporarily) from the monitor model assumptions.
In this paper we analyze the effects of the impedance mismatch between the monitor and the underlying program for the detection of conjunctive predicates. We find that there is a small interval where the monitor assumptions are hypersensitive to the underlying program environment. We provide analytical derivations for this interval, and also provide simulation support for exploring the sensitivity of predicate detection to the impedance mismatch between the monitor and the program under a partially synchronous system.
KeywordsFalse Positive Rate Linear Temporal Logic Clock Synchronization Impedance Mismatch Asynchronous System
This work is supported in part by NSF CNS-1329807, NSF CNS-1318678, NSF XPS-1533870, and NSF XPS-1533802.
- 1.Fan, R., Lynch, N.: Gradient clock synchronization. In: PODC, pp. 320–327 (2004)Google Scholar
- 2.Patt-Shamir, B., Rajsbaum, S.: A theory of clock synchronization (extended abstract). In: ACM Symposium on Theory of Computing (STOC), pp. 810–819 (1994)Google Scholar
- 6.Yingchareonthawornchai, S., Nguyen, D., Valapil, V.T., Kulkarni, S.S., Demirbas, M.: Precision, recall, sensitivity of monitoring partially synchronous distributed systems. CoRR, abs/1607.03369 (2016). http://arxiv.org/abs/1607.03369
- 7.Garg, V.K., Chase, C.: Distributed algorithms for detecting conjunctive predicates. In: International Conference on Distributed Computing Systems, pp. 423–430, June 1995Google Scholar
- 8.Lu, H., Veeraraghavan, K., Ajoux, P., Hunt, J., Song, Y.-J., Tobagus, W., Kumar, S., Lloyd, W.: Existential consistency: measuring and understanding consistency at Facebook. In: Proceedings of the 25th Symposium on Operating Systems Principles, pp. 295–310. ACM (2015)Google Scholar
- 9.Sigelman, B., Barroso, L., Burrows, M., Stephenson, P., Plakal, M., Beaver, D., Jaspan, S., Shanbhag, C.: Dapper, a large-scale distributed systems tracing infrastructure. Google Inc., Technical report (2010). http://research.google.com/archive/papers/dapper-2010-1.pdf
- 10.Chow, M., Meisner, D., Flinn, J., Peek, D., Wenisch, T.: The mystery machine: end-to-end performance analysis of large-scale internet services. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 217–231 (2014)Google Scholar
- 12.Verissimo, P.: Real-time communication. In: Distributed Systems, vol. 2 (1993)Google Scholar
- 15.Demirbas, M., Kulkarni, S.: Beyond truetime: using augmentedtime for improving google spanner. In: 7th Workshop on Large-Scale Distributed Systems and Middleware, LADIS (2013)Google Scholar
- 17.Yingchareonthawornchai, S., Kulkarni, S., Demirbas, M.: Analysis of bounds on hybrid vector clocks. In: 19th International Conference on Principles of Distributed Systems, OPODIS (2015)Google Scholar
- 20.Mostafa, M., Bonakdarpour, B.: Decentralized runtime verification of LTL specifications in distributed systems. In: 2015 IEEE International Parallel, Distributed Processing Symposium, IPDPS 2015, 25–29 May 2015 Hyderabad, India, pp. 494–503 (2015). http://dx.doi.org/10.1109/IPDPS.2015.95