Abstract
In this paper we study the MapReduce Class (MRC) defined by Karloff et al., which is a formal complexity-theoretic model of MapReduce. We show that constant-round MRC computations can decide regular languages and simulate sublogarithmic space-bounded Turing machines. In addition, we prove hierarchy theorems for MRC under certain complexity-theoretic assumptions. These theorems show that sufficiently increasing the number of rounds or the amount of time per processor strictly increases the computational power of MRC. Our work lays the foundation for further analysis relating MapReduce to established complexity classes. Our results also hold for Valiant’s BSP model of parallel computation and the MPC model of Beame et al.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Andoni, A., Nikolov, A., Onak, K., Yaroslavtsev, G.: Parallel algorithms for geometric graph problems. In: STOC, pp. 574–583 (2014)
Beame, P., Koutris, P., Suciu, D.: Communication steps for parallel query processing. In: PODS, pp. 273–284 (2013)
Chu, C.-T., Kim, S.K., Lin, Y.-A., Yu, Y., Bradski, G.R., Ng, A.Y., Olukotun, K.: Map-reduce for machine learning on multicore. In: NIPS, pp. 281–288 (2006)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Farahat, A.K., Elgohary, A., Ghodsi, A., Kamel, M.S.: Distributed column subset selection on mapreduce. In: ICDM, pp. 171–180 (2013)
Feldman, J., Muthukrishnan, S., Sidiropoulos, A., Stein, C., Svitkina, Z.: On distributing symmetric streaming computations. ACM Transactions on Algorithms, 6(4) (2010)
Fortnow, L.: Time-space tradeoffs for satisfiability. J. Comput. Syst. Sci. 60(2), 337–353 (2000)
Goodrich, M.T., Sitchinava, N., Zhang, Q.: Sorting, searching, and simulation in the mapreduce framework. In: Asano, T., Nakano, S., Okamoto, Y., Watanabe, O. (eds.) ISAAC 2011. LNCS, vol. 7074, pp. 374–383. Springer, Heidelberg (2011)
Impagliazzo, R., Paturi, R.: The complexity of k-sat. In: 2012 IEEE 27th Conference on Computational Complexity, p. 237 (1999)
Impagliazzo, R., Paturi, R., Zane, F.: Which problems have strongly exponential complexity? J. Comput. Syst. Sci. 63(4), 512–530 (2001)
Kamara, S., Raykova, M.: Parallel homomorphic encryption. In: Financial Cryptography Workshops, pp. 213–225 (2013)
Karloff, H., Suri, S., Vassilvitskii, S.: A model of computation for mapreduce. In: SODA 2010, pp. 938–948. Society for Industrial and Applied Mathematics, Philadelphia (2010)
Kumar, R., Moseley, B., Vassilvitskii, S., Vattani, A.: Fast greedy algorithms in mapreduce and streaming. In: SPAA 2013, pp. 1–10. ACM, New York (2013)
Lokshtanov, D., Marx, D., Saurabh, S.: Lower bounds based on the exponential time hypothesis. Bulletin of the EATCS 105, 41–72 (2011)
Pace, M.F.: BSP vs mapreduce. In: Proceedings of the International Conference on Computational Science, ICCS 2012, Omaha, Nebraska, USA, June 4–6, 2012, pp. 246–255 (2012)
Sarma, A.D., Afrati, F.N., Salihoglu, S., Ullman, J.D.: Upper and lower bounds on the cost of a map-reduce computation. In: PVLDB 2013, pp. 277–288. VLDB Endowment (2013)
Shepherdson, J.C.: The reduction of two-way automata to one-way automata. IBM J. Res. Dev. 3(2), 198–200 (1959)
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Khatib, M.G., He, X., Factor, M. (eds.) MSST, pp. 1–10. IEEE Computer Society (2010)
Szepietowski, A.: Turing machines with sublogarithmic space. Ernst Schering Research Foundation Workshops. Springer (1994)
Valiant, L.G.: A bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990)
Wagner, K., Wechsung, G.: Computational Complexity. Mathematics and itsApplications. Springer (1986)
Williams, R.: Time-space tradeoffs for counting NP solutions modulo integers. Computational Complexity 17(2), 179–219 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fish, B., Kun, J., Lelkes, Á.D., Reyzin, L., Turán, G. (2015). On the Computational Complexity of MapReduce. In: Moses, Y. (eds) Distributed Computing. DISC 2015. Lecture Notes in Computer Science(), vol 9363. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48653-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-662-48653-5_1
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-48652-8
Online ISBN: 978-3-662-48653-5
eBook Packages: Computer ScienceComputer Science (R0)