Abstract
This paper describes a patent storage and analysis system based on Hadoop and SQLite. This system is proposed to solve data loading overhead in Map Phase of patent analysis applications. System proposed in this paper utilizes SQLite data structure as HDFS block container to enhance patent documents storage and query performance. In addition, a hierarchical index-based patent data processing approach has been implemented in this system to support patent analysis applications. This paper presents a data storing mechanism which enhances query jobs by at least 2x faster than original Hadoop. And our mechanism decreases 90 % data loading overhead in map phase.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
CNET News Internet & Media Twitter hits 400 million tweets per day, mostly. http://news.cnet.com/8301-1023_3-57448388-93/twitter-hits-400-million-tweets-per-day-mostly-mobile/
Facebook is collecting your data - 500 terabytes a day. http://gigaom.com/2012/08/22/facebook-is-collecting-your-data-500-terabytes-a-day/
Huang, L., Yuan, Y., Zhao, Z.: A study on the application of data mining in the patent information analysis for company. In: Second International Workshop on Education Technology and Computer Science (ETCS), Wuhan, vol. 1 (2010)
Karki, M.M.S.: Patent citation analysis: a policy analysis tool. World Patent Inf. 19(4), 269–272 (1997)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Buck, J.B., Watkins, N., LeFevre, J., Ioannidou, K., Maltzahn, C., Polyzotis, N., Brandt, S.: SciHadoop: array-based query processing in hadoop. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, Seattle, p. 66 (2011)
Tseng, Y.-H., Lin, C.-J., Parker, D.S.: Text mining techniques for patent analysis. Inf. Proc. Manage. 43(5), 1216–1247 (2007)
Yang, H.-C., Dasdan, A., Hsiao, R.-L., Parker, D. S.: Map-reduce-merge: simplified relational data processing on large clusters. In: ACM SIGMOD International Conference on Management of Data, Beijing, pp. 1029−1040 (2007)
Herodotou, H., Babu, S.: Profiling, what-if analysis, and cost-based optimization of MapReduce programs. In: Very Large Database Endowment, vol. 4, pp. 1111−1122 (2011)
Lin, M., Zhang, L., Wierman, A., Tan, J.: Joint optimization of overlapping phases in MapReduce. Perform. Eval. 70(10), 720–735 (2013)
Java Database Connector for SQLite. https://bitbucket.org/xerial/sqlite-jdbc
USPTO Bulk Downloads: Patent Grant Full Text. http://www.google.com/googlebooks/uspto-patents-grants-text.html
The SQLite Database File Format. https://www.sqlite.org/fileformat.html
Acknowledgement
This work (Grants No. C0141582) was supported by Business for Cooperative R&D between Industry, Academy, and Research Institute funded Korea Small and Medium Business Administration in 2014.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Rui, X., Kim, B., Min, D. (2014). An Efficient Patent Storing Mechanism Based on SQLite on Hadoop Platform. In: Chen, Y., et al. Web-Age Information Management. WAIM 2014. Lecture Notes in Computer Science(), vol 8597. Springer, Cham. https://doi.org/10.1007/978-3-319-11538-2_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-11538-2_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11537-5
Online ISBN: 978-3-319-11538-2
eBook Packages: Computer ScienceComputer Science (R0)