Abstract
Knowledge graphs, powerful assets for enhancing search and various data integration, are being essential in both academia and industry. In this paper we will demonstrate that knowledge graph abilities are much wider than search and data integration. We will do it in a twofold manner: 1) we will show how to build knowledge graph in Spark instead of using SPARQL language and how to explore data in DataFrames and GraphFrames; and 2) we will reveal Spark knowledge graph as a bridge between logical thinking and graph thinking for data mining.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bradley, A.: Semantics 2017. https://2017.semantics.cc/aaron-bradley-eamonn-glass
The Limitations of SPARQL. http://horicky.blogspot.com/2010/08/limitations-of-sparql.html
Apache Spark. https://databricks.com/spark/about
Chambers, B., Zaharia, M.: Spark: The Definitive Guide: Big Data Processing Made Simple
Databricks Community Edition. https://databricks.com/blog/2016/02/17/introducing-databricks-community-edition-apache-spark-for-all.html
Spark DataFrames. https://databricks.com/blog/2015/02/17/introducing-dataframes-in-spark-for-large-scale-data-science.html
Spark GraphFrames. https://databricks.com/blog/2016/03/03/introducing-graphframes.html
Industry-scale Knowledge Graphs: Lessons and Challenges. https://queue.acm.org/detail.cfm?id=3332266
Kaggle dataset ‘Museum of Modern Art Collection’. https://www.kaggle.com/momanyc/museum-collection
MoMA exhibition: Inventing Abstraction 1910–1925. https://www.moma.org/interactives/exhibitions/2012/inventingabstraction/?page=artists
Timeline of the ‘Modern Art Movements’. https://drawpaintacademy.com/modern-art-movements/
“Knowledge Graph for Data Mining” post. http://sparklingdataocean.com/2019/09/24/knowledgeGraphDataAnalysis/
Drawing graphs with dot. https://www.ocf.berkeley.edu/~eek/index.html/tiny_examples/thinktank/src/gv1.7c/doc/dotguide.pdf
Visual network analysis with Gephi. https://medium.com/@EthnographicMachines/visual-network-analysis-with-gephi-d6241127a336
Motifs Findings in GraphFrames. https://www.waitingforcode.com/apache-spark-graphframes/motifs-finding-graphframes/read
“Knowledge Graph for Data Integration” post. http://sparklingdataocean.com/2020/02/02/knowledgeGraphIntegration/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Romanova, A. (2020). Building Knowledge Graph in Spark Without SPARQL. In: Kotsis, G., et al. Database and Expert Systems Applications. DEXA 2020. Communications in Computer and Information Science, vol 1285. Springer, Cham. https://doi.org/10.1007/978-3-030-59028-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-59028-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59027-7
Online ISBN: 978-3-030-59028-4
eBook Packages: Computer ScienceComputer Science (R0)