Skip to main content
SpringerLink
Log in

On-chain and Off-chain Blockchain Data Collection

  • Chapter
  • First Online:
Blockchain Intelligence

Abstract

After the introduction of blockchain intelligence in Chap. 1, in this chapter, we present an overview of blockchain data collection. We first review the data growth brought about by the rapid development of blockchain in recent years, then analyze the data processing and exploration challenges caused by this phenomenon, and finally propose our solution XBlock-ETH, well-processed up-to-date on-chain datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://xblock.pro/on-chain-data/.

  2. 2.

    https://tether.to/.

  3. 3.

    https://makerdao.com/.

  4. 4.

    https://ethernodes.org.

  5. 5.

    https://etherscan.io.

  6. 6.

    https://eips.ethereum.org/EIPS/eip-20.

  7. 7.

    https://eips.ethereum.org/EIPS/eip-721.

  8. 8.

    https://etherchain.org.

  9. 9.

    https://coinmarketcap.com/all/views/all/.

  10. 10.

    https://dapp.review.

References

  • Abraham, J., Higdon, D., Nelson, J., & Ibarra, J. (2018). Cryptocurrency price prediction using tweet volumes and sentiment analysis. SMU Data Science Review, 1(3), 1.

    Google Scholar 

  • Bartoletti, M., Carta, S., Cimoli, T., & Saia, R. (2020). Dissecting Ponzi schemes on Ethereum: Identification, analysis, and impact. Future Generation Computer Systems, 102, 259–277.

    Article  Google Scholar 

  • Buterin, V. (2013). Ethereum white paper. Ethereum.

    Google Scholar 

  • Buterin, V., & Fabian, V. (2015). ERC20 Token Standard. https://theethereum.wiki/w/index.php/ERC20_Token_Standard

    Google Scholar 

  • Cao, Q., Sirivianos, M., Yang, X., & Pregueiro, T. (2012). Aiding the detection of fake accounts in large scale social online services. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (p. 15). Berkeley, CA: USENIX Association.

    Google Scholar 

  • Chen, T., Li, X., Wang, Y., Chen, J., Li, Z., Luo, X., et al. (2017). An adaptive gas cost mechanism for Ethereum to defend against under-priced DOS attacks. In International Conference on Information Security Practice and Experience (pp. 3–24). Berlin: Springer.

    Chapter  Google Scholar 

  • Chen, T., Zhu, Y., Li, Z., Chen, J., Li, X., Luo, X., et al. (2018a). Understanding Ethereum via graph analysis. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications (pp. 1484–1492). Piscataway, NJ: IEEE.

    Chapter  Google Scholar 

  • Chen, W., Zheng, Z., Cui, J., Ngai, E., Zheng, P., & Zhou, Y. (2018b). Detecting Ponzi schemes on Ethereum: Towards healthier blockchain technology. In Proceedings of the 27th International Conference on World Wide Web, WWW. New York, NY: ACM.

    Google Scholar 

  • Chen, W., Zheng, Z., Ngai, E. C.-H., Zheng, P., & Zhou, Y. (2019). Exploiting blockchain data to detect smart Ponzi schemes on Ethereum. IEEE Access, 7, 37575–37586.

    Article  Google Scholar 

  • Chilowicz, M., Duris, E., & Roussel, G. (2009). Syntax tree fingerprinting for source code similarity detection. In 2009 IEEE 17th International Conference on Program Comprehension (pp. 243–247). Piscataway, NJ: IEEE.

    Chapter  Google Scholar 

  • Dai, H.-N., Zheng, Z., & Zhang, Y. (2019). Blockchain for internet of things: A survey. IEEE Internet of Things Journal, 6(5), 8076–8094.

    Article  Google Scholar 

  • Dinh, T. T. A., Wang, J., Chen, G., Liu, R., Ooi, B. C., & Tan, K.-L. (2017). Blockbench: A framework for analyzing private blockchains. In Proceedings of the 2017 ACM International Conference on Management of Data (pp. 1085–1100).

    Google Scholar 

  • Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The rise of social bots. Communications of the ACM, 59(7), 96–104.

    Article  Google Scholar 

  • Gencer, A. E., Basu, S., Eyal, I., Van Renesse, R., & Sirer, E. G. (2018). Decentralization in bitcoin and Ethereum networks. Preprint, arXiv:1801.03998.

    Google Scholar 

  • Gilad, Y., Hemo, R., Micali, S., Vlachos, G., & Zeldovich, N. (2017). Algorand: Scaling byzantine agreements for cryptocurrencies. In Proceedings of the 26th Symposium on Operating Systems Principles (pp. 51–68). New York, NY: ACM.

    Chapter  Google Scholar 

  • Howell, S. T., Niessner, M., & Yermack, D. (2018). Initial coin offerings: Financing growth with cryptocurrency token sales. Technical report, National Bureau of Economic Research.

    Google Scholar 

  • Huang, Y., Kong, Q., Jia, N., Chen, X., & Zheng, Z. (2019). Recommending differentiated code to support smart contract update. In Proceedings of the 27th International Conference on Program Comprehension (pp. 260–270). Piscataway, NJ: IEEE Press.

    Google Scholar 

  • Jiang, B., Liu, Y., & Chan, W. K. (2018). Contractfuzzer: Fuzzing smart contracts for vulnerability detection. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (pp. 259–269). New York, NY: ACM.

    Chapter  Google Scholar 

  • Jin, S. (2018). Ethereum gas price analysis.

    Google Scholar 

  • Kalra, S., Goel, S., Dhawan, M., & Sharma, S. (2018). Zeus: Analyzing safety of smart contracts. In Network and Distributed Systems Security (NDSS) Symposium.

    Google Scholar 

  • Kharif, O. (2017). Cryptokitties mania overwhelms Ethereum network’s processing. New York, NY: Bloomberg.

    Google Scholar 

  • Kokoris-Kogias, E., Jovanovic, P., Gasser, L., Gailly, N., Syta, E., & Ford, B. (2018). Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP) (pp. 583–598). Piscataway, NJ: IEEE.

    Chapter  Google Scholar 

  • Krupp, J., & Rossow, C. (2018). Teether: Gnawing at Ethereum to automatically exploit smart contracts. In 27th USENIX Security Symposium, Security) (pp. 1317–1333).

    Google Scholar 

  • Lamon, C., Nielsen, E., & Redondo, E. (2017). Cryptocurrency price prediction using news and social media sentiment. SMU Data Science Review, 1(3), 1–22.

    Google Scholar 

  • Layman, L., Williams, L., & St Amant, R. (2007). Toward reducing fault fix time: Understanding developer behavior for the design of automated fault detection tools. In First International Symposium on Empirical Software Engineering and Measurement, ESEM (pp. 176–185). Piscataway, NJ: IEEE.

    Google Scholar 

  • Liu, H., Liu, C., Zhao, W., Jiang, Y., & Sun, J. (2018). S-gram: Towards semantic-aware security auditing for Ethereum smart contracts. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (pp. 814–819). New York, NY: ACM.

    Chapter  Google Scholar 

  • Luo, L., Ming, J., Wu, D., Liu, P., & Zhu, S. (2014). Semantics-based obfuscation-resilient binary code similarity comparison with applications to software plagiarism detection. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (pp. 389–400). New York, NY: ACM.

    Chapter  Google Scholar 

  • Luu, L., Chu, D.-H., Olickel, H., Saxena, P., & Hobor, A. (2016). Making smart contracts smarter. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS (pp. 254–269). New York, NY: ACM.

    Google Scholar 

  • Luu, Y. V. L. (2017) . Kybernetwork: A trustless decentralized exchange and payment service. https://home.kyber.network/assets/KyberNetworkWhitepaper.pdf

    Google Scholar 

  • Majuri, Y. (2018). Simply explained: Ethereum gas.

    Google Scholar 

  • Mehar, M. I., Shier, C. L., Giambattista, A., Gong, E., Fletcher, G., Sanayhie, R., et al. (2019). Understanding a revolutionary and flawed grand experiment in blockchain: The DAO attack. Journal of Cases on Information Technology, 21(1), 19–32.

    Article  Google Scholar 

  • Meneely, A., Williams, L., Snipes, W., & Osborne, J. (2008). Predicting failures with developer networks and social network analysis. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (pp. 13–23).

    Google Scholar 

  • Mensi, W., Al-Yahyaee, K. H., & Kang, S. H. (2019). Structural breaks and double long memory of cryptocurrency prices: A comparative analysis from bitcoin and Ethereum. Finance Research Letters, 29, 222–230.

    Article  Google Scholar 

  • Merton, R. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63.

    Article  Google Scholar 

  • Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system.

    Google Scholar 

  • Somin, S., Gordon, G., & Altshuler, Y. (2018). Network analysis of ERC20 tokens trading on Ethereum blockchain. In International Conference on Complex Systems (pp. 439–450). Berlin: Springer.

    Google Scholar 

  • Su, Z., Wang, Y., Xu, Q., & Zhang, N. (2020). LVBS: Lightweight vehicular blockchain for secure data sharing in disaster rescue. IEEE Transactions on Dependable and Secure Computing, 1. https://doi.org/10.1109/TDSC.2020.2980255

  • Szabo, N. (1997). The idea of smart contracts.

    Google Scholar 

  • Torres, C. F., Steichen, M., & State, R. (2019). The art of the scam: Demystifying honeypots in Ethereum smart contracts. In Proceedings of the 28th USENIX Conference on Security Symposium, SEC’19 (pp. 1591–1607). Berkeley, CA: USENIX Association. ISBN 978-1-939133-06-9. https://dl.acm.org/citation.cfm?id=3361338.3361449

    Google Scholar 

  • van Valkenburgh, P. (2017). A token airdrop may not spare you from securities regulation.

    Google Scholar 

  • Varol, O., Ferrara, E., Davis, C. A., Menczer, F., & Flammini, A. (2017). Online human-bot interactions: Detection, estimation, and characterization. In Eleventh International AAAI Conference on Web and Social Media.

    Google Scholar 

  • Victor, F., & Lüders, B. K. (2019). Measuring Ethereum-based ERC20 token networks. In International Conference on Financial Cryptography and Data Security.

    Google Scholar 

  • Wang, C., Chu, X., & Yang, Q. (2019). Measurement and analysis of the bitcoin networks: A view from mining pools. Preprint, arXiv:1902.07549.

    Google Scholar 

  • Wang, J., & Wang, H. (2019). Monoxide: Scale out blockchains with asynchronous consensus zones. In 16th USENIX Symposium on Networked Systems Design and Implementation, NSDI) (pp. 95–112).

    Google Scholar 

  • Weyuker, E. J., Ostrand, T. J., & Bell, R. M. (2007). Using developer information as a factor for fault prediction. In Proceedings of the Third International Workshop on Predictor Models in Software Engineering (p. 8).

    Google Scholar 

  • Xu, Q., Su, Z., Dai, M., & Yu, S. (2019). APIS: Privacy-preserving incentive for sensing task allocation in cloud and edge-cooperation mobile internet of things with SDN. IEEE Internet of Things Journal,7(7), 5892–5905.

    Google Scholar 

  • Xu, Q., Su, Z., & Lu, R. (2020a). Game theory and reinforcement learning based secure edge caching in mobile social networks. IEEE Transactions on Information Forensics and Security, 15, 3415–3429.

    Article  Google Scholar 

  • Xu, Q., Su, Z., & Yang, Q. (2020b). Blockchain-based trustworthy edge caching scheme for mobile cyber-physical system. IEEE Internet of Things Journal, 7(2), 1098–1110.

    Article  Google Scholar 

  • Zamani, M., Movahedi, M., & Raykova, M. (2018). Rapidchain: Scaling blockchain via full sharding. In Proceedings of the 2018 ACMSIGSAC Conference on Computer and Communications Security (pp. 931–948). New York, NY: ACM.

    Google Scholar 

  • Zheng, P., Zheng, Z., Luo, X., Chen, X., & Liu, X. (2018a). A detailed and real-time performance monitoring framework for blockchain systems. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP (pp. 134–143). New York, NY: ACM.

    Chapter  Google Scholar 

  • Zheng, Z., Xie, S., Dai, H.-N., Chen, X., & Wang, H. (2018b). Blockchain challenges and opportunities: A survey. International Journal of Web and Grid Services, 14(4), 352–375. https://doi.org/10.1504/IJWGS.2018.095647

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sun Yat-sen University, Guangzhou, China

    Peilin Zheng

  2. National Engineering Research Center of Digital Life, Sun Yat-sen University, Guangzhou, China

    Zibin Zheng & Jiajing Wu

  3. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China

    Jiajing Wu

  4. Macau University of Science and Technology, Macau SAR, China

    Hong-Ning Dai

Authors
  1. Peilin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zibin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiajing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong-Ning Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zibin Zheng .

Editor information

Editors and Affiliations

  1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, Guangdong, China

    Zibin Zheng

  2. Faculty of Information Technology, Macau University of Science and Technology, Macao, China

    Hong-Ning Dai

  3. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, Guangdong, China

    Jiajing Wu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zheng, P., Zheng, Z., Wu, J., Dai, HN. (2021). On-chain and Off-chain Blockchain Data Collection. In: Zheng, Z., Dai, HN., Wu, J. (eds) Blockchain Intelligence. Springer, Singapore. https://doi.org/10.1007/978-981-16-0127-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-0127-9_2

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-0126-2

  • Online ISBN: 978-981-16-0127-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation