Crawled Data Analysis on Baidu API Website for Improving SaaS Platform (Short Paper)

  • Lei YuEmail author
  • Shanshan Liang
  • Shiping Chen
  • Yaoyao Wen
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 268)


SaaS (Software-as-a-Service) is a cloud computing model, which is sometimes referred to as “on-demand software”. Existing SaaS platforms are investigated before building new distributed SaaS platform. The service data mining and evaluation on existing SaaS platforms improve our new SaaS platform. For SaaS that provide various APIs, we analysis their website data in this paper by our data mining method and related software. We wrote a crawler program to obtain data from these websites. The websites include Baidu API and ProgrammableWeb API. After ETL (Extract-Transform-Load), the obtained and processed data is ready to be analyzed. Statistical methods including non-linear regression and outlier detection are used to evaluate the websites performance, and give suggestions to improve the design and development of our API website. All figures and tables in this paper are generated from IBM SPSS statistical software. The work helps us improve our own API website by comprehensively analyzing other successful API websites.


SaaS (Software-as-a-Service) Baidu API Data analysis Regression Micro service 



This work was supported by grants from Natural Science Foundation of Inner Mongolia Autonomous Region (2015BS0603) and Open Foundation of State key Laboratory of Networking and Switching Technology (Beijing University of Posts and Telecommunications, SKLNST-2016-1-01).


  1. 1.
    Couto, R.S., Sadok, H., Cruz, P., Silva, F.F.D., Sciammarella, T., Campista, M.E.M., et al.: Building an IaaS cloud with droplets: a collaborative experience with openstack. J. Netw. Comput. Appl. (2018)Google Scholar
  2. 2.
    Kaufman, S., Garber, D.: Pro Windows Server AppFabric. Apress, New York (2010)CrossRefGoogle Scholar
  3. 3.
    Malawski, M., Gajek, A., Zima, A., Balis, B., Figiela, K.: Serverless execution of scientific workflows: experiments with hyperflow, AWS lambda and Google cloud functions. Future Gener. Comput. Syst. (2017)Google Scholar
  4. 4.
    Villamizar, M., Ochoa, L., Castro, H., Salamanca, L., Verano, M., Lang, M., et al.: Cost comparison of running web applications in the cloud using monolithic, microservice, and aws lambda architectures. SOCA 11(2), 1–15 (2017)CrossRefGoogle Scholar
  5. 5.
    Abrahao, S., Insfran, E.: Models@runtime for monitoring cloud services in Google App Engine. In: IEEE 13th World Congress on Services, pp. 30–35 (2017)Google Scholar
  6. 6.
    Nishida, S., Shinkawa, Y.: A performance prediction model for Google App Engine. In: IEEE International Conference on P2p, Parallel, Grid, Cloud and Internet Computing, vol.3, pp. 134–140 (2014)Google Scholar
  7. 7.
    Basu, A., Vaidya, J., Dimitrakos, T., Kikuchi, H.: Feasibility of a privacy preserving collaborative filtering scheme on the Google App Engine: a performance case study. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp. 447–452 (2012)Google Scholar
  8. 8.
    Prodan, R., Sperk, M., Ostermann, S.: Evaluating high-performance computing on Google App Engine. IEEE Computer Society Press (2012)Google Scholar
  9. 9.
    Prodan, R., Sperk, M.: Scientific computing with Google App Engine. Future Gener. Comput. Syst. 29(7), 1851–1859 (2013)CrossRefGoogle Scholar
  10. 10.
    Park, S.H., Synn, J., Kwon, O.H., Sung, Y.: Apriori-based text mining method for the advancement of the transportation management plan in expressway work zones. J. Supercomput. 74(3), 1283–1298 (2018)CrossRefGoogle Scholar
  11. 11.
    Huang, P., et al.: Locality-regularized linear regression discriminant analysis for feature extraction. Inf. Sci. 429, 164–176 (2018)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bravi, L., Piccialli, V., Sciandrone, M.: An optimization-based method for feature ranking in nonlinear regression problems. IEEE Trans. Neural Netw. 28(4), 1005–1010 (2017)CrossRefGoogle Scholar
  13. 13.
    Yu, L., Junxing, Z., Yu, P.S.: Service recommendation based on topics and trend prediction. In: Wang, S., Zhou, A. (eds.) CollaborateCom 2016. LNICST, vol. 201, pp. 343–352. Springer, Cham (2017). Scholar
  14. 14.

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Lei Yu
    • 1
    Email author
  • Shanshan Liang
    • 1
  • Shiping Chen
    • 2
  • Yaoyao Wen
    • 1
  1. 1.Department of Computer ScienceInner Mongolia UniversityHohhotChina
  2. 2.Commonwealth Scientific and Industrial Research OrganizationCanberraAustralia

Personalised recommendations