Chapter

Advances in Web Intelligence and Data Mining

Volume 23 of the series Studies in Computational Intelligence pp 1-10

DataRover: An Automated System for Extracting Product Information From Online Catalogs

  • Syed Toufeeq AhmedAffiliated withDepartment of Computer Science and Engineering, Arizona State University
  • , Srinivas VadrevuAffiliated withDepartment of Computer Science and Engineering, Arizona State University
  • , Hasan DavulcuAffiliated withDepartment of Computer Science and Engineering, Arizona State University

* Final gross prices may vary according to local VAT.

Get Access

Abstract

The increasing number of e-commerce Web sites on the Web introduces numerous challenges in organizing and searching the product information across multiple Web sites. This problem is further exacerbated by various presentation templates that different Web sites use in presenting their product information, and different ways of product information they store in their catalogs. This paper describes the DataRover system, which can automatically crawl and extract all products from online catalogs. DataRover is based on pattern mining algorithms and domain specific heuristics which utilize the navigational and presentation regularities to identify taxonomy, list-of-product and single-product segments within an online catalog. Next, it uses the inferred patterns to extract data from all such data segments and to automatically transform an online catalog into a database of categorized products. We also provide experimental results to demonstrate the efficacy of the DataRover.