Advertisement

Website Scraping with Python

Using BeautifulSoup and Scrapy

  • Gábor László Hajba

Table of contents

  1. Front Matter
    Pages i-xviii
  2. Gábor László Hajba
    Pages 1-18
  3. Gábor László Hajba
    Pages 19-39
  4. Gábor László Hajba
    Pages 41-96
  5. Gábor László Hajba
    Pages 97-167
  6. Gábor László Hajba
    Pages 169-192
  7. Gábor László Hajba
    Pages 193-217
  8. Back Matter
    Pages 219-223

About this book

Introduction

Closely examine website scraping and data processing: the technique of extracting data from websites in a format suitable for further analysis. You'll review which tools to use, and compare their features and efficiency. Focusing on BeautifulSoup4 and Scrapy, this concise, focused book highlights common problems and suggests solutions that readers can implement on their own.

Website Scraping with Python starts by introducing and installing the scraping tools and explaining the features of the full application that readers will build throughout the book. You'll see how to use BeautifulSoup4 and Scrapy individually or together to achieve the desired results. Because many sites use JavaScript, you'll also employ Selenium with a browser emulator to render these sites and make them ready for scraping. 

By the end of this book, you'll have a complete scraping application to use and rewrite to suit your needs. As a bonus, the author shows you options of how to deploy your spiders into the Cloud to leverage your computer from long-running scraping tasks.


Keywords

Python Web Scraping BeautifulSoup 4 Scrapy Chrome Developer Tools Selenium Web Driver Splash Spiders Cloud ScrapingHub XML HTML CSS XPath data processing

Authors and affiliations

  • Gábor László Hajba
    • 1
  1. 1.SopronHungary

Bibliographic information