Abstract
This chapter introduces a concurrent web crawler written in OCaml, which traverses a web server and finds all the local href links. It then outputs information about which pages link together. A web crawler is different from a web browser in that the web crawler is automated. Both are web (or HTTP) clients and are quite similar, but this automation versus interactivity is the important distinction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Rights and permissions
Copyright information
© 2006 Joshua B. Smith
About this chapter
Cite this chapter
(2006). Practical: A Concurrent Web Crawler. In: Practical OCaml. Apress. https://doi.org/10.1007/978-1-4302-0244-8_24
Download citation
DOI: https://doi.org/10.1007/978-1-4302-0244-8_24
Publisher Name: Apress
Print ISBN: 978-1-59059-620-3
Online ISBN: 978-1-4302-0244-8
eBook Packages: Professional and Applied ComputingProfessional and Applied Computing (R0)Apress Access Books