Decision Tree Regression with Pandas, Scikit-Learn, and PySpark

Abstract

In this chapter, we continue with our exploration of supervised learning with a focus on regression tasks. Specifically, we will be building a regression model using the decision tree algorithm—an alternative to the multiple linear regression model we used in the previous chapter. We will use both Scikit-Learn and PySpark to train and evaluate the model and then use it to predict the sale price of houses based on several features such as the size of property and the number of bedrooms, bathrooms, and stories, among others. Additionally, we will compare the performance of Pandas and PySpark in data loading and exploration tasks to better understand their similarities and differences.