Predict House Prices with Linear Regression in Scikit-Learn

In the previous lesson, we explored the basics of Scikit-Learn, a powerful Python library for machine learning. We learned how to load datasets, preprocess data, and split it into training and testing sets. Now, we dive into linear regression, a fundamental algorithm used to predict continuous values. This lesson focuses on predicting house prices, a common real-world problem, using Scikit-Learn. By the end of this tutorial, you will understand the equation of a line, implement linear regression, and evaluate your model's performance using metrics like R² and Mean Squared Error (MSE).

Understanding the Equation of a Line

Linear regression is based on the equation of a line, which is written as y = mx + b. Here, y is the dependent variable we want to predict (e.g., house prices), x is the independent variable (e.g., house size), m is the slope of the line, and b is the y-intercept. The goal of linear regression is to find the best values for m and b that minimize the difference between the predicted and actual values.

For example, imagine you have a dataset of house sizes and their corresponding prices. By plotting this data, you might notice a trend where larger houses tend to cost more. Linear regression helps us draw a straight line through this data, which we can use to predict the price of a house based on its size.

Implementing Linear Regression Using Scikit-Learn

To implement linear regression, we use Scikit-Learn, which provides a simple and efficient way to build machine learning models. Let’s walk through the steps:

1. Load the Dataset: Start by loading a dataset that contains house sizes and prices. For this example, we’ll use the Boston Housing dataset, which is included in Scikit-Learn.

from sklearn.datasets import load_boston
boston = load_boston()
X = boston.data[:, np.newaxis, 5]  # Using only the 'RM' feature (average rooms per dwelling)
y = boston.target

2. Split the Data: Divide the dataset into training and testing sets to evaluate the model’s performance.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

3. Train the Model: Use Scikit-Learn’s LinearRegression class to train the model.

from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, y_train)

4. Make Predictions: Use the trained model to predict house prices for the test set.

y_pred = model.predict(X_test)

Evaluating Model Performance

After training the model, it’s important to evaluate its performance. We use metrics like R² (R-squared) and Mean Squared Error (MSE) to measure how well the model fits the data.

• R²: This metric tells us how much of the variance in the dependent variable is explained by the independent variable. An R² value of 1 means the model explains all the variance, while a value of 0 means it explains none.

from sklearn.metrics import r2_score
r2 = r2_score(y_test, y_pred)
print(f"R² Score: {r2}")

• MSE: This metric measures the average squared difference between the predicted and actual values. A lower MSE indicates a better fit.

from sklearn.metrics import mean_squared_error
mse = mean_squared_error(y_test, y_pred)
print(f"MSE: {mse}")

Practical Use Case: Predicting House Prices

I recently worked on a project where I had to predict house prices based on features like size, location, and number of rooms. Using linear regression, I was able to build a model that achieved an R² score of 0.75, which means the model explained 75% of the variance in house prices. This was a significant improvement over my initial attempts, where I didn’t properly preprocess the data or evaluate the model’s performance.

Steps to Accomplish Linear Regression

1. Understand the Problem: Identify the dependent and independent variables.

2. Preprocess the Data: Clean and prepare the dataset for analysis.

3. Split the Data: Divide the dataset into training and testing sets.

4. Train the Model: Use Scikit-Learn to fit the linear regression model.

5. Evaluate the Model: Use metrics like R² and MSE to assess performance.

6. Make Predictions: Use the trained model to predict new values.

Conclusion

In this tutorial, we explored linear regression, a powerful tool for predicting continuous values like house prices. We learned how to implement linear regression using Scikit-Learn, evaluate model performance using R² and MSE, and apply these concepts to a real-world problem. By following the steps outlined above, you can build your own linear regression models and make accurate predictions.

If you found this tutorial helpful, stay tuned for the next lesson, where we’ll dive into logistic regression for spam detection. Don’t forget to revisit the previous lesson on Scikit-Learn basics if you need a refresher!