Supervised Learning – Linear Regression

Linear Regression

Linear regression predicts a continuous target by fitting a line (or hyperplane) to the data.

Mathematical formulation

y = β0 + β1·x1 + β2·x2 + … + ε

Using scikit‑learn

from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_absolute_error

lin_reg = LinearRegression()
lin_reg.fit(X_train, y_train)

y_pred = lin_reg.predict(X_test)
print('R^2:', r2_score(y_test, y_pred))
print('MAE:', mean_absolute_error(y_test, y_pred))

Assumptions

• Linear relationship between predictors and target.
• Homoscedasticity (constant variance of errors).
• No multicollinearity among features.
• Errors are normally distributed.

Diagnostics plots

import matplotlib.pyplot as plt
import seaborn as sns

residuals = y_test - y_pred
sns.scatterplot(x=y_pred, y=residuals)
plt.axhline(0, color='red', linestyle='--')
plt.xlabel('Predicted')
plt.ylabel('Residual')
plt.title('Residual Plot')
plt.show()

Regularization (optional)

• Ridge: L2 penalty
• Lasso: L1 penalty (feature selection)

from sklearn.linear_model import Ridge, Lasso
ridge = Ridge(alpha=1.0).fit(X_train, y_train)
lasso = Lasso(alpha=0.1).fit(X_train, y_train)