PyTorch Regression Pipeline — California Housing (7 Steps)
End-to-end PyTorch regression pipeline: leakage-safe scaling, Huber/SmoothL1 loss, DataLoader, MLP, and RMSE/MAE on California Housing.
PRACTICE THIS PYTORCH PIPELINE IN YOUR BROWSER
Write each milestone in the retro IDE. Audit tensor shapes, loss, and autograd with sub-second AI diagnostics.
California Housing Price Prediction: End-to-End PyTorch Deep Regression Pipeline
While classification is the most frequently taught deep learning task, continuous regression—predicting continuous real-valued targets such as home values, asset prices, temperature, and customer lifetime value—powers vast segments of industry applications.
Deep regression presents unique challenges: unconstrained output spaces, susceptibility to extreme outlier target values, and the risk of catastrophic loss scaling when errors square in Mean Squared Error (MSE).
In this tutorial, we build a 7-step Deep Tabular Regression Pipeline on the California Housing dataset using PyTorch, HuberLoss, and dynamic learning rate annealing.
#1. Regression vs Classification: Fundamental Differences
┌─────────────────────────────────┬──────────────────────────────────┐
│ Binary / Multiclass Tasks │ Continuous Regression Tasks │
├─────────────────────────────────┼──────────────────────────────────┤
│ Output layer: Logits + Softmax │ Output layer: Unbounded Linear │
│ Loss: BCEWithLogits / CrossEnt │ Loss: MSELoss, L1Loss, HuberLoss │
│ Metrics: Accuracy, ROC-AUC, F1 │ Metrics: RMSE, MAE, R² Score │
│ Target shape: [N] class indices │ Target shape: [N, 1] floats │
└─────────────────────────────────┴──────────────────────────────────┘#2. Step-by-Step Deep Regression Pipeline
Step 1: Data Ingestion & Target Distribution Analysis
import pandas as pd
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
# Ingest California Housing feature matrix
housing = fetch_california_housing(as_frame=True)
df = housing.frame
print(df.head())
print(df.describe())
# Target: MedHouseVal (in $100,000s)
# Inspect target distribution skewness
print("Target Mean:", df['MedHouseVal'].mean(), "| Std:", df['MedHouseVal'].std())Step 2: Feature Transformation & Train/Val/Test Split
feature_cols = housing.feature_names
X = df[feature_cols].values
y = df['MedHouseVal'].values
# Stratified-style split by target quantiles or random split
X_train_raw, X_val_raw, y_train, y_val = train_test_split(
X, y, test_size=0.20, random_state=42
)
# Standardize inputs: zero-mean, unit variance
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train_raw)
X_val = scaler.transform(X_val_raw)Step 3: Tensor Datasets & Batched Loading
import torch
from torch.utils.data import TensorDataset, DataLoader
# Ensure y has shape [N, 1] to avoid silent broadcasting!
train_dataset = TensorDataset(
torch.tensor(X_train, dtype=torch.float32),
torch.tensor(y_train, dtype=torch.float32).unsqueeze(1)
)
val_dataset = TensorDataset(
torch.tensor(X_val, dtype=torch.float32),
torch.tensor(y_val, dtype=torch.float32).unsqueeze(1)
)
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=64, shuffle=False)Step 4: Regression MLP Architecture
import torch.nn as nn
class HousingRegressor(nn.Module):
def __init__(self, input_dim: int = 8, hidden_dim: int = 64):
super(HousingRegressor, self).__init__()
self.net = nn.Sequential(
nn.Linear(input_dim, hidden_dim),
nn.LayerNorm(hidden_dim),
nn.ReLU(),
nn.Dropout(0.1),
nn.Linear(hidden_dim, 32),
nn.LayerNorm(32),
nn.ReLU(),
nn.Linear(32, 16),
nn.ReLU(),
nn.Linear(16, 1) # Single continuous scalar output without activation
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)
model = HousingRegressor(input_dim=8)Step 5: Why Huber Loss Outperforms MSE for Real Estate Data
Mean Squared Error (L_2) squares residuals (y - \hat{y})^2. In housing data with luxury multi-million-dollar outlier properties, a large residual produces huge gradient spikes that destabilize weights.
Huber Loss (Smooth L1 Loss) acts quadratic for small errors and linear for large errors (\delta = 1.0):
import torch.optim as optim
criterion = nn.HuberLoss(delta=1.0)
optimizer = optim.Adam(model.parameters(), lr=0.005, weight_decay=1e-4)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=5)Step 6: Epoch Training Loop with Dynamic Learning Rate Decay
EPOCHS = 40
for epoch in range(1, EPOCHS + 1):
model.train()
running_loss = 0.0
for bx, by in train_loader:
optimizer.zero_grad()
preds = model(bx)
loss = criterion(preds, by)
loss.backward()
optimizer.step()
running_loss += loss.item() * bx.size(0)
epoch_loss = running_loss / len(train_loader.dataset)
scheduler.step(epoch_loss)
if epoch % 10 == 0:
print(f"Epoch [{epoch:02d}/{EPOCHS}] | Huber Loss: {epoch_loss:.4f} | LR: {optimizer.param_groups[0]['lr']:.6f}")Step 7: Regression Diagnostics (RMSE, MAE, R²)
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
model.eval()
val_preds, val_targets = [], []
with torch.no_grad():
for bx, by in val_loader:
preds = model(bx)
val_preds.extend(preds.squeeze().tolist())
val_targets.extend(by.squeeze().tolist())
rmse = np.sqrt(mean_squared_error(val_targets, val_preds))
mae = mean_absolute_error(val_targets, val_preds)
r2 = r2_score(val_targets, val_preds)
print(f"--- REGRESSION EVALUATION ---")
print(f"Root Mean Squared Error (RMSE): ${rmse * 100_000:,.2f}")
print(f"Mean Absolute Error (MAE): ${mae * 100_000:,.2f}")
print(f"Coefficient of Determination (R²): {r2:.4f}")#Launch the Interactive Regression Lab
Practice continuous tabular regression in your browser with real-time AI validation.
FAQ
What loss should I use for a PyTorch regression pipeline?
Use MSE when the target tail is clean. Use Huber or SmoothL1 on California Housing when outliers exist. Targets are float32 with shape [N, 1], never class indices.
How do I report housing pipeline quality?
RMSE and MAE on a held-out split after leakage-safe scaling. Do not report training loss as the score.
RUN THE PIPELINE IN THE INTERACTIVE LAB
Same 7 steps as this guide. No install. AI validates each milestone.