L1 Regularization (LASSO): Creates a diamond-shaped constraint region with sharp corners on the axes. The optimal solution often occurs at these corners, setting parameters to exactly zero. This creates sparse models perfect for feature selection.
Controls where your data "wants" the parameters to be without regularization (the orange dot). This simulates different data scenarios:
• 0°: Data suggests w₁ should be large, w₂ ≈ 0
• 45°: Data suggests w₁ ≈ w₂ (equal importance)
• 90°: Data suggests w₂ should be large, w₁ ≈ 0
Try this: Set angle to 30° with L1 - notice it chooses w₁ only. Slowly increase to 60° - it suddenly jumps to choosing w₂ only!