Jupyter notebooks for the book "The Elements of Statistical Learning".

This repository contains Jupyter notebooks implementing the algorithms found in the book, proofs and summary of the textbook.

Requirements

• jupyter
• pandas
• numpy
• matplotlib
• scipy
• tensorflow 2 - temporarily until I have a lot of free time to implement them from scratch and it is used only in Chapter 11.

Table of Contents

• Chapter 2

  • 2.3 Least Squares and Nearest Neighbors (nbviewer)
  • 2.4 Statistical Decision Theory (nbviewer)
  • 2.5 Local Methods in High Dimensions (nbviewer)
  • 2.6 Statistical Models, Supervised Learning and Function Approximation (nbviewer)
  • 2.7 Structured Regression Models (nbviewer)
  • 2.8 Classes of Restricted Estimators (nbviewer)
  • 2.9 Model Selection and the Bias-Variance Tradeoff (nbviewer)

• Chapter 3

  • 3.1 Introduction (nbviewer)
  • 3.2 Linear Regression Models and Least Squares (nbviewer)
  • 3.2.1 Example Prostate Cancer (nbviewer)
  • 3.2.2 The Gauss–Markov Theorem (nbviewer)
  • 3.2.3 Multiple Regression From Simple Univariate Regression (nbviewer)
  • 3.2.4 Multiple Outputs (nbviewer)
  • 3.3 Subset Selection (nbviewer)
  • 3.4 Shrinkage Methods (nbviewer)
  • 3.4.1 Ridge Regression (nbviewer)
  • 3.4.2 The Lasso (nbviewer)
  • TODO: 3.4.3 Discussion: Subset Selection, Ridge Regression and the Lasso (nbviewer)
  • 3.4.4 Least Angle Regression (nbviewer)
  • 3.5 Methods Using Derived Input Directions (nbviewer)

• Chapter 4

  • 4.1 Introduction (nbviewer)
  • 4.2 Linear Regression of an Indicator Matrix (nbviewer)
  • 4.3 Linear Discriminant Analysis (nbviewer)
  • 4.3.1 Regularized Discriminant Analysis nbviewer)
  • 4.3.2 Computations for LDA (nbviewer)
  • 4.3.3 Reduced-Rank Linear Discriminant Analysis (nbviewer)
  • 4.4 Logistic Regression (nbviewer)
  • 4.4.1 Fitting Logistic Regression Models (nbviewer)
  • 4.4.2 Example: South African Heart Disease (nbviewer)
  • 4.4.3 Quadratic Approximations and Inference (nbviewer)
  • 4.4.4 L1 Regularized Logistic Regression (nbviewer)

• Chapter 11

  • WIP: 11.7 Example: ZIP Code Data (nbviewer)