ST 790A - Introduction to Smoothing Methods and Nonparametric Regression

Announcement:

04/07, a list of suggested papers for the final project is posted.

Course Information

Syllabus
Instructor: Hao Zhang | Office hours: MW 4:20-5:20pm, 216 Patterson Hall

Reference Books

Spline models for Observational Data -- Wahba (1990)
Generalized Additive Models -- Hastie and Tibshirani (1990)
Nonparametric Regression and Generalized Linear Models -- Green and Silverman (1994)
Smoothing Spline ANOVA Models -- Gu (2002)

Lecture Notes

Weeks 1-2: Overview of various smoothers
- Lecture 1 (.pdf): introduction
- Lecture 2 (.pdf): regressogram and histogram
- Lecture 3 (.pdf): local polynomial regression and kernel estimators
  Reading 1: Robust locally weighted regression and smoothing scatterplots (Cleveland 1979, JASA)
  Reading 2: Locally weighted regression: an approach to regression analysis by local fitting (Cleveland and Delvin 1988, JASA)
Weeks 3-4: Theoretical Properties of smoothers
- Lecture 4 (.pdf): Consistency and Linear Estimators
  Reading 3: Consistent Nonparametric Regression (Stone 1977, Annals of Statistics)
- Lecture 5 (.pdf): Tuning Parameter Selection
  Reading 4: Generalized cross-validation as a method for choosing a good ridge parameter (Golub and Heath and Wahba 1979, Technometrics)
Weeks 5-9: Regression Splines
- Lecture 6 (.pdf): Regression splines and Natural Splines
- Lecture 7 (.pdf): Knot Selection
- Lecture 8 (.pdf): Introduction on B-splines
  Reading 5: Free-knot polynomial splines with confidence intervals (Mao and Zhao 2003, JRSSB)
Weeks 11-14: Smoothing Splines
- Lecture 9 (.pdf): Introduction on cubic smoothing splines
- Lecture 10 (.pdf): Functional analysis overview
  Reading 6: Theory of reproducing kernels (Aronszajn, 1950)
- Lecture 11 (.pdf): Theory of reproducing kernel Hilbert space (RKHS)
  Reading 7: Spline models for observational data (Wahba, 1990), page 1-30.
- Lecture 12 (.pdf): Another Look at Smoothing Spline models
- Lecture 13 (.pdf): Smoothing Spline ANOVA and Thin plate splines
Week 15: Generalized Additive Models
- Lecture 14 (.pdf): Introduction on Generalized Additive Models

Software

Here is the R code for various smoothers. You can get help on R here.

Assignments

Homework 1 (.pdf) assigned on 01/24/07, due on 02/07/07. Solution is available here.
Homework 2 (.pdf) assigned on 02/07/07, due on 02/21/07. First part of solution is here.
Homework 3 (.pdf) assigned on 02/21/07, due on 03/14/07. Second part of solution for HW2 here.
Homework 4 (.pdf) assigned on 03/19/07, due on 04/02/07. Solution is available here.
Homework 5 (.pdf) assigned on 04/02/07, due on 04/16/07
Homework 6 (.pdf) assigned on 04/18/07, due on 05/02/07

Final Project, due on 05/02/2007.
Suggested paper list (pdf file)