ST 790A - Introduction to Smoothing Methods and Nonparametric Regression

About the course: Smoothing techniques make an important class of tools for identifying the true signal hidden in highly noisy data. They offer the art of nonlinear curve/surface estimation by relaxing the linear assumption in regression and have very broad applications in many areas including biostatistics and bioinformatics. This course gives a thorough overview of various smoothing and nonparametric regression methodologies, with emphasis on both theoretical and computational aspects. Classical and modern techniques are introduced for nonparametric regression, classification, and density estimations. Main topics include regression and smoothing splines, kernel smoothers, generalized additive models, and other regularization techniques such as support vector machines used in data mining. Special treatment is given to important issues like parameter tuning and model selection. The course demands a moderate amount of programming with R language for data analysis. Various real datasets are used in class and homework. By the end of this course, the students are expected to get familiar with common smoothing techniques and gain hand-on experience with popular software and packages.

Course Information

Syllabus
Instructor: Hao Helen Zhang | Office hours: MW 2-3pm

Reference Books

Spline models for Observational Data -- Wahba (1990)
Generalized Additive Models -- Hastie and Tibshirani (1990)
Nonparametric Regression and Generalized Linear Models -- Green and Silverman (1994)
Smoothing Spline ANOVA Models -- Gu (2002)

Software and Datasets

Comprehensive R Archive Network | R Tutorial from Dr. Dennis Boos
LAPACK--lapack double | lapack search | SAS
LaTex -- Comprehensive Tex Archive Network (CTAN) | PROSPER | HTML
Datasets -- UPITT | UCI Machine Learning

Lecture Notes

Weeks 1-3: Overview of various smoothers
- Handout 1 (.pdf): Motivating Example
- Handout 2 (.pdf): More Examples and Homework 1, Solution (.pdf)
- Handout 3 (.pdf): Homework 2, Solution (.pdf)
- Handout 4 (.pdf): Moving average and LOWESS examples
- Handout 4 (part II) (.pdf): Homework 3, Solution (.pdf), Running Line R code
- Handout 5 (.pdf): Kernel Smoother
- Handout 5 - parts II (.pdf): R code for various smoothers
Weeks 4-8: Splines
- Handout 6 (.pdf): Function Approximation and Spline (I)
- Handout 7 (.pdf): More Splines
- Homework 4 (.pdf): Homework 4, Solution (.ps)
- Homework 5 (.pdf): Homework 5, Solution (.pdf)
- Handout 8 (.pdf): Cubic Splines
- Handout 9 (.pdf): B-splines
Weeks 9-11: Parameter Tuning
- Homework 6 (.pdf): Homework 6, Solution (.pdf)
Weeks 12-14: Multivariate Smoothers
- Homework 7 (.pdf): Homework 7, Solution (.pdf)

Reading Assignment

Cleveland (1979): Robust locally weighted regression and smoothing scatterplots (JASA)
Aroniszajn (1950): Theory of reproducing kernels.

Final Project, due on 05/04/2009.
Suggested paper list (pdf file)