## Adaptive LASSO in R

The adaptive lasso was introduced by Zou (2006, JASA) for linear regression and
by Zhang and Lu (2007, Biometrika) for proportional hazards regression
(R code from these latter authors).
For linear regression, we provide a simple R program
that uses the lars package after reweighting the X matrix. We choose the tuning parameter
using the "known variance" version of BIC with full model MSE for
estimating the error variance.
Example call and output and
plot of BIC versus model size. This "known variance" version of BIC
seemed to work well in simulations and matches up with the software below.

An important approximate adaptive lasso approach for many types of regression modeling was proposed
by Wang and Leng (2007, JASA). We have taken their Least Squares Approximation (LSA)
software from the second author's
website and created a
wrapper for linear regression. However, the software is easy to
use without the wrapper: create a regression object in R, say "obj," and then lsa(obj).
Example call and output. Comparing this output with the above
adaptive lasso output
using lars illustrates that the two programs give identical coefficient estimates for linear regression.
We also checked this identity in simulations. Of course, in logistic regression and other regression
situations the LSA adaptive lasso will not be identical to the full adaptive lasso based on the log likelihood
(or loss function) plus penalty. Here is example use of LSA with logistic regression for the
lucency data:

> read.table("lucency.data.txt",header=T)->lucency
> glm(lucency~blade_size+I(1/tt_area)+I(1/tt_width)+age+wt_kg+pin+unibi,family=binomial,data=lucency)->out
> out
Call: glm(formula = lucency ~ blade_size + I(1/tt_area) + I(1/tt_width) +
age + wt_kg + pin + unibi, family = binomial, data = lucency)
Coefficients:
(Intercept) blade_size I(1/tt_area) I(1/tt_width) age
-12.18910 0.01898 -37.97180 36.88735 0.25408
wt_kg pin unibi
0.06356 1.13983 2.49374
Degrees of Freedom: 218 Total (i.e. Null); 211 Residual
Null Deviance: 129.2
Residual Deviance: 98.37 AIC: 114.4
> source("LSA.R.txt")
> lsa(out)
$beta.ols
(Intercept) blade_size I(1/tt_area) I(1/tt_width) age
-12.18910399 0.01897884 -37.97180291 36.88735217 0.25408239
wt_kg pin unibi
0.06355784 1.13982947 2.49373744
$beta.bic
[1] -2.1782283 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000
[8] 0.8468798
$beta.aic
[1] -11.45609123 0.00000000 0.00000000 33.49036320 0.25048200 0.06380635
[7] 1.10105494 2.47889917