Adaptive LASSO in R

The adaptive lasso was introduced by Zou (2006, JASA) for linear regression and by Zhang and Lu (2007, Biometrika) for proportional hazards regression (R code from these latter authors).

For linear regression, we provide a simple R program that uses the lars package after reweighting the X matrix. We choose the tuning parameter using the "known variance" version of BIC with full model MSE for estimating the error variance. Example call and output and plot of BIC versus model size. This "known variance" version of BIC seemed to work well in simulations and matches up with the software below.

An important approximate adaptive lasso approach for many types of regression modeling was proposed by Wang and Leng (2007, JASA). We have taken their Least Squares Approximation (LSA) software from the second author's website and created a wrapper for linear regression. However, the software is easy to use without the wrapper: create a regression object in R, say "obj," and then lsa(obj). Example call and output. Comparing this output with the above adaptive lasso output using lars illustrates that the two programs give identical coefficient estimates for linear regression. We also checked this identity in simulations. Of course, in logistic regression and other regression situations the LSA adaptive lasso will not be identical to the full adaptive lasso based on the log likelihood (or loss function) plus penalty. Here is example use of LSA with logistic regression for the lucency data: 

> read.table("lucency.data.txt",header=T)->lucency
> glm(lucency~blade_size+I(1/tt_area)+I(1/tt_width)+age+wt_kg+pin+unibi,family=binomial,data=lucency)->out
> out

Call:  glm(formula = lucency ~ blade_size + I(1/tt_area) + I(1/tt_width) +
            age + wt_kg + pin + unibi, family = binomial, data = lucency)

Coefficients:
  (Intercept)     blade_size   I(1/tt_area)  I(1/tt_width)            age
    -12.18910        0.01898      -37.97180       36.88735        0.25408
        wt_kg            pin          unibi
      0.06356        1.13983        2.49374

Degrees of Freedom: 218 Total (i.e. Null);  211 Residual
Null Deviance:      129.2
Residual Deviance: 98.37        AIC: 114.4

> source("LSA.R.txt")
> lsa(out)
$beta.ols
  (Intercept)    blade_size  I(1/tt_area) I(1/tt_width)           age
 -12.18910399    0.01897884  -37.97180291   36.88735217    0.25408239
        wt_kg           pin         unibi
   0.06355784    1.13982947    2.49373744

$beta.bic
[1] -2.1782283  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000
[8]  0.8468798

$beta.aic
[1] -11.45609123   0.00000000   0.00000000  33.49036320   0.25048200   0.06380635
[7]   1.10105494   2.47889917