Skip to content

Add alpha and lambda (L1/L2 regularization) parameters to xgbTree and xgbDART#1415

Open
dgenio wants to merge 2 commits intotopepo:masterfrom
dgenio:add-alpha-lambda-xgbtree-xgbdart
Open

Add alpha and lambda (L1/L2 regularization) parameters to xgbTree and xgbDART#1415
dgenio wants to merge 2 commits intotopepo:masterfrom
dgenio:add-alpha-lambda-xgbtree-xgbdart

Conversation

@dgenio
Copy link
Copy Markdown

@dgenio dgenio commented Feb 19, 2026

What changed

Adds alpha (L1 regularization) and lambda (L2 regularization) as tunable parameters to the xgbTree and xgbDART model definitions. These regularization parameters are valid for the tree booster in XGBoost (confirmed in xgboost#6763) but were previously only exposed in xgbLinear.

Files modified:

  • models/files/xgbTree.R — added lambda/alpha to parameters, grid, loop, fit (all 3 objective branches), sort, and tags
  • models/files/xgbDART.R — same changes
  • RegressionTests/Code/xgbTree.R — updated test grid with lambda = 1, alpha = 0 (XGBoost defaults)
  • RegressionTests/Code/xgbDART.R — updated test grid with lambda = 1, alpha = 0
  • pkg/caret/inst/models/models.RData — regenerated via parseModels.R

Why

Closes #1187. XGBoost's alpha and lambda regularization parameters apply to the tree booster, not just the linear booster. Users want to tune these for xgbTree/xgbDART via train() and random search.

How verified

  • Sourced both model files in R — no parse errors
  • Validated grid() produces correct columns for both grid and random search
  • Validated models.RData contains updated parameter definitions (239 models total, all intact)
  • Verified xgbLinear was not affected
  • Regression test grids updated with XGBoost default values (lambda = 1, alpha = 0)

Tradeoffs / risks

  • Grid size: Default grid uses XGBoost defaults only (lambda = 1, alpha = 0) to avoid grid explosion. Full regularization range is explored via random search (10^runif(len, -5, 0)). Users can explore custom values via tuneGrid.
  • Backward compat: Users with custom tuneGrid data frames that omit lambda/alpha will need to add them. This is expected caret behavior when new tuning parameters are added.

Scope notes (Mode A)

Changes are strictly limited to adding alpha/lambda to xgbTree and xgbDART. Also added L1 Regularization Models and L2 Regularization Models tags to both models for consistency with xgbLinear.

… xgbDART

Adds alpha (L1 regularization) and lambda (L2 regularization) as tunable
parameters to the xgbTree and xgbDART model definitions, matching the
existing pattern in xgbLinear.

Changes:
- models/files/xgbTree.R: added lambda/alpha to parameters, grid, loop, fit, sort, tags
- models/files/xgbDART.R: same changes
- RegressionTests/Code/xgbTree.R: updated test grid with lambda=1, alpha=0
- RegressionTests/Code/xgbDART.R: updated test grid with lambda=1, alpha=0
- pkg/caret/inst/models/models.RData: regenerated via parseModels.R

Closes topepo#1187
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add new parameters to xgboostTree

1 participant