'gbm3' gives much narrower predictions than 'gbm' pkg

[AMBarbosa]

Hi,
I'm trying to transition to gbm3, as prompted by the message that's now displayed when loading the gbm package. However, I get visibly different predictions for the same data. Here's a simple reproducible example based on random data:

set.seed(1)
N <- 1000
data <- data.frame(Y=sample(c(0, 1), N, replace = TRUE), 
                   X1=runif(N), X2=2*runif(N), X3=3*runif(N))

gbm1 <-  gbm::gbm(Y~X1+X2+X3, data=data)
gbm2 <- gbm3::gbm(Y~X1+X2+X3, data=data)

pred1 <- predict(gbm1, data, type = "response", n.trees = 100)
pred2 <- predict(gbm2, data, type = "response", n.trees = 100)

range(pred1)
# 0.2253441 0.6708913

range(pred2)
# 0.4887668 0.5017359

In this and other cases I've tried, gbm3 predicts a much narrower and (for my ecological data) less plausible range of values. What are these differences due to? Do I need to do something different to get my expected results with gbm3?

[brandongreenwell-8451]

I believe, depending on version at least, that they have pretty different defaults, which would go along way in causing such a difference. I'd go back and rerun with fixing interaction depth, learning rate, etc. to the same values and check the difference again.

[AMBarbosa]

Thanks. shrinkage (which went from a 0.1 to a 0.001 default value) seems to be the most influential parameter here: if I do gbm3::gbm with shrinkage=0.1 (the default in gbm::gbm), I get much more similar (even if still not equal) results.

Is there a reason for such a drastic change in the default shrinkage, especially given that it seems to provide (at least in my case) poorer default predictions?

Regards,

[gregridgeway]

Brandon is correct. Make sure the settings are the same and then predictions will be similar. set.seed(1) N <- 1000 data <- data.frame(Y=sample(c(0, 1), N, replace = TRUE), X1=runif(N), X2=2*runif(N), X3=3*runif(N)) set.seed(1) gbm1 <- gbm::gbm(Y~X1+X2+X3, distribution="bernoulli", data=data, n.trees=100, interaction.depth=3, shrinkage=0.01) set.seed(1) gbm2 <- gbm3::gbmt(Y~X1+X2+X3, distribution=gbm3::gbm_dist("Bernoulli"), data=data, train_params=gbm3::training_params(num_trees=100, interaction_depth = 3, shrinkage=0.01, num_train=N, num_features=3)) pred1 <- predict(gbm1, data, type = "response", n.trees = 100) pred2 <- predict(gbm2, data, type = "response", n.trees = 100) range(pred1) [1] 0.3862258 0.5648238 range(pred2) [1] 0.3981332 0.5628039 Smaller values of shrinkage will almost always get you better predictive performance. However, smaller values of shrinkage mean that you need num_trees to be larger. There’s a decreasing marginal return on smaller and smaller values of the shrinkage parameter. You need to make a predictive performance vs. computational performance tradeoff decision. I always make shrinkage as small as I can, but still get gbm to give me a model within a few minutes. From: AMBarbosa ***@***.***> Sent: Wednesday, August 7, 2024 3:09 PM To: gbm-developers/gbm3 ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [gbm-developers/gbm3] 'gbm3' gives much narrower predictions than 'gbm' pkg (Issue #165) Thanks. shrinkage (which went from a 0.1 to a 0.001 default value) seems to be the most influential parameter here: if I do gbm3::gbm with shrinkage=0.1 (the default in gbm::gbm), I get much more similar (even if still not equal) results. Is there a reason for such a drastic change in the default shrinkage, especially given that it seems to provide (at least in my case) poorer default predictions? Regards, — Reply to this email directly, view it on GitHub <#165 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACERTQGILUXGSEKFNS2MCALZQJWGLAVCNFSM6AAAAABMEUKDIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZUGE3DKOJXGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>