Trevor Hastie - Gradient Boosting Machine Learning
Why Cross Validated Stack Exchange recommends it
Stats Stack Exchange (Cross Validated) users link it when explaining gradient boosting and boosting residuals, and as an authoritative walkthrough from one of the authors of the standard textbook.
Recommended by 3 people across 4 different months, 2015 to 2019.
What it is
A 44-minute talk by Stanford professor Trevor Hastie, hosted on the H2O.ai channel, covering ensemble learners for classification. It moves from decision trees and random forests to boosting and gradient boosting machines.
What people said
Recommends meh · 2015-11 · Cross Validated Stack Exchange · on “how to interpret gradient boost model & results in R?”
That is the right link. There is more than one page devoted to gbm. I would actually read the discussion on bagging and probably rf too. There is also implementation in the "R lab' section. Have a look at this video (Tibshirani) for a nice explanation [this video] . Since it's about 35 min (as I recall) it should give you more information.
Recommends Abraham Horowitz · 2017-05 · Cross Validated Stack Exchange · on “Evaluating mean-squared error”
I don't know for sure whether or not there is something wrong with your numbers/code, but if your error is really high, this would actually be pretty standard when you're working with isolated regression trees. If your error really is high though, try gradient boosting. The algorithm is a subset of boosting algorithms (look into AdaBoost too; it's in the video below). It essentially fits more trees to the residuals of your other models. Check out this video [this video] by Prof. Trevor Hastie of H20.ai (& Stanford). Also, if you want to excel, read through this paper [LINK] by Prof. Robert Schapire and Prof. Yoav Freund, founders of AdaBoost. It isn't about gradient boosting, but maybe you will f
Recommends Abraham Horowitz · 2017-06 · Cross Validated Stack Exchange · on “How to validate an ANN model?”
If you are creating a forecast of a continuous time series, you can do that. This really depends on your objective though. If you are comparing your network model to the ARIMA, then you will probably want to use a similar metric so you can compare their accuracy. You're not treating it as an ARIMA model if you look at residuals. Residual analysis is widely used, as you probably know from GARCH models. This said, have you ever heard of gradient boosting? It essentially defines an additive model where each model forecasts the residuals of the previous model's objective. To be clear, the first learner models the series, then the second learner models the first's residuals, and then the third mo
Recommends Jonathan · 2019-09 · Cross Validated Stack Exchange · on “Using LASSO on random forest”
You must be on to something...Hastie himself suggests post-processing trees from random forest or boosting using LASSO. He mentioned is in this video [this video] at 30:10.