| adaboost.M1 {adabag} | R Documentation |
Fits the Adaboost.M1 algorithm proposed by Freund and Schapire in 1996 using classification trees as single classifiers.
adaboost.M1(formula, data, boos = TRUE, mfinal = 100, coeflearn = 'Breiman',
minsplit = 5, cp = 0.01, maxdepth = nlevels(vardep))
formula |
a formula, as in the lm function. |
data |
a data frame in which to interpret the variables named in formula. |
boos |
if TRUE (by default), a bootstrap sample of the training set is drawn using
the weights for each observation on that iteration. If FALSE, every observation
is used with its weights. |
mfinal |
an integer, the number of iterations for which boosting is run
or the number of trees to use. Defaults to mfinal=100 iterations. |
coeflearn |
if 'Breiman'(by default), alpha=1/2ln((1-err)/err) is used.
If 'Freund' alpha=ln((1-err)/err) is used. Where alpha is the weight updating coefficient. |
minsplit |
the minimum number of observations that must exist in a node in order for a split to be attempted. |
cp |
complexity parameter. Any split that does not decrease the overall
lack of fit by a factor of cp is not attempted. |
maxdepth |
set the maximum depth of any node of the final tree, with the root node counted as depth 0 (past 30 rpart will give nonsense results on 32-bit machines). Defaults to the number of classes. |
Adaboost.M1 is a simple generalization of Adaboost for more than two classes
An object of class adaboost.M1, which is a list with the following components:
formula |
the formula used. |
trees |
the trees grown along the iterations. |
weights |
a vector with the weighting of the trees of all iterations. |
votes |
a matrix describing, for each observation, the number of trees that assigned it to each class, weighting each tree by its alpha coefficient. |
class |
the class predicted by the ensemble classifier. |
importance |
returns the relative importance of each variable in the classification task. This measure is the number of times each variable is selected to split. |
Esteban Alfaro Cortes Esteban.Alfaro@uclm.es, Matias Gamez Martinez Matias.Gamez@uclm.es and Noelia Garcia Rubio Noelia.Garcia@uclm.es
Alfaro, E., Gamez, M. and Garcia, N. (2007): ``Multiclass corporate failure prediction by Adaboost.M1''. International Advances in Economic Research, Vol 13, 3, pp. 301–312.
Freund, Y. and Schapire, R.E. (1996): ``Experiments with a New Boosting Algorithm''. In Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156, Morgan Kaufmann.
Breiman, L. (1998): ``Arcing classifiers''. The Annals of Statistics, Vol 26, 3, pp. 801–849.
## rpart library should be loaded
library(rpart)
data(iris)
names(iris)<-c("LS","AS","LP","AP","Especies")
iris.adaboost <- adaboost.M1(Especies~LS +AS +LP+ AP, data=iris, boos=TRUE,
mfinal=10)
## rpart and mlbench libraries should be loaded
## Comparing the test error of rpart and adaboost.M1
library(rpart)
library(mlbench)
data(BreastCancer)
l <- length(BreastCancer[,1])
sub <- sample(1:l,2*l/3)
BC.rpart <- rpart(Class~.,data=BreastCancer[sub,-1], maxdepth=3)
BC.rpart.pred <- predict(BC.rpart,newdata=BreastCancer[-sub,-1],type="class")
tb <-table(BC.rpart.pred,BreastCancer$Class[-sub])
error.rpart <- 1-(sum(diag(tb))/sum(tb))
tb
error.rpart
BC.adaboost <- adaboost.M1(Class ~.,data=BreastCancer[,-1],mfinal=25, maxdepth=3)
BC.adaboost.pred <- predict.boosting(BC.adaboost,newdata=BreastCancer[-sub,-1])
BC.adaboost.pred[-1]
## Data Vehicle (four classes)
library(rpart)
library(mlbench)
data(Vehicle)
l <- length(Vehicle[,1])
sub <- sample(1:l,2*l/3)
mfinal <- 25
maxdepth <- 5
Vehicle.rpart <- rpart(Class~.,data=Vehicle[sub,],maxdepth=maxdepth)
Vehicle.rpart.pred <- predict(Vehicle.rpart,newdata=Vehicle[-sub, ],type="class")
tb <- table(Vehicle.rpart.pred,Vehicle$Class[-sub])
error.rpart <- 1-(sum(diag(tb))/sum(tb))
tb
error.rpart
Vehicle.adaboost <- adaboost.M1(Class ~.,data=Vehicle[sub, ],mfinal=mfinal,
maxdepth=maxdepth)
Vehicle.adaboost.pred <- predict.boosting(Vehicle.adaboost,newdata=Vehicle[-sub, ])
Vehicle.adaboost.pred[-1]