SVMBench              package:relations              R Documentation

_S_V_M _B_e_n_c_h_m_a_r_k_i_n_g _D_a_t_a _a_n_d _C_o_n_s_e_n_s_u_s _R_e_l_a_t_i_o_n_s

_D_e_s_c_r_i_p_t_i_o_n:

     'SVM_Benchmarking_Classification' and
     'SVM_Benchmarking_Regression' represent the results of a benchmark
     study comparing Support Vector Machines to other predictive
     methods on real and artificial data sets involving classification
     and regression methods, respectively.
     'SVM_Benchmarking_Classification_Consensus' and
     'SVM_Benchmarking_Regression_Consensus' are consensus rankings
     derived from these data.

_U_s_a_g_e:

     data("SVM_Benchmarking_Classification")
     data("SVM_Benchmarking_Regression")
     data("SVM_Benchmarking_Classification_Consensus")
     data("SVM_Benchmarking_Regression_Consensus")

_F_o_r_m_a_t:

     'SVM_Benchmarking_Classification' ('SVM_Benchmarking_Regression')
     is an ensemble of 21 (12) relations representing pairwise
     comparisons of 17 classification (10 regression) methods on 21
     (12) data sets.  Each relation of the  ensemble summarizes the
     results for a particular data set.  The relations are reflexive
     endorelations on the set of methods employed, with a pair (a, b)
     of distinct methods contained in a relation iff both delivered
     results on the corresponding data set and a did not perform
     significantly better than b at the 5% level.  Since some methods
     failed on some data sets, the relations are not guaranteed to be
     complete or transitive.  See Meyer et al. (2003) for details on
     the experimental design of the benchmark study, and Hornik and
     Meyer (2007) for the pairwise comparisons.

     'SVM_Benchmarking_Classification_Consensus' and
     'SVM_Benchmarking_Regression_Consensus' are lists of ensembles of
     consensus relations fitted to the benchmark results. For each of
     the following three endorelation families: 'SD/L' (linear
     orders), 'SD/O' (partial orders), and 'SD/W' (weak orders),
     _all_ possible consensus relations have been computed (see
     'relation_consensus'). For both classification and regression, the
     three relation ensembles obtained are provided as a named list of
     length 3. See Hornik et Meyer (2007) for details on the
     meta-analysis.

_S_o_u_r_c_e:

     D. Meyer, F. Leisch, and K. Hornik (2003), The support vector
     machine under test. _Neurocomputing_, *55*, 169-186.

     K. Hornik and D. Meyer (2007), Deriving consensus rankings from
     benchmarking experiments. In R. Decker and H.-J. Lenz, _Advances
     in Data Analysis_. Studies in Classification, Data Analysis, and
     Knowledge Organization. Springer-Verlag: Heidelberg, 163-170.

_E_x_a_m_p_l_e_s:

     data("SVM_Benchmarking_Classification")

     ## 21 data sets
     names(SVM_Benchmarking_Classification)

     ## 17 methods
     relation_domain(SVM_Benchmarking_Classification)

     ## select weak orders
     weak_orders <-
         Filter(relation_is_weak_order, SVM_Benchmarking_Classification)

     ## only the artifical data sets yield weak orders
     names(weak_orders)

     ## visualize them using Hasse diagrams
     if(require("Rgraphviz")) plot(weak_orders)

     ## Same for regression:
     data("SVM_Benchmarking_Regression")

     ## 12 data sets
     names(SVM_Benchmarking_Regression)

     ## 10 methods
     relation_domain(SVM_Benchmarking_Regression)

     ## select weak orders
     weak_orders <-
         Filter(relation_is_weak_order, SVM_Benchmarking_Regression)

     ## only two of the artifical data sets yield weak orders
     names(weak_orders)

     ## visualize them using Hasse diagrams
     if(require("Rgraphviz")) plot(weak_orders)

     ## Consensus solutions:

     data("SVM_Benchmarking_Classification_Consensus")
     data("SVM_Benchmarking_Regression_Consensus")

     ## The solutions for the three families are not unique
     print(SVM_Benchmarking_Classification_Consensus)
     print(SVM_Benchmarking_Regression_Consensus)

     ## visualize the consensus weak orders
     classW <- SVM_Benchmarking_Classification_Consensus$W
     regrW <- SVM_Benchmarking_Regression_Consensus$W
     if(require("Rgraphviz")) {
         plot(classW)
         plot(regrW)
     }

     ## in tabular style:
     ranking <- function(x) rev(names(sort(relation_class_ids(x))))
     sapply(classW, ranking)
     sapply(regrW, ranking)

     ## (prettier and more informative:)
     relation_classes(classW[[1L]])

