Adapting Algorithms for CAISEr

Introduction

This is a short guide to adapting existing algorithms and problem instances for running an experiment using CAISEr. In this document, we cover:

• Definition of instance lists
• Adaptation of existing algorithms
• Some stuff you can do with the results

A general description of the CAISE methodology is available in our paper.¹

Assembling an instance list

As stated in the documentation of both run_experiment and calc_nreps2, each instance must be a named list containing all relevant parameters that define the problem instance. This list must contain at least the field instance$FUN, with the name of the problem instance function, that is, a routine that calculates \(y = f(x)\). If the instance requires additional parameters, these must also be provided as named fields. Each instance can also have an alias, a unique name to distinguish it from other instances. If no alias is provided, the name of the function (instance$FUN) is used as the instance ID.

The Instance.list parameter for run_experiment() is simply a vector of these instance lists.

To illustrate how to adapt existing implementations to this structure, we assume that we are interested in comparing two multiobjective optimization algorithms for a (hypothetical) problem class that is well-represented by problems UF1 - UF7 (in dimensions between 10 and 40) from package smoof, . For this implementation to work with the MOEADr::moead() routine (see next section) some manipulation is necessary, but the instance list in this case is simply a list with each element containing the name of the routine as field $FUN (since all function names are different, no need for aliases).

suppressPackageStartupMessages(library(smoof))
suppressPackageStartupMessages(library(MOEADr))

### Build function names (instances: UF1 - UF7, dimensions 10 - 40)
fname   <- paste0("UF_", 1:7)
dims    <- c(10:40)
allfuns <- expand.grid(fname, dims, stringsAsFactors = FALSE)

# Assemble instances list
Instance.list <- vector(nrow(allfuns), mode = "list")
for (i in 1:length(Instance.list)){
  Instance.list[[i]]$FUN <- paste0(allfuns[i,1], "_", allfuns[i,2])
}

### Build the functions listed in Instance.list
# (so that they can be properly used)
for (i in 1:nrow(allfuns)){
  assign(x = Instance.list[[i]]$FUN,
     value = MOEADr::make_vectorized_smoof(prob.name  = "UF",
                    dimensions = allfuns[i, 2],
                    id = as.numeric(strsplit(allfuns[i, 1], "_")[[1]][2])))
}

Adaptation of an existing algorithm implementation

We will use the MOEA/D implementation available in the MOEADr package as our base algorithm, and assume that we are interested in comparing the performance of two versions of this algorithm: the original MOEA/D and the MOEA/D-DE (see the documentation of MOEADr and references therein for details of these methods) as solvers of the hypothetical problem class represented by the available test instances. The performance of each algorithm on each instance will be measured according to an indicator known as Inverted Generational Distance (IGD - details here), for which smaller = better.

As described in the documentation of both run_experiment() and calc_nreps2(), an algorithm must contain an algorithm$FUN field (the name of the function that calls the algorithm) and any other elements/parameters that algorithm$FUN requires (e.g., stop criteria, operator names and parameters, etc.). An additional field, algorithm$alias, can be used to provide the algorithm with a unique identifier.

Supposing that the list in algorithm has fields algorithm$FUN = myalgo, algorithm$par1 = "a", algorithm$par2 = 5, then the function in algorithm$FUN must have the following structure:

myalgo <- function(par1, par2, instance, ...){
  # do stuff
  return(results)
}

That is, it must be able to run if called as:

# remove '$FUN' and '$alias' field from list of arguments
# and include the problem definition as field 'instance'
myargs          <- algorithm[names(algorithm) != "FUN"]
myargs          <- myargs[names(myargs) != "alias"]
myargs$instance <- instance

# call 'algorithm$FUN' with the arguments in 'myargs'
do.call(algorithm$FUN, args = myargs)

Finally, the algorithm$FUN routine must return a list object containing (at least) the performance value of the final solution obtained after a given run, in a field named value (e.g., result$value) .

To build the algorithm functions to be used in run_experiment(), we encapsulate (almost) all algorithm parameters within a myalgo() function, which receives only two inputs: the instance to be solved (i.e., one element from Instance.list) and the specification of which version of the algorithm is to be run (the original MOEA/D or the MOEA/D-DE).

# Prepare algorithm function to be used in run_experiment():
myalgo <- function(type, instance){
  # Input parameters:
  #     - type (variant to use: "original" or "moead.de")
  #     - instance (instance to be solved, e.g., instance = Instance.list[[i]])
  # All other parameters are set internally

  ## Extract instance information to build the MOEADr problem format
  fdef  <- unlist(strsplit(instance$FUN, split = "_"))
  uffun <- smoof::makeUFFunction(dimensions = as.numeric(fdef[3]),
                                 id         = as.numeric(fdef[2]))
  fattr    <- attr(uffun, "par.set")
  prob.dim <- fattr$pars$x$len
  
  ## Build MOEADr problem list
  problem <- list(name = instance$FUN,
                  xmin = fattr$pars$x$lower,
                  xmax = fattr$pars$x$upper,
                  m    = attr(uffun, "n.objectives"))

  ## Load presets for the algorithm provided in input 'type' and 
  ## modify whatever is needed for this particular experiment
  
  algo.preset <- MOEADr::preset_moead(type)
  algo.preset$decomp$H <- 99 # <-- set population size
  algo.preset$stopcrit[[1]]$name <- "maxeval" # <-- type of stop criterion
  algo.preset$stopcrit[[1]]$maxeval <- 2000 * prob.dim # stop crit.
  poly.ind <- which(sapply(algo.preset$variation,
                           function(x){x$name == "polymut"}))
  algo.preset$variation[[poly.ind]]$pm <- 1 / prob.dim # <--- pm = 1/d
  
  ## Run algorithm on "instance"
  out <- MOEADr::moead(preset = algo.preset, problem = problem,
                       showpars = list(show.iters = "none"))

  ## Read reference data to calculate the IGD
  Yref  <- as.matrix(read.table(paste0("../inst/extdata/pf_data/",
                                       fdef[1], fdef[2], ".dat")))
  IGD = MOEADr::calcIGD(Y = out$Y, Yref = Yref)

  ## Return IGD as field "value" in the output list
  return(list(value = IGD))
}

Finally, the Algorithm.list parameter must be assembled as a list of algorithm objects (each containing fields $FUN, $alias and, in this case, $type).

# Assemble Algorithm.list. Notice that we need to provide an alias for each 
# method, since both algorithms have the same '$FUN' argument.
Algorithm.list <- list(list(FUN   = "myalgo", 
                            alias = "Algorithm 1", 
                            type  = "original"), 
                       list(FUN   = "myalgo", 
                            alias = "Algorithm 2", 
                            type  = "moead.de"))

Running an experiment using CAISEr

With the definitions above it is possible now to run an experiment using the iterative sample size determination implemented in CAISEr. For that, all we have to do is define the desired experimental parameters and use run_experiment():

#library(CAISEr)
my.results <- run_experiment(Instance.list = Instance.list,
                             Algorithm.list = Algorithm.list,
                             power = 0.8,      # Desired power: 80%
                             d = 0.5,          # to detect differences greater
                                               # than 0.5 standard deviations
                             sig.level = 0.05, # at a 95% confidence level. 
                             se.max = 0.05,    # Measurement error: 5% 
                             dif = "perc",     # on the paired percent 
                                               # differences of means,
                             method = "boot",  # calculated using bootstrap.
                             nstart = 15,      # Start with 20 runs/algo/inst
                             nmax   = 200,     # and do no more than 200 runs/inst
                             seed   = 1234)    # PRNG seed (for reproducibility)

After that we can interrogate the results and perform inference, if we are so inclined. For instance, we can check if our sample of paired differences in performance is (at least approximately) Normal, so that we can assume a Normal sampling distribution of the means and use a t test with a clean conscience:

# Take a look at the data summary:
print(my.results)

## $Configuration
## $Configuration[[1]]
## run_experiment
## 
## $Configuration$Instance.list
## Instance.list
## 
## $Configuration$Algorithm.list
## Algorithm.list
## 
## $Configuration$power
## [1] 0.8
## 
## $Configuration$d
## [1] 0.5
## 
## $Configuration$sig.level
## [1] 0.05
## 
## $Configuration$se.max
## [1] 0.05
## 
## $Configuration$dif
## [1] "perc"
## 
## $Configuration$method
## [1] "boot"
## 
## $Configuration$nstart
## [1] 15
## 
## $Configuration$nmax
## [1] 200
## 
## $Configuration$seed
## [1] 1234
## 
## 
## $data.raw
##        Algorithm Instance Observation
## 1    Algorithm 1  UF_4_13  0.06668496
## 2    Algorithm 1  UF_4_13  0.07174461
## 3    Algorithm 1  UF_4_13  0.06519550
## 4    Algorithm 1  UF_4_13  0.05780341
## 5    Algorithm 1  UF_4_13  0.06114531
## 6    Algorithm 1  UF_4_13  0.05794732
## 7    Algorithm 1  UF_4_13  0.06176148
## 8    Algorithm 1  UF_4_13  0.06379853
## 9    Algorithm 1  UF_4_13  0.06544827
## 10   Algorithm 1  UF_4_13  0.06096137
## 11   Algorithm 1  UF_4_13  0.06458852
## 12   Algorithm 1  UF_4_13  0.06307042
## 13   Algorithm 1  UF_4_13  0.06124503
## 14   Algorithm 1  UF_4_13  0.06576863
## 15   Algorithm 1  UF_4_13  0.06729768
## 16   Algorithm 2  UF_4_13  0.05345008
## 17   Algorithm 2  UF_4_13  0.05834417
## 18   Algorithm 2  UF_4_13  0.06173589
## 19   Algorithm 2  UF_4_13  0.05767850
## 20   Algorithm 2  UF_4_13  0.05557940
## 21   Algorithm 2  UF_4_13  0.04882276
## 22   Algorithm 2  UF_4_13  0.05724819
## 23   Algorithm 2  UF_4_13  0.04741506
## 24   Algorithm 2  UF_4_13  0.05437620
## 25   Algorithm 2  UF_4_13  0.05197991
## 26   Algorithm 2  UF_4_13  0.05434006
## 27   Algorithm 2  UF_4_13  0.05372360
## 28   Algorithm 2  UF_4_13  0.05173860
## 29   Algorithm 2  UF_4_13  0.05166588
## 30   Algorithm 2  UF_4_13  0.06224151
## 31   Algorithm 1  UF_2_29  0.04034199
## 32   Algorithm 1  UF_2_29  0.08482150
## 33   Algorithm 1  UF_2_29  0.02928047
## 34   Algorithm 1  UF_2_29  0.03286747
## 35   Algorithm 1  UF_2_29  0.02816334
## 36   Algorithm 1  UF_2_29  0.07032420
## 37   Algorithm 1  UF_2_29  0.07918861
## 38   Algorithm 1  UF_2_29  0.07324642
## 39   Algorithm 1  UF_2_29  0.08379163
## 40   Algorithm 1  UF_2_29  0.03008440
## 41   Algorithm 1  UF_2_29  0.10424497
## 42   Algorithm 1  UF_2_29  0.02883648
## 43   Algorithm 1  UF_2_29  0.02916721
## 44   Algorithm 1  UF_2_29  0.03335048
## 45   Algorithm 1  UF_2_29  0.03984722
## 46   Algorithm 1  UF_2_29  0.03055968
## 47   Algorithm 1  UF_2_29  0.07853463
## 48   Algorithm 1  UF_2_29  0.04703420
## 49   Algorithm 1  UF_2_29  0.06217556
## 50   Algorithm 1  UF_2_29  0.03434102
## 51   Algorithm 1  UF_2_29  0.03060020
## 52   Algorithm 1  UF_2_29  0.03262545
## 53   Algorithm 1  UF_2_29  0.04979062
## 54   Algorithm 1  UF_2_29  0.18867565
## 55   Algorithm 1  UF_2_29  0.04288892
## 56   Algorithm 1  UF_2_29  0.02914426
## 57   Algorithm 1  UF_2_29  0.07506079
## 58   Algorithm 1  UF_2_29  0.03035634
## 59   Algorithm 1  UF_2_29  0.03221542
## 60   Algorithm 1  UF_2_29  0.09093811
## 61   Algorithm 1  UF_2_29  0.08148270
## 62   Algorithm 1  UF_2_29  0.03151895
## 63   Algorithm 1  UF_2_29  0.09164118
## 64   Algorithm 1  UF_2_29  0.02895830
## 65   Algorithm 1  UF_2_29  0.07594967
## 66   Algorithm 1  UF_2_29  0.08147212
## 67   Algorithm 1  UF_2_29  0.07022693
## 68   Algorithm 1  UF_2_29  0.07324860
## 69   Algorithm 1  UF_2_29  0.04400419
## 70   Algorithm 1  UF_2_29  0.05548376
## 71   Algorithm 1  UF_2_29  0.07330740
## 72   Algorithm 1  UF_2_29  0.14449624
## 73   Algorithm 1  UF_2_29  0.07168279
## 74   Algorithm 1  UF_2_29  0.09493847
## 75   Algorithm 1  UF_2_29  0.02709823
## 76   Algorithm 1  UF_2_29  0.03358611
## 77   Algorithm 1  UF_2_29  0.03103623
## 78   Algorithm 1  UF_2_29  0.06128940
## 79   Algorithm 1  UF_2_29  0.02872702
## 80   Algorithm 1  UF_2_29  0.03138152
## 81   Algorithm 1  UF_2_29  0.08037487
## 82   Algorithm 1  UF_2_29  0.05041394
## 83   Algorithm 1  UF_2_29  0.07927069
## 84   Algorithm 1  UF_2_29  0.03112332
## 85   Algorithm 1  UF_2_29  0.02573133
## 86   Algorithm 1  UF_2_29  0.04781369
## 87   Algorithm 1  UF_2_29  0.03207411
## 88   Algorithm 1  UF_2_29  0.09784276
## 89   Algorithm 1  UF_2_29  0.04085161
## 90   Algorithm 1  UF_2_29  0.09050657
## 91   Algorithm 1  UF_2_29  0.03756956
## 92   Algorithm 1  UF_2_29  0.03150457
## 93   Algorithm 1  UF_2_29  0.02680941
## 94   Algorithm 1  UF_2_29  0.09625425
## 95   Algorithm 1  UF_2_29  0.04165846
## 96   Algorithm 2  UF_2_29  0.03521839
## 97   Algorithm 2  UF_2_29  0.03080864
## 98   Algorithm 2  UF_2_29  0.03017560
## 99   Algorithm 2  UF_2_29  0.04653534
## 100  Algorithm 2  UF_2_29  0.03637654
## 101  Algorithm 2  UF_2_29  0.03905905
## 102  Algorithm 2  UF_2_29  0.03799805
## 103  Algorithm 2  UF_2_29  0.04376631
## 104  Algorithm 2  UF_2_29  0.03458192
## 105  Algorithm 2  UF_2_29  0.03054443
## 106  Algorithm 2  UF_2_29  0.03664902
## 107  Algorithm 2  UF_2_29  0.03471599
## 108  Algorithm 2  UF_2_29  0.03215568
## 109  Algorithm 2  UF_2_29  0.04999694
## 110  Algorithm 2  UF_2_29  0.02968448
## 111  Algorithm 1  UF_5_28  0.31418503
## 112  Algorithm 1  UF_5_28  0.36043217
## 113  Algorithm 1  UF_5_28  0.40764830
## 114  Algorithm 1  UF_5_28  0.38299393
## 115  Algorithm 1  UF_5_28  0.23798117
## 116  Algorithm 1  UF_5_28  0.40111422
## 117  Algorithm 1  UF_5_28  0.38956883
## 118  Algorithm 1  UF_5_28  0.31733214
## 119  Algorithm 1  UF_5_28  0.35470902
## 120  Algorithm 1  UF_5_28  0.39298365
## 121  Algorithm 1  UF_5_28  0.39080700
## 122  Algorithm 1  UF_5_28  0.39558147
## 123  Algorithm 1  UF_5_28  0.48442203
## 124  Algorithm 1  UF_5_28  0.29354717
## 125  Algorithm 1  UF_5_28  0.34466974
## 126  Algorithm 1  UF_5_28  0.38568634
## 127  Algorithm 1  UF_5_28  0.34445052
## 128  Algorithm 1  UF_5_28  0.47067123
## 129  Algorithm 1  UF_5_28  0.40286730
## 130  Algorithm 1  UF_5_28  0.37579615
## 131  Algorithm 1  UF_5_28  0.43448529
## 132  Algorithm 1  UF_5_28  0.38917758
## 133  Algorithm 1  UF_5_28  0.41727851
## 134  Algorithm 1  UF_5_28  0.32752972
## 135  Algorithm 1  UF_5_28  0.40069034
## 136  Algorithm 1  UF_5_28  0.38819321
## 137  Algorithm 1  UF_5_28  0.45456313
## 138  Algorithm 1  UF_5_28  0.38114293
## 139  Algorithm 1  UF_5_28  0.43110816
## 140  Algorithm 1  UF_5_28  0.45454948
## 141  Algorithm 1  UF_5_28  0.23506711
## 142  Algorithm 1  UF_5_28  0.45456657
## 143  Algorithm 1  UF_5_28  0.38918752
## 144  Algorithm 1  UF_5_28  0.38420066
## 145  Algorithm 1  UF_5_28  0.23589024
## 146  Algorithm 1  UF_5_28  0.44399440
## 147  Algorithm 1  UF_5_28  0.32487443
## 148  Algorithm 1  UF_5_28  0.23812081
## 149  Algorithm 1  UF_5_28  0.26140033
## 150  Algorithm 1  UF_5_28  0.43051879
## 151  Algorithm 1  UF_5_28  0.22618751
## 152  Algorithm 1  UF_5_28  0.36089234
## 153  Algorithm 1  UF_5_28  0.30742528
## 154  Algorithm 1  UF_5_28  0.36308815
## 155  Algorithm 1  UF_5_28  0.38963148
## 156  Algorithm 1  UF_5_28  0.21119894
## 157  Algorithm 1  UF_5_28  0.34360884
## 158  Algorithm 1  UF_5_28  0.37859872
## 159  Algorithm 1  UF_5_28  0.26323488
## 160  Algorithm 1  UF_5_28  0.23912361
## 161  Algorithm 1  UF_5_28  0.34086502
## 162  Algorithm 1  UF_5_28  0.38979188
## 163  Algorithm 1  UF_5_28  0.38228217
## 164  Algorithm 1  UF_5_28  0.22906087
## 165  Algorithm 1  UF_5_28  0.25137446
## 166  Algorithm 1  UF_5_28  0.34248434
## 167  Algorithm 1  UF_5_28  0.34572988
## 168  Algorithm 1  UF_5_28  0.57577792
## 169  Algorithm 1  UF_5_28  0.28831185
## 170  Algorithm 1  UF_5_28  0.35109824
## 171  Algorithm 1  UF_5_28  0.38267523
## 172  Algorithm 1  UF_5_28  0.48658894
## 173  Algorithm 1  UF_5_28  0.38962900
## 174  Algorithm 1  UF_5_28  0.34166645
## 175  Algorithm 1  UF_5_28  0.41456356
## 176  Algorithm 1  UF_5_28  0.42187494
## 177  Algorithm 1  UF_5_28  0.31688655
## 178  Algorithm 1  UF_5_28  0.40358298
## 179  Algorithm 1  UF_5_28  0.32524689
## 180  Algorithm 1  UF_5_28  0.43394548
## 181  Algorithm 1  UF_5_28  0.28633755
## 182  Algorithm 1  UF_5_28  0.27371692
## 183  Algorithm 1  UF_5_28  0.19127030
## 184  Algorithm 1  UF_5_28  0.44000926
## 185  Algorithm 1  UF_5_28  0.36145331
## 186  Algorithm 1  UF_5_28  0.32725276
## 187  Algorithm 1  UF_5_28  0.40826109
## 188  Algorithm 1  UF_5_28  0.39171818
## 189  Algorithm 1  UF_5_28  0.45867835
## 190  Algorithm 1  UF_5_28  0.30763754
## 191  Algorithm 2  UF_5_28  0.65106130
## 192  Algorithm 2  UF_5_28  0.59313343
## 193  Algorithm 2  UF_5_28  0.10603429
## 194  Algorithm 2  UF_5_28  0.67314213
## 195  Algorithm 2  UF_5_28  0.83807464
## 196  Algorithm 2  UF_5_28  0.99860606
## 197  Algorithm 2  UF_5_28  0.51336319
## 198  Algorithm 2  UF_5_28  0.44791345
## 199  Algorithm 2  UF_5_28  0.58330388
## 200  Algorithm 2  UF_5_28  0.55583159
## 201  Algorithm 2  UF_5_28  0.69133464
## 202  Algorithm 2  UF_5_28  0.70263352
## 203  Algorithm 2  UF_5_28  0.45475384
## 204  Algorithm 2  UF_5_28  0.60643257
## 205  Algorithm 2  UF_5_28  0.61319932
## 206  Algorithm 2  UF_5_28  0.65544674
## 207  Algorithm 2  UF_5_28  0.71176229
## 208  Algorithm 2  UF_5_28  0.62746928
## 209  Algorithm 2  UF_5_28  0.47396727
## 210  Algorithm 2  UF_5_28  0.49501237
## 211  Algorithm 2  UF_5_28  0.59813851
## 212  Algorithm 2  UF_5_28  0.49443735
## 213  Algorithm 2  UF_5_28  0.62039137
## 214  Algorithm 2  UF_5_28  0.91085082
## 215  Algorithm 2  UF_5_28  0.61241038
## 216  Algorithm 2  UF_5_28  0.68001481
## 217  Algorithm 2  UF_5_28  0.51795796
## 218  Algorithm 2  UF_5_28  0.72628335
## 219  Algorithm 2  UF_5_28  0.64558739
## 220  Algorithm 2  UF_5_28  0.47290837
## 221  Algorithm 2  UF_5_28  0.44886000
## 222  Algorithm 2  UF_5_28  0.58466173
## 223  Algorithm 2  UF_5_28  0.74549083
## 224  Algorithm 2  UF_5_28  0.74066089
## 225  Algorithm 2  UF_5_28  0.58503785
## 226  Algorithm 2  UF_5_28  0.60734755
## 227  Algorithm 2  UF_5_28  0.48915696
## 228  Algorithm 2  UF_5_28  0.46016858
## 229  Algorithm 2  UF_5_28  0.70461591
## 230  Algorithm 2  UF_5_28  0.77616399
## 231  Algorithm 2  UF_5_28  0.68965699
## 232  Algorithm 2  UF_5_28  0.52656329
## 233  Algorithm 2  UF_5_28  0.74801734
## 234  Algorithm 2  UF_5_28  0.54066704
## 235  Algorithm 2  UF_5_28  0.39196324
## 236  Algorithm 2  UF_5_28  0.64936114
## 237  Algorithm 2  UF_5_28  0.57381021
## 238  Algorithm 2  UF_5_28  0.47273522
## 239  Algorithm 2  UF_5_28  0.73955162
## 240  Algorithm 2  UF_5_28  0.51666917
## 241  Algorithm 2  UF_5_28  0.95779358
## 242  Algorithm 2  UF_5_28  0.56914304
## 243  Algorithm 2  UF_5_28  0.78221531
## 244  Algorithm 2  UF_5_28  0.67513996
## 245  Algorithm 2  UF_5_28  0.61450298
## 246  Algorithm 2  UF_5_28  0.61516815
## 247  Algorithm 2  UF_5_28  0.54712648
## 248  Algorithm 2  UF_5_28  0.54111707
## 249  Algorithm 2  UF_5_28  0.60746841
## 250  Algorithm 2  UF_5_28  0.66999863
## 251  Algorithm 2  UF_5_28  0.75475758
## 252  Algorithm 2  UF_5_28  0.84201911
## 253  Algorithm 2  UF_5_28  0.71941448
## 254  Algorithm 2  UF_5_28  0.48489491
## 255  Algorithm 2  UF_5_28  0.66812959
## 256  Algorithm 2  UF_5_28  0.49830723
## 257  Algorithm 2  UF_5_28  0.48619180
## 258  Algorithm 2  UF_5_28  0.67976218
## 259  Algorithm 2  UF_5_28  0.59709340
## 260  Algorithm 2  UF_5_28  0.53727034
## 261  Algorithm 2  UF_5_28  0.61687045
## 262  Algorithm 2  UF_5_28  0.56772397
## 263  Algorithm 2  UF_5_28  0.90018399
## 264  Algorithm 2  UF_5_28  0.51045208
## 265  Algorithm 2  UF_5_28  0.44902949
## 266  Algorithm 2  UF_5_28  0.80138435
## 267  Algorithm 2  UF_5_28  0.69484727
## 268  Algorithm 2  UF_5_28  0.52543506
## 269  Algorithm 2  UF_5_28  0.58084608
## 270  Algorithm 2  UF_5_28  0.62927665
## 271  Algorithm 2  UF_5_28  0.62419444
## 272  Algorithm 2  UF_5_28  0.55311301
## 273  Algorithm 2  UF_5_28  0.55495439
## 274  Algorithm 2  UF_5_28  0.60072481
## 275  Algorithm 2  UF_5_28  0.65231928
## 276  Algorithm 2  UF_5_28  0.69289110
## 277  Algorithm 2  UF_5_28  0.69577417
## 278  Algorithm 2  UF_5_28  0.60408674
## 279  Algorithm 2  UF_5_28  0.58851832
## 280  Algorithm 2  UF_5_28  0.34357199
## 281  Algorithm 2  UF_5_28  0.75770296
## 282  Algorithm 2  UF_5_28  0.54843495
## 283  Algorithm 2  UF_5_28  0.54897481
## 284  Algorithm 2  UF_5_28  0.37409346
## 285  Algorithm 2  UF_5_28  0.83076449
## 286  Algorithm 2  UF_5_28  0.58824427
## 287  Algorithm 2  UF_5_28  0.60686481
## 288  Algorithm 2  UF_5_28  0.39088280
## 289  Algorithm 2  UF_5_28  0.59911602
## 290  Algorithm 2  UF_5_28  0.52230976
## 291  Algorithm 2  UF_5_28  0.39295374
## 292  Algorithm 2  UF_5_28  0.76299457
## 293  Algorithm 2  UF_5_28  0.66596785
## 294  Algorithm 2  UF_5_28  0.79616153
## 295  Algorithm 2  UF_5_28  0.47038921
## 296  Algorithm 2  UF_5_28  0.28973261
## 297  Algorithm 2  UF_5_28  0.55908501
## 298  Algorithm 2  UF_5_28  0.72015076
## 299  Algorithm 2  UF_5_28  0.51853937
## 300  Algorithm 2  UF_5_28  0.66269414
## 301  Algorithm 2  UF_5_28  0.69629310
## 302  Algorithm 2  UF_5_28  0.54792902
## 303  Algorithm 2  UF_5_28  0.52993300
## 304  Algorithm 2  UF_5_28  0.84193993
## 305  Algorithm 2  UF_5_28  0.58557940
## 306  Algorithm 2  UF_5_28  0.57800482
## 307  Algorithm 2  UF_5_28  0.65949926
## 308  Algorithm 2  UF_5_28  0.44620837
## 309  Algorithm 2  UF_5_28  0.59111152
## 310  Algorithm 2  UF_5_28  0.67327912
## 311  Algorithm 1  UF_1_29  0.17079563
## 312  Algorithm 1  UF_1_29  0.10305189
## 313  Algorithm 1  UF_1_29  0.08562614
## 314  Algorithm 1  UF_1_29  0.13576414
## 315  Algorithm 1  UF_1_29  0.09885127
## 316  Algorithm 1  UF_1_29  0.09108061
## 317  Algorithm 1  UF_1_29  0.12478635
## 318  Algorithm 1  UF_1_29  0.09304815
## 319  Algorithm 1  UF_1_29  0.12785744
## 320  Algorithm 1  UF_1_29  0.07230859
## 321  Algorithm 1  UF_1_29  0.13264596
## 322  Algorithm 1  UF_1_29  0.17913137
## 323  Algorithm 1  UF_1_29  0.21385808
## 324  Algorithm 1  UF_1_29  0.48731078
## 325  Algorithm 1  UF_1_29  0.10811966
## 326  Algorithm 1  UF_1_29  0.09861468
## 327  Algorithm 1  UF_1_29  0.16108209
## 328  Algorithm 1  UF_1_29  0.09278349
## 329  Algorithm 1  UF_1_29  0.17056863
## 330  Algorithm 1  UF_1_29  0.13008200
## 331  Algorithm 1  UF_1_29  0.13481016
## 332  Algorithm 1  UF_1_29  0.20699227
## 333  Algorithm 1  UF_1_29  0.09833294
## 334  Algorithm 1  UF_1_29  0.10841241
## 335  Algorithm 1  UF_1_29  0.18423834
## 336  Algorithm 2  UF_1_29  0.03817489
## 337  Algorithm 2  UF_1_29  0.08405970
## 338  Algorithm 2  UF_1_29  0.03992789
## 339  Algorithm 2  UF_1_29  0.08001466
## 340  Algorithm 2  UF_1_29  0.06862952
## 341  Algorithm 2  UF_1_29  0.05970668
## 342  Algorithm 2  UF_1_29  0.05595731
## 343  Algorithm 2  UF_1_29  0.05366178
## 344  Algorithm 2  UF_1_29  0.05758635
## 345  Algorithm 2  UF_1_29  0.04146082
## 346  Algorithm 2  UF_1_29  0.05549422
## 347  Algorithm 2  UF_1_29  0.04532183
## 348  Algorithm 2  UF_1_29  0.04104027
## 349  Algorithm 2  UF_1_29  0.04251228
## 350  Algorithm 2  UF_1_29  0.03329520
## 351  Algorithm 1  UF_2_36  0.03441510
## 352  Algorithm 1  UF_2_36  0.03876661
## 353  Algorithm 1  UF_2_36  0.04194750
## 354  Algorithm 1  UF_2_36  0.11271591
## 355  Algorithm 1  UF_2_36  0.09337724
## 356  Algorithm 1  UF_2_36  0.05181730
## 357  Algorithm 1  UF_2_36  0.03201601
## 358  Algorithm 1  UF_2_36  0.05956520
## 359  Algorithm 1  UF_2_36  0.04486447
## 360  Algorithm 1  UF_2_36  0.03400949
## 361  Algorithm 1  UF_2_36  0.03350702
## 362  Algorithm 1  UF_2_36  0.03201656
## 363  Algorithm 1  UF_2_36  0.03255274
## 364  Algorithm 1  UF_2_36  0.03056980
## 365  Algorithm 1  UF_2_36  0.02731216
## 366  Algorithm 1  UF_2_36  0.08948446
## 367  Algorithm 1  UF_2_36  0.02934978
## 368  Algorithm 1  UF_2_36  0.04111900
## 369  Algorithm 1  UF_2_36  0.04162181
## 370  Algorithm 1  UF_2_36  0.03259478
## 371  Algorithm 1  UF_2_36  0.09638080
## 372  Algorithm 1  UF_2_36  0.03163931
## 373  Algorithm 1  UF_2_36  0.03819944
## 374  Algorithm 1  UF_2_36  0.03020800
## 375  Algorithm 1  UF_2_36  0.03100325
## 376  Algorithm 1  UF_2_36  0.08873506
## 377  Algorithm 1  UF_2_36  0.04335315
## 378  Algorithm 1  UF_2_36  0.08944417
## 379  Algorithm 1  UF_2_36  0.03008220
## 380  Algorithm 1  UF_2_36  0.07352627
## 381  Algorithm 1  UF_2_36  0.03694254
## 382  Algorithm 1  UF_2_36  0.03395909
## 383  Algorithm 1  UF_2_36  0.03655852
## 384  Algorithm 1  UF_2_36  0.03349721
## 385  Algorithm 1  UF_2_36  0.03233938
## 386  Algorithm 1  UF_2_36  0.08642831
## 387  Algorithm 1  UF_2_36  0.03770713
## 388  Algorithm 1  UF_2_36  0.03854461
## 389  Algorithm 1  UF_2_36  0.02846811
## 390  Algorithm 1  UF_2_36  0.02894436
## 391  Algorithm 1  UF_2_36  0.08874924
## 392  Algorithm 1  UF_2_36  0.16861583
## 393  Algorithm 1  UF_2_36  0.04021527
## 394  Algorithm 1  UF_2_36  0.03053868
## 395  Algorithm 1  UF_2_36  0.06616631
## 396  Algorithm 1  UF_2_36  0.09609435
## 397  Algorithm 1  UF_2_36  0.09209959
## 398  Algorithm 1  UF_2_36  0.03763834
## 399  Algorithm 1  UF_2_36  0.03259443
## 400  Algorithm 1  UF_2_36  0.06769211
## 401  Algorithm 1  UF_2_36  0.06895552
## 402  Algorithm 1  UF_2_36  0.08142635
## 403  Algorithm 1  UF_2_36  0.05186895
## 404  Algorithm 1  UF_2_36  0.09952544
## 405  Algorithm 1  UF_2_36  0.03359356
## 406  Algorithm 1  UF_2_36  0.03653540
## 407  Algorithm 1  UF_2_36  0.03552658
## 408  Algorithm 1  UF_2_36  0.06242334
## 409  Algorithm 1  UF_2_36  0.03440577
## 410  Algorithm 1  UF_2_36  0.05416082
## 411  Algorithm 1  UF_2_36  0.08957604
## 412  Algorithm 1  UF_2_36  0.03055216
## 413  Algorithm 1  UF_2_36  0.06465175
## 414  Algorithm 1  UF_2_36  0.07058542
## 415  Algorithm 1  UF_2_36  0.08028116
## 416  Algorithm 1  UF_2_36  0.03005397
## 417  Algorithm 1  UF_2_36  0.09708527
## 418  Algorithm 1  UF_2_36  0.06055014
## 419  Algorithm 1  UF_2_36  0.06375896
## 420  Algorithm 1  UF_2_36  0.06599654
## 421  Algorithm 1  UF_2_36  0.03662137
## 422  Algorithm 2  UF_2_36  0.04141037
## 423  Algorithm 2  UF_2_36  0.05351244
## 424  Algorithm 2  UF_2_36  0.03610849
## 425  Algorithm 2  UF_2_36  0.03939121
## 426  Algorithm 2  UF_2_36  0.03259172
## 427  Algorithm 2  UF_2_36  0.03489153
## 428  Algorithm 2  UF_2_36  0.04458333
## 429  Algorithm 2  UF_2_36  0.04489122
## 430  Algorithm 2  UF_2_36  0.03970704
## 431  Algorithm 2  UF_2_36  0.03803534
## 432  Algorithm 2  UF_2_36  0.03377483
## 433  Algorithm 2  UF_2_36  0.03301098
## 434  Algorithm 2  UF_2_36  0.03983152
## 435  Algorithm 2  UF_2_36  0.04184384
## 436  Algorithm 2  UF_2_36  0.02544661
## 437  Algorithm 2  UF_2_36  0.03291922
## 438  Algorithm 1  UF_3_29  0.13485979
## 439  Algorithm 1  UF_3_29  0.10206843
## 440  Algorithm 1  UF_3_29  0.07472094
## 441  Algorithm 1  UF_3_29  0.09577617
## 442  Algorithm 1  UF_3_29  0.15385904
## 443  Algorithm 1  UF_3_29  0.10000981
## 444  Algorithm 1  UF_3_29  0.10262553
## 445  Algorithm 1  UF_3_29  0.09990467
## 446  Algorithm 1  UF_3_29  0.15149806
## 447  Algorithm 1  UF_3_29  0.08571304
## 448  Algorithm 1  UF_3_29  0.13445214
## 449  Algorithm 1  UF_3_29  0.11971940
## 450  Algorithm 1  UF_3_29  0.09581176
## 451  Algorithm 1  UF_3_29  0.07642467
## 452  Algorithm 1  UF_3_29  0.11710521
## 453  Algorithm 1  UF_3_29  0.09883760
## 454  Algorithm 1  UF_3_29  0.14214975
## 455  Algorithm 1  UF_3_29  0.11161669
## 456  Algorithm 1  UF_3_29  0.17094952
## 457  Algorithm 1  UF_3_29  0.11327628
## 458  Algorithm 1  UF_3_29  0.12426388
## 459  Algorithm 1  UF_3_29  0.13546382
## 460  Algorithm 1  UF_3_29  0.08993221
## 461  Algorithm 1  UF_3_29  0.09169918
## 462  Algorithm 1  UF_3_29  0.10849825
## 463  Algorithm 1  UF_3_29  0.15933083
## 464  Algorithm 1  UF_3_29  0.13845079
## 465  Algorithm 1  UF_3_29  0.11287348
## 466  Algorithm 1  UF_3_29  0.11608015
## 467  Algorithm 1  UF_3_29  0.17478963
## 468  Algorithm 1  UF_3_29  0.21609988
## 469  Algorithm 1  UF_3_29  0.14298520
## 470  Algorithm 1  UF_3_29  0.14638206
## 471  Algorithm 1  UF_3_29  0.08846388
## 472  Algorithm 1  UF_3_29  0.09833903
## 473  Algorithm 1  UF_3_29  0.12372334
## 474  Algorithm 1  UF_3_29  0.10634481
## 475  Algorithm 1  UF_3_29  0.12812203
## 476  Algorithm 1  UF_3_29  0.10252423
## 477  Algorithm 1  UF_3_29  0.12072432
## 478  Algorithm 1  UF_3_29  0.10986797
## 479  Algorithm 1  UF_3_29  0.12439233
## 480  Algorithm 1  UF_3_29  0.09865325
## 481  Algorithm 1  UF_3_29  0.11761155
## 482  Algorithm 1  UF_3_29  0.11005573
## 483  Algorithm 1  UF_3_29  0.10190078
## 484  Algorithm 1  UF_3_29  0.16029106
## 485  Algorithm 1  UF_3_29  0.09820187
## 486  Algorithm 1  UF_3_29  0.11031301
## 487  Algorithm 1  UF_3_29  0.12242608
## 488  Algorithm 1  UF_3_29  0.48119821
## 489  Algorithm 1  UF_3_29  0.13640272
## 490  Algorithm 1  UF_3_29  0.08975813
## 491  Algorithm 1  UF_3_29  0.13018791
## 492  Algorithm 1  UF_3_29  0.10109603
## 493  Algorithm 1  UF_3_29  0.14886520
## 494  Algorithm 1  UF_3_29  0.12962382
## 495  Algorithm 1  UF_3_29  0.09808411
## 496  Algorithm 1  UF_3_29  0.09233720
## 497  Algorithm 1  UF_3_29  0.09972590
## 498  Algorithm 1  UF_3_29  0.17701927
## 499  Algorithm 1  UF_3_29  0.11158489
## 500  Algorithm 1  UF_3_29  0.10291353
## 501  Algorithm 1  UF_3_29  0.11843044
## 502  Algorithm 1  UF_3_29  0.23901969
## 503  Algorithm 1  UF_3_29  0.14318706
## 504  Algorithm 1  UF_3_29  0.10577634
## 505  Algorithm 1  UF_3_29  0.11188581
## 506  Algorithm 1  UF_3_29  0.14605218
## 507  Algorithm 1  UF_3_29  0.09764713
## 508  Algorithm 1  UF_3_29  0.12974414
## 509  Algorithm 1  UF_3_29  0.12200796
## 510  Algorithm 1  UF_3_29  0.28929009
## 511  Algorithm 1  UF_3_29  0.12945296
## 512  Algorithm 1  UF_3_29  0.17038442
## 513  Algorithm 1  UF_3_29  0.09833576
## 514  Algorithm 1  UF_3_29  0.17917405
## 515  Algorithm 1  UF_3_29  0.15512351
## 516  Algorithm 1  UF_3_29  0.13556987
## 517  Algorithm 1  UF_3_29  0.16295540
## 518  Algorithm 1  UF_3_29  0.11308877
## 519  Algorithm 1  UF_3_29  0.08509735
## 520  Algorithm 1  UF_3_29  0.08802555
## 521  Algorithm 1  UF_3_29  0.15082430
## 522  Algorithm 1  UF_3_29  0.12369275
## 523  Algorithm 1  UF_3_29  0.10068139
## 524  Algorithm 1  UF_3_29  0.09316523
## 525  Algorithm 1  UF_3_29  0.12488037
## 526  Algorithm 1  UF_3_29  0.12052388
## 527  Algorithm 1  UF_3_29  0.12234219
## 528  Algorithm 1  UF_3_29  0.09664046
## 529  Algorithm 1  UF_3_29  0.10133131
## 530  Algorithm 1  UF_3_29  0.20309962
## 531  Algorithm 1  UF_3_29  0.09673441
## 532  Algorithm 1  UF_3_29  0.12987671
## 533  Algorithm 1  UF_3_29  0.09150593
## 534  Algorithm 1  UF_3_29  0.10660809
## 535  Algorithm 1  UF_3_29  0.19627255
## 536  Algorithm 1  UF_3_29  0.12269570
## 537  Algorithm 2  UF_3_29  0.17179585
## 538  Algorithm 2  UF_3_29  0.14935235
## 539  Algorithm 2  UF_3_29  0.08976703
## 540  Algorithm 2  UF_3_29  0.07986184
## 541  Algorithm 2  UF_3_29  0.17287715
## 542  Algorithm 2  UF_3_29  0.15091010
## 543  Algorithm 2  UF_3_29  0.21834300
## 544  Algorithm 2  UF_3_29  0.05693440
## 545  Algorithm 2  UF_3_29  0.24294369
## 546  Algorithm 2  UF_3_29  0.15570224
## 547  Algorithm 2  UF_3_29  0.20061253
## 548  Algorithm 2  UF_3_29  0.09380417
## 549  Algorithm 2  UF_3_29  0.14570548
## 550  Algorithm 2  UF_3_29  0.09119522
## 551  Algorithm 2  UF_3_29  0.13784852
## 552  Algorithm 2  UF_3_29  0.12336010
## 553  Algorithm 2  UF_3_29  0.21364514
## 554  Algorithm 2  UF_3_29  0.09859683
## 555  Algorithm 2  UF_3_29  0.13689709
## 556  Algorithm 2  UF_3_29  0.10546887
## 557  Algorithm 2  UF_3_29  0.11866019
## 558  Algorithm 2  UF_3_29  0.04726745
## 559  Algorithm 2  UF_3_29  0.12907583
## 560  Algorithm 2  UF_3_29  0.14997826
## 561  Algorithm 2  UF_3_29  0.08379961
## 562  Algorithm 2  UF_3_29  0.12823083
## 563  Algorithm 2  UF_3_29  0.10963671
## 564  Algorithm 2  UF_3_29  0.18413519
## 565  Algorithm 2  UF_3_29  0.20033710
## 566  Algorithm 2  UF_3_29  0.21029523
## 567  Algorithm 2  UF_3_29  0.12307836
## 568  Algorithm 2  UF_3_29  0.14992596
## 569  Algorithm 2  UF_3_29  0.11267252
## 570  Algorithm 2  UF_3_29  0.18136523
## 571  Algorithm 2  UF_3_29  0.07413493
## 572  Algorithm 2  UF_3_29  0.11028110
## 573  Algorithm 2  UF_3_29  0.08736141
## 574  Algorithm 2  UF_3_29  0.06081145
## 575  Algorithm 2  UF_3_29  0.14929344
## 576  Algorithm 2  UF_3_29  0.14109291
## 577  Algorithm 2  UF_3_29  0.13293422
## 578  Algorithm 2  UF_3_29  0.10631891
## 579  Algorithm 2  UF_3_29  0.06397775
## 580  Algorithm 2  UF_3_29  0.11659815
## 581  Algorithm 2  UF_3_29  0.08072298
## 582  Algorithm 2  UF_3_29  0.08469108
## 583  Algorithm 2  UF_3_29  0.14249538
## 584  Algorithm 2  UF_3_29  0.12127790
## 585  Algorithm 2  UF_3_29  0.06284180
## 586  Algorithm 2  UF_3_29  0.08685965
## 587  Algorithm 2  UF_3_29  0.15710756
## 588  Algorithm 2  UF_3_29  0.13389627
## 589  Algorithm 2  UF_3_29  0.05992256
## 590  Algorithm 2  UF_3_29  0.08705484
## 591  Algorithm 2  UF_3_29  0.10014311
## 592  Algorithm 2  UF_3_29  0.15054485
## 593  Algorithm 2  UF_3_29  0.21781253
## 594  Algorithm 2  UF_3_29  0.11001830
## 595  Algorithm 2  UF_3_29  0.17948330
## 596  Algorithm 2  UF_3_29  0.12243941
## 597  Algorithm 2  UF_3_29  0.21775619
## 598  Algorithm 2  UF_3_29  0.08728484
## 599  Algorithm 2  UF_3_29  0.12805215
## 600  Algorithm 2  UF_3_29  0.25833195
## 601  Algorithm 2  UF_3_29  0.12205996
## 602  Algorithm 2  UF_3_29  0.25501416
## 603  Algorithm 2  UF_3_29  0.18502751
## 604  Algorithm 2  UF_3_29  0.31734201
## 605  Algorithm 2  UF_3_29  0.10377003
## 606  Algorithm 2  UF_3_29  0.06480213
## 607  Algorithm 2  UF_3_29  0.07935004
## 608  Algorithm 2  UF_3_29  0.11782088
## 609  Algorithm 2  UF_3_29  0.12889231
## 610  Algorithm 2  UF_3_29  0.13407144
## 611  Algorithm 2  UF_3_29  0.11565925
## 612  Algorithm 2  UF_3_29  0.13820761
## 613  Algorithm 2  UF_3_29  0.19109675
## 614  Algorithm 2  UF_3_29  0.24604979
## 615  Algorithm 2  UF_3_29  0.12089089
## 616  Algorithm 2  UF_3_29  0.09260491
## 617  Algorithm 2  UF_3_29  0.12692324
## 618  Algorithm 2  UF_3_29  0.21288388
## 619  Algorithm 2  UF_3_29  0.07983592
## 620  Algorithm 2  UF_3_29  0.10948601
## 621  Algorithm 2  UF_3_29  0.17447620
## 622  Algorithm 2  UF_3_29  0.12536783
## 623  Algorithm 2  UF_3_29  0.04000001
## 624  Algorithm 2  UF_3_29  0.11032864
## 625  Algorithm 2  UF_3_29  0.11386252
## 626  Algorithm 2  UF_3_29  0.12681584
## 627  Algorithm 2  UF_3_29  0.08638529
## 628  Algorithm 2  UF_3_29  0.13240447
## 629  Algorithm 2  UF_3_29  0.21648460
## 630  Algorithm 2  UF_3_29  0.15807440
## 631  Algorithm 2  UF_3_29  0.07970772
## 632  Algorithm 2  UF_3_29  0.12711245
## 633  Algorithm 2  UF_3_29  0.14833358
## 634  Algorithm 2  UF_3_29  0.14594639
## 635  Algorithm 2  UF_3_29  0.10294851
## 636  Algorithm 2  UF_3_29  0.16289247
## 637  Algorithm 2  UF_3_29  0.13221329
## 638  Algorithm 1  UF_3_10  0.21261113
## 639  Algorithm 1  UF_3_10  0.29895429
## 640  Algorithm 1  UF_3_10  0.23084350
## 641  Algorithm 1  UF_3_10  0.18611964
## 642  Algorithm 1  UF_3_10  0.27618336
## 643  Algorithm 1  UF_3_10  0.27727948
## 644  Algorithm 1  UF_3_10  0.25683815
## 645  Algorithm 1  UF_3_10  0.42839175
## 646  Algorithm 1  UF_3_10  0.37219036
## 647  Algorithm 1  UF_3_10  0.20448010
## 648  Algorithm 1  UF_3_10  0.28635020
## 649  Algorithm 1  UF_3_10  0.24647764
## 650  Algorithm 1  UF_3_10  0.23002484
## 651  Algorithm 1  UF_3_10  0.18230544
## 652  Algorithm 1  UF_3_10  0.25248777
## 653  Algorithm 1  UF_3_10  0.24499462
## 654  Algorithm 1  UF_3_10  0.17263053
## 655  Algorithm 1  UF_3_10  0.46137637
## 656  Algorithm 1  UF_3_10  0.19857867
## 657  Algorithm 1  UF_3_10  0.19995401
## 658  Algorithm 1  UF_3_10  0.22729567
## 659  Algorithm 1  UF_3_10  0.23244152
## 660  Algorithm 1  UF_3_10  0.25275378
## 661  Algorithm 1  UF_3_10  0.18546551
## 662  Algorithm 1  UF_3_10  0.32573540
## 663  Algorithm 1  UF_3_10  0.28838125
## 664  Algorithm 1  UF_3_10  0.19719456
## 665  Algorithm 1  UF_3_10  0.19401104
## 666  Algorithm 1  UF_3_10  0.17582304
## 667  Algorithm 1  UF_3_10  0.37284891
## 668  Algorithm 1  UF_3_10  0.23979182
## 669  Algorithm 1  UF_3_10  0.25752895
## 670  Algorithm 1  UF_3_10  0.25951982
## 671  Algorithm 1  UF_3_10  0.28908056
## 672  Algorithm 1  UF_3_10  0.36965661
## 673  Algorithm 1  UF_3_10  0.33768411
## 674  Algorithm 1  UF_3_10  0.21983339
## 675  Algorithm 1  UF_3_10  0.25207412
## 676  Algorithm 1  UF_3_10  0.19580014
## 677  Algorithm 1  UF_3_10  0.18501337
## 678  Algorithm 1  UF_3_10  0.21725800
## 679  Algorithm 1  UF_3_10  0.44850388
## 680  Algorithm 1  UF_3_10  0.26232790
## 681  Algorithm 1  UF_3_10  0.25373492
## 682  Algorithm 1  UF_3_10  0.23827010
## 683  Algorithm 1  UF_3_10  0.33846039
## 684  Algorithm 1  UF_3_10  0.23006682
## 685  Algorithm 1  UF_3_10  0.18223609
## 686  Algorithm 1  UF_3_10  0.18442585
## 687  Algorithm 1  UF_3_10  0.23355307
## 688  Algorithm 1  UF_3_10  0.21203698
## 689  Algorithm 1  UF_3_10  0.32369435
## 690  Algorithm 1  UF_3_10  0.33974954
## 691  Algorithm 1  UF_3_10  0.33892079
## 692  Algorithm 1  UF_3_10  0.26487471
## 693  Algorithm 1  UF_3_10  0.26664721
## 694  Algorithm 1  UF_3_10  0.24778361
## 695  Algorithm 2  UF_3_10  0.33997001
## 696  Algorithm 2  UF_3_10  0.31487208
## 697  Algorithm 2  UF_3_10  0.29220692
## 698  Algorithm 2  UF_3_10  0.27039131
## 699  Algorithm 2  UF_3_10  0.21124338
## 700  Algorithm 2  UF_3_10  0.17855591
## 701  Algorithm 2  UF_3_10  0.34303997
## 702  Algorithm 2  UF_3_10  0.34004086
## 703  Algorithm 2  UF_3_10  0.26147582
## 704  Algorithm 2  UF_3_10  0.34300722
## 705  Algorithm 2  UF_3_10  0.33978810
## 706  Algorithm 2  UF_3_10  0.32939652
## 707  Algorithm 2  UF_3_10  0.14449738
## 708  Algorithm 2  UF_3_10  0.33574298
## 709  Algorithm 2  UF_3_10  0.26036389
## 710  Algorithm 2  UF_3_10  0.21023651
## 711  Algorithm 2  UF_3_10  0.33093256
## 712  Algorithm 2  UF_3_10  0.34639752
## 713  Algorithm 2  UF_3_10  0.34832671
## 714  Algorithm 2  UF_3_10  0.34058037
## 715  Algorithm 2  UF_3_10  0.20811065
## 716  Algorithm 2  UF_3_10  0.16038942
## 717  Algorithm 2  UF_3_10  0.27385015
## 718  Algorithm 2  UF_3_10  0.20493942
## 719  Algorithm 2  UF_3_10  0.33848381
## 720  Algorithm 2  UF_3_10  0.32383683
## 721  Algorithm 2  UF_3_10  0.12533315
## 722  Algorithm 2  UF_3_10  0.29106978
## 723  Algorithm 2  UF_3_10  0.28406942
## 724  Algorithm 2  UF_3_10  0.33849774
## 725  Algorithm 2  UF_3_10  0.34661065
## 726  Algorithm 2  UF_3_10  0.33603363
## 727  Algorithm 2  UF_3_10  0.20375244
## 728  Algorithm 2  UF_3_10  0.33723771
## 729  Algorithm 2  UF_3_10  0.18293321
## 730  Algorithm 2  UF_3_10  0.33175677
## 731  Algorithm 2  UF_3_10  0.31874093
## 732  Algorithm 2  UF_3_10  0.34248399
## 733  Algorithm 2  UF_3_10  0.34043753
## 734  Algorithm 2  UF_3_10  0.13829614
## 735  Algorithm 2  UF_3_10  0.34040530
## 736  Algorithm 2  UF_3_10  0.16391707
## 737  Algorithm 2  UF_3_10  0.29605923
## 738  Algorithm 2  UF_3_10  0.34416609
## 739  Algorithm 2  UF_3_10  0.18572599
## 740  Algorithm 2  UF_3_10  0.30950668
## 741  Algorithm 2  UF_3_10  0.31844760
## 742  Algorithm 2  UF_3_10  0.30609215
## 743  Algorithm 2  UF_3_10  0.34198565
## 744  Algorithm 2  UF_3_10  0.24911329
## 745  Algorithm 2  UF_3_10  0.34880029
## 746  Algorithm 2  UF_3_10  0.19205396
## 747  Algorithm 2  UF_3_10  0.20053106
## 748  Algorithm 2  UF_3_10  0.33282104
## 749  Algorithm 2  UF_3_10  0.16099665
## 750  Algorithm 2  UF_3_10  0.33864796
## 751  Algorithm 2  UF_3_10  0.17493419
## 752  Algorithm 2  UF_3_10  0.22689130
## 753  Algorithm 1  UF_7_16  0.34239563
## 754  Algorithm 1  UF_7_16  0.33050389
## 755  Algorithm 1  UF_7_16  0.34379216
## 756  Algorithm 1  UF_7_16  0.34848111
## 757  Algorithm 1  UF_7_16  0.32683561
## 758  Algorithm 1  UF_7_16  0.34138221
## 759  Algorithm 1  UF_7_16  0.34878661
## 760  Algorithm 1  UF_7_16  0.35504147
## 761  Algorithm 1  UF_7_16  0.35818776
## 762  Algorithm 1  UF_7_16  0.34668495
## 763  Algorithm 1  UF_7_16  0.34745398
## 764  Algorithm 1  UF_7_16  0.33937478
## 765  Algorithm 1  UF_7_16  0.39901176
## 766  Algorithm 1  UF_7_16  0.34971566
## 767  Algorithm 1  UF_7_16  0.35123767
## 768  Algorithm 2  UF_7_16  0.01403717
## 769  Algorithm 2  UF_7_16  0.01562596
## 770  Algorithm 2  UF_7_16  0.01567629
## 771  Algorithm 2  UF_7_16  0.01402523
## 772  Algorithm 2  UF_7_16  0.01448069
## 773  Algorithm 2  UF_7_16  0.02262519
## 774  Algorithm 2  UF_7_16  0.01457875
## 775  Algorithm 2  UF_7_16  0.02471940
## 776  Algorithm 2  UF_7_16  0.02056412
## 777  Algorithm 2  UF_7_16  0.01987860
## 778  Algorithm 2  UF_7_16  0.02204249
## 779  Algorithm 2  UF_7_16  0.02034747
## 780  Algorithm 2  UF_7_16  0.01987121
## 781  Algorithm 2  UF_7_16  0.01884027
## 782  Algorithm 2  UF_7_16  0.01943830
## 783  Algorithm 1  UF_7_29  0.34182458
## 784  Algorithm 1  UF_7_29  0.35663732
## 785  Algorithm 1  UF_7_29  0.52732558
## 786  Algorithm 1  UF_7_29  0.56975282
## 787  Algorithm 1  UF_7_29  0.59761349
## 788  Algorithm 1  UF_7_29  0.40466139
## 789  Algorithm 1  UF_7_29  0.36367979
## 790  Algorithm 1  UF_7_29  0.35907023
## 791  Algorithm 1  UF_7_29  0.35662430
## 792  Algorithm 1  UF_7_29  0.35881693
## 793  Algorithm 1  UF_7_29  0.34606098
## 794  Algorithm 1  UF_7_29  0.35785914
## 795  Algorithm 1  UF_7_29  0.54617604
## 796  Algorithm 1  UF_7_29  0.36115163
## 797  Algorithm 1  UF_7_29  0.53726879
## 798  Algorithm 2  UF_7_29  0.03090315
## 799  Algorithm 2  UF_7_29  0.01686651
## 800  Algorithm 2  UF_7_29  0.02210778
## 801  Algorithm 2  UF_7_29  0.27676492
## 802  Algorithm 2  UF_7_29  0.02755434
## 803  Algorithm 2  UF_7_29  0.03114942
## 804  Algorithm 2  UF_7_29  0.02725695
## 805  Algorithm 2  UF_7_29  0.02849915
## 806  Algorithm 2  UF_7_29  0.02226347
## 807  Algorithm 2  UF_7_29  0.01950048
## 808  Algorithm 2  UF_7_29  0.02586767
## 809  Algorithm 2  UF_7_29  0.03733569
## 810  Algorithm 2  UF_7_29  0.02230300
## 811  Algorithm 2  UF_7_29  0.02895698
## 812  Algorithm 2  UF_7_29  0.02652764
## 813  Algorithm 1  UF_2_25  0.03094333
## 814  Algorithm 1  UF_2_25  0.03000915
## 815  Algorithm 1  UF_2_25  0.02789043
## 816  Algorithm 1  UF_2_25  0.08605605
## 817  Algorithm 1  UF_2_25  0.04275696
## 818  Algorithm 1  UF_2_25  0.13119906
## 819  Algorithm 1  UF_2_25  0.04145332
## 820  Algorithm 1  UF_2_25  0.03074464
## 821  Algorithm 1  UF_2_25  0.03211799
## 822  Algorithm 1  UF_2_25  0.07378741
## 823  Algorithm 1  UF_2_25  0.03744218
## 824  Algorithm 1  UF_2_25  0.09128189
## 825  Algorithm 1  UF_2_25  0.09543743
## 826  Algorithm 1  UF_2_25  0.02969880
## 827  Algorithm 1  UF_2_25  0.03006556
## 828  Algorithm 1  UF_2_25  0.05481254
## 829  Algorithm 1  UF_2_25  0.08585325
## 830  Algorithm 1  UF_2_25  0.07910609
## 831  Algorithm 1  UF_2_25  0.02885818
## 832  Algorithm 1  UF_2_25  0.02973658
## 833  Algorithm 1  UF_2_25  0.02903613
## 834  Algorithm 1  UF_2_25  0.03147280
## 835  Algorithm 1  UF_2_25  0.03712773
## 836  Algorithm 1  UF_2_25  0.03550769
## 837  Algorithm 1  UF_2_25  0.05613123
## 838  Algorithm 1  UF_2_25  0.03639893
## 839  Algorithm 1  UF_2_25  0.03630912
## 840  Algorithm 1  UF_2_25  0.04259047
## 841  Algorithm 1  UF_2_25  0.08913257
## 842  Algorithm 1  UF_2_25  0.03000228
## 843  Algorithm 1  UF_2_25  0.02716640
## 844  Algorithm 1  UF_2_25  0.08948897
## 845  Algorithm 1  UF_2_25  0.02825884
## 846  Algorithm 1  UF_2_25  0.02922361
## 847  Algorithm 1  UF_2_25  0.03655307
## 848  Algorithm 1  UF_2_25  0.03663795
## 849  Algorithm 1  UF_2_25  0.09029349
## 850  Algorithm 1  UF_2_25  0.08896406
## 851  Algorithm 1  UF_2_25  0.08150821
## 852  Algorithm 1  UF_2_25  0.02836609
## 853  Algorithm 1  UF_2_25  0.03254869
## 854  Algorithm 1  UF_2_25  0.05620543
## 855  Algorithm 1  UF_2_25  0.03801131
## 856  Algorithm 1  UF_2_25  0.16415257
## 857  Algorithm 1  UF_2_25  0.02891167
## 858  Algorithm 1  UF_2_25  0.02851139
## 859  Algorithm 1  UF_2_25  0.04057341
## 860  Algorithm 1  UF_2_25  0.03147554
## 861  Algorithm 1  UF_2_25  0.03197201
## 862  Algorithm 1  UF_2_25  0.08775909
## 863  Algorithm 1  UF_2_25  0.02978161
## 864  Algorithm 1  UF_2_25  0.04187708
## 865  Algorithm 1  UF_2_25  0.14073992
## 866  Algorithm 1  UF_2_25  0.02929343
## 867  Algorithm 1  UF_2_25  0.03704238
## 868  Algorithm 1  UF_2_25  0.04884736
## 869  Algorithm 1  UF_2_25  0.02893671
## 870  Algorithm 1  UF_2_25  0.09230550
## 871  Algorithm 1  UF_2_25  0.07504791
## 872  Algorithm 1  UF_2_25  0.03303044
## 873  Algorithm 1  UF_2_25  0.03330396
## 874  Algorithm 1  UF_2_25  0.02683304
## 875  Algorithm 1  UF_2_25  0.07757438
## 876  Algorithm 1  UF_2_25  0.04600811
## 877  Algorithm 1  UF_2_25  0.02960550
## 878  Algorithm 1  UF_2_25  0.10287454
## 879  Algorithm 2  UF_2_25  0.02684243
## 880  Algorithm 2  UF_2_25  0.03190485
## 881  Algorithm 2  UF_2_25  0.03439615
## 882  Algorithm 2  UF_2_25  0.02857159
## 883  Algorithm 2  UF_2_25  0.03680506
## 884  Algorithm 2  UF_2_25  0.03706374
## 885  Algorithm 2  UF_2_25  0.03243283
## 886  Algorithm 2  UF_2_25  0.03285117
## 887  Algorithm 2  UF_2_25  0.02793493
## 888  Algorithm 2  UF_2_25  0.03237850
## 889  Algorithm 2  UF_2_25  0.03145263
## 890  Algorithm 2  UF_2_25  0.03197610
## 891  Algorithm 2  UF_2_25  0.02975796
## 892  Algorithm 2  UF_2_25  0.04239616
## 893  Algorithm 2  UF_2_25  0.03304485
## 894  Algorithm 1  UF_4_30  0.06899149
## 895  Algorithm 1  UF_4_30  0.07720830
## 896  Algorithm 1  UF_4_30  0.07680575
## 897  Algorithm 1  UF_4_30  0.06756990
## 898  Algorithm 1  UF_4_30  0.07525072
## 899  Algorithm 1  UF_4_30  0.07638802
## 900  Algorithm 1  UF_4_30  0.06946953
## 901  Algorithm 1  UF_4_30  0.08088519
## 902  Algorithm 1  UF_4_30  0.07955435
## 903  Algorithm 1  UF_4_30  0.07585220
## 904  Algorithm 1  UF_4_30  0.07337896
## 905  Algorithm 1  UF_4_30  0.07948034
## 906  Algorithm 1  UF_4_30  0.08107560
## 907  Algorithm 1  UF_4_30  0.05949978
## 908  Algorithm 1  UF_4_30  0.08358997
## 909  Algorithm 2  UF_4_30  0.06530389
## 910  Algorithm 2  UF_4_30  0.07140367
## 911  Algorithm 2  UF_4_30  0.06964194
## 912  Algorithm 2  UF_4_30  0.07042767
## 913  Algorithm 2  UF_4_30  0.07316897
## 914  Algorithm 2  UF_4_30  0.06781261
## 915  Algorithm 2  UF_4_30  0.07832749
## 916  Algorithm 2  UF_4_30  0.07586210
## 917  Algorithm 2  UF_4_30  0.07047986
## 918  Algorithm 2  UF_4_30  0.07177145
## 919  Algorithm 2  UF_4_30  0.06857085
## 920  Algorithm 2  UF_4_30  0.07746472
## 921  Algorithm 2  UF_4_30  0.07372832
## 922  Algorithm 2  UF_4_30  0.06933716
## 923  Algorithm 2  UF_4_30  0.07500989
## 924  Algorithm 1  UF_1_26  0.09800765
## 925  Algorithm 1  UF_1_26  0.09844016
## 926  Algorithm 1  UF_1_26  0.10795866
## 927  Algorithm 1  UF_1_26  0.11788732
## 928  Algorithm 1  UF_1_26  0.34710320
## 929  Algorithm 1  UF_1_26  0.20622943
## 930  Algorithm 1  UF_1_26  0.17325298
## 931  Algorithm 1  UF_1_26  0.07931058
## 932  Algorithm 1  UF_1_26  0.09534811
## 933  Algorithm 1  UF_1_26  0.21149598
## 934  Algorithm 1  UF_1_26  0.09536897
## 935  Algorithm 1  UF_1_26  0.18980243
## 936  Algorithm 1  UF_1_26  0.12961563
## 937  Algorithm 1  UF_1_26  0.13937697
## 938  Algorithm 1  UF_1_26  0.09248966
## 939  Algorithm 2  UF_1_26  0.03931478
## 940  Algorithm 2  UF_1_26  0.05013548
## 941  Algorithm 2  UF_1_26  0.07533751
## 942  Algorithm 2  UF_1_26  0.06090974
## 943  Algorithm 2  UF_1_26  0.04675321
## 944  Algorithm 2  UF_1_26  0.05605795
## 945  Algorithm 2  UF_1_26  0.03958948
## 946  Algorithm 2  UF_1_26  0.07504439
## 947  Algorithm 2  UF_1_26  0.05222853
## 948  Algorithm 2  UF_1_26  0.05639071
## 949  Algorithm 2  UF_1_26  0.03492968
## 950  Algorithm 2  UF_1_26  0.04779897
## 951  Algorithm 2  UF_1_26  0.05203207
## 952  Algorithm 2  UF_1_26  0.04245498
## 953  Algorithm 2  UF_1_26  0.03712472
## 954  Algorithm 1  UF_2_18  0.02687181
## 955  Algorithm 1  UF_2_18  0.04470041
## 956  Algorithm 1  UF_2_18  0.04038913
## 957  Algorithm 1  UF_2_18  0.08282422
## 958  Algorithm 1  UF_2_18  0.06111107
## 959  Algorithm 1  UF_2_18  0.03114082
## 960  Algorithm 1  UF_2_18  0.10277213
## 961  Algorithm 1  UF_2_18  0.02728864
## 962  Algorithm 1  UF_2_18  0.02974868
## 963  Algorithm 1  UF_2_18  0.05043122
## 964  Algorithm 1  UF_2_18  0.09455486
## 965  Algorithm 1  UF_2_18  0.03265708
## 966  Algorithm 1  UF_2_18  0.03370670
## 967  Algorithm 1  UF_2_18  0.05026110
## 968  Algorithm 1  UF_2_18  0.02757064
## 969  Algorithm 1  UF_2_18  0.02834260
## 970  Algorithm 1  UF_2_18  0.02361871
## 971  Algorithm 1  UF_2_18  0.07467500
## 972  Algorithm 1  UF_2_18  0.08567590
## 973  Algorithm 1  UF_2_18  0.03644772
## 974  Algorithm 1  UF_2_18  0.03231066
## 975  Algorithm 1  UF_2_18  0.08661764
## 976  Algorithm 1  UF_2_18  0.04267055
## 977  Algorithm 1  UF_2_18  0.10098167
## 978  Algorithm 1  UF_2_18  0.02716095
## 979  Algorithm 1  UF_2_18  0.07983843
## 980  Algorithm 1  UF_2_18  0.03395053
## 981  Algorithm 1  UF_2_18  0.02831690
## 982  Algorithm 1  UF_2_18  0.02994789
## 983  Algorithm 1  UF_2_18  0.03755208
## 984  Algorithm 1  UF_2_18  0.09684179
## 985  Algorithm 1  UF_2_18  0.04164197
## 986  Algorithm 1  UF_2_18  0.02795233
## 987  Algorithm 1  UF_2_18  0.04156414
## 988  Algorithm 1  UF_2_18  0.02644570
## 989  Algorithm 1  UF_2_18  0.08494499
## 990  Algorithm 1  UF_2_18  0.02884323
## 991  Algorithm 1  UF_2_18  0.07137627
## 992  Algorithm 1  UF_2_18  0.07203910
## 993  Algorithm 1  UF_2_18  0.05883526
## 994  Algorithm 2  UF_2_18  0.02615053
## 995  Algorithm 2  UF_2_18  0.02608111
## 996  Algorithm 2  UF_2_18  0.03269484
## 997  Algorithm 2  UF_2_18  0.03109422
## 998  Algorithm 2  UF_2_18  0.02258887
## 999  Algorithm 2  UF_2_18  0.02382149
## 1000 Algorithm 2  UF_2_18  0.02251701
## 1001 Algorithm 2  UF_2_18  0.01995230
## 1002 Algorithm 2  UF_2_18  0.02419279
## 1003 Algorithm 2  UF_2_18  0.02852803
## 1004 Algorithm 2  UF_2_18  0.03076944
## 1005 Algorithm 2  UF_2_18  0.02701743
## 1006 Algorithm 2  UF_2_18  0.03733098
## 1007 Algorithm 2  UF_2_18  0.02783179
## 1008 Algorithm 2  UF_2_18  0.02942199
## 1009 Algorithm 1  UF_7_36  0.36283397
## 1010 Algorithm 1  UF_7_36  0.40367524
## 1011 Algorithm 1  UF_7_36  0.33230389
## 1012 Algorithm 1  UF_7_36  0.36183014
## 1013 Algorithm 1  UF_7_36  0.70444665
## 1014 Algorithm 1  UF_7_36  0.35296129
## 1015 Algorithm 1  UF_7_36  0.34737677
## 1016 Algorithm 1  UF_7_36  0.40652206
## 1017 Algorithm 1  UF_7_36  0.53498524
## 1018 Algorithm 1  UF_7_36  0.70506956
## 1019 Algorithm 1  UF_7_36  0.59840072
## 1020 Algorithm 1  UF_7_36  0.36252971
## 1021 Algorithm 1  UF_7_36  0.54043714
## 1022 Algorithm 1  UF_7_36  0.58976566
## 1023 Algorithm 1  UF_7_36  0.70370474
## 1024 Algorithm 2  UF_7_36  0.03128883
## 1025 Algorithm 2  UF_7_36  0.02150318
## 1026 Algorithm 2  UF_7_36  0.02589815
## 1027 Algorithm 2  UF_7_36  0.03524419
## 1028 Algorithm 2  UF_7_36  0.03403753
## 1029 Algorithm 2  UF_7_36  0.01864268
## 1030 Algorithm 2  UF_7_36  0.01950871
## 1031 Algorithm 2  UF_7_36  0.02330434
## 1032 Algorithm 2  UF_7_36  0.01984352
## 1033 Algorithm 2  UF_7_36  0.04035095
## 1034 Algorithm 2  UF_7_36  0.02458901
## 1035 Algorithm 2  UF_7_36  0.02045718
## 1036 Algorithm 2  UF_7_36  0.17989645
## 1037 Algorithm 2  UF_7_36  0.03146327
## 1038 Algorithm 2  UF_7_36  0.03549736
## 1039 Algorithm 1  UF_4_18  0.07135943
## 1040 Algorithm 1  UF_4_18  0.07013495
## 1041 Algorithm 1  UF_4_18  0.07430603
## 1042 Algorithm 1  UF_4_18  0.07419233
## 1043 Algorithm 1  UF_4_18  0.06913871
## 1044 Algorithm 1  UF_4_18  0.07693806
## 1045 Algorithm 1  UF_4_18  0.06906136
## 1046 Algorithm 1  UF_4_18  0.07550941
## 1047 Algorithm 1  UF_4_18  0.09182527
## 1048 Algorithm 1  UF_4_18  0.07864215
## 1049 Algorithm 1  UF_4_18  0.06080577
## 1050 Algorithm 1  UF_4_18  0.06465215
## 1051 Algorithm 1  UF_4_18  0.06819287
## 1052 Algorithm 1  UF_4_18  0.07047517
## 1053 Algorithm 1  UF_4_18  0.06993621
## 1054 Algorithm 2  UF_4_18  0.05535791
## 1055 Algorithm 2  UF_4_18  0.05696925
## 1056 Algorithm 2  UF_4_18  0.06177844
## 1057 Algorithm 2  UF_4_18  0.06123496
## 1058 Algorithm 2  UF_4_18  0.05534115
## 1059 Algorithm 2  UF_4_18  0.05968352
## 1060 Algorithm 2  UF_4_18  0.05933566
## 1061 Algorithm 2  UF_4_18  0.06463361
## 1062 Algorithm 2  UF_4_18  0.06246089
## 1063 Algorithm 2  UF_4_18  0.06538523
## 1064 Algorithm 2  UF_4_18  0.06283626
## 1065 Algorithm 2  UF_4_18  0.06598971
## 1066 Algorithm 2  UF_4_18  0.05529079
## 1067 Algorithm 2  UF_4_18  0.06206258
## 1068 Algorithm 2  UF_4_18  0.05745316
## 1069 Algorithm 1  UF_2_34  0.03935866
## 1070 Algorithm 1  UF_2_34  0.07819420
## 1071 Algorithm 1  UF_2_34  0.09700939
## 1072 Algorithm 1  UF_2_34  0.03082474
## 1073 Algorithm 1  UF_2_34  0.04957996
## 1074 Algorithm 1  UF_2_34  0.05721314
## 1075 Algorithm 1  UF_2_34  0.06623194
## 1076 Algorithm 1  UF_2_34  0.08118620
## 1077 Algorithm 1  UF_2_34  0.02973160
## 1078 Algorithm 1  UF_2_34  0.02874883
## 1079 Algorithm 1  UF_2_34  0.04392976
## 1080 Algorithm 1  UF_2_34  0.03259528
## 1081 Algorithm 1  UF_2_34  0.08734551
## 1082 Algorithm 1  UF_2_34  0.09690031
## 1083 Algorithm 1  UF_2_34  0.09311586
## 1084 Algorithm 1  UF_2_34  0.04637322
## 1085 Algorithm 1  UF_2_34  0.03153827
## 1086 Algorithm 1  UF_2_34  0.08299036
## 1087 Algorithm 1  UF_2_34  0.03120060
## 1088 Algorithm 1  UF_2_34  0.18334988
## 1089 Algorithm 1  UF_2_34  0.05040548
## 1090 Algorithm 1  UF_2_34  0.09998270
## 1091 Algorithm 1  UF_2_34  0.03534562
## 1092 Algorithm 1  UF_2_34  0.04231811
## 1093 Algorithm 1  UF_2_34  0.07963446
## 1094 Algorithm 1  UF_2_34  0.07929059
## 1095 Algorithm 1  UF_2_34  0.03554327
## 1096 Algorithm 1  UF_2_34  0.03558154
## 1097 Algorithm 1  UF_2_34  0.10858877
## 1098 Algorithm 1  UF_2_34  0.05635853
## 1099 Algorithm 1  UF_2_34  0.05968655
## 1100 Algorithm 1  UF_2_34  0.03594967
## 1101 Algorithm 1  UF_2_34  0.04471301
## 1102 Algorithm 1  UF_2_34  0.08177387
## 1103 Algorithm 1  UF_2_34  0.03087765
## 1104 Algorithm 1  UF_2_34  0.04649282
## 1105 Algorithm 1  UF_2_34  0.09356856
## 1106 Algorithm 1  UF_2_34  0.09093488
## 1107 Algorithm 1  UF_2_34  0.03751002
## 1108 Algorithm 1  UF_2_34  0.03027751
## 1109 Algorithm 1  UF_2_34  0.09674447
## 1110 Algorithm 1  UF_2_34  0.03508532
## 1111 Algorithm 1  UF_2_34  0.06576767
## 1112 Algorithm 1  UF_2_34  0.09281265
## 1113 Algorithm 2  UF_2_34  0.04243291
## 1114 Algorithm 2  UF_2_34  0.03757650
## 1115 Algorithm 2  UF_2_34  0.03505778
## 1116 Algorithm 2  UF_2_34  0.03506574
## 1117 Algorithm 2  UF_2_34  0.03077859
## 1118 Algorithm 2  UF_2_34  0.03448647
## 1119 Algorithm 2  UF_2_34  0.03930687
## 1120 Algorithm 2  UF_2_34  0.03616759
## 1121 Algorithm 2  UF_2_34  0.03385390
## 1122 Algorithm 2  UF_2_34  0.03972591
## 1123 Algorithm 2  UF_2_34  0.03805128
## 1124 Algorithm 2  UF_2_34  0.03985157
## 1125 Algorithm 2  UF_2_34  0.04417166
## 1126 Algorithm 2  UF_2_34  0.04663509
## 1127 Algorithm 2  UF_2_34  0.03264524
## 1128 Algorithm 1  UF_2_39  0.03093071
## 1129 Algorithm 1  UF_2_39  0.03128476
## 1130 Algorithm 1  UF_2_39  0.04815801
## 1131 Algorithm 1  UF_2_39  0.03090203
## 1132 Algorithm 1  UF_2_39  0.10463275
## 1133 Algorithm 1  UF_2_39  0.09181518
## 1134 Algorithm 1  UF_2_39  0.04536654
## 1135 Algorithm 1  UF_2_39  0.18460527
## 1136 Algorithm 1  UF_2_39  0.10494294
## 1137 Algorithm 1  UF_2_39  0.03103229
## 1138 Algorithm 1  UF_2_39  0.03504527
## 1139 Algorithm 1  UF_2_39  0.03132444
## 1140 Algorithm 1  UF_2_39  0.03490366
## 1141 Algorithm 1  UF_2_39  0.03235873
## 1142 Algorithm 1  UF_2_39  0.07876201
## 1143 Algorithm 1  UF_2_39  0.05028283
## 1144 Algorithm 1  UF_2_39  0.03072089
## 1145 Algorithm 1  UF_2_39  0.03189853
## 1146 Algorithm 1  UF_2_39  0.04190666
## 1147 Algorithm 1  UF_2_39  0.05610102
## 1148 Algorithm 1  UF_2_39  0.08396469
## 1149 Algorithm 1  UF_2_39  0.08112024
## 1150 Algorithm 1  UF_2_39  0.03548716
## 1151 Algorithm 1  UF_2_39  0.04683573
## 1152 Algorithm 1  UF_2_39  0.08051210
## 1153 Algorithm 1  UF_2_39  0.08545027
## 1154 Algorithm 1  UF_2_39  0.03319449
## 1155 Algorithm 1  UF_2_39  0.03144649
## 1156 Algorithm 1  UF_2_39  0.09651847
## 1157 Algorithm 1  UF_2_39  0.09188922
## 1158 Algorithm 1  UF_2_39  0.03390673
## 1159 Algorithm 1  UF_2_39  0.03026143
## 1160 Algorithm 1  UF_2_39  0.08513249
## 1161 Algorithm 1  UF_2_39  0.03738877
## 1162 Algorithm 1  UF_2_39  0.08102169
## 1163 Algorithm 1  UF_2_39  0.03345063
## 1164 Algorithm 1  UF_2_39  0.03409181
## 1165 Algorithm 1  UF_2_39  0.03409215
## 1166 Algorithm 1  UF_2_39  0.03263014
## 1167 Algorithm 1  UF_2_39  0.03184338
## 1168 Algorithm 1  UF_2_39  0.03495952
## 1169 Algorithm 1  UF_2_39  0.03437677
## 1170 Algorithm 1  UF_2_39  0.03157692
## 1171 Algorithm 1  UF_2_39  0.03203987
## 1172 Algorithm 1  UF_2_39  0.03059812
## 1173 Algorithm 1  UF_2_39  0.03374986
## 1174 Algorithm 1  UF_2_39  0.08791002
## 1175 Algorithm 1  UF_2_39  0.04708382
## 1176 Algorithm 1  UF_2_39  0.05189724
## 1177 Algorithm 1  UF_2_39  0.10205945
## 1178 Algorithm 1  UF_2_39  0.08159649
## 1179 Algorithm 1  UF_2_39  0.03812531
## 1180 Algorithm 1  UF_2_39  0.03429483
## 1181 Algorithm 1  UF_2_39  0.04043496
## 1182 Algorithm 1  UF_2_39  0.03538721
## 1183 Algorithm 1  UF_2_39  0.08576006
## 1184 Algorithm 1  UF_2_39  0.06129355
## 1185 Algorithm 1  UF_2_39  0.05469734
## 1186 Algorithm 1  UF_2_39  0.02967428
## 1187 Algorithm 1  UF_2_39  0.03177627
## 1188 Algorithm 1  UF_2_39  0.03393455
## 1189 Algorithm 1  UF_2_39  0.03668224
## 1190 Algorithm 1  UF_2_39  0.08758847
## 1191 Algorithm 1  UF_2_39  0.03917503
## 1192 Algorithm 1  UF_2_39  0.03277562
## 1193 Algorithm 1  UF_2_39  0.15134399
## 1194 Algorithm 1  UF_2_39  0.08962600
## 1195 Algorithm 1  UF_2_39  0.03575246
## 1196 Algorithm 1  UF_2_39  0.10099779
## 1197 Algorithm 1  UF_2_39  0.03481408
## 1198 Algorithm 1  UF_2_39  0.08060940
## 1199 Algorithm 2  UF_2_39  0.03322346
## 1200 Algorithm 2  UF_2_39  0.03954014
## 1201 Algorithm 2  UF_2_39  0.04192228
## 1202 Algorithm 2  UF_2_39  0.03570221
## 1203 Algorithm 2  UF_2_39  0.04157639
## 1204 Algorithm 2  UF_2_39  0.04134390
## 1205 Algorithm 2  UF_2_39  0.03803307
## 1206 Algorithm 2  UF_2_39  0.03660917
## 1207 Algorithm 2  UF_2_39  0.03673834
## 1208 Algorithm 2  UF_2_39  0.04106130
## 1209 Algorithm 2  UF_2_39  0.03687262
## 1210 Algorithm 2  UF_2_39  0.04486406
## 1211 Algorithm 2  UF_2_39  0.03669031
## 1212 Algorithm 2  UF_2_39  0.03849836
## 1213 Algorithm 2  UF_2_39  0.04402458
## 1214 Algorithm 1  UF_5_17  0.35287259
## 1215 Algorithm 1  UF_5_17  0.23955321
## 1216 Algorithm 1  UF_5_17  0.44808876
## 1217 Algorithm 1  UF_5_17  0.29055995
## 1218 Algorithm 1  UF_5_17  0.27601146
## 1219 Algorithm 1  UF_5_17  0.40566888
## 1220 Algorithm 1  UF_5_17  0.25603699
## 1221 Algorithm 1  UF_5_17  0.29897739
## 1222 Algorithm 1  UF_5_17  0.45121370
## 1223 Algorithm 1  UF_5_17  0.37788127
## 1224 Algorithm 1  UF_5_17  0.32468823
## 1225 Algorithm 1  UF_5_17  0.22704101
## 1226 Algorithm 1  UF_5_17  0.38710819
## 1227 Algorithm 1  UF_5_17  0.43793479
## 1228 Algorithm 1  UF_5_17  0.33816441
## 1229 Algorithm 1  UF_5_17  0.36180104
## 1230 Algorithm 1  UF_5_17  0.46883720
## 1231 Algorithm 1  UF_5_17  0.42541190
## 1232 Algorithm 1  UF_5_17  0.33051862
## 1233 Algorithm 1  UF_5_17  0.39578070
## 1234 Algorithm 1  UF_5_17  0.40019439
## 1235 Algorithm 1  UF_5_17  0.33807370
## 1236 Algorithm 1  UF_5_17  0.37357954
## 1237 Algorithm 1  UF_5_17  0.38312285
## 1238 Algorithm 1  UF_5_17  0.36194764
## 1239 Algorithm 1  UF_5_17  0.30114009
## 1240 Algorithm 1  UF_5_17  0.42653016
## 1241 Algorithm 1  UF_5_17  0.24249536
## 1242 Algorithm 1  UF_5_17  0.70480685
## 1243 Algorithm 1  UF_5_17  0.34341075
## 1244 Algorithm 1  UF_5_17  0.39898267
## 1245 Algorithm 1  UF_5_17  0.43157560
## 1246 Algorithm 1  UF_5_17  0.39584332
## 1247 Algorithm 1  UF_5_17  0.36481683
## 1248 Algorithm 1  UF_5_17  0.32330553
## 1249 Algorithm 1  UF_5_17  0.31599173
## 1250 Algorithm 1  UF_5_17  0.44772821
## 1251 Algorithm 1  UF_5_17  0.39061294
## 1252 Algorithm 1  UF_5_17  0.36467252
## 1253 Algorithm 1  UF_5_17  0.33932665
## 1254 Algorithm 1  UF_5_17  0.28534739
## 1255 Algorithm 1  UF_5_17  0.36039221
## 1256 Algorithm 1  UF_5_17  0.37996457
## 1257 Algorithm 1  UF_5_17  0.38846563
## 1258 Algorithm 1  UF_5_17  0.17855765
## 1259 Algorithm 1  UF_5_17  0.34446971
## 1260 Algorithm 1  UF_5_17  0.34257744
## 1261 Algorithm 1  UF_5_17  0.47739378
## 1262 Algorithm 1  UF_5_17  0.23256287
## 1263 Algorithm 1  UF_5_17  0.38549912
## 1264 Algorithm 1  UF_5_17  0.28974813
## 1265 Algorithm 1  UF_5_17  0.20723410
## 1266 Algorithm 1  UF_5_17  0.37120226
## 1267 Algorithm 1  UF_5_17  0.24297097
## 1268 Algorithm 1  UF_5_17  0.45457684
## 1269 Algorithm 1  UF_5_17  0.45458153
## 1270 Algorithm 1  UF_5_17  0.48305714
## 1271 Algorithm 1  UF_5_17  0.64430561
## 1272 Algorithm 1  UF_5_17  0.34296329
## 1273 Algorithm 1  UF_5_17  0.29996400
## 1274 Algorithm 1  UF_5_17  0.38479061
## 1275 Algorithm 1  UF_5_17  0.41743189
## 1276 Algorithm 1  UF_5_17  0.23649963
## 1277 Algorithm 1  UF_5_17  0.37345843
## 1278 Algorithm 1  UF_5_17  0.23479365
## 1279 Algorithm 1  UF_5_17  0.33933905
## 1280 Algorithm 1  UF_5_17  0.44129592
## 1281 Algorithm 1  UF_5_17  0.21286555
## 1282 Algorithm 1  UF_5_17  0.35341553
## 1283 Algorithm 1  UF_5_17  0.39797876
## 1284 Algorithm 1  UF_5_17  0.40745384
## 1285 Algorithm 1  UF_5_17  0.38616746
## 1286 Algorithm 1  UF_5_17  0.53282950
## 1287 Algorithm 1  UF_5_17  0.28523344
## 1288 Algorithm 1  UF_5_17  0.30059312
## 1289 Algorithm 1  UF_5_17  0.50081472
## 1290 Algorithm 1  UF_5_17  0.21907968
## 1291 Algorithm 1  UF_5_17  0.40638490
## 1292 Algorithm 1  UF_5_17  0.38505630
## 1293 Algorithm 1  UF_5_17  0.48357905
## 1294 Algorithm 1  UF_5_17  0.30180354
## 1295 Algorithm 1  UF_5_17  0.33863041
## 1296 Algorithm 1  UF_5_17  0.37856446
## 1297 Algorithm 2  UF_5_17  0.18642884
## 1298 Algorithm 2  UF_5_17  0.45840918
## 1299 Algorithm 2  UF_5_17  0.65165995
## 1300 Algorithm 2  UF_5_17  0.52658541
## 1301 Algorithm 2  UF_5_17  0.54366971
## 1302 Algorithm 2  UF_5_17  0.49814493
## 1303 Algorithm 2  UF_5_17  0.43816353
## 1304 Algorithm 2  UF_5_17  0.59757003
## 1305 Algorithm 2  UF_5_17  0.74168131
## 1306 Algorithm 2  UF_5_17  0.45359961
## 1307 Algorithm 2  UF_5_17  0.50060983
## 1308 Algorithm 2  UF_5_17  0.52689337
## 1309 Algorithm 2  UF_5_17  0.74767978
## 1310 Algorithm 2  UF_5_17  0.67485451
## 1311 Algorithm 2  UF_5_17  0.40360537
## 1312 Algorithm 2  UF_5_17  0.61701156
## 1313 Algorithm 2  UF_5_17  0.61981652
## 1314 Algorithm 2  UF_5_17  0.62072889
## 1315 Algorithm 2  UF_5_17  0.54576453
## 1316 Algorithm 2  UF_5_17  0.49104053
## 1317 Algorithm 2  UF_5_17  0.65863458
## 1318 Algorithm 2  UF_5_17  0.66284927
## 1319 Algorithm 2  UF_5_17  0.55657131
## 1320 Algorithm 2  UF_5_17  0.65968516
## 1321 Algorithm 2  UF_5_17  1.00446327
## 1322 Algorithm 2  UF_5_17  0.48181004
## 1323 Algorithm 2  UF_5_17  0.66778057
## 1324 Algorithm 2  UF_5_17  0.46405181
## 1325 Algorithm 2  UF_5_17  0.48923268
## 1326 Algorithm 2  UF_5_17  0.41676222
## 1327 Algorithm 2  UF_5_17  0.50635565
## 1328 Algorithm 2  UF_5_17  0.61779641
## 1329 Algorithm 2  UF_5_17  0.47943359
## 1330 Algorithm 2  UF_5_17  0.50795743
## 1331 Algorithm 2  UF_5_17  0.45866081
## 1332 Algorithm 2  UF_5_17  0.54504615
## 1333 Algorithm 2  UF_5_17  0.62374748
## 1334 Algorithm 2  UF_5_17  0.73202373
## 1335 Algorithm 2  UF_5_17  0.59857729
## 1336 Algorithm 2  UF_5_17  0.13138399
## 1337 Algorithm 2  UF_5_17  0.59902847
## 1338 Algorithm 2  UF_5_17  0.60045652
## 1339 Algorithm 2  UF_5_17  0.57912643
## 1340 Algorithm 2  UF_5_17  0.68380498
## 1341 Algorithm 2  UF_5_17  0.56406553
## 1342 Algorithm 2  UF_5_17  0.66660829
## 1343 Algorithm 2  UF_5_17  0.47016247
## 1344 Algorithm 2  UF_5_17  0.45797021
## 1345 Algorithm 2  UF_5_17  0.49714721
## 1346 Algorithm 2  UF_5_17  0.41344616
## 1347 Algorithm 2  UF_5_17  0.31851151
## 1348 Algorithm 2  UF_5_17  0.71363641
## 1349 Algorithm 2  UF_5_17  0.50271066
## 1350 Algorithm 2  UF_5_17  0.26926169
## 1351 Algorithm 2  UF_5_17  0.53700330
## 1352 Algorithm 2  UF_5_17  0.41655963
## 1353 Algorithm 2  UF_5_17  0.82841671
## 1354 Algorithm 2  UF_5_17  0.64589890
## 1355 Algorithm 2  UF_5_17  0.49846446
## 1356 Algorithm 2  UF_5_17  0.40414764
## 1357 Algorithm 2  UF_5_17  0.54591384
## 1358 Algorithm 2  UF_5_17  0.62949797
## 1359 Algorithm 2  UF_5_17  0.58995666
## 1360 Algorithm 2  UF_5_17  0.54381024
## 1361 Algorithm 2  UF_5_17  0.56841473
## 1362 Algorithm 2  UF_5_17  0.65815717
## 1363 Algorithm 2  UF_5_17  0.59287434
## 1364 Algorithm 2  UF_5_17  0.62762074
## 1365 Algorithm 2  UF_5_17  0.64151323
## 1366 Algorithm 2  UF_5_17  0.62437241
## 1367 Algorithm 2  UF_5_17  0.57720162
## 1368 Algorithm 2  UF_5_17  0.52699616
## 1369 Algorithm 2  UF_5_17  0.53580697
## 1370 Algorithm 2  UF_5_17  0.31126052
## 1371 Algorithm 2  UF_5_17  0.49997230
## 1372 Algorithm 2  UF_5_17  0.49031109
## 1373 Algorithm 2  UF_5_17  0.64116542
## 1374 Algorithm 2  UF_5_17  0.59045949
## 1375 Algorithm 2  UF_5_17  0.37141491
## 1376 Algorithm 2  UF_5_17  0.47768511
## 1377 Algorithm 2  UF_5_17  0.69752893
## 1378 Algorithm 2  UF_5_17  0.60259944
## 1379 Algorithm 2  UF_5_17  0.62453110
## 1380 Algorithm 2  UF_5_17  0.76626362
## 1381 Algorithm 2  UF_5_17  0.20590968
## 1382 Algorithm 2  UF_5_17  0.46496812
## 1383 Algorithm 2  UF_5_17  0.40892905
## 1384 Algorithm 2  UF_5_17  0.44690239
## 1385 Algorithm 2  UF_5_17  0.67317536
## 1386 Algorithm 2  UF_5_17  0.39906688
## 1387 Algorithm 2  UF_5_17  0.60521195
## 1388 Algorithm 2  UF_5_17  0.70934562
## 1389 Algorithm 2  UF_5_17  0.23645970
## 1390 Algorithm 2  UF_5_17  0.35613211
## 1391 Algorithm 2  UF_5_17  0.38341422
## 1392 Algorithm 2  UF_5_17  0.45122518
## 1393 Algorithm 2  UF_5_17  0.43716721
## 1394 Algorithm 2  UF_5_17  0.48578136
## 1395 Algorithm 2  UF_5_17  0.20558550
## 1396 Algorithm 2  UF_5_17  0.25669954
## 1397 Algorithm 2  UF_5_17  0.65335789
## 1398 Algorithm 2  UF_5_17  0.53936673
## 1399 Algorithm 2  UF_5_17  0.74466178
## 1400 Algorithm 2  UF_5_17  0.49718147
## 1401 Algorithm 2  UF_5_17  0.53996575
## 1402 Algorithm 2  UF_5_17  0.61716003
## 1403 Algorithm 2  UF_5_17  0.66077972
## 1404 Algorithm 2  UF_5_17  0.43631220
## 1405 Algorithm 2  UF_5_17  0.62054483
## 1406 Algorithm 2  UF_5_17  0.43674646
## 1407 Algorithm 2  UF_5_17  0.68755279
## 1408 Algorithm 2  UF_5_17  0.35336726
## 1409 Algorithm 2  UF_5_17  0.52298100
## 1410 Algorithm 2  UF_5_17  0.67290285
## 1411 Algorithm 2  UF_5_17  0.46982332
## 1412 Algorithm 2  UF_5_17  0.14577856
## 1413 Algorithm 2  UF_5_17  0.52136782
## 1414 Algorithm 1  UF_3_15  0.29000648
## 1415 Algorithm 1  UF_3_15  0.22259278
## 1416 Algorithm 1  UF_3_15  0.23147668
## 1417 Algorithm 1  UF_3_15  0.22798573
## 1418 Algorithm 1  UF_3_15  0.20530921
## 1419 Algorithm 1  UF_3_15  0.30285258
## 1420 Algorithm 1  UF_3_15  0.22169861
## 1421 Algorithm 1  UF_3_15  0.25831919
## 1422 Algorithm 1  UF_3_15  0.21497320
## 1423 Algorithm 1  UF_3_15  0.25033231
## 1424 Algorithm 1  UF_3_15  0.24111625
## 1425 Algorithm 1  UF_3_15  0.15668966
## 1426 Algorithm 1  UF_3_15  0.37327553
## 1427 Algorithm 1  UF_3_15  0.23532235
## 1428 Algorithm 1  UF_3_15  0.23566732
## 1429 Algorithm 1  UF_3_15  0.22990281
## 1430 Algorithm 1  UF_3_15  0.26954363
## 1431 Algorithm 1  UF_3_15  0.22954214
## 1432 Algorithm 1  UF_3_15  0.27162609
## 1433 Algorithm 1  UF_3_15  0.20280339
## 1434 Algorithm 1  UF_3_15  0.21631901
## 1435 Algorithm 1  UF_3_15  0.20770243
## 1436 Algorithm 1  UF_3_15  0.39279428
## 1437 Algorithm 1  UF_3_15  0.25928228
## 1438 Algorithm 1  UF_3_15  0.19456376
## 1439 Algorithm 1  UF_3_15  0.36261478
## 1440 Algorithm 1  UF_3_15  0.16612928
## 1441 Algorithm 1  UF_3_15  0.19695997
## 1442 Algorithm 1  UF_3_15  0.33446922
## 1443 Algorithm 1  UF_3_15  0.19599918
## 1444 Algorithm 1  UF_3_15  0.20891803
## 1445 Algorithm 1  UF_3_15  0.22701278
## 1446 Algorithm 1  UF_3_15  0.25117497
## 1447 Algorithm 1  UF_3_15  0.20647033
## 1448 Algorithm 1  UF_3_15  0.34889126
## 1449 Algorithm 1  UF_3_15  0.24149505
## 1450 Algorithm 1  UF_3_15  0.25693436
## 1451 Algorithm 1  UF_3_15  0.24034821
## 1452 Algorithm 1  UF_3_15  0.28703108
## 1453 Algorithm 1  UF_3_15  0.19247202
## 1454 Algorithm 2  UF_3_15  0.18284190
## 1455 Algorithm 2  UF_3_15  0.28981168
## 1456 Algorithm 2  UF_3_15  0.25465209
## 1457 Algorithm 2  UF_3_15  0.15269134
## 1458 Algorithm 2  UF_3_15  0.29591931
## 1459 Algorithm 2  UF_3_15  0.14500138
## 1460 Algorithm 2  UF_3_15  0.34013984
## 1461 Algorithm 2  UF_3_15  0.13839694
## 1462 Algorithm 2  UF_3_15  0.18703447
## 1463 Algorithm 2  UF_3_15  0.17951046
## 1464 Algorithm 2  UF_3_15  0.24942870
## 1465 Algorithm 2  UF_3_15  0.29927756
## 1466 Algorithm 2  UF_3_15  0.17061441
## 1467 Algorithm 2  UF_3_15  0.22619912
## 1468 Algorithm 2  UF_3_15  0.33736364
## 1469 Algorithm 2  UF_3_15  0.17558850
## 1470 Algorithm 2  UF_3_15  0.21687428
## 1471 Algorithm 2  UF_3_15  0.16022485
## 1472 Algorithm 2  UF_3_15  0.31894183
## 1473 Algorithm 2  UF_3_15  0.21865929
## 1474 Algorithm 2  UF_3_15  0.15709828
## 1475 Algorithm 2  UF_3_15  0.31932754
## 1476 Algorithm 2  UF_3_15  0.14872501
## 1477 Algorithm 2  UF_3_15  0.18444741
## 1478 Algorithm 2  UF_3_15  0.31241434
## 1479 Algorithm 2  UF_3_15  0.18095746
## 1480 Algorithm 2  UF_3_15  0.21044794
## 1481 Algorithm 2  UF_3_15  0.33367269
## 1482 Algorithm 2  UF_3_15  0.19360129
## 1483 Algorithm 2  UF_3_15  0.16201441
## 1484 Algorithm 2  UF_3_15  0.08611372
## 1485 Algorithm 2  UF_3_15  0.25739255
## 1486 Algorithm 2  UF_3_15  0.21059341
## 1487 Algorithm 2  UF_3_15  0.13390572
## 1488 Algorithm 2  UF_3_15  0.34361741
## 1489 Algorithm 2  UF_3_15  0.33996019
## 1490 Algorithm 2  UF_3_15  0.32811894
## 1491 Algorithm 2  UF_3_15  0.19979762
## 1492 Algorithm 2  UF_3_15  0.25905367
## 1493 Algorithm 2  UF_3_15  0.24948943
## 1494 Algorithm 2  UF_3_15  0.11591314
## 1495 Algorithm 2  UF_3_15  0.33283752
## 1496 Algorithm 2  UF_3_15  0.14311561
## 1497 Algorithm 2  UF_3_15  0.34373873
## 1498 Algorithm 2  UF_3_15  0.21071887
## 1499 Algorithm 2  UF_3_15  0.15733441
## 1500 Algorithm 2  UF_3_15  0.24760815
## 1501 Algorithm 2  UF_3_15  0.27202924
## 1502 Algorithm 2  UF_3_15  0.32514558
## 1503 Algorithm 2  UF_3_15  0.21386848
## 1504 Algorithm 2  UF_3_15  0.16298839
## 1505 Algorithm 2  UF_3_15  0.14661225
## 1506 Algorithm 2  UF_3_15  0.24419356
## 1507 Algorithm 1  UF_4_16  0.05493201
## 1508 Algorithm 1  UF_4_16  0.06336805
## 1509 Algorithm 1  UF_4_16  0.06559547
## 1510 Algorithm 1  UF_4_16  0.06641982
## 1511 Algorithm 1  UF_4_16  0.06915655
## 1512 Algorithm 1  UF_4_16  0.06278642
## 1513 Algorithm 1  UF_4_16  0.05877090
## 1514 Algorithm 1  UF_4_16  0.07659814
## 1515 Algorithm 1  UF_4_16  0.06587625
## 1516 Algorithm 1  UF_4_16  0.06599309
## 1517 Algorithm 1  UF_4_16  0.07055077
## 1518 Algorithm 1  UF_4_16  0.06099361
## 1519 Algorithm 1  UF_4_16  0.07363596
## 1520 Algorithm 1  UF_4_16  0.07718932
## 1521 Algorithm 1  UF_4_16  0.07200404
## 1522 Algorithm 2  UF_4_16  0.06100680
## 1523 Algorithm 2  UF_4_16  0.06428126
## 1524 Algorithm 2  UF_4_16  0.06280575
## 1525 Algorithm 2  UF_4_16  0.06691924
## 1526 Algorithm 2  UF_4_16  0.05987861
## 1527 Algorithm 2  UF_4_16  0.06097958
## 1528 Algorithm 2  UF_4_16  0.05431484
## 1529 Algorithm 2  UF_4_16  0.05652381
## 1530 Algorithm 2  UF_4_16  0.05282301
## 1531 Algorithm 2  UF_4_16  0.05645443
## 1532 Algorithm 2  UF_4_16  0.05760218
## 1533 Algorithm 2  UF_4_16  0.05821081
## 1534 Algorithm 2  UF_4_16  0.06148058
## 1535 Algorithm 2  UF_4_16  0.06208412
## 1536 Algorithm 2  UF_4_16  0.05698077
## 1537 Algorithm 1  UF_7_18  0.34966204
## 1538 Algorithm 1  UF_7_18  0.36028843
## 1539 Algorithm 1  UF_7_18  0.34324295
## 1540 Algorithm 1  UF_7_18  0.35034670
## 1541 Algorithm 1  UF_7_18  0.34665304
## 1542 Algorithm 1  UF_7_18  0.35573576
## 1543 Algorithm 1  UF_7_18  0.34957461
## 1544 Algorithm 1  UF_7_18  0.34294613
## 1545 Algorithm 1  UF_7_18  0.35098644
## 1546 Algorithm 1  UF_7_18  0.35697207
## 1547 Algorithm 1  UF_7_18  0.36582900
## 1548 Algorithm 1  UF_7_18  0.65889623
## 1549 Algorithm 1  UF_7_18  0.60107017
## 1550 Algorithm 1  UF_7_18  0.30178980
## 1551 Algorithm 1  UF_7_18  0.33171650
## 1552 Algorithm 1  UF_7_18  0.33821842
## 1553 Algorithm 1  UF_7_18  0.35081516
## 1554 Algorithm 1  UF_7_18  0.36642865
## 1555 Algorithm 1  UF_7_18  0.70330062
## 1556 Algorithm 1  UF_7_18  0.31814677
## 1557 Algorithm 1  UF_7_18  0.46613831
## 1558 Algorithm 1  UF_7_18  0.32364889
## 1559 Algorithm 1  UF_7_18  0.54420201
## 1560 Algorithm 1  UF_7_18  0.48376391
## 1561 Algorithm 1  UF_7_18  0.46086328
## 1562 Algorithm 1  UF_7_18  0.34098773
## 1563 Algorithm 1  UF_7_18  0.40319898
## 1564 Algorithm 1  UF_7_18  0.65288318
## 1565 Algorithm 1  UF_7_18  0.35221818
## 1566 Algorithm 1  UF_7_18  0.34918949
## 1567 Algorithm 1  UF_7_18  0.70142017
## 1568 Algorithm 1  UF_7_18  0.35886332
## 1569 Algorithm 1  UF_7_18  0.34195115
## 1570 Algorithm 1  UF_7_18  0.35741759
## 1571 Algorithm 1  UF_7_18  0.35541632
## 1572 Algorithm 1  UF_7_18  0.40075442
## 1573 Algorithm 1  UF_7_18  0.34784322
## 1574 Algorithm 1  UF_7_18  0.36498111
## 1575 Algorithm 1  UF_7_18  0.46365932
## 1576 Algorithm 1  UF_7_18  0.34137416
## 1577 Algorithm 1  UF_7_18  0.43110866
## 1578 Algorithm 2  UF_7_18  0.01693527
## 1579 Algorithm 2  UF_7_18  0.02192156
## 1580 Algorithm 2  UF_7_18  0.01237511
## 1581 Algorithm 2  UF_7_18  0.02620278
## 1582 Algorithm 2  UF_7_18  0.01511154
## 1583 Algorithm 2  UF_7_18  0.02131654
## 1584 Algorithm 2  UF_7_18  0.01482729
## 1585 Algorithm 2  UF_7_18  0.65564927
## 1586 Algorithm 2  UF_7_18  0.01815887
## 1587 Algorithm 2  UF_7_18  0.02351001
## 1588 Algorithm 2  UF_7_18  0.24407903
## 1589 Algorithm 2  UF_7_18  0.01895075
## 1590 Algorithm 2  UF_7_18  0.02040084
## 1591 Algorithm 2  UF_7_18  0.02207076
## 1592 Algorithm 2  UF_7_18  0.01632826
## 1593 Algorithm 2  UF_7_18  0.01876780
## 1594 Algorithm 2  UF_7_18  0.01882200
## 1595 Algorithm 2  UF_7_18  0.02114886
## 1596 Algorithm 2  UF_7_18  0.02335623
## 1597 Algorithm 2  UF_7_18  0.02025934
## 1598 Algorithm 2  UF_7_18  0.01945582
## 1599 Algorithm 2  UF_7_18  0.02038049
## 1600 Algorithm 2  UF_7_18  0.02053908
## 1601 Algorithm 2  UF_7_18  0.01719338
## 1602 Algorithm 2  UF_7_18  0.01680179
## 1603 Algorithm 2  UF_7_18  0.01583491
## 1604 Algorithm 2  UF_7_18  0.01561060
## 1605 Algorithm 2  UF_7_18  0.01666466
## 1606 Algorithm 2  UF_7_18  0.01246764
## 1607 Algorithm 2  UF_7_18  0.01846354
## 1608 Algorithm 2  UF_7_18  0.02185068
## 1609 Algorithm 2  UF_7_18  0.01784678
## 1610 Algorithm 2  UF_7_18  0.01728164
## 1611 Algorithm 1  UF_7_38  0.34852456
## 1612 Algorithm 1  UF_7_38  0.34353379
## 1613 Algorithm 1  UF_7_38  0.69933073
## 1614 Algorithm 1  UF_7_38  0.40206265
## 1615 Algorithm 1  UF_7_38  0.35603370
## 1616 Algorithm 1  UF_7_38  0.37248876
## 1617 Algorithm 1  UF_7_38  0.35157148
## 1618 Algorithm 1  UF_7_38  0.40607183
## 1619 Algorithm 1  UF_7_38  0.69120471
## 1620 Algorithm 1  UF_7_38  0.58708779
## 1621 Algorithm 1  UF_7_38  0.54304058
## 1622 Algorithm 1  UF_7_38  0.34983240
## 1623 Algorithm 1  UF_7_38  0.35897428
## 1624 Algorithm 1  UF_7_38  0.35961617
## 1625 Algorithm 1  UF_7_38  0.40367518
## 1626 Algorithm 1  UF_7_38  0.33952968
## 1627 Algorithm 1  UF_7_38  0.36531221
## 1628 Algorithm 1  UF_7_38  0.34206102
## 1629 Algorithm 1  UF_7_38  0.03729959
## 1630 Algorithm 1  UF_7_38  0.34264656
## 1631 Algorithm 1  UF_7_38  0.36174276
## 1632 Algorithm 1  UF_7_38  0.36218052
## 1633 Algorithm 1  UF_7_38  0.36158240
## 1634 Algorithm 1  UF_7_38  0.36121486
## 1635 Algorithm 1  UF_7_38  0.35593765
## 1636 Algorithm 1  UF_7_38  0.36998243
## 1637 Algorithm 1  UF_7_38  0.68308205
## 1638 Algorithm 1  UF_7_38  0.33846636
## 1639 Algorithm 1  UF_7_38  0.39920599
## 1640 Algorithm 1  UF_7_38  0.35755613
## 1641 Algorithm 1  UF_7_38  0.60539157
## 1642 Algorithm 1  UF_7_38  0.35873246
## 1643 Algorithm 2  UF_7_38  0.01915522
## 1644 Algorithm 2  UF_7_38  0.02914570
## 1645 Algorithm 2  UF_7_38  0.31164578
## 1646 Algorithm 2  UF_7_38  0.02700643
## 1647 Algorithm 2  UF_7_38  0.21880636
## 1648 Algorithm 2  UF_7_38  0.02543032
## 1649 Algorithm 2  UF_7_38  0.01941986
## 1650 Algorithm 2  UF_7_38  0.02548521
## 1651 Algorithm 2  UF_7_38  0.02536312
## 1652 Algorithm 2  UF_7_38  0.02906629
## 1653 Algorithm 2  UF_7_38  0.01706825
## 1654 Algorithm 2  UF_7_38  0.01564606
## 1655 Algorithm 2  UF_7_38  0.03200816
## 1656 Algorithm 2  UF_7_38  0.04540685
## 1657 Algorithm 2  UF_7_38  0.02240327
## 1658 Algorithm 2  UF_7_38  0.02406361
## 1659 Algorithm 1  UF_4_14  0.07185440
## 1660 Algorithm 1  UF_4_14  0.06505940
## 1661 Algorithm 1  UF_4_14  0.07236999
## 1662 Algorithm 1  UF_4_14  0.08580576
## 1663 Algorithm 1  UF_4_14  0.07494096
## 1664 Algorithm 1  UF_4_14  0.07520456
## 1665 Algorithm 1  UF_4_14  0.07199709
## 1666 Algorithm 1  UF_4_14  0.07347395
## 1667 Algorithm 1  UF_4_14  0.06102935
## 1668 Algorithm 1  UF_4_14  0.07431553
## 1669 Algorithm 1  UF_4_14  0.05988419
## 1670 Algorithm 1  UF_4_14  0.06354947
## 1671 Algorithm 1  UF_4_14  0.06518621
## 1672 Algorithm 1  UF_4_14  0.06945230
## 1673 Algorithm 1  UF_4_14  0.07430802
## 1674 Algorithm 2  UF_4_14  0.05569611
## 1675 Algorithm 2  UF_4_14  0.05158953
## 1676 Algorithm 2  UF_4_14  0.05447310
## 1677 Algorithm 2  UF_4_14  0.05572043
## 1678 Algorithm 2  UF_4_14  0.04924262
## 1679 Algorithm 2  UF_4_14  0.05300738
## 1680 Algorithm 2  UF_4_14  0.06196498
## 1681 Algorithm 2  UF_4_14  0.06226014
## 1682 Algorithm 2  UF_4_14  0.05489685
## 1683 Algorithm 2  UF_4_14  0.05455697
## 1684 Algorithm 2  UF_4_14  0.05566547
## 1685 Algorithm 2  UF_4_14  0.05144929
## 1686 Algorithm 2  UF_4_14  0.05980390
## 1687 Algorithm 2  UF_4_14  0.05495355
## 1688 Algorithm 2  UF_4_14  0.06490967
## 1689 Algorithm 1  UF_1_11  0.26650253
## 1690 Algorithm 1  UF_1_11  0.10812059
## 1691 Algorithm 1  UF_1_11  0.11950407
## 1692 Algorithm 1  UF_1_11  0.16040014
## 1693 Algorithm 1  UF_1_11  0.15306183
## 1694 Algorithm 1  UF_1_11  0.19044697
## 1695 Algorithm 1  UF_1_11  0.15763980
## 1696 Algorithm 1  UF_1_11  0.10535335
## 1697 Algorithm 1  UF_1_11  0.23795922
## 1698 Algorithm 1  UF_1_11  0.13306989
## 1699 Algorithm 1  UF_1_11  0.33747227
## 1700 Algorithm 1  UF_1_11  0.09688814
## 1701 Algorithm 1  UF_1_11  0.08416188
## 1702 Algorithm 1  UF_1_11  0.14874590
## 1703 Algorithm 1  UF_1_11  0.16084223
## 1704 Algorithm 2  UF_1_11  0.02700769
## 1705 Algorithm 2  UF_1_11  0.02814077
## 1706 Algorithm 2  UF_1_11  0.03053129
## 1707 Algorithm 2  UF_1_11  0.02717723
## 1708 Algorithm 2  UF_1_11  0.03852262
## 1709 Algorithm 2  UF_1_11  0.02807750
## 1710 Algorithm 2  UF_1_11  0.02539741
## 1711 Algorithm 2  UF_1_11  0.02652545
## 1712 Algorithm 2  UF_1_11  0.02943445
## 1713 Algorithm 2  UF_1_11  0.03241197
## 1714 Algorithm 2  UF_1_11  0.03822682
## 1715 Algorithm 2  UF_1_11  0.02928790
## 1716 Algorithm 2  UF_1_11  0.02917528
## 1717 Algorithm 2  UF_1_11  0.02369976
## 1718 Algorithm 2  UF_1_11  0.02894602
## 1719 Algorithm 1  UF_1_16  0.22057914
## 1720 Algorithm 1  UF_1_16  0.17443781
## 1721 Algorithm 1  UF_1_16  0.14421278
## 1722 Algorithm 1  UF_1_16  0.10503528
## 1723 Algorithm 1  UF_1_16  0.12946121
## 1724 Algorithm 1  UF_1_16  0.14048116
## 1725 Algorithm 1  UF_1_16  0.06664685
## 1726 Algorithm 1  UF_1_16  0.19465441
## 1727 Algorithm 1  UF_1_16  0.09462311
## 1728 Algorithm 1  UF_1_16  0.11588488
## 1729 Algorithm 1  UF_1_16  0.09118281
## 1730 Algorithm 1  UF_1_16  0.18533341
## 1731 Algorithm 1  UF_1_16  0.15613686
## 1732 Algorithm 1  UF_1_16  0.13006203
## 1733 Algorithm 1  UF_1_16  0.17138465
## 1734 Algorithm 2  UF_1_16  0.03299933
## 1735 Algorithm 2  UF_1_16  0.03745205
## 1736 Algorithm 2  UF_1_16  0.03358959
## 1737 Algorithm 2  UF_1_16  0.02434512
## 1738 Algorithm 2  UF_1_16  0.03268770
## 1739 Algorithm 2  UF_1_16  0.03664486
## 1740 Algorithm 2  UF_1_16  0.02669603
## 1741 Algorithm 2  UF_1_16  0.03838134
## 1742 Algorithm 2  UF_1_16  0.05457198
## 1743 Algorithm 2  UF_1_16  0.03032361
## 1744 Algorithm 2  UF_1_16  0.03704150
## 1745 Algorithm 2  UF_1_16  0.04390376
## 1746 Algorithm 2  UF_1_16  0.03207142
## 1747 Algorithm 2  UF_1_16  0.03005495
## 1748 Algorithm 2  UF_1_16  0.03567334
## 1749 Algorithm 1  UF_2_32  0.06774802
## 1750 Algorithm 1  UF_2_32  0.03717321
## 1751 Algorithm 1  UF_2_32  0.08336262
## 1752 Algorithm 1  UF_2_32  0.05010000
## 1753 Algorithm 1  UF_2_32  0.03400893
## 1754 Algorithm 1  UF_2_32  0.10714931
## 1755 Algorithm 1  UF_2_32  0.08893593
## 1756 Algorithm 1  UF_2_32  0.02613952
## 1757 Algorithm 1  UF_2_32  0.03918550
## 1758 Algorithm 1  UF_2_32  0.04991091
## 1759 Algorithm 1  UF_2_32  0.03339389
## 1760 Algorithm 1  UF_2_32  0.03165187
## 1761 Algorithm 1  UF_2_32  0.03120470
## 1762 Algorithm 1  UF_2_32  0.09995471
## 1763 Algorithm 1  UF_2_32  0.08921339
## 1764 Algorithm 1  UF_2_32  0.03322310
## 1765 Algorithm 1  UF_2_32  0.04436443
## 1766 Algorithm 1  UF_2_32  0.10788865
## 1767 Algorithm 1  UF_2_32  0.03943491
## 1768 Algorithm 1  UF_2_32  0.09133607
## 1769 Algorithm 1  UF_2_32  0.04040033
## 1770 Algorithm 1  UF_2_32  0.08682982
## 1771 Algorithm 1  UF_2_32  0.04271256
## 1772 Algorithm 1  UF_2_32  0.05853770
## 1773 Algorithm 1  UF_2_32  0.04123578
## 1774 Algorithm 1  UF_2_32  0.07436007
## 1775 Algorithm 1  UF_2_32  0.04969498
## 1776 Algorithm 1  UF_2_32  0.03830173
## 1777 Algorithm 1  UF_2_32  0.08137475
## 1778 Algorithm 1  UF_2_32  0.08512850
## 1779 Algorithm 1  UF_2_32  0.03080766
## 1780 Algorithm 1  UF_2_32  0.05230546
## 1781 Algorithm 1  UF_2_32  0.08869262
## 1782 Algorithm 1  UF_2_32  0.05816919
## 1783 Algorithm 1  UF_2_32  0.04344546
## 1784 Algorithm 1  UF_2_32  0.04877102
## 1785 Algorithm 1  UF_2_32  0.03046607
## 1786 Algorithm 1  UF_2_32  0.10256112
## 1787 Algorithm 1  UF_2_32  0.02937215
## 1788 Algorithm 1  UF_2_32  0.03307258
## 1789 Algorithm 1  UF_2_32  0.03485604
## 1790 Algorithm 1  UF_2_32  0.04335289
## 1791 Algorithm 1  UF_2_32  0.02888661
## 1792 Algorithm 1  UF_2_32  0.03485709
## 1793 Algorithm 1  UF_2_32  0.07652189
## 1794 Algorithm 1  UF_2_32  0.03797834
## 1795 Algorithm 1  UF_2_32  0.03320024
## 1796 Algorithm 1  UF_2_32  0.08143253
## 1797 Algorithm 1  UF_2_32  0.05169338
## 1798 Algorithm 1  UF_2_32  0.09219034
## 1799 Algorithm 1  UF_2_32  0.07753612
## 1800 Algorithm 2  UF_2_32  0.03528807
## 1801 Algorithm 2  UF_2_32  0.04208163
## 1802 Algorithm 2  UF_2_32  0.03507537
## 1803 Algorithm 2  UF_2_32  0.03560949
## 1804 Algorithm 2  UF_2_32  0.03289879
## 1805 Algorithm 2  UF_2_32  0.05435531
## 1806 Algorithm 2  UF_2_32  0.03441163
## 1807 Algorithm 2  UF_2_32  0.03354939
## 1808 Algorithm 2  UF_2_32  0.03367490
## 1809 Algorithm 2  UF_2_32  0.04798833
## 1810 Algorithm 2  UF_2_32  0.03657164
## 1811 Algorithm 2  UF_2_32  0.03725882
## 1812 Algorithm 2  UF_2_32  0.03197220
## 1813 Algorithm 2  UF_2_32  0.03197075
## 1814 Algorithm 2  UF_2_32  0.03284690
## 1815 Algorithm 1  UF_3_24  0.18936345
## 1816 Algorithm 1  UF_3_24  0.15294819
## 1817 Algorithm 1  UF_3_24  0.12223744
## 1818 Algorithm 1  UF_3_24  0.17797057
## 1819 Algorithm 1  UF_3_24  0.24873908
## 1820 Algorithm 1  UF_3_24  0.17930078
## 1821 Algorithm 1  UF_3_24  0.16681832
## 1822 Algorithm 1  UF_3_24  0.11937914
## 1823 Algorithm 1  UF_3_24  0.19118439
## 1824 Algorithm 1  UF_3_24  0.16646918
## 1825 Algorithm 1  UF_3_24  0.14282842
## 1826 Algorithm 1  UF_3_24  0.13465299
## 1827 Algorithm 1  UF_3_24  0.17407880
## 1828 Algorithm 1  UF_3_24  0.36156100
## 1829 Algorithm 1  UF_3_24  0.18035367
## 1830 Algorithm 1  UF_3_24  0.15348957
## 1831 Algorithm 1  UF_3_24  0.20794366
## 1832 Algorithm 1  UF_3_24  0.12218537
## 1833 Algorithm 1  UF_3_24  0.18683504
## 1834 Algorithm 1  UF_3_24  0.19260175
## 1835 Algorithm 1  UF_3_24  0.16176352
## 1836 Algorithm 1  UF_3_24  0.15831605
## 1837 Algorithm 1  UF_3_24  0.18272553
## 1838 Algorithm 1  UF_3_24  0.16016214
## 1839 Algorithm 1  UF_3_24  0.11974160
## 1840 Algorithm 1  UF_3_24  0.20360774
## 1841 Algorithm 1  UF_3_24  0.15149624
## 1842 Algorithm 1  UF_3_24  0.15087901
## 1843 Algorithm 1  UF_3_24  0.16324122
## 1844 Algorithm 1  UF_3_24  0.17301754
## 1845 Algorithm 1  UF_3_24  0.20075482
## 1846 Algorithm 1  UF_3_24  0.19717688
## 1847 Algorithm 1  UF_3_24  0.25096687
## 1848 Algorithm 1  UF_3_24  0.16527747
## 1849 Algorithm 1  UF_3_24  0.17111440
## 1850 Algorithm 1  UF_3_24  0.17556232
## 1851 Algorithm 1  UF_3_24  0.17904961
## 1852 Algorithm 1  UF_3_24  0.18279726
## 1853 Algorithm 1  UF_3_24  0.12247645
## 1854 Algorithm 1  UF_3_24  0.12951807
## 1855 Algorithm 1  UF_3_24  0.21936787
## 1856 Algorithm 1  UF_3_24  0.13984526
## 1857 Algorithm 2  UF_3_24  0.15170889
## 1858 Algorithm 2  UF_3_24  0.18292617
## 1859 Algorithm 2  UF_3_24  0.14667867
## 1860 Algorithm 2  UF_3_24  0.20625284
## 1861 Algorithm 2  UF_3_24  0.13380133
## 1862 Algorithm 2  UF_3_24  0.19164266
## 1863 Algorithm 2  UF_3_24  0.08879042
## 1864 Algorithm 2  UF_3_24  0.16249278
## 1865 Algorithm 2  UF_3_24  0.06505074
## 1866 Algorithm 2  UF_3_24  0.09433747
## 1867 Algorithm 2  UF_3_24  0.14706641
## 1868 Algorithm 2  UF_3_24  0.22393959
## 1869 Algorithm 2  UF_3_24  0.14266127
## 1870 Algorithm 2  UF_3_24  0.04130638
## 1871 Algorithm 2  UF_3_24  0.25769199
## 1872 Algorithm 2  UF_3_24  0.07128074
## 1873 Algorithm 2  UF_3_24  0.14458483
## 1874 Algorithm 2  UF_3_24  0.11006956
## 1875 Algorithm 2  UF_3_24  0.22769087
## 1876 Algorithm 2  UF_3_24  0.15318049
## 1877 Algorithm 2  UF_3_24  0.16753929
## 1878 Algorithm 2  UF_3_24  0.15981896
## 1879 Algorithm 2  UF_3_24  0.13227531
## 1880 Algorithm 2  UF_3_24  0.11581989
## 1881 Algorithm 2  UF_3_24  0.12383583
## 1882 Algorithm 2  UF_3_24  0.18804829
## 1883 Algorithm 2  UF_3_24  0.09514209
## 1884 Algorithm 2  UF_3_24  0.21515447
## 1885 Algorithm 2  UF_3_24  0.11262531
## 1886 Algorithm 2  UF_3_24  0.08900236
## 1887 Algorithm 2  UF_3_24  0.07416887
## 1888 Algorithm 2  UF_3_24  0.13762733
## 1889 Algorithm 2  UF_3_24  0.17638186
## 1890 Algorithm 2  UF_3_24  0.23060900
## 1891 Algorithm 2  UF_3_24  0.12487329
## 1892 Algorithm 2  UF_3_24  0.04950580
## 1893 Algorithm 2  UF_3_24  0.14876443
## 1894 Algorithm 2  UF_3_24  0.14484876
## 1895 Algorithm 2  UF_3_24  0.17351649
## 1896 Algorithm 2  UF_3_24  0.13162563
## 1897 Algorithm 2  UF_3_24  0.07392650
## 1898 Algorithm 2  UF_3_24  0.09805138
## 1899 Algorithm 2  UF_3_24  0.16673588
## 1900 Algorithm 2  UF_3_24  0.17773109
## 1901 Algorithm 2  UF_3_24  0.11492746
## 1902 Algorithm 2  UF_3_24  0.18152154
## 1903 Algorithm 2  UF_3_24  0.10630532
## 1904 Algorithm 1  UF_6_34  0.31798621
## 1905 Algorithm 1  UF_6_34  0.31903518
## 1906 Algorithm 1  UF_6_34  0.18272759
## 1907 Algorithm 1  UF_6_34  0.34921571
## 1908 Algorithm 1  UF_6_34  0.31934885
## 1909 Algorithm 1  UF_6_34  0.16104220
## 1910 Algorithm 1  UF_6_34  0.29457158
## 1911 Algorithm 1  UF_6_34  0.18911404
## 1912 Algorithm 1  UF_6_34  0.19053688
## 1913 Algorithm 1  UF_6_34  0.21926039
## 1914 Algorithm 1  UF_6_34  0.31948751
## 1915 Algorithm 1  UF_6_34  0.18460403
## 1916 Algorithm 1  UF_6_34  0.15711069
## 1917 Algorithm 1  UF_6_34  0.58218139
## 1918 Algorithm 1  UF_6_34  0.18686945
## 1919 Algorithm 2  UF_6_34  0.03357852
## 1920 Algorithm 2  UF_6_34  0.14833575
## 1921 Algorithm 2  UF_6_34  0.04965231
## 1922 Algorithm 2  UF_6_34  0.03082528
## 1923 Algorithm 2  UF_6_34  0.07104525
## 1924 Algorithm 2  UF_6_34  0.06199547
## 1925 Algorithm 2  UF_6_34  0.05640778
## 1926 Algorithm 2  UF_6_34  0.03664978
## 1927 Algorithm 2  UF_6_34  0.05008014
## 1928 Algorithm 2  UF_6_34  0.05319856
## 1929 Algorithm 2  UF_6_34  0.16483762
## 1930 Algorithm 2  UF_6_34  0.03718492
## 1931 Algorithm 2  UF_6_34  0.05328763
## 1932 Algorithm 2  UF_6_34  0.16274610
## 1933 Algorithm 2  UF_6_34  0.03731301
## 1934 Algorithm 1  UF_4_32  0.07993465
## 1935 Algorithm 1  UF_4_32  0.07310928
## 1936 Algorithm 1  UF_4_32  0.07818879
## 1937 Algorithm 1  UF_4_32  0.07227283
## 1938 Algorithm 1  UF_4_32  0.08173018
## 1939 Algorithm 1  UF_4_32  0.07796214
## 1940 Algorithm 1  UF_4_32  0.07933505
## 1941 Algorithm 1  UF_4_32  0.07086909
## 1942 Algorithm 1  UF_4_32  0.08757509
## 1943 Algorithm 1  UF_4_32  0.07279727
## 1944 Algorithm 1  UF_4_32  0.06648456
## 1945 Algorithm 1  UF_4_32  0.07751133
## 1946 Algorithm 1  UF_4_32  0.07717086
## 1947 Algorithm 1  UF_4_32  0.06714934
## 1948 Algorithm 1  UF_4_32  0.06974507
## 1949 Algorithm 2  UF_4_32  0.07150581
## 1950 Algorithm 2  UF_4_32  0.07997446
## 1951 Algorithm 2  UF_4_32  0.08401103
## 1952 Algorithm 2  UF_4_32  0.07771901
## 1953 Algorithm 2  UF_4_32  0.08171862
## 1954 Algorithm 2  UF_4_32  0.07574977
## 1955 Algorithm 2  UF_4_32  0.07696594
## 1956 Algorithm 2  UF_4_32  0.06549513
## 1957 Algorithm 2  UF_4_32  0.07338036
## 1958 Algorithm 2  UF_4_32  0.07228396
## 1959 Algorithm 2  UF_4_32  0.07116104
## 1960 Algorithm 2  UF_4_32  0.06650342
## 1961 Algorithm 2  UF_4_32  0.07756846
## 1962 Algorithm 2  UF_4_32  0.07432470
## 1963 Algorithm 2  UF_4_32  0.07848214
## 1964 Algorithm 1  UF_2_11  0.02970006
## 1965 Algorithm 1  UF_2_11  0.02694752
## 1966 Algorithm 1  UF_2_11  0.02670063
## 1967 Algorithm 1  UF_2_11  0.06694157
## 1968 Algorithm 1  UF_2_11  0.03893101
## 1969 Algorithm 1  UF_2_11  0.02973091
## 1970 Algorithm 1  UF_2_11  0.06269185
## 1971 Algorithm 1  UF_2_11  0.12661527
## 1972 Algorithm 1  UF_2_11  0.05735948
## 1973 Algorithm 1  UF_2_11  0.08141940
## 1974 Algorithm 1  UF_2_11  0.04636835
## 1975 Algorithm 1  UF_2_11  0.02389078
## 1976 Algorithm 1  UF_2_11  0.02026786
## 1977 Algorithm 1  UF_2_11  0.02278117
## 1978 Algorithm 1  UF_2_11  0.03087219
## 1979 Algorithm 1  UF_2_11  0.02306465
## 1980 Algorithm 1  UF_2_11  0.02410220
## 1981 Algorithm 1  UF_2_11  0.02646633
## 1982 Algorithm 1  UF_2_11  0.02915197
## 1983 Algorithm 1  UF_2_11  0.02285612
## 1984 Algorithm 1  UF_2_11  0.08700133
## 1985 Algorithm 1  UF_2_11  0.03141637
## 1986 Algorithm 1  UF_2_11  0.03177810
## 1987 Algorithm 1  UF_2_11  0.02447761
## 1988 Algorithm 1  UF_2_11  0.02328748
## 1989 Algorithm 1  UF_2_11  0.02738428
## 1990 Algorithm 1  UF_2_11  0.09284998
## 1991 Algorithm 1  UF_2_11  0.08290730
## 1992 Algorithm 1  UF_2_11  0.02334707
## 1993 Algorithm 1  UF_2_11  0.02106947
## 1994 Algorithm 1  UF_2_11  0.02451959
## 1995 Algorithm 1  UF_2_11  0.09747840
## 1996 Algorithm 1  UF_2_11  0.03070217
## 1997 Algorithm 1  UF_2_11  0.03716631
## 1998 Algorithm 1  UF_2_11  0.03513670
## 1999 Algorithm 1  UF_2_11  0.07124948
## 2000 Algorithm 1  UF_2_11  0.02587610
## 2001 Algorithm 1  UF_2_11  0.02397752
## 2002 Algorithm 1  UF_2_11  0.02460819
## 2003 Algorithm 1  UF_2_11  0.03887568
## 2004 Algorithm 1  UF_2_11  0.02694919
## 2005 Algorithm 1  UF_2_11  0.03394983
## 2006 Algorithm 1  UF_2_11  0.03372086
## 2007 Algorithm 1  UF_2_11  0.04056791
## 2008 Algorithm 1  UF_2_11  0.02898277
## 2009 Algorithm 1  UF_2_11  0.03957239
## 2010 Algorithm 2  UF_2_11  0.02395845
## 2011 Algorithm 2  UF_2_11  0.02238487
## 2012 Algorithm 2  UF_2_11  0.02238412
## 2013 Algorithm 2  UF_2_11  0.02462377
## 2014 Algorithm 2  UF_2_11  0.02084492
## 2015 Algorithm 2  UF_2_11  0.02384229
## 2016 Algorithm 2  UF_2_11  0.01794329
## 2017 Algorithm 2  UF_2_11  0.01868283
## 2018 Algorithm 2  UF_2_11  0.02602986
## 2019 Algorithm 2  UF_2_11  0.01847829
## 2020 Algorithm 2  UF_2_11  0.02180715
## 2021 Algorithm 2  UF_2_11  0.02075790
## 2022 Algorithm 2  UF_2_11  0.02235055
## 2023 Algorithm 2  UF_2_11  0.02347349
## 2024 Algorithm 2  UF_2_11  0.01840677
## 2025 Algorithm 1  UF_2_22  0.04687716
## 2026 Algorithm 1  UF_2_22  0.05978902
## 2027 Algorithm 1  UF_2_22  0.03350498
## 2028 Algorithm 1  UF_2_22  0.02791371
## 2029 Algorithm 1  UF_2_22  0.09173168
## 2030 Algorithm 1  UF_2_22  0.03338613
## 2031 Algorithm 1  UF_2_22  0.05136548
## 2032 Algorithm 1  UF_2_22  0.03492631
## 2033 Algorithm 1  UF_2_22  0.02856083
## 2034 Algorithm 1  UF_2_22  0.07721396
## 2035 Algorithm 1  UF_2_22  0.21678092
## 2036 Algorithm 1  UF_2_22  0.02828499
## 2037 Algorithm 1  UF_2_22  0.04255131
## 2038 Algorithm 1  UF_2_22  0.02667376
## 2039 Algorithm 1  UF_2_22  0.08830552
## 2040 Algorithm 1  UF_2_22  0.03025670
## 2041 Algorithm 1  UF_2_22  0.08135058
## 2042 Algorithm 1  UF_2_22  0.03592255
## 2043 Algorithm 1  UF_2_22  0.07013544
## 2044 Algorithm 1  UF_2_22  0.02864841
## 2045 Algorithm 1  UF_2_22  0.09395718
## 2046 Algorithm 1  UF_2_22  0.05971615
## 2047 Algorithm 1  UF_2_22  0.03097521
## 2048 Algorithm 1  UF_2_22  0.03914360
## 2049 Algorithm 1  UF_2_22  0.03150270
## 2050 Algorithm 1  UF_2_22  0.22598213
## 2051 Algorithm 1  UF_2_22  0.07149465
## 2052 Algorithm 1  UF_2_22  0.02951162
## 2053 Algorithm 1  UF_2_22  0.08954708
## 2054 Algorithm 1  UF_2_22  0.10148721
## 2055 Algorithm 1  UF_2_22  0.02730899
## 2056 Algorithm 1  UF_2_22  0.09833251
## 2057 Algorithm 1  UF_2_22  0.10660501
## 2058 Algorithm 1  UF_2_22  0.04759199
## 2059 Algorithm 1  UF_2_22  0.07825080
## 2060 Algorithm 1  UF_2_22  0.03353387
## 2061 Algorithm 1  UF_2_22  0.08484702
## 2062 Algorithm 1  UF_2_22  0.02999330
## 2063 Algorithm 1  UF_2_22  0.08691252
## 2064 Algorithm 1  UF_2_22  0.03931096
## 2065 Algorithm 1  UF_2_22  0.07784196
## 2066 Algorithm 1  UF_2_22  0.03958675
## 2067 Algorithm 1  UF_2_22  0.08030694
## 2068 Algorithm 1  UF_2_22  0.09373298
## 2069 Algorithm 2  UF_2_22  0.02851019
## 2070 Algorithm 2  UF_2_22  0.02665587
## 2071 Algorithm 2  UF_2_22  0.02904327
## 2072 Algorithm 2  UF_2_22  0.03866564
## 2073 Algorithm 2  UF_2_22  0.02962962
## 2074 Algorithm 2  UF_2_22  0.03553411
## 2075 Algorithm 2  UF_2_22  0.02772292
## 2076 Algorithm 2  UF_2_22  0.02820782
## 2077 Algorithm 2  UF_2_22  0.02939777
## 2078 Algorithm 2  UF_2_22  0.03018175
## 2079 Algorithm 2  UF_2_22  0.02277412
## 2080 Algorithm 2  UF_2_22  0.02409353
## 2081 Algorithm 2  UF_2_22  0.02314675
## 2082 Algorithm 2  UF_2_22  0.03758412
## 2083 Algorithm 2  UF_2_22  0.03121549
## 2084 Algorithm 1  UF_1_17  0.18050074
## 2085 Algorithm 1  UF_1_17  0.16237024
## 2086 Algorithm 1  UF_1_17  0.13713107
## 2087 Algorithm 1  UF_1_17  0.11342100
## 2088 Algorithm 1  UF_1_17  0.08967701
## 2089 Algorithm 1  UF_1_17  0.13113710
## 2090 Algorithm 1  UF_1_17  0.13257657
## 2091 Algorithm 1  UF_1_17  0.21240270
## 2092 Algorithm 1  UF_1_17  0.09312394
## 2093 Algorithm 1  UF_1_17  0.16368360
## 2094 Algorithm 1  UF_1_17  0.14430066
## 2095 Algorithm 1  UF_1_17  0.15855083
## 2096 Algorithm 1  UF_1_17  0.07156048
## 2097 Algorithm 1  UF_1_17  0.10800398
## 2098 Algorithm 1  UF_1_17  0.17386612
## 2099 Algorithm 2  UF_1_17  0.02701273
## 2100 Algorithm 2  UF_1_17  0.05162414
## 2101 Algorithm 2  UF_1_17  0.03875826
## 2102 Algorithm 2  UF_1_17  0.03339056
## 2103 Algorithm 2  UF_1_17  0.03294722
## 2104 Algorithm 2  UF_1_17  0.07555167
## 2105 Algorithm 2  UF_1_17  0.04337048
## 2106 Algorithm 2  UF_1_17  0.04552125
## 2107 Algorithm 2  UF_1_17  0.03298608
## 2108 Algorithm 2  UF_1_17  0.03527235
## 2109 Algorithm 2  UF_1_17  0.03769468
## 2110 Algorithm 2  UF_1_17  0.03480704
## 2111 Algorithm 2  UF_1_17  0.03408000
## 2112 Algorithm 2  UF_1_17  0.02922823
## 2113 Algorithm 2  UF_1_17  0.03938640
## 2114 Algorithm 1  UF_1_18  0.20723284
## 2115 Algorithm 1  UF_1_18  0.17192208
## 2116 Algorithm 1  UF_1_18  0.08975677
## 2117 Algorithm 1  UF_1_18  0.13067803
## 2118 Algorithm 1  UF_1_18  0.20015283
## 2119 Algorithm 1  UF_1_18  0.13727666
## 2120 Algorithm 1  UF_1_18  0.16578073
## 2121 Algorithm 1  UF_1_18  0.13968145
## 2122 Algorithm 1  UF_1_18  0.15816050
## 2123 Algorithm 1  UF_1_18  0.07991706
## 2124 Algorithm 1  UF_1_18  0.07646092
## 2125 Algorithm 1  UF_1_18  0.08129121
## 2126 Algorithm 1  UF_1_18  0.20528810
## 2127 Algorithm 1  UF_1_18  0.09177844
## 2128 Algorithm 1  UF_1_18  0.09428341
## 2129 Algorithm 2  UF_1_18  0.04772118
## 2130 Algorithm 2  UF_1_18  0.03748537
## 2131 Algorithm 2  UF_1_18  0.04819580
## 2132 Algorithm 2  UF_1_18  0.02939061
## 2133 Algorithm 2  UF_1_18  0.04437567
## 2134 Algorithm 2  UF_1_18  0.04244938
## 2135 Algorithm 2  UF_1_18  0.03397885
## 2136 Algorithm 2  UF_1_18  0.04260431
## 2137 Algorithm 2  UF_1_18  0.04215961
## 2138 Algorithm 2  UF_1_18  0.03538750
## 2139 Algorithm 2  UF_1_18  0.03806308
## 2140 Algorithm 2  UF_1_18  0.04984098
## 2141 Algorithm 2  UF_1_18  0.05373220
## 2142 Algorithm 2  UF_1_18  0.03727865
## 2143 Algorithm 2  UF_1_18  0.04898188
## 2144 Algorithm 1  UF_3_23  0.17254052
## 2145 Algorithm 1  UF_3_23  0.14917832
## 2146 Algorithm 1  UF_3_23  0.20535996
## 2147 Algorithm 1  UF_3_23  0.18539838
## 2148 Algorithm 1  UF_3_23  0.13695726
## 2149 Algorithm 1  UF_3_23  0.18551667
## 2150 Algorithm 1  UF_3_23  0.20876919
## 2151 Algorithm 1  UF_3_23  0.18288555
## 2152 Algorithm 1  UF_3_23  0.20349347
## 2153 Algorithm 1  UF_3_23  0.27351131
## 2154 Algorithm 1  UF_3_23  0.22260575
## 2155 Algorithm 1  UF_3_23  0.16796494
## 2156 Algorithm 1  UF_3_23  0.15296183
## 2157 Algorithm 1  UF_3_23  0.18624877
## 2158 Algorithm 1  UF_3_23  0.15751989
## 2159 Algorithm 1  UF_3_23  0.23959994
## 2160 Algorithm 1  UF_3_23  0.21903606
## 2161 Algorithm 1  UF_3_23  0.20206172
## 2162 Algorithm 1  UF_3_23  0.23000011
## 2163 Algorithm 1  UF_3_23  0.17752137
## 2164 Algorithm 1  UF_3_23  0.12750401
## 2165 Algorithm 1  UF_3_23  0.16292583
## 2166 Algorithm 1  UF_3_23  0.17697100
## 2167 Algorithm 1  UF_3_23  0.17853810
## 2168 Algorithm 1  UF_3_23  0.17825094
## 2169 Algorithm 1  UF_3_23  0.17582031
## 2170 Algorithm 1  UF_3_23  0.20534143
## 2171 Algorithm 1  UF_3_23  0.26862070
## 2172 Algorithm 1  UF_3_23  0.15357743
## 2173 Algorithm 1  UF_3_23  0.12744153
## 2174 Algorithm 1  UF_3_23  0.22460286
## 2175 Algorithm 1  UF_3_23  0.19072299
## 2176 Algorithm 1  UF_3_23  0.18455831
## 2177 Algorithm 2  UF_3_23  0.14476837
## 2178 Algorithm 2  UF_3_23  0.11908844
## 2179 Algorithm 2  UF_3_23  0.32413524
## 2180 Algorithm 2  UF_3_23  0.11442373
## 2181 Algorithm 2  UF_3_23  0.11635878
## 2182 Algorithm 2  UF_3_23  0.15662261
## 2183 Algorithm 2  UF_3_23  0.22706824
## 2184 Algorithm 2  UF_3_23  0.28859693
## 2185 Algorithm 2  UF_3_23  0.08073338
## 2186 Algorithm 2  UF_3_23  0.24856070
## 2187 Algorithm 2  UF_3_23  0.14867209
## 2188 Algorithm 2  UF_3_23  0.16622054
## 2189 Algorithm 2  UF_3_23  0.08239823
## 2190 Algorithm 2  UF_3_23  0.20421840
## 2191 Algorithm 2  UF_3_23  0.21340642
## 2192 Algorithm 2  UF_3_23  0.07210873
## 2193 Algorithm 2  UF_3_23  0.17731140
## 2194 Algorithm 2  UF_3_23  0.25548736
## 2195 Algorithm 2  UF_3_23  0.14402585
## 2196 Algorithm 2  UF_3_23  0.19408905
## 2197 Algorithm 2  UF_3_23  0.13971992
## 2198 Algorithm 2  UF_3_23  0.13621560
## 2199 Algorithm 2  UF_3_23  0.09720724
## 2200 Algorithm 2  UF_3_23  0.13400297
## 2201 Algorithm 2  UF_3_23  0.13390058
## 2202 Algorithm 2  UF_3_23  0.10922962
## 2203 Algorithm 2  UF_3_23  0.22048604
## 2204 Algorithm 2  UF_3_23  0.11893266
## 2205 Algorithm 2  UF_3_23  0.06211160
## 2206 Algorithm 2  UF_3_23  0.23162837
## 2207 Algorithm 2  UF_3_23  0.14912236
## 2208 Algorithm 2  UF_3_23  0.14910503
## 2209 Algorithm 2  UF_3_23  0.15810634
## 2210 Algorithm 2  UF_3_23  0.10614790
## 2211 Algorithm 2  UF_3_23  0.06777581
## 2212 Algorithm 2  UF_3_23  0.09899449
## 2213 Algorithm 2  UF_3_23  0.14091183
## 2214 Algorithm 2  UF_3_23  0.21534429
## 2215 Algorithm 2  UF_3_23  0.13555046
## 2216 Algorithm 2  UF_3_23  0.05439559
## 2217 Algorithm 2  UF_3_23  0.19699857
## 2218 Algorithm 2  UF_3_23  0.06304065
## 2219 Algorithm 2  UF_3_23  0.04935538
## 2220 Algorithm 2  UF_3_23  0.15392999
## 2221 Algorithm 2  UF_3_23  0.15672519
## 2222 Algorithm 2  UF_3_23  0.13506642
## 2223 Algorithm 2  UF_3_23  0.11424355
## 2224 Algorithm 2  UF_3_23  0.08358825
## 2225 Algorithm 2  UF_3_23  0.31276332
## 2226 Algorithm 2  UF_3_23  0.17979631
## 2227 Algorithm 2  UF_3_23  0.15169587
## 2228 Algorithm 2  UF_3_23  0.16437242
## 2229 Algorithm 2  UF_3_23  0.16357566
## 2230 Algorithm 2  UF_3_23  0.15663369
## 2231 Algorithm 2  UF_3_23  0.13764472
## 2232 Algorithm 2  UF_3_23  0.26580363
## 2233 Algorithm 2  UF_3_23  0.16335892
## 2234 Algorithm 2  UF_3_23  0.16719870
## 
## $data.summary
##    Instance        phi.j     std.err n1j n2j
## 1   UF_4_13 -0.140520374 0.021244572  15  15
## 2   UF_2_29 -0.355067018 0.049778592  65  15
## 3   UF_5_28  0.687112139 0.051752761  80 120
## 4   UF_1_29 -0.632128519 0.047547812  25  15
## 5   UF_2_36 -0.294324240 0.049541869  71  16
## 6   UF_3_29  0.046833947 0.054367040  99 101
## 7   UF_3_10  0.067378637 0.048449127  57  58
## 8   UF_7_16 -0.947072631 0.002562324  15  15
## 9   UF_7_29 -0.899153444 0.041669953  15  15
## 10  UF_2_25 -0.377597096 0.047448660  66  15
## 11  UF_4_30 -0.041501786 0.023717906  15  15
## 12  UF_1_26 -0.648848834 0.046931423  15  15
## 13  UF_2_18 -0.462644651 0.047951855  40  15
## 14  UF_7_36 -0.923150755 0.021584219  15  15
## 15  UF_4_18 -0.165279882 0.023559291  15  15
## 16  UF_2_34 -0.397055947 0.047815242  44  15
## 17  UF_2_39 -0.293336878 0.049015022  71  15
## 18  UF_5_17  0.463440974 0.053598852  83 117
## 19  UF_3_15 -0.076297160 0.049925757  40  53
## 20  UF_4_16 -0.111094622 0.025666088  15  15
## 21  UF_7_18 -0.889747249 0.048849417  41  33
## 22  UF_7_38 -0.862621393 0.048313917  32  16
## 23  UF_4_14 -0.206193064 0.024435256  15  15
## 24  UF_1_11 -0.820109032 0.020082045  15  15
## 25  UF_1_16 -0.751694495 0.022388751  15  15
## 26  UF_2_32 -0.347339456 0.049116212  51  15
## 27  UF_3_24 -0.188829906 0.049475209  42  47
## 28  UF_6_34 -0.736442498 0.049411387  15  15
## 29  UF_4_32 -0.004410248 0.025967302  15  15
## 30  UF_2_11 -0.467062561 0.049853866  46  15
## 31  UF_2_22 -0.541752393 0.049869899  44  15
## 32  UF_1_17 -0.714505921 0.030266726  15  15
## 33  UF_1_18 -0.688792828 0.030398146  15  15
## 34  UF_3_23 -0.180249684 0.048622757  33  58
## 
## $N
## [1] 34
## 
## $N.star
## [1] 34
## 
## $instances.sampled
##  [1] "UF_4_13" "UF_2_29" "UF_5_28" "UF_1_29" "UF_2_36" "UF_3_29" "UF_3_10"
##  [8] "UF_7_16" "UF_7_29" "UF_2_25" "UF_4_30" "UF_1_26" "UF_2_18" "UF_7_36"
## [15] "UF_4_18" "UF_2_34" "UF_2_39" "UF_5_17" "UF_3_15" "UF_4_16" "UF_7_18"
## [22] "UF_7_38" "UF_4_14" "UF_1_11" "UF_1_16" "UF_2_32" "UF_3_24" "UF_6_34"
## [29] "UF_4_32" "UF_2_11" "UF_2_22" "UF_1_17" "UF_1_18" "UF_3_23"
## 
## $Underpowered
## [1] FALSE

suppressPackageStartupMessages(library(car))
car::qqPlot(my.results$data.summary$phi.j, 
            pch = 20, las = 1, 
            ylab = "observed results", xlab = "theoretical quantiles")

The normal QQ plot indicates that no expressive deviations of normality are present, which gives us confidence in using the t test as our inferential procedure of choice (as the sampling distribution of the means will be even more “well-behaved” than the data distribution).

It is also interesting to observe a few things from the summary table. First, we observed negative values of phi.j in the majority of instances tested, which suggests an advantage of the MOEA/D-DE over the original MOEA/D (remember, smaller = better for the quality indicator used). The MOEA/D-DE seems to require smaller sample sizes in most instances, suggesting a smaller variance of performance, which is also a desirable feature. Also, in three instances (UF_5_28, UF_3_29 and UF__7) the maximum number of runs/instance (nmax = 200 in the run_experiment() call) was not enough to reduce the standard error (our “measurement error” on the values of phi.j) below the predefined threshold of \(0.05\). There is no reason to worry in this particular case, however, since the resulting standard errors were not particularly high, and therefore their effect on the test power (resulting from the increased uncertainty in the estimation of these particular phi.j values) will be insignificant.

Since our observations phi.j already express paired differences per instance, we can compare the two algorithms using a simple, one-sample t.test:

t.test(my.results$data.summary$phi.j)

## 
##  One Sample t-test
## 
## data:  my.results$data.summary$phi.j
## t = -5.627, df = 33, p-value = 2.897e-06
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.5165943 -0.2422327
## sample estimates:
##  mean of x 
## -0.3794135

which indicates a statistically significant advantage of the MOEA/D-DE over the original MOEA/D (\(p = 2.90\times10^{-6}, df=33\)), with estimated mean IGD gains of \(37.94\%\) (\(CI_{0.95} = [24.22\%,51.66\%]\)) over the original MOEA/D for instances belonging to the problem class of interest.

These results could also be used to motivate further analyses. For instance, we can observe in the summary table that the only two cases for which the MOEA/D was substantially better than the MOEA/D-DE were for different dimensions of problem UF_5, which could suggest that some specific feature of this problem jeopardizes the latter algorithm’s search mechanism. This could motivate research on which particular aspect of this problem results in this loss of performance, and on how to improve the MOEA/D-DE.

Finally, the full data of the experiment is contained in other fields of the output list my.results, and the user is encouraged to explore these. As an example, we can generate box plots and confidence intervals on the mean performance of each algorithm on each sampled instance, which could inspire new questions for the researcher.

suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(ggridges))

# Adjust instance names for plotting
mydata <- my.results$data.raw
mydata$Instance <- gsub(pattern = "UF\\_", replacement = "UF", mydata$Instance)
mydata$Instance <- gsub(pattern = "\\_", replacement = " (", mydata$Instance)
mydata$Instance <- sapply(mydata$Instance, FUN = function(x){paste0(x, ")")})

ggplot2::ggplot(mydata, 
                aes(x = Observation, y = Instance, fill = Algorithm)) + 
  ggridges::geom_density_ridges(alpha = 0.7) + 
  ggplot2::ggtitle("Estimated IGD distribution", 
                   subtitle = "for each algorithm on each instance") + 
  ggplot2::theme(legend.position = "bottom")

## Picking joint bandwidth of 0.0132

# Calculate confidence intervals for each instance
algos <- unique(mydata$Algorithm)
ninstances <- length(my.results$instances.sampled)
CIs <- data.frame(instance  = rep(unique(mydata$Instance, times = 2)),
                  algorithm = rep(algos, each = ninstances),
                  x.est     = 0, CI.l = 0, CI.u = 0)

for (i in 1:ninstances){
  tmpdata <- mydata %>% 
    filter(Instance == unique(mydata$Instance)[i])
  myt1 <- t.test(tmpdata$Observation[tmpdata$Algorithm == algos[1]])
  myt2 <- t.test(tmpdata$Observation[tmpdata$Algorithm == algos[2]])
  
  CIs[i,3:5] <- c(myt1$estimate, as.numeric(myt1$conf.int))
  CIs[i + ninstances,3:5] <- c(myt2$estimate, as.numeric(myt2$conf.int))
}

# Plot individual confidence intervals for each instance
myplot <- ggplot2::ggplot(CIs, aes(x = instance, 
                                   y = x.est, ymin = CI.l, ymax = CI.u,
                                   group = algorithm, colour = algorithm,
                                   fill = algorithm))
myplot + 
  ggplot2::geom_pointrange(position = position_dodge(width = 0.5), alpha = 0.7) + 
  ggplot2::xlab("Instance") + ggplot2::ylab("IGD") + 
  ggplot2::ggtitle("Estimated mean IGD", 
                   subtitle = "for each algorithm on each instance") + 
  ggplot2::theme(legend.position = "bottom",
                 axis.text.x = element_text(angle = 55, hjust = 1, size = 6))