weka.classifiers.functions

## Class SMO

- java.lang.Object
- - weka.classifiers.AbstractClassifier
  - - weka.classifiers.functions.SMO

- All Implemented Interfaces:
    java.io.Serializable, java.lang.Cloneable, Classifier, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, TechnicalInformationHandler, WeightedInstancesHandler

      public class SMO
      extends AbstractClassifier
      implements WeightedInstancesHandler, TechnicalInformationHandler

  Implements John Platt's sequential minimal optimization algorithm for training a support vector classifier.
  This implementation globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes by default. (In that case the coefficients in the output are based on the normalized data, not the original data --- this is important for interpreting the classifier.)
  Multi-class problems are solved using pairwise classification (aka 1-vs-1).
  To obtain proper probability estimates, use the option that fits calibration models to the outputs of the support vector machine. In the multi-class case, the predicted probabilities are coupled using Hastie and Tibshirani's pairwise coupling method.
  Note: for improved speed normalization should be turned off when operating on SparseInstances.
  For more information on the SMO algorithm, see
  J. Platt: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In B. Schoelkopf and C. Burges and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning, 1998.
  S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, K.R.K. Murthy (2001). Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation. 13(3):637-649.
  Trevor Hastie, Robert Tibshirani: Classification by Pairwise Coupling. In: Advances in Neural Information Processing Systems, 1998.
  BibTeX:

       @incollection{Platt1998,
          author = {J. Platt},
          booktitle = {Advances in Kernel Methods - Support Vector Learning},
          editor = {B. Schoelkopf and C. Burges and A. Smola},
          publisher = {MIT Press},
          title = {Fast Training of Support Vector Machines using Sequential Minimal Optimization},
          year = {1998},
          URL = {http://research.microsoft.com/\~jplatt/smo.html},
          PS = {http://research.microsoft.com/\~jplatt/smo-book.ps.gz},
          PDF = {http://research.microsoft.com/\~jplatt/smo-book.pdf}
       }

       @article{Keerthi2001,
          author = {S.S. Keerthi and S.K. Shevade and C. Bhattacharyya and K.R.K. Murthy},
          journal = {Neural Computation},
          number = {3},
          pages = {637-649},
          title = {Improvements to Platt's SMO Algorithm for SVM Classifier Design},
          volume = {13},
          year = {2001},
          PS = {http://guppy.mpe.nus.edu.sg/\~mpessk/svm/smo_mod_nc.ps.gz}
       }

       @inproceedings{Hastie1998,
          author = {Trevor Hastie and Robert Tibshirani},
          booktitle = {Advances in Neural Information Processing Systems},
          editor = {Michael I. Jordan and Michael J. Kearns and Sara A. Solla},
          publisher = {MIT Press},
          title = {Classification by Pairwise Coupling},
          volume = {10},
          year = {1998},
          PS = {http://www-stat.stanford.edu/\~hastie/Papers/2class.ps}
       }

  Valid options are:

       -no-checks
        Turns off all checks - use with caution!
        Turning them off assumes that data is purely numeric, doesn't
        contain any missing values, and has a nominal class. Turning them
        off also means that no header information will be stored if the
        machine is linear. Finally, it also assumes that no instance has
        a weight equal to 0.
        (default: checks on)

       -C <double>
        The complexity constant C. (default 1)

       -N
        Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

       -L <double>
        The tolerance parameter. (default 1.0e-3)

       -P <double>
        The epsilon for round-off error. (default 1.0e-12)

       -M
        Fit calibration models to SVM outputs.

       -V <double>
        The number of folds for the internal
        cross-validation. (default -1, use training data)

       -W <double>
        The random number seed. (default 1)

       -K <classname and parameters>
        The Kernel to use.
        (default: weka.classifiers.functions.supportVector.PolyKernel)

       -calibrator <scheme specification>
        Full name of calibration model, followed by options.
        (default: "weka.classifiers.functions.Logistic")

       -output-debug-info
        If set, classifier is run in debug mode and
        may output additional info to the console

       -do-not-check-capabilities
        If set, classifier capabilities are not checked before classifier is built
        (use with caution).

       -num-decimal-places
        The number of decimal places for the output of numbers in the model (default 2).

       Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:

       -E <num>
        The Exponent to use.
        (default: 1.0)

       -L
        Use lower-order terms.
        (default: no)

       -C <num>
        The size of the cache (a prime number), 0 for full cache and
        -1 to turn it off.
        (default: 250007)

       -output-debug-info
        Enables debugging output (if available) to be printed.
        (default: off)

       -no-checks
        Turns off all checks - use with caution!
        (default: checks on)

       Options specific to calibrator weka.classifiers.functions.Logistic:

       -C
        Use conjugate gradient descent rather than BFGS updates.

       -R <ridge>
        Set the ridge in the log-likelihood.

       -M <number>
        Set the maximum number of iterations (default -1, until convergence).

       -output-debug-info
        If set, classifier is run in debug mode and
        may output additional info to the console

       -do-not-check-capabilities
        If set, classifier capabilities are not checked before classifier is built
        (use with caution).

       -num-decimal-places
        The number of decimal places for the output of numbers in the model (default 2).

  Version:
    $Revision: 12558 $
  Author:
    Eibe Frank (eibe@cs.waikato.ac.nz), Shane Legg (shane@intelligenesis.net) (sparse vector code), Stuart Inglis (stuart@reeltwo.com) (sparse vector code)
  See Also:
    Serialized Form

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `class` | `SMO.BinarySMO` Class for building a binary support vector machine.

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static int` | `FILTER_NONE` filter: No normalization/standardization
    `static int` | `FILTER_NORMALIZE` filter: Normalize training data
    `static int` | `FILTER_STANDARDIZE` filter: Standardize training data
    `static Tag[]` | `TAGS_FILTER` The filter to apply to the training data

    - ### Fields inherited from class weka.classifiers.AbstractClassifier

      `BATCH_SIZE_DEFAULT, NUM_DECIMAL_PLACES_DEFAULT`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `SMO()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.lang.String[][][]` | `attributeNames()` Returns the attribute names.
    `double[][]` | `bias()` Returns the bias of each binary SMO.
    `java.lang.String` | `buildCalibrationModelsTipText()` Returns the tip text for this property
    `void` | `buildClassifier(Instances insts)` Method for building the classifier.
    `java.lang.String` | `calibratorTipText()` Returns the tip text for this property
    `java.lang.String` | `checksTurnedOffTipText()` Returns the tip text for this property
    `java.lang.String[]` | `classAttributeNames()`
    `java.lang.String` | `cTipText()` Returns the tip text for this property
    `double[]` | `distributionForInstance(Instance inst)` Estimates class probabilities for given instance.
    `java.lang.String` | `epsilonTipText()` Returns the tip text for this property
    `java.lang.String` | `filterTypeTipText()` Returns the tip text for this property
    `boolean` | `getBuildCalibrationModels()` Get the value of buildCalibrationModels.
    `double` | `getC()` Get the value of C.
    `Classifier` | `getCalibrator()` Returns the calibrator to use
    `Capabilities` | `getCapabilities()` Returns default capabilities of the classifier.
    `boolean` | `getChecksTurnedOff()` Returns whether the checks are turned off or not.
    `double` | `getEpsilon()` Get the value of epsilon.
    `SelectedTag` | `getFilterType()` Gets how the training data will be transformed.
    `Kernel` | `getKernel()` Returns the kernel to use
    `int` | `getNumFolds()` Get the value of numFolds.
    `java.lang.String[]` | `getOptions()` Gets the current settings of the classifier.
    `int` | `getRandomSeed()` Get the value of randomSeed.
    `java.lang.String` | `getRevision()` Returns the revision string.
    `TechnicalInformation` | `getTechnicalInformation()` Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
    `double` | `getToleranceParameter()` Get the value of tolerance parameter.
    `java.lang.String` | `globalInfo()` Returns a string describing classifier
    `java.lang.String` | `kernelTipText()` Returns the tip text for this property
    `java.util.Enumeration<Option>` | `listOptions()` Returns an enumeration describing the available options.
    `static void` | `main(java.lang.String[] argv)` Main method for testing this class.
    `int` | `numClassAttributeValues()`
    `java.lang.String` | `numFoldsTipText()` Returns the tip text for this property
    `int[]` | `obtainVotes(Instance inst)` Returns an array of votes for the given instance.
    `java.lang.String` | `randomSeedTipText()` Returns the tip text for this property
    `void` | `setBuildCalibrationModels(boolean newbuildCalibrationModels)` Set the value of buildCalibrationModels.
    `void` | `setC(double v)` Set the value of C.
    `void` | `setCalibrator(Classifier value)` sets the calibrator to use
    `void` | `setChecksTurnedOff(boolean value)` Disables or enables the checks (which could be time-consuming).
    `void` | `setEpsilon(double v)` Set the value of epsilon.
    `void` | `setFilterType(SelectedTag newType)` Sets how the training data will be transformed.
    `void` | `setKernel(Kernel value)` sets the kernel to use
    `void` | `setNumFolds(int newnumFolds)` Set the value of numFolds.
    `void` | `setOptions(java.lang.String[] options)` Parses a given list of options.
    `void` | `setRandomSeed(int newrandomSeed)` Set the value of randomSeed.
    `void` | `setToleranceParameter(double v)` Set the value of tolerance parameter.
    `int[][][]` | `sparseIndices()` Returns the indices in sparse format.
    `double[][][]` | `sparseWeights()` Returns the weights in sparse format.
    `java.lang.String` | `toleranceParameterTipText()` Returns the tip text for this property
    `java.lang.String` | `toString()` Prints out the classifier.
    `void` | `turnChecksOff()` Turns off checks for missing values, etc.
    `void` | `turnChecksOn()` Turns on checks for missing values, etc.

    - ### Methods inherited from class weka.classifiers.AbstractClassifier

      `batchSizeTipText, classifyInstance, debugTipText, distributionsForInstances, doNotCheckCapabilitiesTipText, forName, getBatchSize, getDebug, getDoNotCheckCapabilities, getNumDecimalPlaces, implementsMoreEfficientBatchPrediction, makeCopies, makeCopy, numDecimalPlacesTipText, postExecution, preExecution, run, runClassifier, setBatchSize, setDebug, setDoNotCheckCapabilities, setNumDecimalPlaces`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### FILTER_NORMALIZE

          public static final int FILTER_NORMALIZE

      filter: Normalize training data
      See Also:
        Constant Field Values

    - #### FILTER_STANDARDIZE

          public static final int FILTER_STANDARDIZE

      filter: Standardize training data
      See Also:
        Constant Field Values

    - #### FILTER_NONE

          public static final int FILTER_NONE

      filter: No normalization/standardization
      See Also:
        Constant Field Values

    - #### TAGS_FILTER

          public static final Tag[] TAGS_FILTER

      The filter to apply to the training data

  - ### Constructor Detail

    - #### SMO

          public SMO()

  - ### Method Detail

    - #### globalInfo

          public java.lang.String globalInfo()

      Returns a string describing classifier
      Returns:
        a description suitable for displaying in the explorer/experimenter gui

    - #### getTechnicalInformation

          public TechnicalInformation getTechnicalInformation()

      Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
      Specified by:
        `getTechnicalInformation` in interface `TechnicalInformationHandler`
      Returns:
        the technical information about this class

    - #### turnChecksOff

          public void turnChecksOff()

      Turns off checks for missing values, etc. Use with caution.

    - #### turnChecksOn

          public void turnChecksOn()

      Turns on checks for missing values, etc.

    - #### getCapabilities

          public Capabilities getCapabilities()

      Returns default capabilities of the classifier.
      Specified by:
        `getCapabilities` in interface `Classifier`
      Specified by:
        `getCapabilities` in interface `CapabilitiesHandler`
      Overrides:
        `getCapabilities` in class `AbstractClassifier`
      Returns:
        the capabilities of this classifier
      See Also:
        `Capabilities`

    - #### buildClassifier

          public void buildClassifier(Instances insts)
                               throws java.lang.Exception

      Method for building the classifier. Implements a one-against-one wrapper for multi-class problems.
      Specified by:
        `buildClassifier` in interface `Classifier`
      Parameters:
        `insts` - the set of training instances
      Throws:
        `java.lang.Exception` - if the classifier can't be built successfully

    - #### distributionForInstance

          public double[] distributionForInstance(Instance inst)
                                           throws java.lang.Exception

      Estimates class probabilities for given instance.
      Specified by:
        `distributionForInstance` in interface `Classifier`
      Overrides:
        `distributionForInstance` in class `AbstractClassifier`
      Parameters:
        `inst` - the instance to compute the probabilities for
      Returns:
        an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction
      Throws:
        `java.lang.Exception` - in case of an error

    - #### obtainVotes

          public int[] obtainVotes(Instance inst)
                            throws java.lang.Exception

      Returns an array of votes for the given instance.
      Parameters:
        `inst` - the instance
      Returns:
        array of votex
      Throws:
        `java.lang.Exception` - if something goes wrong

    - #### sparseWeights

          public double[][][] sparseWeights()

      Returns the weights in sparse format.

    - #### sparseIndices

          public int[][][] sparseIndices()

      Returns the indices in sparse format.

    - #### bias

          public double[][] bias()

      Returns the bias of each binary SMO.

    - #### numClassAttributeValues

          public int numClassAttributeValues()

    - #### classAttributeNames

          public java.lang.String[] classAttributeNames()

    - #### attributeNames

          public java.lang.String[][][] attributeNames()

      Returns the attribute names.

    - #### listOptions

          public java.util.Enumeration<Option> listOptions()

      Returns an enumeration describing the available options.
      Specified by:
        `listOptions` in interface `OptionHandler`
      Overrides:
        `listOptions` in class `AbstractClassifier`
      Returns:
        an enumeration of all the available options.

    - #### setOptions

          public void setOptions(java.lang.String[] options)
                          throws java.lang.Exception

      Parses a given list of options.

      Valid options are:

           -no-checks
              Turns off all checks - use with caution!
              Turning them off assumes that data is purely numeric, doesn't
              contain any missing values, and has a nominal class. Turning them
              off also means that no header information will be stored if the
              machine is linear. Finally, it also assumes that no instance has
              a weight equal to 0.
              (default: checks on)

           -C <double>
              The complexity constant C. (default 1)

           -N
              Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

           -L <double>
              The tolerance parameter. (default 1.0e-3)

           -P <double>
              The epsilon for round-off error. (default 1.0e-12)

           -M
              Fit calibration models to SVM outputs.

           -V <double>
              The number of folds for the internal
              cross-validation. (default -1, use training data)

           -W <double>
              The random number seed. (default 1)

           -K <classname and parameters>
              The Kernel to use.
              (default: weka.classifiers.functions.supportVector.PolyKernel)

           -calibrator <scheme specification>
              Full name of calibration model, followed by options.
              (default: "weka.classifiers.functions.Logistic")

           -output-debug-info
              If set, classifier is run in debug mode and
              may output additional info to the console

           -do-not-check-capabilities
              If set, classifier capabilities are not checked before classifier is built
              (use with caution).

           -num-decimal-places
              The number of decimal places for the output of numbers in the model (default 2).

             Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:

           -E <num>
              The Exponent to use.
              (default: 1.0)

           -L
              Use lower-order terms.
              (default: no)

           -C <num>
              The size of the cache (a prime number), 0 for full cache and
              -1 to turn it off.
              (default: 250007)

           -output-debug-info
              Enables debugging output (if available) to be printed.
              (default: off)

           -no-checks
              Turns off all checks - use with caution!
              (default: checks on)

             Options specific to calibrator weka.classifiers.functions.Logistic:

           -C
              Use conjugate gradient descent rather than BFGS updates.

           -R <ridge>
              Set the ridge in the log-likelihood.

           -M <number>
              Set the maximum number of iterations (default -1, until convergence).

           -output-debug-info
              If set, classifier is run in debug mode and
              may output additional info to the console

           -do-not-check-capabilities
              If set, classifier capabilities are not checked before classifier is built
              (use with caution).

           -num-decimal-places
              The number of decimal places for the output of numbers in the model (default 2).

      Specified by:
        `setOptions` in interface `OptionHandler`
      Overrides:
        `setOptions` in class `AbstractClassifier`
      Parameters:
        `options` - the list of options as an array of strings
      Throws:
        `java.lang.Exception` - if an option is not supported

    - #### getOptions

          public java.lang.String[] getOptions()

      Gets the current settings of the classifier.
      Specified by:
        `getOptions` in interface `OptionHandler`
      Overrides:
        `getOptions` in class `AbstractClassifier`
      Returns:
        an array of strings suitable for passing to setOptions

    - #### setChecksTurnedOff

          public void setChecksTurnedOff(boolean value)

      Disables or enables the checks (which could be time-consuming). Use with caution!
      Parameters:
        `value` - if true turns off all checks

    - #### getChecksTurnedOff

          public boolean getChecksTurnedOff()

      Returns whether the checks are turned off or not.
      Returns:
        true if the checks are turned off

    - #### checksTurnedOffTipText

          public java.lang.String checksTurnedOffTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### kernelTipText

          public java.lang.String kernelTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### setKernel

          public void setKernel(Kernel value)

      sets the kernel to use
      Parameters:
        `value` - the kernel to use

    - #### getKernel

          public Kernel getKernel()

      Returns the kernel to use
      Returns:
        the current kernel

    - #### calibratorTipText

          public java.lang.String calibratorTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### setCalibrator

          public void setCalibrator(Classifier value)

      sets the calibrator to use
      Parameters:
        `value` - the calibrator to use

    - #### getCalibrator

          public Classifier getCalibrator()

      Returns the calibrator to use
      Returns:
        the current calibrator

    - #### cTipText

          public java.lang.String cTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getC

          public double getC()

      Get the value of C.
      Returns:
        Value of C.

    - #### setC

          public void setC(double v)

      Set the value of C.
      Parameters:
        `v` - Value to assign to C.

    - #### toleranceParameterTipText

          public java.lang.String toleranceParameterTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getToleranceParameter

          public double getToleranceParameter()

      Get the value of tolerance parameter.
      Returns:
        Value of tolerance parameter.

    - #### setToleranceParameter

          public void setToleranceParameter(double v)

      Set the value of tolerance parameter.
      Parameters:
        `v` - Value to assign to tolerance parameter.

    - #### epsilonTipText

          public java.lang.String epsilonTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getEpsilon

          public double getEpsilon()

      Get the value of epsilon.
      Returns:
        Value of epsilon.

    - #### setEpsilon

          public void setEpsilon(double v)

      Set the value of epsilon.
      Parameters:
        `v` - Value to assign to epsilon.

    - #### filterTypeTipText

          public java.lang.String filterTypeTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getFilterType

          public SelectedTag getFilterType()

      Gets how the training data will be transformed. Will be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
      Returns:
        the filtering mode

    - #### setFilterType

          public void setFilterType(SelectedTag newType)

      Sets how the training data will be transformed. Should be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
      Parameters:
        `newType` - the new filtering mode

    - #### buildCalibrationModelsTipText

          public java.lang.String buildCalibrationModelsTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getBuildCalibrationModels

          public boolean getBuildCalibrationModels()

      Get the value of buildCalibrationModels.
      Returns:
        Value of buildCalibrationModels.

    - #### setBuildCalibrationModels

          public void setBuildCalibrationModels(boolean newbuildCalibrationModels)

      Set the value of buildCalibrationModels.
      Parameters:
        `newbuildCalibrationModels` - Value to assign to buildCalibrationModels.

    - #### numFoldsTipText

          public java.lang.String numFoldsTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getNumFolds

          public int getNumFolds()

      Get the value of numFolds.
      Returns:
        Value of numFolds.

    - #### setNumFolds

          public void setNumFolds(int newnumFolds)

      Set the value of numFolds.
      Parameters:
        `newnumFolds` - Value to assign to numFolds.

    - #### randomSeedTipText

          public java.lang.String randomSeedTipText()

      Returns the tip text for this property
      Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

    - #### getRandomSeed

          public int getRandomSeed()

      Get the value of randomSeed.
      Returns:
        Value of randomSeed.

    - #### setRandomSeed

          public void setRandomSeed(int newrandomSeed)

      Set the value of randomSeed.
      Parameters:
        `newrandomSeed` - Value to assign to randomSeed.

    - #### toString

          public java.lang.String toString()

      Prints out the classifier.
      Overrides:
        `toString` in class `java.lang.Object`
      Returns:
        a description of the classifier as a string

    - #### getRevision

          public java.lang.String getRevision()

      Returns the revision string.
      Specified by:
        `getRevision` in interface `RevisionHandler`
      Overrides:
        `getRevision` in class `AbstractClassifier`
      Returns:
        the revision

    - #### main

          public static void main(java.lang.String[] argv)

      Main method for testing this class.