%\VignetteIndexEntry{sdpt3r}
% JSS Article Template
\documentclass{article}


%%% Additional Packages %%%

% Algorithms
% http://tex.stackexchange.com/questions/229355/algorithm-algorithmic-algorithmicx-algorithm2e-algpseudocode-confused
\usepackage{algorithm}% http://ctan.org/pkg/algorithms
\usepackage{algpseudocode}% http://ctan.org/pkg/algorithmicx


% Font stuff
\usepackage[T1]{fontenc}% for correct hyphenation and T1 encoding
%\usepackage{lmodern}% latin modern font
\usepackage[american]{babel}% for American English

% Math fonts/symbols
\usepackage{amsmath}% sophisticated mathematical formulas with amstex (includes \text{})
\usepackage{mathtools}% fix amsmath deficiencies
\usepackage{amssymb}% sophisticated mathematical symbols with amstex
\usepackage{amstext}% ams mathematical fonts
\usepackage{amsfonts}% ams mathematical fonts

% Figures
\usepackage{graphicx}% for including figures

%%% Commands %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% vetors
\newcommand{\ve}[1]{\mathbf{#1}}           % for vetors
\newcommand{\sv}[1]{\boldsymbol{#1}}   % for greek letters
\newcommand{\m}[1]{\mathbf{#1}}               % for matrices
\newcommand{\sm}[1]{\boldsymbol{#1}}   % for greek letters
\newcommand{\tr}[1]{{#1}^{\mkern-1.5mu\mathsf{T}}}              % for transpose
\newcommand{\norm}[1]{||{#1}||}              % for transpose
\newcommand*{\mve}{\operatorname{ve}}
\newcommand*{\trace}{\operatorname{trace}}
\newcommand*{\rank}{\operatorname{rank}}
\newcommand*{\diag}{\operatorname{diag}}
\newcommand*{\vspan}{\operatorname{span}}
\newcommand*{\rowsp}{\operatorname{rowsp}}
\newcommand*{\colsp}{\operatorname{colsp}}
\newcommand*{\svd}{\operatorname{svd}}
\newcommand*{\edm}{\operatorname{edm}}  % euclidean distance matrix (D * D)


% statistical
\newcommand{\widebar}[1]{\overline{#1}}  

% 
% operators
\newcommand{\Had}{\circ}
\DeclareMathOperator*{\lmin}{Minimize}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\arginf}{arg\,inf}
\DeclareMathOperator*{\argsup}{arg\,sup}
%\newcommand*{\arginf}{\operatorname*{arginf}}
%\newcommand*{\argsup}{\operatorname*{argsup}}

% Sets
\newcommand*{\intersect}{\cap}
\newcommand*{\union}{\cup}
\let\oldemptyset\emptyset
\let\emptyset\varnothing

% Fields, Reals, etc. etc
\newcommand{\field}[1]{\mathbb{#1}}
\newcommand{\Reals}{\field{R}}
\newcommand{\Integers}{\field{Z}}
\newcommand{\Naturals}{\field{N}}
\newcommand{\Complex}{\field{C}}
\newcommand{\Rationals}{\field{Q}}

% Hyphenation
\hyphenation{Ar-chi-me-dean}

% Editorial
\newcommand*{\TODO}[1]{\textcolor{red}{TODO: #1}}
\newcommand*{\NOTE}[1]{\textcolor{blue}{Note: #1}}
\newtheorem{theorem}{Theorem}
\newtheorem{Proof}{Proof}


\usepackage[total={6.5in,8.75in}, top=0.9in, left=0.7in, right=0.7in, includefoot]{geometry}

% Misc
%\makeatletter
%\newcommand\myisodate{\number\year-\ifcase\month\or 01\or 02\or 03\or 04\or 05\or 06\or 07\or 08\or 09\or 10\or 11\or 12\fi-\ifcase\day\or 01\or 02\or 03\or 04\or 05\or 06\or 07\or 08\or 09\or 10\or 11\or 12\or 13\or 14\or 15\or 16\or 17\or 18\or 19\or 20\or 21\or 22\or 23\or 24\or 25\or 26\or 27\or 28\or 29\or 30\or 31\fi}% create iso date
\makeatother
%\newcommand*{\abstractnoindent}{}% define abstract such that it has no indent
%\let\abstractnoindent\abstract
%\renewcommand*{\abstract}{\let\quotation\quote\let\endquotation\endquote
%  \abstractnoindent}
%\deffootnote[1em]{1em}{1em}{\textsuperscript{\thefootnotemark}}% setting for footnote

\author{Adam Rahman}
\title{Distance Weighted Discrimination}


%% need no \usepackage{Sweave.sty}

\begin{document}
\SweaveOpts{concordance=FALSE}

\maketitle

Given two sets of points in a matrix $\m{X} \in \mathcal{R}^{n}$ with associated class variables [-1,1] in $\m{Y} = diag(\ve{y})$, distance weighted discrimination (\cite{marron2007distance}) seeks to classify the points into two distinct subsets by finding a hyperplane between the two sets of points. Mathematically, the distance weighted discrimination problem seeks a hyperplane defined by a normal vector, $\sv{\omega}$, and position, $\beta$, such that each element in the residual vector $\bar{\ve{r}} = \m{Y}\tr{\m{X}}\sv{\omega} + \beta \ve{y}$ is positive and large. Since the class labels are either 1 or -1, having the residuals be positive is equivalent to having the points on the proper side of the hyperplane.

Of course, it may be impossible to have a perfect separation of points using a linear hyperplane, so an error term $\xi$ is introduced. Thus, the perturbed residuals are defined to be

\[
\ve{r} =  \m{Y}\tr{\m{X}}\sv{\omega} + \beta \ve{y} + \sv{\xi}
\]

Distance Weighted Discrimination solves the following optimization problem to find the optimal hyperplane\cite{marron2007distance}.

\[
\begin{array}{ll}
\underset{\ve{r},~\sv{\omega},~\beta,~\sv{\xi}}{\text{minimize}} & \sum_{i=1}^{n}(1/r_{i}) + C\tr{\ve{1}}\sv{\xi} \\
\text{subject to} & \\
& \begin{array}{rl}
\ve{r} &=~~ \m{Y}\tr{\m{X}}\sv{\omega} + \beta \ve{y} + \sv{\xi} \\
\tr{\sv{\omega}}\sv{\omega} &\leq~~ 1 \\
\ve{r} &\geq~~ \ve{0} \\
\sv{\xi} &\geq~~ \ve{0}
\end{array}
\end{array}
\]

\noindent where $C > 0$ is a penalty parameter to be chosen. 

The function \verb!dwd! takes as input two $n \times p$ matrices \verb!X1! and \verb!X2! containing the points to be separated, as well as a penalty term \verb!C! $\geq 0$ penalizing the movement of a point on the wrong side of the hyperplane to the proper side, and returns the input variables necessary for \verb!sqlp! to solve the distance weighted discrimination problem. 

\begin{verbatim}
R> out <- dwd(X1,X2,C)
R> blk <- out$blk
R> At <- out$At
R> C <- out$C
R> b <- out$b

R> sqlp(blk,At,C,b)
\end{verbatim}

\section*{Numerical Example}

Consider two point configurations - \verb!An! and \verb!Ap! - which we would like to classify using distance weighted discrimination. Each point configuration is a matrix containing 50 points in three dimensional space.

\begin{verbatim}
R> data(Andwd)
R> data(Apdwd)
R> d <- ncol(Andwd)

R> head(Andwd)

         V1     V2     V3
[1,]  0.214 -1.577 -1.525
[2,]  0.480  0.624 -0.501
[3,]  0.088  0.330 -1.213
[4,]  0.444 -0.398 -0.630
[5,] -0.363 -1.081 -1.447
[6,]  0.123 -0.077 -0.167

R> head(Apdwd)

         V1     V2    V3
[1,] -0.687  0.192 0.726
[2,]  0.444  0.782 0.887
[3,]  2.360 -1.114 0.089
[4,]  2.230  1.428 1.369
[5,]  1.555 -0.142 2.138
[6,]  0.259  0.163 1.818
\end{verbatim}

Distane weighed discrimination is used to seperate these two configrurations by specifying an appropriate penalization term. Here, we will take a value of 0.5.

\begin{verbatim}
R> out <- dwd(Apdwd,Andwd,0.5)
R> blk <- out$blk
R> At <- out$At
R> C <- out$C
R> b <- out$b

R> out <- sqlp(blk, At, C, b)
\end{verbatim}

The information defining the seperating hyperplane ($\omega$ and $\beta$) is stored in the \verb!X! output vector. 

\begin{verbatim}
X <- out$X

omega <- X[[1]][2:(d+1)]
beta <- X[[1]][d+3]

omega
          [,1]
[1,] 0.6567689
[2,] 0.4857645
[3,] 0.5767907

beta

[1] -0.7520769
\end{verbatim}

\bibliographystyle{plain}
\bibliography{sdpt3r}
\end{document}