transace                package:Hmisc                R Documentation

_A_d_d_i_t_i_v_e _R_e_g_r_e_s_s_i_o_n _a_n_d _T_r_a_n_s_f_o_r_m_a_t_i_o_n_s _u_s_i_n_g _a_c_e _o_r _a_v_a_s

_D_e_s_c_r_i_p_t_i_o_n:

     'transace' is 'ace' packaged for easily automatically transforming
     all variables in a matrix.  'transace' is a fast one-iteration
     version of 'transcan' without imputation of NAs.  

     'areg.boot' uses 'avas' or 'ace' to fit additive regression models
     allowing all variables in the model (including the
     right-hand-side) to be transformed, with transformations chosen so
     as to optimize certain criteria.  The default method uses 'avas',
     which explicity tries to transform the response variable so as to
     stabilize the variance of the residuals. 
     All-variables-transformed models tend to inflate 'R^2' and it can
     be difficult to get confidence limits for each transformation. 
     'areg.boot' solves both of these problems using the bootstrap.  As
     with the 'validate' function in the Design library, the Efron
     bootstrap is used to estimate the optimism in the apparent 'R^2',
     and this optimism is subtracted from the apparent 'R^2' to optain
     a bias-corrected 'R^2'.  This is done however on the transformed
     response variable scale.

     Tests with 3 predictors show that the AVAS and ACE estimates used
     by 'areg.boot' are unstable unless the sample size exceeds 350. 
     Apparent 'R^2' with low sample sizes can be very inflated, and
     bootstrap estimates of 'R^2' can be even more unstable in such
     cases, resulting in optimism-corrected 'R^2' that are much lower
     even than the actual 'R^2'.  The situation can be improved a
     little by restricting predictor transformations to be monotonic.

     For 'method="avas"' the response transformation is restricted to
     be monotonic.  You can specify restrictions for transformations of
     predictors (and linearity for the response, or its monotonicity if
     using 'ace').  When the first argument is a formula, the function
     automatically determines which variables are categorical (i.e.,
     'factor', 'category', or character vectors).  Specify linear
     transformations by enclosing variables by the identify function
     ('I()'), and specify monotonicity by using 'monotone(variable)'.

     The 'summary' method for 'areg.boot' computes bootstrap estimates
     of standard errors of differences in predicted responses (usually 
     on the original scale) for selected levels of each predictor
     against the lowest level of the predictor.  The smearing estimator
     (see below) can be used here to estimate differences in predicted
     means, medians, or many other statistics.  By default, quartiles
     are used for continuous predictors and all levels are used for
     categorical ones.  See DETAILS below. There is also a 'plot'
     method for plotting transformation estimates, transformations for
     individual bootstrap re-samples, and pointwise confidence limits
     for transformations.  Unless you already have a 'par(mfrow=)' in
     effect with more than one row or column, 'plot' will try to fit
     the plots on one page.  A 'predict' method computes predicted
     values on the original or transformed response scale, or a matrix
     of transformed predictors.  There is a 'Function' method for
     producing a list of S-PLUS functions that perform the final fitted
     transformations.  There is also a 'print' method for 'areg.boot'
     objects.

     When estimated means (or medians or other statistical parameters)
     are requested for models fitted with 'areg.boot' (by
     'summary.areg.boot' or 'predict.areg.boot'), the "smearing"
     estimator of Duan (1983) is used.  Here we estimate the mean of
     the untransformed response by computing the arithmetic mean of
     ginverse(lp + residuals), where ginverse is the inverse of the
     nonparametric transformation of the response (obtained by reverse
     linear interpolation), 'lp' is the linear predictor for an
     individual observation on the transformed scale, and 'residuals'
     is the entire vector of residuals estimated from the fitted model,
     on the transformed scales (n residuals for n original
     observations).  The 'smearingEst' function computes the general
     smearing estimate.  For efficiency 'smearingEst' recognizes that
     quantiles are transformation-preserving, i.e., when one wishes to
     estimate a quantile of the untransformed distribution one just
     needs to compute the inverse transformation of the transformed
     estimate after the chosen quantile of the vector of residuals is
     added to it. When the median is desired, the estimate is
     ginverse(lp + median(residuals)).  See the last example for how
     'smearingEst' can be used outside of 'areg.boot'.

     'Mean' is a generic function that returns an S function to compute
     the estimate of the mean of a variable.  Its input is typically
     some kind of model fit object.  Likewise, 'Quantile' is a generic
     quantile function-producing function.  'Mean.areg.boot' and
     'Quantile.areg.boot' create functions of a vector of linear
     predictors that transform them into the smearing estimates of the
     mean or quantile of the response variable, respectively.
     'Quantile.areg.boot' produces exactly the same value as
     'predict.areg.boot' or 'smearingEst'.  'Mean' approximates the
     mapping of linear predictors to means over an evenly spaced grid
     of by default 200 points.  Linear interpolation is used between
     these points.  This approximate method is much faster than the
     full smearing estimator once 'Mean' creates the function.  These
     functions are especially useful in 'nomogram.Design' (see the
     example on hypothetical data).

_U_s_a_g_e:

     transace(x, monotonic=NULL, categorical=NULL, binary=NULL, pl=TRUE)

     areg.boot(x, y, data, weights, subset, na.action=na.delete, 
               B=100, method=c("avas", "ace"), evaluation=100, valrsq=TRUE, 
               probs=c(.25,.5,.75), ...)

     ## S3 method for class 'areg.boot':
     print(x, ...)

     ## S3 method for class 'areg.boot':
     plot(x, ylim, boot=TRUE, col.boot=2, lwd.boot=.15,
     conf.int=.95, ...)

     smearingEst(transEst, inverseTrans, res,
                 statistic=c('median','quantile','mean','fitted','lp'),
                 q)

     ## S3 method for class 'areg.boot':
     summary(object, conf.int=.95, values, adj.to,
             statistic='median', q, ...)

     ## S3 method for class 'summary.areg.boot':
     print(x, ...)

     ## S3 method for class 'areg.boot':
     predict(object, newdata,
              statistic=c("lp", "median",
                          "quantile", "mean", "fitted", "terms"),
              q=NULL, ...) 

     ## S3 method for class 'areg.boot':
     Function(object, type=c('list','individual'),
              ytype=c('transformed','inverse'),
              prefix='.', suffix='', frame=0, where=1, ...)

     Mean(object, ...)

     Quantile(object, ...)

     ## S3 method for class 'areg.boot':
     Mean(object, evaluation=200, ...)

     ## S3 method for class 'areg.boot':
     Quantile(object, q=.5, ...)

_A_r_g_u_m_e_n_t_s:

       x: for 'transace' a numeric matrix.  For 'areg.boot' 'x' may be
          a numeric matrix or a formula.  For 'print' or 'plot', an
          object created by 'areg.boot'.  For
          'print.summary.areg.boot', and object created by
          'summary.areg.boot'. 

  object: an object created by 'areg.boot', or a model fit object
          suitable for 'Mean' or 'Quantile'. 

transEst: a vector of transformed values.  In log-normal regression
          these could be predicted log(Y) for example. 

inverseTrans: a function specifying the inverse transformation needed
          to change 'transEst' to the original untransformed scale. 
          'inverseTrans' may also be a 2-element list defining a
          mapping from the transformed values to untransformed values. 
          Linear interpolation is used in this case to obtain
          untransform values. 

monotonic: 

categorical: 

  binary: These are vectors of variable names specifying what to assume
          about each column of 'x' for 'transace'.  Binary variables
          are not transformed, of course. 

      pl: set 'pl=FALSE' to prevent 'transace' from plotting each
          fitted transformation 

       y: numeric vector representing the response variable. Not used
          if 'x' is a formula. 

    data: data frame to use if 'x' is a formula and variables are not
          already in the search list 

 weights: a numeric vector of observation weights.  By default, all
          observations are weighted equally. 

  subset: an expression to subset data if 'x' is a formula 

na.action: a function specifying how to handle NAs.  Default is
          'na.delete' (in Hmisc). 

       B: number of bootstrap samples (default=100) 

  method: '"avas"' (the default) or 'ace' 

evaluation: number of equally-spaced points at which to evaluate (and
          save) the nonparametric transformations derived by 'avas' or
          'ace'.  Default is 100.  For 'Mean.areg.boot', 'evaluation'
          is the number of points at which to evaluate exact smearing
          estimates, to approximate them using linear interpolation
          (default is 200). 

  valrsq: set to 'TRUE' to more quickly do bootstrapping without
          validating 'R^2' 

   probs: vector probabilities denoting the quantiles of continuous
          predictors to use in estimating effects of those predictors 

     ...: other arguments to pass to 'avas' or 'ace' (useful if 'x' is
          not a formula) 

     res: a vector of residuals from the transformed model.  Not
          required when 'statistic="lp"' or 'statistic="fitted"'. 

statistic: statistic to estimate with the smearing estimator.  For
          'smearingEst', the default results in computation of the
          sample median of the model residuals, then 'smearingEst' adds
          the median residual and back-transforms to get estimated
          median responses on the original scale.  'statistic="lp"'
          causes predicted transformed responses to be computed.  For
          'smearingEst', the result (for 'statistic="lp"') is the input
          argument 'transEst'.  'statistic="fitted"' gives predicted
          untransformed responses, i.e., ginverse(lp), where ginverse
          is the inverse of the estimated response transformation,
          estimated by reverse linear interpolation on the tabulated
          nonparametric response transformation or by using an explicit
          analytic function. 'statistic="quantile"' generalizes
          '"median"' to any single quantile 'q' which must be
          specified.  "mean"' causes the population mean response to be
          estimated.  For 'predict.areg.boot', 'statistic="terms"'
          returns a matrix of transformed predictors. 'statistic' can
          also be any S-PLUS function that computes a single value on a
          vector of values, such as 'statistic=var'.  Note that in this
          case the function name is not quoted. 

       q: a single quantile of the original response scale to estimate,
          when 'statistic="quantile"', or for 'Quantile.areg.boot'. 

    ylim: 2-vector of y-axis limits 

    boot: set to 'FALSE' to not plot any bootstrapped transformations. 
          Set it to an integer 'k' to plot the first 'k' bootstrap
          estimates. 

col.boot: color for bootstrapped transformations 

lwd.boot: line width for bootstrapped transformations 

conf.int: confidence level (0-1) for pointwise bootstrap confidence
          limits and for estimated effects of predictors in
          'summary.areg.boot'.  The latter assumes normality of the
          estimated effects. 

  values: a list of vectors of settings of the predictors, for
          predictors for which you want to overide settings determined
          from 'probs'.  The list must have named components, with
          names corresponding to the predictors.  Example:
          'values=list(x1=c(2,4,6,8), x2=c(-1,0,1))' specifies that
          'summary' is to estimate the effect on 'y' of changing 'x1'
          from 2 to 4, 2 to 6, 2 to 8, and separately, of changing 'x2'
          from -1 to 0 and -1 to 1. 

  adj.to: a named vector of adjustment constants, for setting all other
          predictors when examining the effect of a single predictor in
          'summary'.  The more nonlinear is the transformation of 'y'
          the more the adjustment settings will matter.  Default values
          are the medians of the values defined by 'values' or 'probs'.
           You only need to name the predictors for which you are
          overriding the default settings. Example:
          'adj.to=c(x2=0,x5=10)' will set 'x2' to 0 and 'x5' to 10 when
          assessing the impact of variation in the other predictors. 

 newdata: a data frame or list containing the same number of values of
          all of the predictors used in the fit.  For 'factor'
          predictors the 'levels' attribute do not need to be in the
          same order as those used in the original fit, and not all
          levels need to be represented.  If 'newdata' is omitted, you
          can still obtain linear predictors (on the transformed
          response scale) and fitted values (on the original response
          scale), but not '"terms"'.  

    type: specifies how 'Function' is to return the series of functions
          that define the transformations of all variables.  By default
          a list is created, with the names of the list elements being
          the names of the variables.  Specify 'type="individual"' to
          have separate functions created in the session frame
          ('frame=0', the default) or in location defined by 'where' if
          'where' is specified.  For the latter method, the names of
          the objects created are the names of the corresponding
          variables, prefixed by 'prefix' and with 'suffix' appended to
          the end. If any of 'frame', 'where', 'prefix', or 'suffix' is
          specified, 'type' is automatically set to '"individual"'. 

   ytype: By default the first function created by 'Function' is the
          y-transformation.  Specify 'ytype="inverse"' to instead
          create the inverse of the transformation, to be able to
          obtain originally scaled y-values. 

  prefix: character string defining the prefix for function names
          created when 'type="individual"'.  By default, the function
          specifying the transformation for variable 'x' will be named
          '.x'. 

  suffix: character string defining the suffix for the function names 

   frame: frame number in which to store functions (see 'assign').  The
          default is frame 0, the session database, which disappears at
          the end of the S-Plus session. 

   where: location in which to store functions (see 'assign').  If
          'where' is specified (e.g., 'where=1' to store functions in
          search position one), 'frame' is ignored.  For R, the value
          of 'where' is passed to 'assign' as the 'pos' argument. 

_D_e_t_a_i_l_s:

     As 'transace' only does one iteration over the predictors, it may
     not find optimal transformations and it will be dependent on the
     order of the predictors in 'x'.

     'ace' and 'avas' standardize transformed variables to have mean
     zero and variance one for each bootstrap sample, so if a predictor
     is not important it will still consistently have a positive
     regression coefficient.  Therefore using the bootstrap to estimate
     standard errors of the additive least squares regression
     coefficients would not help in drawing inferences about the
     importance of the predictors.  To do this, 'summary.areg.boot'
     computes estimates of, e.g., the inter-quartile range effects of
     predictors in predicting the response variable (after
     untransforming it).  As an example, at each bootstrap repetition
     the estimated transformed value of one of the predictors is
     computed at the lower quartile, median, and upper quartile of the
     raw value of the predictor.  These transformed x values are then
     multipled by the least squares estimate of the partial regression
     coefficient for that transformed predictor in predicting
     transformed y.  Then these weighted transformed x values have the
     weighted transformed x value corresponding to the lower quartile
     subtracted from them, to estimate an x effect accounting for
     nonlinearity.  The last difference computed is then the
     standardized effect of raising x from its lowest to its highest
     quartile.  Before computing differences, predicted values are
     back-transformed to be on the original y scale in a way depending
     on 'statistic' and 'q'. The sample standard deviation of these
     effects (differences) is taken over the bootstrap samples, and
     this is used to compute approximate confidence intervals for
     effects and approximate P-values, both assuming normality.

     'predict' does not re-insert NAs corresponding to observations
     that were dropped before the fit, when 'newdata' is omitted.

     'statistic="fitted"' estimates the same quantity as
     'statistic="median"' if the residuals on the transformed response
     have a symmetric distribution. The two provide identical estimates
     when the sample median of the residuals is exactly zero. The
     sample mean of the residuals is constrained to be exactly zero
     although this does not simplify anything.

_V_a_l_u_e:

     'transace' returns a matrix like 'x' but containing transformed
     values.  This matrix has attributes 'rsq' (vector of 'R^2' with
     which each variable can be predicted from the others) and
     'omitted' (row numbers of 'x' that were deleted due to NAs).

     'areg.boot' returns a list of class '"areg.boot"' containing many
     elements, including (if 'valrsq' is 'TRUE') 'rsquare.app' and
     'rsquare.val'.  'summary.areg.boot' returns a list of class
     '"summary.areg.boot"' containing a matrix of results for each
     predictor and a vector of adjust-to settings.  It also contains
     the call and a 'label' for the statistic that was computed.  A
     'print' method for these objects handles the printing. 
     'predict.areg.boot' returns a vector unless 'statistic="terms"',
     in which case it returns a matrix.  'Function.areg.boot' returns
     by default a list of functions whose argument is one of the
     variables (on the original scale) and whose returned values are
     the corresponding transformed values.  The names of the list of
     functions correspond to the names of the original variables.  When
     'type="individual"', 'Function.areg.boot' invisibly returns the
     vector of names of the created function objects. 'Mean.areg.boot'
     and 'Quantile.areg.boot' also return functions.

     'smearingEst' returns a vector of estimates of distribution
     parameters of class '"labelled"' so that 'print.labelled' wil
     print a label documenting the estimate that was used (see
     'label').  This label can be retrieved for other purposes by using
     e.g. 'label(obj)', where 'obj' was the vector returned by
     'smearingEst'.

_A_u_t_h_o_r(_s):

     Frank Harrell 
      Department of Biostatistics 
      Vanderbilt University School of Medicine 
      f.harrell@vanderbilt.edu

_R_e_f_e_r_e_n_c_e_s:

     Harrell FE, Lee KL, Mark DB (1996): Stat in Med 15:361-387.

     Duan N (1983): Smearing estimate: A nonparametric retransformation
     method.  JASA 78:605-610.

     Wang N, Ruppert D (1995): Nonparametric estimation of the
     transformation in the transform-both-sides regression model.  JASA
     90:522-534. 

     See 'avas', 'ace' for primary references.

_S_e_e _A_l_s_o:

     'avas', 'ace', 'ols', 'validate', 'predab.resample', 'label',
     'nomogram'

_E_x_a_m_p_l_e_s:

     # xtrans <- transace(cbind(age,sex,blood.pressure,race.code),
     #                    binary='sex', monotonic='age',
     #                    categorical='race.code')

     # Generate random data from the model y = exp(x1 + epsilon/3) where
     # x1 and epsilon are Gaussian(0,1)
     set.seed(171)  # to be able to reproduce example
     x1 <- rnorm(200)
     x2 <- runif(200)  # a variable that is really unrelated to y]
     x3 <- factor(sample(c('cat','dog','cow'), 200,TRUE))  # also unrelated to y
     y  <- exp(x1 + rnorm(200)/3)
     f  <- areg.boot(y ~ x1 + x2 + x3, B=40)
     f
     plot(f)
     # Note that the fitted transformation of y is very nearly log(y)
     # (the appropriate one), the transformation of x1 is nearly linear,
     # and the transformations of x2 and x3 are essentially flat 
     # (specifying monotone(x2) would have resulted in a smaller 
     # confidence band for x2)

     summary(f)

     # use summary(f, values=list(x2=c(.2,.5,.8))) for example if you
     # want to use nice round values for judging effects

     # Plot Y hat vs. Y (this doesn't work if there were NAs)
     plot(fitted(f), y)  # or: plot(predict(f,statistic='fitted'), y)

     # Show fit of model by varying x1 on the x-axis and creating separate
     # panels for x2 and x3.  For x2 using only a few discrete values
     newdat <- expand.grid(x1=seq(-2,2,length=100),x2=c(.25,.75),
                           x3=c('cat','dog','cow'))
     yhat <- predict(f, newdat, statistic='fitted')  
     # statistic='mean' to get estimated mean rather than simple inverse trans.
     xYplot(yhat ~ x1 | x2, groups=x3, type='l', data=newdat)

     ## Not run: 
     # Another example, on hypothetical data
     f <- areg.boot(response ~ I(age) + monotone(blood.pressure) + race)
     # use I(response) to not transform the response variable
     plot(f, conf.int=.9)
     # Check distribution of residuals
     plot(fitted(f), resid(f))
     qqnorm(resid(f))
     # Refit this model using ols so that we can draw a nomogram of it.
     # The nomogram will show the linear predictor, median, mean.
     # The last two are smearing estimators.
     Function(f, type='individual')  # create transformation functions
     f.ols <- ols(.response(response) ~ age + 
                  .blood.pressure(blood.pressure) + .race(race))
     # Note: This model is almost exactly the same as f but there
     # will be very small differences due to interpolation of
     # transformations
     meanr <- Mean(f)      # create function of lp computing mean response
     medr  <- Quantile(f)  # default quantile is .5
     nomogram(f.ols, fun=list(Mean=meanr,Median=medr))

     # Create S-PLUS functions that will do the transformations
     # This is a table look-up with linear interpolation
     g <- Function(f)
     plot(blood.pressure, g$blood.pressure(blood.pressure))
     # produces the central curve in the last plot done by plot(f)
     ## End(Not run)

     # Another simulated example, where y has a log-normal distribution
     # with mean x and variance 1.  Untransformed y thus has median
     # exp(x) and mean exp(x + .5sigma^2) = exp(x + .5)
     # First generate data from the model y = exp(x + epsilon),
     # epsilon ~ Gaussian(0, 1)

     set.seed(139)
     n <- 1000
     x <- rnorm(n)
     y <- exp(x + rnorm(n))
     f <- areg.boot(y ~ x, B=20)
     plot(f)       # note log shape for y, linear for x.  Good!
     xs <- c(-2, 0, 2)
     d <- data.frame(x=xs)
     predict(f, d, 'fitted')
     predict(f, d, 'median')   # almost same; median residual=-.003
     exp(xs)                   # population medians
     predict(f, d, 'mean')
     exp(xs + .5)              # population means

     # Show how smearingEst works
     res <- c(-1,0,1)          # define residuals
     y <- 1:5
     ytrans <- log(y)
     ys <- seq(.1,15,length=50)
     trans.approx <- list(x=log(ys), y=ys)
     options(digits=4)
     smearingEst(ytrans, exp, res, 'fitted')          # ignores res
     smearingEst(ytrans, trans.approx, res, 'fitted') # ignores res 
     smearingEst(ytrans, exp, res, 'median')          # median res=0
     smearingEst(ytrans, exp, res+.1, 'median')       # median res=.1
     smearingEst(ytrans, trans.approx, res, 'median')
     smearingEst(ytrans, exp, res, 'mean')
     mean(exp(ytrans[2] + res))                       # should equal 2nd # above
     smearingEst(ytrans, trans.approx, res, 'mean')
     smearingEst(ytrans, trans.approx, res, mean)
     # Last argument can be any statistical function operating
     # on a vector that returns a single value

