modelsCompare {accuracy} | R Documentation

## Functions for comparing results: LRE, compare, agrees

### Description

Use these functions to compare a set of results from an analysis, to check if they agree to a certain number of significant digits.

### Usage

            # compare numeric values against correct values
            LRE(x,correct, use.LAE=TRUE, cutoff=16)

            # compare the results from two possibly identical models
            modelsAgree(model, model2=NULL, digits=3, ...)
            modelsCompare(model, model2=NULL, param.function=modelBetas,
                                     similarity.function=LRE, summary.function=min)

            # extract betas or entire coefficient summaries
            modelBetas(model)
            modelSummary(model)

### Arguments

`x` | vector of computed values
`correct` | vector of correct values
`use.LAE` | use log absolute error if $c_i$==0. If false, returns NA where $c_i$==0
`cutoff` | In LRE results, convert INF to cutoff. Since perfect agreement yields an infinite LRE otherwise. Default value of 16 is based on the number of significant digits for doubles.
`model` | first model for comparison, or if model2=NULL, supply a list of models for comparison
`model2` | second model for comparison, if
`digits` | number of digits to use for comparison
`param.function` | function used to extract model parameters for comparison
`similarity.function` | function used to compute similarity between sets of model parameters
`summary.function` | function used to summarize differences
`...` | parameters to pass from modelsAgree to modelsCompare

### Details

modelsAgree is a convenienence function that calls modelsCompare to summarize similarities between models results.

### Value

Returns a new vector of log-relative-errors (log absolute error where $c_i$ ==0). The resulting values are roughly interpretable as the number of significant digits of agreement between c and x. Larger numbers indicate that x is a relatively more accurate approximation of c. Numbers less than or equal to zero indicate complete disagreement between x and c.

### Author(s)

Micah Altman Micah_Altman@harvard.edu http://www.hmdc.harvard.edu/micah_altman/, Michael McDonald

### References

Altman, M., J. Gill and M. P. McDonald. 2003. Numerical Issues in Statistical Computing for the Social Scientist. John Wiley & Sons. http://www.hmdc.harvard.edu/numerical_issues/

### See Also

See Also as `ttst`

### Examples

    # simple LRE examples
    LRE(1.001,1) # roughly 3 significant digits agreement
    LRE(1,1) # complete agreement
    LRE(20,1) # complete disagreement

    # compare two models

    if (is.R()) {
     hasNmle=require(nlme,quietly=TRUE)
    }  else {
      hasNmle=require(nlme3,quietly=TRUE)
    }

    if (hasNmle) {
      O2<-Orthodont
      O3<-Orthodont

      O2[,2]<-O2[,2]+.1
      O3[,2]<-O2[,2]+.11
      if (is.R()) {
        fm1<- lmList(Orthodont)
        fm2<- lmList(O2)
        fm3<- lmList(O2)
      } else {
        # workaround bug in lmlist in S-Plus
        fm1<- eval(substitute(lmList(Orthodont)))
        fm2<- eval(substitute(lmList(O2)))
        fm3<- eval(substitute(lmList(O2)))
      }

      # do the three models agree?
      print(modelsAgree(list(fm1,fm2,fm2)))

      # show details of diagreement between first 2 models
      print(modelsCompare(fm1,fm2))

      #compare betas at 2 significant digits
      print(modelsAgree(fm1,fm2,digits=2,param.function=modelBetas))
      #compare betas at 1 significant digit
      print(modelsAgree(fm1,fm2,digits=1,param.function=modelBetas))
    }

[Package accuracy version 1.35 Index]