% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/boot_test.R
\name{boot_test}
\alias{boot_test}
\title{Test for equivalence of a metric}
\source{
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F.,
Sanchez, J.-C., & Müller, M. (2011). pROC: An open-source package for R
and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 77.
https://doi.org/10.1186/1471-2105-12-77
}
\usage{
boot_test(x, y = NULL, variable = "AUC", in_bag = TRUE, correction = "holm")
}
\arguments{
\item{x}{A cutpointr object with bootstrap results}

\item{y}{If x does not contain subgroups another cutpointr object}

\item{variable}{The variable for testing}

\item{in_bag}{Whether the in-bag or out-of-bag results should be used for testing}

\item{correction}{The type of correction for multiple testing. Possible
values are as in p.adjust.methods}
}
\value{
A data.frame (a tibble) with the columns test_var, p, d, sd_d, z
and in_bag. If a grouped cutpointr object was tested, the additional
columns subgroup1, subgroup2 and p_adj are returned.
}
\description{
This function performs a significance test based on the bootstrap results
of cutpointr to test whether a chosen metric is equal between subgroups
or between two cutpointr objects. The test statistic is calculated as
the standardized difference of the metric between groups. If \code{x}
contains subgroups, the test is run on all possible pairings of subgroups.
An additional adjusted p-value is returned in that case.
}
\details{
The variable name is looked up in the columns of the bootstrap results
where the suffixes _b and _oob indicate in-bag and out-of-bag estimates,
respectively (controlled via the \code{in_bag} argument).
Possible values are optimal_cutpoint, AUC,
acc, sensitivity, specificity, and the metric that was selected
in \code{cutpointr}. Note that there is no "out-of-bag optimal cutpoint", so
when selecting \code{variable = optimal_cutpoint} the test will be based on
the in-bag data.

The test statistic is calculated as z = (t1 - t2) / sd(t1 - t2) where t1 and
t2 are the metric values on the full sample and sd(t1 - t2) is the standard
deviation of the differences of the metric values per bootstrap repetition.
The test is two-sided.

If two cutpointr objects are compared and the numbers of bootstrap repetitions
differ, the smaller number will be used.

Since pairwise differences are calculated for this test, the test function
does not support multiple optimal cutpoints, because it is unclear how the
differences should be calculated in that case.
}
\examples{
\dontrun{
library(cutpointr)
library(dplyr)
set.seed(734)
cp_f <- cutpointr(suicide \%>\% filter(gender == "female"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
set.seed(928)
cp_m <- cutpointr(suicide \%>\% filter(gender == "male"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
# No significant differences:
boot_test(cp_f, cp_m, AUC, in_bag = TRUE)
boot_test(cp_f, cp_m, sum_sens_spec, in_bag = FALSE)

set.seed(135)
cp <- cutpointr(suicide, dsi, suicide, gender, boot_runs = 1000,
  boot_stratify = TRUE)
# Roughly same result as above:
boot_test(cp, variable = AUC, in_bag = TRUE)
boot_test(cp, variable = sum_sens_spec, in_bag = FALSE)
}

}
\seealso{
Other main cutpointr functions: 
\code{\link{add_metric}()},
\code{\link{boot_ci}()},
\code{\link{cutpointr}()},
\code{\link{multi_cutpointr}()},
\code{\link{predict.cutpointr}()},
\code{\link{roc}()}
}
\concept{main cutpointr functions}