% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/boot_test.R
\name{boot_test}
\alias{boot_test}
\title{Test for equivalence of a metric}
\source{
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F.,
Sanchez, J.-C., & Müller, M. (2011). pROC: An open-source package for R
and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 77.
https://doi.org/10.1186/1471-2105-12-77
}
\usage{
boot_test(x, y = NULL, variable = "AUC", in_bag = TRUE, correction = "holm")
}
\arguments{
\item{x}{A cutpointr object with bootstrap results}

\item{y}{If x does not contain subgroups another cutpointr object}

\item{variable}{The variable for testing}

\item{in_bag}{Whether the in-bag or out-of-bag results should be used for testing}

\item{correction}{The type of correction for multiple testing. Possible
values are as in p.adjust.methods}
}
\value{
A data.frame (a tibble) with the columns test_var, p, d, sd_d, z
and in_bag. If a grouped cutpointr object was tested, the additional
columns subgroup1, subgroup2 and p_adj are returned.
}
\description{
This function performs a significance test based on the bootstrap results
of cutpointr to test whether a chosen metric is equal between subgroups
or between two cutpointr objects. The test statistic is calculated as
the standardized difference of the metric between groups. If \code{x}
contains subgroups, the test is run on all possible pairings of subgroups.
An additional adjusted p-value is returned in that case.
}
\details{
The variable name is looked up in the columns of the bootstrap results
where the suffixes _b and _oob indicate in-bag and out-of-bag estimates,
respectively (controlled via the \code{in_bag} argument).
Possible values are optimal_cutpoint, AUC,
acc, sensitivity, specificity, and the metric that was selected
in \code{cutpointr}. Note that there is no "out-of-bag optimal cutpoint", so
when selecting \code{variable = optimal_cutpoint} the test will be based on
the in-bag data.

The test statistic is calculated as z = (t1 - t2) / sd(t1 - t2) where t1 and
t2 are the metric values on the full sample and sd(t1 - t2) is the standard
deviation of the differences of the metric values per bootstrap repetition.
The test is two-sided.

If two cutpointr objects are compared and the numbers of bootstrap repetitions
differ, the smaller number will be used.
}
\examples{
\dontrun{
library(cutpointr)
library(dplyr)
set.seed(734)
cp_f <- cutpointr(suicide \%>\% filter(gender == "female"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
set.seed(928)
cp_m <- cutpointr(suicide \%>\% filter(gender == "male"), dsi, suicide,
  boot_runs = 1000, boot_stratify = TRUE)
# No significant differences:
boot_test(cp_f, cp_m, AUC, in_bag = TRUE)
boot_test(cp_f, cp_m, sum_sens_spec, in_bag = FALSE)

set.seed(135)
cp <- cutpointr(suicide, dsi, suicide, gender, boot_runs = 1000,
  boot_stratify = TRUE)
# Roughly same result as above:
boot_test(cp, variable = AUC, in_bag = TRUE)
boot_test(cp, variable = sum_sens_spec, in_bag = FALSE)
}

}
\seealso{
Other main cutpointr functions: 
\code{\link{add_metric}()},
\code{\link{boot_ci}()},
\code{\link{cutpointr}()},
\code{\link{multi_cutpointr}()},
\code{\link{predict.cutpointr}()},
\code{\link{roc}()}
}
\concept{main cutpointr functions}