| Title: | A Workflow for Statistical Testing and Interpretation |
|---|---|
| Description: | Provides a unified workflow for choosing, running, and interpreting common statistical tests, from group comparisons and analysis of variance to regression, survival analysis, and diagnostic and agreement statistics. The package combines assumption checks, test selection, effect sizes, formatted results, plain-language interpretation, and a sample-size planning module covering continuous, binary, survival, ordinal, bioequivalence, and precision-based designs. Implemented methods follow standard references including Casella and Berger (2002, ISBN:9780534243128), Hollander et al. (2013, ISBN:9781118553299), Agresti (2013, ISBN:9780470463635), Cohen (1988, ISBN:9780805802832), Hosmer, Lemeshow and Sturdivant (2013, ISBN:9780470582473), and Julious (2010, ISBN:9781584887393). |
| Authors: | Imad El Badisy [aut, cre] |
| Maintainer: | Imad El Badisy <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.0.2 |
| Built: | 2026-09-19 23:30:30 UTC |
| Source: | https://github.com/ielbadisy/testflow |
Convert a testflow object to a one-row tibble
as_tibble(x, ...)as_tibble(x, ...)
x |
A testflow object. |
... |
Unused. |
A one-row tibble with the workflow name, design, variables, recommended test, null hypothesis, statistic, degrees of freedom when available, p-value, confidence interval when available, effect-size fields, and decision text.
Simulate a small cardiovascular teaching dataset
make_cardio_data(n = 180, seed = 2026)make_cardio_data(n = 180, seed = 2026)
n |
Number of rows. |
seed |
Random seed. |
A tibble with example numeric and categorical variables.
Plot a testflow object
## S3 method for class 'testflow' plot(x, title = NULL, subtitle = NULL, caption = NULL, ...)## S3 method for class 'testflow' plot(x, title = NULL, subtitle = NULL, caption = NULL, ...)
x |
A testflow object. |
title |
Optional plot title override. Defaults to the stored title. |
subtitle |
Optional plot subtitle override. Defaults to the stored subtitle. |
caption |
Optional plot caption override. Defaults to the stored caption. |
... |
Unused. |
A ggplot object stored in the testflow object, with optional
title, subtitle, and caption overrides applied. If the workflow was created
with plot = FALSE, returns NULL.
Print a summary for a sample size object
## S3 method for class 'summary.sample_size' print(x, ...)## S3 method for class 'summary.sample_size' print(x, ...)
x |
A |
... |
Unused. |
The object invisibly.
Print a testflow object.
## S3 method for class 'testflow' print(x, ...)## S3 method for class 'testflow' print(x, ...)
x |
A testflow object. |
... |
Unused. |
Console colors are enabled by default in interactive sessions. Use
options(testflow.cli_colors = FALSE) to disable colors, or
options(testflow.cli_colors = TRUE) to force colors in non-interactive
output.
The input testflow object, invisibly. Called for its side effect of
printing a formatted workflow summary to the console.
Prints a sumtab() result without column truncation, unlike the default
tibble print method, which hides columns and truncates cell text (e.g.
"58.0 (10.4)…") once the table exceeds the console width.
## S3 method for class 'testflow_sumtab' print(x, ...)## S3 method for class 'testflow_sumtab' print(x, ...)
x |
A |
... |
Unused. |
The input object, invisibly.
Return a ready-to-use testflow report
report(x, ...)report(x, ...)
x |
A testflow object. |
... |
Unused. |
A length-one character vector containing a plain-language summary of the workflow result, including the design, recommended test, p-value, effect size when reported, and null hypothesis when available.
Return a ready-to-use testflow report
report_test(x)report_test(x)
x |
A testflow object. |
A length-one character vector containing the same report text as
report() for a testflow object.
Planning functions for common trial designs across continuous, binary, survival, and ordinal endpoints.
sample_size( endpoint = c("continuous", "binary", "survival", "ordinal"), design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), ... ) sample_size_continuous( design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), delta = NULL, sd = NULL, sd_diff = NULL, expected_difference = 0, margin = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, n_time = 2, correlation = 0.5 ) sample_size_binary( design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), p1, p2, margin = NULL, method = c("pooled", "anticipated"), discordant_or = NULL, discordance_rate = NULL, p10 = NULL, p01 = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, n_time = 2 ) sample_size_survival( design = c("parallel"), objective = c("superiority", "noninferiority", "equivalence"), hr, margin_hr = NULL, lower = NULL, upper = NULL, survival_a = NULL, survival_b = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, method = c("exponential", "ph_only"), accrual_duration = NULL, follow_up = NULL ) sample_size_ordinal( design = c("parallel"), objective = c("superiority"), p_superiority = NULL, alpha = 0.05, power = 0.9, dropout = 0 ) sample_size_adjust_dropout(n, dropout = 0)sample_size( endpoint = c("continuous", "binary", "survival", "ordinal"), design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), ... ) sample_size_continuous( design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), delta = NULL, sd = NULL, sd_diff = NULL, expected_difference = 0, margin = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, n_time = 2, correlation = 0.5 ) sample_size_binary( design = c("parallel", "paired", "repeated"), objective = c("superiority", "noninferiority", "equivalence"), p1, p2, margin = NULL, method = c("pooled", "anticipated"), discordant_or = NULL, discordance_rate = NULL, p10 = NULL, p01 = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, n_time = 2 ) sample_size_survival( design = c("parallel"), objective = c("superiority", "noninferiority", "equivalence"), hr, margin_hr = NULL, lower = NULL, upper = NULL, survival_a = NULL, survival_b = NULL, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, method = c("exponential", "ph_only"), accrual_duration = NULL, follow_up = NULL ) sample_size_ordinal( design = c("parallel"), objective = c("superiority"), p_superiority = NULL, alpha = 0.05, power = 0.9, dropout = 0 ) sample_size_adjust_dropout(n, dropout = 0)
endpoint |
Endpoint family: continuous, binary, survival, or ordinal. |
design |
Study design: parallel, paired, or repeated. Repeated designs are routed to the paired formulas when |
objective |
Planning objective. |
delta |
Effect size or margin on the continuous scale. |
sd |
Between-subject standard deviation. |
sd_diff |
Paired-difference standard deviation. |
expected_difference |
Planned difference from the null boundary. |
margin |
Non-inferiority or equivalence margin on the risk-difference or continuous scale. |
method |
For |
p1, p2
|
Anticipated binary response probabilities. |
discordant_or |
Discordant odds ratio for paired binary planning. |
discordance_rate |
Probability of a discordant pair. |
p10, p01
|
Alternative paired-binary inputs for the discordant cells. |
hr |
Hazard ratio. |
margin_hr |
Non-inferiority hazard-ratio margin. |
lower, upper
|
Equivalence bounds on the hazard-ratio scale. |
survival_a, survival_b
|
Survival probabilities at the planning horizon. |
p_superiority |
Probability that a randomly selected subject from group A exceeds one from group B. |
alpha |
Type I error rate. |
power |
Target power. |
allocation |
Allocation ratio |
dropout |
Expected dropout proportion. |
n_time |
Number of repeated measures. |
correlation |
Within-subject correlation for repeated measures when |
n |
Evaluable sample size before dropout adjustment. |
accrual_duration |
Uniform accrual (enrollment) duration for |
follow_up |
Additional follow-up duration after accrual ends, for |
... |
Endpoint-specific arguments passed to the selected helper. |
The implementation currently covers:
Continuous endpoints: parallel and paired superiority, non-inferiority, and equivalence, plus repeated-measures approximation for 3+ time points.
Binary endpoints: parallel risk-difference planning, paired discordant-pair planning, non-inferiority, and equivalence.
Survival endpoints: event-based superiority, non-inferiority, and equivalence.
Ordinal endpoints: Noether's superiority approximation.
The helper sample_size_adjust_dropout() applies .
See the individual function help pages (e.g. ?sample_size_continuous) for the underlying formulas and their literature references.
A sample_size object with the raw sample size, dropout-adjusted sample size, method label, assumptions, report text, and a plot-ready summary. Formulas and literature references are documented in the individual function help pages, not returned in the object.
Julious SA. Sample Sizes for Clinical Trials. Chapman & Hall/CRC; 2010.
sample_size_continuous( design = "paired", objective = "superiority", delta = 5, sd_diff = 10, alpha = 0.05, power = 0.90 ) sample_size_binary( design = "paired", objective = "superiority", p10 = 0.2, p01 = 0.1, alpha = 0.05, power = 0.90 ) sample_size_continuous( design = "repeated", n_time = 4, correlation = 0.5, objective = "superiority", delta = 5, sd_diff = 10, alpha = 0.05, power = 0.90 )sample_size_continuous( design = "paired", objective = "superiority", delta = 5, sd_diff = 10, alpha = 0.05, power = 0.90 ) sample_size_binary( design = "paired", objective = "superiority", p10 = 0.2, p01 = 0.1, alpha = 0.05, power = 0.90 ) sample_size_continuous( design = "repeated", n_time = 4, correlation = 0.5, objective = "superiority", delta = 5, sd_diff = 10, alpha = 0.05, power = 0.90 )
Sample size for average bioequivalence
sample_size_bioequivalence( design = c("crossover", "parallel"), gmr = 1, cv_within = NULL, cv_between = NULL, lower = 0.8, upper = 1.25, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, method = c("iterative_tost", "normal_approx") )sample_size_bioequivalence( design = c("crossover", "parallel"), gmr = 1, cv_within = NULL, cv_between = NULL, lower = 0.8, upper = 1.25, alpha = 0.05, power = 0.9, allocation = 1, dropout = 0, method = c("iterative_tost", "normal_approx") )
design |
|
gmr |
Anticipated geometric mean ratio (test/reference). |
cv_within |
Within-subject coefficient of variation on the raw
(untransformed) scale. Required for |
cv_between |
Between-subject coefficient of variation on the raw
scale. Required for |
lower |
Lower bioequivalence bound (ratio scale), usually |
upper |
Upper bioequivalence bound (ratio scale), usually |
alpha |
Type I error rate (one-sided; bioequivalence uses two
one-sided tests, each at |
power |
Target power. |
allocation |
Allocation ratio |
dropout |
Expected dropout proportion. |
method |
|
On the log scale, with , ,
, and (must be positive: the anticipated GMR must lie
strictly inside the bioequivalence bounds):
method = "iterative_tost" searches for the smallest such that
the exact two-one-sided-test power is at least the target. Let
for a crossover design (total ,
) or for a
parallel design (per-arm , , ).
The two one-sided test statistics
and each follow a noncentral t
distribution with the above and noncentrality parameters
,
not a location-shifted central t, and not a normal approximation, since
the estimated carries its own sampling variability. Power is
where is the noncentral t CDF (stats::pt(..., ncp = delta)) - this is Phillips's (1990) formula, verified in package
tests to match PowerTOST::power.TOST() (the regulatory-standard,
Owen's-Q-based calculation) to numerical precision for balanced designs.
Coefficients of variation are converted to log-scale standard deviations
via .
method = "normal_approx" uses the closed-form approximation
(crossover) or, per arm,
(parallel), switching to in place of
when exactly, matching the analogous special case in
sample_size_continuous(). iterative_tost is preferred because this
approximation - which treats as known and uses normal
rather than noncentral-t quantiles - can under-size a study by a
material margin (verified up to ~20% at small n) away from
or at small n.
A sample_size object.
Julious SA. Sample Sizes for Clinical Trials. Chapman & Hall/CRC; 2010.
Phillips KF. Power of the two one-sided tests procedure in bioequivalence. Journal of Pharmacokinetics and Biopharmaceutics. 1990;18(2):137-144.
Adjust an individually randomized sample size for cluster randomization
sample_size_cluster_adjust(n_ind, m, rho, cv_m = NULL)sample_size_cluster_adjust(n_ind, m, rho, cv_m = NULL)
n_ind |
Individually randomized sample size (e.g. from
|
m |
Average cluster size. |
rho |
Intracluster correlation. |
cv_m |
Coefficient of variation of cluster sizes, for unequal
cluster sizes. |
Equal cluster size:
Unequal cluster size (approximate), with coefficient of variation
:
The cluster-adjusted sample size.
Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. Arnold; 2000.
Sample size for confidence-interval precision
sample_size_precision( endpoint = c("continuous", "binary"), design = c("one_sample", "two_sample", "odds_ratio"), width, sd = NULL, p = NULL, p1 = NULL, p2 = NULL, alpha = 0.05, allocation = 1, dropout = 0, conservative = FALSE, method = c("wald", "wilson", "exact"), criterion = c("expected", "worst_case"), min_expected_events = 5, max_n = 1e+07 )sample_size_precision( endpoint = c("continuous", "binary"), design = c("one_sample", "two_sample", "odds_ratio"), width, sd = NULL, p = NULL, p1 = NULL, p2 = NULL, alpha = 0.05, allocation = 1, dropout = 0, conservative = FALSE, method = c("wald", "wilson", "exact"), criterion = c("expected", "worst_case"), min_expected_events = 5, max_n = 1e+07 )
endpoint |
Endpoint family: |
design |
|
width |
Desired confidence-interval half-width, on the scale of the
estimate (or of the log odds ratio for |
sd |
Standard deviation (continuous endpoint). |
p |
Anticipated proportion (binary, |
p1 |
Anticipated proportion in arm A (binary, |
p2 |
Anticipated proportion in arm B (binary, |
alpha |
Type I error rate (two-sided CI coverage is |
allocation |
Allocation ratio |
dropout |
Expected dropout proportion. |
conservative |
For |
method |
For |
criterion |
For |
min_expected_events |
For |
max_n |
For |
Unlike the other sample_size_*() functions, precision-based planning is
driven by a target confidence-interval half-width rather than
power against an effect size.
Continuous, one sample:
Continuous, two independent means:
Binary, one proportion, method = "wald" (normal approximation; the
original, backward-compatible formula):
(or for the conservative case)
Binary, one proportion, method = "wilson" or "exact": there is no
closed form. Instead, the smallest integer is found such that the
relevant confidence interval, evaluated at (a) the anticipated event count
when criterion = "expected", or (b) every
event count in a prevalence-local band qbinom(c(0.001, 0.999), n, p)
when criterion = "worst_case", has maximum one-sided half-width
. Because Wilson and
Clopper-Pearson intervals are generally asymmetric around ,
this maximum one-sided distance - not half of the total interval width -
is the planning criterion. The search itself is a deterministic
doubling-then-bisection integer search with a bounded backward repair scan
(see sample_size_binomial_search()), not a closed-form calculation, and
a Clopper-Pearson interval guarantees at-least-nominal coverage at the
evaluated count(s) - this is not the same as guaranteeing the achieved
width at every conceivable outcome. The achieved half-width is not
perfectly monotone in (integer event counts round differently as
changes), so a plain bisection search can converge on an integer
more than one step above the true minimum; the backward repair scan
guards against this (verified against an exhaustive brute-force scan in
the package tests) without falling back to an unbounded linear search.
A precision-based sample size answers "how wide will my confidence interval be", not "how likely am I to observe any events at all"; for rare-prevalence planning (e.g. newborn-screening or rare-adverse-event studies) the expected event count, and the probability of observing zero events, are reported as diagnostics precisely because a narrow interval does not guarantee that any events will be observed.
Binary, two independent proportions, risk-difference half-width, with
allocation ratio :
(reduces to the earlier equal-allocation-only formula when ).
Binary, log-odds-ratio half-width:
A sample_size object. For one-sample binary precision planning,
x$diagnostics additionally holds rare-event and achieved-precision
diagnostics (see Details).
Julious SA. Sample Sizes for Clinical Trials. Chapman & Hall/CRC; 2010.
Wilson EB. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association. 1927;22(158):209-212.
Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934;26(4):404-413.
# 1) Common prevalence, Wald (backward-compatible default) sample_size_precision(endpoint = "binary", design = "one_sample", width = 0.05, p = 0.30) # 2) Rare prevalence, Wilson score, iterative search sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.02, p = 0.01, method = "wilson" ) # 3) Rare prevalence, exact Clopper-Pearson, worst-case criterion sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.005, p = 0.001, method = "exact", criterion = "worst_case" ) # 4) Dropout inflation: complete_case_n vs. adjusted_n x <- sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.02, p = 0.05, method = "wilson", dropout = 0.10 ) x$diagnostics$complete_case_n x$diagnostics$adjusted_n # 5) Two independent proportions, unequal allocation sample_size_precision( endpoint = "binary", design = "two_sample", width = 0.08, p1 = 0.4, p2 = 0.3, allocation = 2 )# 1) Common prevalence, Wald (backward-compatible default) sample_size_precision(endpoint = "binary", design = "one_sample", width = 0.05, p = 0.30) # 2) Rare prevalence, Wilson score, iterative search sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.02, p = 0.01, method = "wilson" ) # 3) Rare prevalence, exact Clopper-Pearson, worst-case criterion sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.005, p = 0.001, method = "exact", criterion = "worst_case" ) # 4) Dropout inflation: complete_case_n vs. adjusted_n x <- sample_size_precision( endpoint = "binary", design = "one_sample", width = 0.02, p = 0.05, method = "wilson", dropout = 0.10 ) x$diagnostics$complete_case_n x$diagnostics$adjusted_n # 5) Two independent proportions, unequal allocation sample_size_precision( endpoint = "binary", design = "two_sample", width = 0.08, p1 = 0.4, p2 = 0.3, allocation = 2 )
Summarize a testflow object.
## S3 method for class 'testflow' summary(object, ...)## S3 method for class 'testflow' summary(object, ...)
object |
A testflow object. |
... |
Unused. |
Console colors follow the same testflow.cli_colors option used by
print.testflow().
A summary.testflow list containing the workflow metadata,
descriptives, assumptions, recommended test, primary and alternative test
results, post-hoc results when available, effect size, decision, and report
text.
Builds a compact table of descriptive summaries using a formula interface. Numeric variables are summarized as mean (SD); median [Q1, Q3]; n. Categorical variables are summarized as n (percent).
sumtab( formula, data, p_value = FALSE, overall = TRUE, digits = 1, p_digits = 3, alpha = 0.05, fisher_threshold = 5, na.rm = TRUE )sumtab( formula, data, p_value = FALSE, overall = TRUE, digits = 1, p_digits = 3, alpha = 0.05, fisher_threshold = 5, na.rm = TRUE )
formula |
A one-sided formula such as |
data |
A data frame. |
p_value |
Logical; add a p-value column when a grouping variable is supplied. |
overall |
Logical; include an overall summary column. |
digits |
Number of digits for summary statistics. |
p_digits |
Number of digits for formatted p-values. |
alpha |
Significance level used by automatic test selection. |
fisher_threshold |
Expected-count threshold for Fisher's exact test. |
na.rm |
Logical; remove missing values before summaries and tests. |
When p_value = TRUE and a grouping variable is supplied,
sumtab() chooses the p-value test automatically. Numeric variables use
Student t-test, Welch t-test, or Wilcoxon rank-sum test for two groups, and
one-way ANOVA, Welch ANOVA, or Kruskal-Wallis test for more than two groups.
Categorical variables use a chi-square test unless expected counts fall below
fisher_threshold, in which case Fisher's exact test is used.
A tibble with one row per numeric variable and one row per categorical level.
dat <- make_cardio_data(80, seed = 1) sumtab(~ age + sex | treatment, dat, p_value = TRUE)dat <- make_cardio_data(80, seed = 1) sumtab(~ age + sex | treatment, dat, p_value = TRUE)
Inter-rater agreement workflow (Cohen's kappa)
test_agreement(data, rater1, rater2, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_agreement(data, rater1, rater2, alpha = 0.05, plot = TRUE, na.rm = TRUE)
data |
A data frame. |
rater1 |
First rater's categorical column. |
rater2 |
Second rater's categorical column, using the same category
set as |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
With observed agreement (the proportion of subjects on which
both raters agree) and chance-expected agreement (from the
marginal category proportions):
Two different standard errors are used for two different purposes, since
the sampling variance of depends on itself:
The confidence interval uses the large-sample SE of Fleiss, Cohen &
Everitt (1969), evaluated at , which accounts for the
full marginal covariance structure (not the simpler
approximation sometimes seen,
which understates the variance when categories are unevenly
distributed).
The z-test of uses a different, smaller-magnitude
SE (Fleiss, 1981), evaluated under the null rather than
at : . Using the
confidence-interval SE for this test instead (a mistake this function
previously made) understates the null SE whenever agreement is
materially above chance - the typical case - inflating the z-statistic
and anti-conservatively shrinking the p-value; verified in package
tests to match irr::kappa2()'s reported z exactly.
A testflow object with class testflow_agreement. The object
is a list containing the cleaned data, categorical descriptives, the test
of kappa against 0 as the primary result with null hypothesis, the
agreement (confusion) table, Cohen's kappa as the effect size, optional
ggplot, original call, and report text.
Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20(1):37-46.
Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin. 1969;72(5):323-327.
Fleiss JL. Statistical Methods for Rates and Proportions (2nd ed.). Wiley; 1981. (Null-hypothesis SE for testing kappa = 0.)
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174. (Magnitude convention.)
Test association between two categorical variables
test_categorical( formula, data, y = NULL, alpha = 0.05, fisher_threshold = 5, plot = TRUE, na.rm = TRUE )test_categorical( formula, data, y = NULL, alpha = 0.05, fisher_threshold = 5, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or a first categorical column when using data-first style. |
y |
Second categorical column. Optional when using formula style. |
alpha |
Significance level. |
fisher_threshold |
Expected-count threshold for Fisher's exact test. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_categorical. The object is
a list containing the cleaned data, categorical descriptives, assumption
checks, recommended association test, primary test result with null
hypothesis, alternative chi-square and Fisher results, effect size, optional
ggplot, original call, and report text.
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, 50(302), 157-175.
Fisher, R. A. (1922). On the interpretation of chi-square from contingency tables, and the calculation of P. Journal of the Royal Statistical Society, 85(1), 87-94.
Cramer, H. (1946). Mathematical Methods of Statistics. Princeton.
Test correlation between two numeric variables
test_correlation( formula, data, y = NULL, method = c("auto", "pearson", "spearman", "kendall"), alpha = 0.05, plot = TRUE, na.rm = TRUE )test_correlation( formula, data, y = NULL, method = c("auto", "pearson", "spearman", "kendall"), alpha = 0.05, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or a first numeric column when using data-first style. |
y |
Second numeric column. Optional when using formula style. |
method |
Correlation method or |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_correlation. The object is
a list containing the cleaned complete-case data, numeric descriptives,
assumption checks, recommended correlation method, primary correlation test
with null hypothesis, Pearson/Spearman/Kendall results, a correlation table,
effect size, optional ggplot, original call, and report text.
Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. Proceedings of the Royal Society of London, 58, 240-242.
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72-101.
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2), 81-93.
Test a correlation matrix
test_correlation_matrix( data, vars, method = c("spearman", "pearson", "kendall"), alpha = 0.05, plot = TRUE, na.rm = TRUE )test_correlation_matrix( data, vars, method = c("spearman", "pearson", "kendall"), alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
vars |
Numeric columns. |
method |
Correlation method. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_correlation_matrix. The
object is a list containing the cleaned data, numeric descriptives,
screening assumptions, selected correlation-matrix method, pairwise
correlation matrix, pairwise p-value table, maximum absolute correlation as
an effect-size summary, optional heatmap ggplot, original call, and report
text.
Cox proportional hazards regression workflow
test_cox(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_cox(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)
formula |
A formula such as |
data |
A data frame. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
With predictors , the Cox model leaves the baseline
hazard unspecified:
Coefficients are reported as hazard ratios . The
overall model test is the likelihood-ratio test against the model with no
predictors, on degrees of freedom (or more, for multi-level
factor terms).
The proportional-hazards assumption is checked via the correlation between scaled Schoenfeld residuals and time (Grambsch & Therneau, 1994); a significant test suggests a predictor's effect on the hazard changes over time.
A testflow object with class testflow_cox. The object is a
list containing the cleaned data, descriptive statistics, a proportional-
hazards assumption check, the overall likelihood-ratio test as the
primary result with null hypothesis, the per-term hazard-ratio
coefficient table, the concordance index as the effect size, optional
ggplot, original call, and report text.
Cox DR. Regression models and life-tables. Journal of the Royal Statistical Society: Series B. 1972;34(2):187-202.
Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika. 1994;81(3):515-526.
Diagnostic test accuracy workflow
test_diagnostic(data, test, reference, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_diagnostic(data, test, reference, alpha = 0.05, plot = TRUE, na.rm = TRUE)
data |
A data frame. |
test |
Binary test-result column (two levels; the alphabetically second level is treated as "positive"). |
reference |
Binary gold-standard column (two levels; the alphabetically second level is treated as "positive"). |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
With true positives , false positives , false negatives
, and true negatives :
Sensitivity, specificity, PPV, NPV, and accuracy each get an exact (Clopper-Pearson) confidence interval. Likelihood ratios get the closed-form log-scale interval of Simel, Samsa & Matchar (1991):
The primary test compares overall accuracy to the no-information rate
(the larger of the two reference-class proportions) via an exact binomial
test, following the same convention as caret::confusionMatrix().
A testflow object with class testflow_diagnostic. The object
is a list containing the cleaned data, categorical descriptives, the
accuracy-vs-no-information-rate test as the primary result with null
hypothesis, a table of sensitivity/specificity/predictive values/
likelihood ratios with confidence intervals, the confusion matrix,
accuracy as the effect size, optional ggplot, original call, and report
text.
Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. Journal of Clinical Epidemiology. 1991;44(8):763-770.
Altman DG, Bland JM. Diagnostic tests 1: sensitivity and specificity. BMJ. 1994;308(6943):1552.
Run a factorial ANOVA workflow
test_factorial( formula, data, factors = NULL, alpha = 0.05, type = 2, plot = TRUE, na.rm = TRUE )test_factorial( formula, data, factors = NULL, alpha = 0.05, type = 2, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or the outcome column when using data-first style. |
factors |
Factor columns selected with tidyselect syntax. Optional when using formula style. |
alpha |
Significance level. |
type |
Sums-of-squares type: |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_factorial. The object is a
list containing the cleaned data, descriptive statistics, residual and
variance assumption checks, recommended factorial ANOVA, primary ANOVA term
result with null hypothesis, ANOVA table, effect size, optional ggplot,
original call, and report text.
Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum.
Compare a numeric outcome across more than two groups
test_groups( formula, data, group = NULL, alpha = 0.05, posthoc = TRUE, plot = TRUE, na.rm = TRUE )test_groups( formula, data, group = NULL, alpha = 0.05, posthoc = TRUE, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or an outcome column when using data-first style. |
group |
Grouping column. Optional when using formula style. |
alpha |
Significance level. |
posthoc |
Logical; compute post-hoc comparisons. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_groups. The object is a
list containing the cleaned data, grouped descriptive statistics, assumption
checks, recommended omnibus test, primary test result with null hypothesis,
alternative omnibus results, post-hoc comparisons when requested, effect
size, optional ggplot, original call, and report text.
Intraclass correlation coefficient workflow
test_icc(data, measures, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_icc(data, measures, alpha = 0.05, plot = TRUE, na.rm = TRUE)
data |
A data frame in wide format (one column per rater/measurement, one row per subject). |
measures |
Rater/measurement columns selected with tidyselect syntax. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
Following Shrout & Fleiss (1979), variance components come from one-way
(, ) and two-way (, , )
ANOVA decompositions of the subjects by raters:
ICC(2,1) (absolute agreement, allowing raters to differ systematically) is reported as the primary/effect-size estimate, following the single-measurement, absolute-agreement, two-way random-effects recommendation of Koo & Li (2016) for typical reliability studies. Confidence intervals use the F-distribution-based formulas of McGraw & Wong (1996); ICC(2,1)'s interval uses a Satterthwaite-approximated denominator degrees of freedom.
A testflow object with class testflow_icc. The object is a
list containing the cleaned long-format data, descriptive statistics, the
F-test for ICC(2,1) as the primary result with null hypothesis, a table
of the one-way (ICC1), two-way random/absolute-agreement (ICC2), and
two-way fixed/consistency (ICC3) estimates with confidence intervals,
ICC(2,1) as the effect size, optional ggplot, original call, and report
text.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420-428.
McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996;1(1):30-46.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine. 2016;15(2):155-163.
Multiple linear regression workflow
test_linear_regression(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_linear_regression(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)
formula |
A formula such as |
data |
A data frame. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
For predictors and observations, ordinary
least squares fits:
The overall model test compares the fitted model to an intercept-only model:
A testflow object with class testflow_linear_regression. The
object is a list containing the cleaned data, descriptive statistics,
residual/homoscedasticity/multicollinearity assumption checks, the overall
model F-test as the primary result with null hypothesis, the per-term
coefficient table, R squared and adjusted R squared as the effect size,
optional ggplot, original call, and report text.
Draper NR, Smith H. Applied Regression Analysis (3rd ed.). Wiley; 1998.
Logistic regression workflow
test_logistic_regression( formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_logistic_regression( formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
With outcome and predictors
:
Coefficients are reported on the log-odds scale and, exponentiated, as
odds ratios .
The overall model test is the likelihood-ratio test against the
intercept-only model, using each model's residual deviance
:
McFadden's pseudo R squared:
A testflow object with class testflow_logistic_regression. The
object is a list containing the cleaned data, descriptive statistics,
multicollinearity/influential-point assumption checks, the likelihood-ratio
test as the primary result with null hypothesis, the per-term coefficient
tables on the log-odds and odds-ratio scale, McFadden's pseudo R squared as
the effect size, optional ggplot, original call, and report text.
Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression (3rd ed.). Wiley; 2013.
Test multinomial goodness of fit
test_multinomial( data, outcome, p = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_multinomial( data, outcome, p = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
outcome |
Categorical outcome column. |
p |
Expected probabilities, or |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_multinomial. The object is
a list containing the cleaned data, categorical descriptives, assumption
checks, recommended goodness-of-fit test, primary chi-square result with null
hypothesis, BH-adjusted pairwise binomial checks, effect-size summary, optional
ggplot, original call, and report text.
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, 50(302), 157-175.
Test one numeric sample against a reference value
test_one_sample( data, outcome, mu = 0, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )test_one_sample( data, outcome, mu = 0, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
outcome |
Numeric outcome column. |
mu |
Reference value. |
alternative |
Alternative hypothesis. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_one_sample. The object is
a list containing the cleaned data, descriptive statistics, assumption
checks, recommended test, primary test result with null hypothesis,
alternative test results, effect size, optional ggplot, original call, and
report text.
Gosset, W. S. (1908). The probable error of a mean. Biometrika, 6(1), 1-25.
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80-83.
Detect numeric outliers
test_outliers( formula, data, group = NULL, method = c("iqr", "mahalanobis", "both"), plot = TRUE, na.rm = TRUE )test_outliers( formula, data, group = NULL, method = c("iqr", "mahalanobis", "both"), plot = TRUE, na.rm = TRUE )
formula |
Numeric columns to screen for outliers. |
data |
A data frame. |
group |
Optional grouping column. |
method |
Outlier method. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_outliers. The object is a
list containing the cleaned data, numeric descriptives, screening
assumptions, selected outlier-screening method, flagged IQR and/or
Mahalanobis rows, outlier-count summary, optional ggplot, original call,
and report text. This is a screening workflow, not a single hypothesis test.
Compare paired before and after numeric measurements
test_paired( formula, data, after = NULL, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )test_paired( formula, data, after = NULL, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or the before column when using data-first style. |
after |
After column. Optional when using formula style. |
alternative |
Alternative hypothesis. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_paired. The object is a
list containing the cleaned paired data, paired difference, descriptive
statistics, assumption checks, recommended paired test, primary test result
with null hypothesis, alternative test results, effect size, optional
ggplot, original call, and report text.
Test paired categorical measurements
test_paired_categorical( data, before, after, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_paired_categorical( data, before, after, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
before |
Before categorical column. |
after |
After categorical column. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_paired_categorical. The
object is a list containing the cleaned paired categorical data, categorical
descriptives, assumption checks, McNemar test result, discordant-pair table,
optional ggplot, original call, and report text.
Test a one-sample proportion
test_proportion( data, outcome, success, p = 0.5, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_proportion( data, outcome, success, p = 0.5, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
outcome |
Categorical outcome column. |
success |
Value counted as success. |
p |
Reference probability. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_proportion. The object is
a list containing the cleaned data, categorical descriptives, assumption
checks, recommended exact or approximate one-sample proportion test, primary
test result with null hypothesis, alternative exact and approximate results,
observed proportion summary, optional ggplot, original call, and report
text.
Clopper, C. J., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4), 404-413.
Run a repeated-measures workflow from wide data
test_repeated( data, measures, id = NULL, between = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_repeated( data, measures, id = NULL, between = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
measures |
Repeated numeric columns selected with tidyselect syntax. |
id |
Optional subject identifier. |
between |
Optional between-subject factor. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_repeated. The object is a
list containing long-form repeated-measures data, numeric descriptives,
assumption checks, recommended repeated-measures ANOVA or Friedman test,
primary test result with null hypothesis, alternative repeated-measures
results, post-hoc paired comparisons, effect size, optional ggplot,
original call, and report text.
Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd.
Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675-701.
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80-83.
Girden, E. R. (1992). ANOVA: Repeated Measures. Sage.
Test repeated categorical measurements
test_repeated_categorical( data, measures, id = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_repeated_categorical( data, measures, id = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
measures |
Repeated binary columns selected with tidyselect syntax. |
id |
Optional subject identifier. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_repeated_categorical. The
object is a list containing the cleaned repeated categorical data,
descriptive counts, assumption checks, Cochran Q test result with null
hypothesis, pairwise McNemar post-hoc comparisons, effect size, optional
ggplot, original call, and report text.
Cochran, W. G. (1950). The comparison of percentages in matched samples. Biometrika, 37(3/4), 256-266.
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157.
Run a repeated-measures workflow from long data
test_repeated_long( data, outcome, within, id, between = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )test_repeated_long( data, outcome, within, id, between = NULL, alpha = 0.05, plot = TRUE, na.rm = TRUE )
data |
A data frame. |
outcome |
Numeric outcome column. |
within |
Within-subject time/condition column. |
id |
Subject identifier column. |
between |
Optional between-subject factor. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_repeated. The object is a
list containing the cleaned long-format data, numeric descriptives,
assumption checks, recommended repeated-measures ANOVA or Friedman test,
primary test result with null hypothesis, alternative repeated-measures
results, post-hoc paired comparisons, effect size, optional ggplot,
original call, and report text.
Receiver operating characteristic (ROC) curve workflow
test_roc(data, predictor, outcome, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_roc(data, predictor, outcome, alpha = 0.05, plot = TRUE, na.rm = TRUE)
data |
A data frame. |
predictor |
Numeric predictor/biomarker column; higher values are assumed to indicate the positive class. |
outcome |
Binary outcome column (two levels; the alphabetically second level is treated as the positive class). |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
The area under the ROC curve equals the probability that a randomly chosen positive-class observation has a higher predictor value than a randomly chosen negative-class observation, and is computed from the Mann-Whitney U statistic:
The standard error uses the closed-form Hanley & McNeil (1982) formula (an asymptotically equivalent, but not numerically identical, alternative to DeLong's method):
The Youden's J statistic identifies the threshold maximizing
.
A testflow object with class testflow_roc. The object is a
list containing the cleaned data, descriptive statistics by outcome
class, the test of AUC against 0.5 as the primary result with null
hypothesis, the full ROC curve and the Youden's-J-optimal threshold, AUC
as the effect size, optional ggplot, original call, and report text.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36.
Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32-35.
Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression (3rd ed.). Wiley; 2013. (AUC magnitude convention, p. 177.)
Kaplan-Meier and log-rank survival workflow
test_survival(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)test_survival(formula, data, alpha = 0.05, plot = TRUE, na.rm = TRUE)
formula |
A formula such as |
data |
A data frame. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
For groups A and B, the log-rank test compares observed to expected events under the null of equal survival distributions:
The companion effect size is the hazard ratio from a univariate Cox model
on the same grouping factor, , reported with its
Wald confidence interval; the hazard ratio is not itself part of the
log-rank test and can disagree with it under strongly non-proportional
hazards.
A testflow object with class testflow_survival. The object is
a list containing the cleaned data, per-group descriptive survival
statistics, the log-rank test as the primary result with null hypothesis,
the Kaplan-Meier curve data, a companion univariate hazard ratio as the
effect size, optional ggplot, original call, and report text.
Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association. 1958;53(282):457-481.
Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemotherapy Reports. 1966;50(3):163-170.
Compare a numeric outcome between two independent groups
test_two_groups( formula, data, group = NULL, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )test_two_groups( formula, data, group = NULL, alternative = c("two.sided", "less", "greater"), alpha = 0.05, plot = TRUE, na.rm = TRUE )
formula |
A formula such as |
data |
A data frame, or an outcome column when using data-first style. |
group |
Two-level grouping column. Optional when using formula style. |
alternative |
Alternative hypothesis. |
alpha |
Significance level. |
plot |
Logical; include a ggplot object. |
na.rm |
Logical; remove missing values. |
A testflow object with class testflow_two_groups. The object is
a list containing the cleaned data, descriptive statistics by group,
assumption checks, recommended test, primary test result with null
hypothesis, alternative test results, effect size, optional ggplot,
original call, and report text.