Revision 9985109256c08d654edb46adb9cb20c913fb1888 authored by Dominique Makowski on 29 May 2019, 14:10:02 UTC, committed by cran-robot on 29 May 2019, 14:10:02 UTC
1 parent fe07bfa
guidelines.Rmd
---
title: "Reporting Guidelines"
output:
github_document:
toc: true
fig_width: 10.08
fig_height: 6
rmarkdown::html_vignette:
toc: true
fig_width: 10.08
fig_height: 6
tags: [r, bayesian, posterior, test]
vignette: >
%\VignetteIndexEntry{Reporting Guidelines}
\usepackage[utf8]{inputenc}
%\VignetteEngine{knitr::rmarkdown}
editor_options:
chunk_output_type: console
---
```{r message=FALSE, warning=FALSE, include=FALSE}
library(knitr)
options(knitr.kable.NA = '')
knitr::opts_chunk$set(comment=">")
options(digits=2)
```
These guidelines can be referred to by citing the package:
- Makowski, D., Ben-Shachar M. S. \& Lüdecke, D. (2019). *Understand and Describe Bayesian Models and Posterior Distributions using bayestestR*. Available from https://github.com/easystats/bayestestR. DOI: [10.5281/zenodo.2556486](https://zenodo.org/record/2556486).
---
# Reporting Guidelines
Based on the previous [**comparison of point-estimates**](https://easystats.github.io/bayestestR/articles/indicesEstimationComparison.html) and [**indices of effect existence**](https://easystats.github.io/bayestestR/articles/indicesExistenceComparison.html), we can draw the following recommendations.
To minimally **describe the posterior distribution** of a parameter, we suggest reporting 1) the **median** as an index of centrality, 2) the **89\% CI** (using HDI rather than quantiles) as an index of centrality and, in the context of null-hypothesis testing, the **Probability of Direction (*pd*)** for effect existence and, especially in the context of confirmatory analyses, the **ROPE percentage** (full, *i.e.*, based on the full posterior distribution) with an explicitly specified range for effect significance.
# Effect *Existence* and *significance*
The ***pd*** and the ***ROPË*** are two indices that give different and independent information: The ***pd*** is a marker of **existence**, consistency and direction of a parameter (whether a parameter has a consistent effect in one or another direction), whereas the percentage in **ROPE** is a index of **significance** (in its primary meaning); informing us whether a parameter is related or not to a non-negligible change (in terms of magnitude) in the outcome.
## Interpretation Rules of Thumb
**The following thresholds are presented as landmarks only, and any use of such "labels" should be explicitly justified. Please consider with caution.**
- **Probability of Direction (*p*d)**: In most cases, it seems that the *pd* corresponds to the frequentist one-sided *p* value through the formula `p value = (1-pd/100)` and to the two-sided *p* value (the most commonly reported) through the formula `p value = 2*(1-pd/100)`. Thus, a `pd` of `95%`, `97.5%` `99.5%` and `99.95%` corresponds approximately to a two-sided *p* value of respectively `.1`, `.05`, `.01` and `.001`. Thus, for convience, we recommend using the following reference values:
- *pd* **\<= 95\%** ~ *p* \> .1: uncertain
- *pd* **\> 95\%** ~ *p* \< .1: possibly existing
- *pd* **\> 97\%**: likely existing
- *pd* **\> 99\%**: probably existing
- *pd* **\> 99.9\%**: certainly existing
- **ROPE (full)**: Extra caution is required as its interpretation highly depends on other parameters such as sample size and ROPE range.
- **\> 99\%** in ROPE: negligible (we can accept the null hypothesis)
- **\> 97.5\%** in ROPE: probably negligible
- **\<= 97.5\%** \& **\>= 2.5\%** in ROPE: undecided significance
- **\< 2.5\%** in ROPE: probably significant
- **\< 1\%** in ROPE: significant (we can reject the null hypothesis)
*Note: If you have any advice, opinion or such, we encourage you to let us know by opening an [discussion thread](https://github.com/easystats/bayestestR/issues) or making a pull request.*
## Template Sentence
Based on these suggestions, a template sentence for minimal reporting of a parameter based on its posterior distribution could be:
- "the effect of *X* has a probability of ***pd*** of being *negative* (Median = ***median***, 89\% CI [***...***, ***...***]) and can be considered as *significant* (***ROPE***\% in ROPE)."
Computing file changes ...