Raw File
Tip revision: 9985109256c08d654edb46adb9cb20c913fb1888 authored by Dominique Makowski on 29 May 2019, 14:10:02 UTC
version 0.2.0
Tip revision: 9985109
title: "Reporting Guidelines"
    toc: true
    fig_width: 10.08
    fig_height: 6
    toc: true
    fig_width: 10.08
    fig_height: 6
tags: [r, bayesian, posterior, test]
vignette: >
  %\VignetteIndexEntry{Reporting Guidelines}
  chunk_output_type: console

```{r message=FALSE, warning=FALSE, include=FALSE}
options(knitr.kable.NA = '')

These guidelines can be referred to by citing the package:

- Makowski, D., Ben-Shachar M. S. \& Lüdecke, D. (2019). *Understand and Describe Bayesian Models and Posterior Distributions using bayestestR*. Available from DOI: [10.5281/zenodo.2556486](


# Reporting Guidelines

Based on the previous [**comparison of point-estimates**]( and [**indices of effect existence**](, we can draw the following recommendations.

To minimally **describe the posterior distribution** of a parameter, we suggest reporting 1) the **median** as an index of centrality, 2) the **89\% CI** (using HDI rather than quantiles) as an index of centrality and, in the context of null-hypothesis testing, the **Probability of Direction (*pd*)** for effect existence and, especially in the context of confirmatory analyses, the **ROPE percentage** (full, *i.e.*, based on the full posterior distribution) with an explicitly specified range for effect significance. 

# Effect *Existence* and *significance*

The ***pd*** and the ***ROPË*** are two indices that give different and independent information: The ***pd*** is a marker of **existence**, consistency and direction of a parameter (whether a parameter has a consistent effect in one or another direction), whereas the percentage in **ROPE** is a index of **significance** (in its primary meaning); informing us whether a parameter is related or not to a non-negligible change (in terms of magnitude) in the outcome.

## Interpretation Rules of Thumb

**The following thresholds are presented as landmarks only, and any use of such "labels" should be explicitly justified. Please consider with caution.**

- **Probability of Direction (*p*d)**: In most cases, it seems that the *pd* corresponds to the frequentist one-sided *p* value through the formula `p value = (1-pd/100)` and to the two-sided *p* value (the most commonly reported) through the formula `p value = 2*(1-pd/100)`. Thus, a `pd` of `95%`, `97.5%` `99.5%` and `99.95%` corresponds approximately to a two-sided *p* value of respectively `.1`, `.05`, `.01` and `.001`. Thus, for convience, we recommend using the following reference values:

    - *pd* **\<= 95\%** ~ *p* \> .1: uncertain
    - *pd* **\> 95\%** ~ *p* \< .1: possibly existing
    - *pd* **\> 97\%**: likely existing
    - *pd* **\> 99\%**: probably existing
    - *pd* **\> 99.9\%**: certainly existing

- **ROPE (full)**: Extra caution is required as its interpretation highly depends on other parameters such as sample size and ROPE range.

    - **\> 99\%** in ROPE: negligible (we can accept the null hypothesis)
    - **\> 97.5\%** in ROPE: probably negligible
    - **\<= 97.5\%** \& **\>= 2.5\%** in ROPE: undecided significance
    - **\< 2.5\%** in ROPE: probably significant
    - **\< 1\%** in ROPE: significant (we can reject the null hypothesis)

*Note: If you have any advice, opinion or such, we encourage you to let us know by opening an [discussion thread]( or making a pull request.*

## Template Sentence

Based on these suggestions, a template sentence for minimal reporting of a parameter based on its posterior distribution could be:

- "the effect of *X* has a probability of ***pd*** of being *negative* (Median = ***median***, 89\% CI [***...***, ***...***]) and can be considered as *significant* (***ROPE***\% in ROPE)."
back to top