P values and scientific communication: a (very) small step

Nearly everyone recognizes the shortcomings of p values and the associated null hypothesis significance testing framework. Of course, much has been written about it, including potential alternatives. While change is needed, it is hard — and it will take time.

At present, I would advocate that the scientific community adopt the suggestion by John Carlin to remove the s-word:

  • “Eliminate the term ‘statistical significance’ from the scientific discourse”

In the past few years, I have been using something similar, but I think a stronger stance is needed by all of us (see also Carlin’s suggestions). In the research from my lab, we use more and more expressions such as “an interaction effect was detected (F value, p value)”.

As important, I suggest avoiding (like the plague) sentences saying that “there was no difference between conditions 1 and 2” — not to mention statements to the effect that conditions 1 and 2 are equivalent. A much better way of expressing this is to say that differences between conditions 1 and 2 were not detected (t value, p value)”.

This is a tiny step. By itself it won’t do much, but in conjunction with educating a new generation of researchers in alternative methods, it will help start changing scientific reporting — an hopefully improve science.