P values and scientific communication: a (very) small step

Nearly everyone recognizes the shortcomings of p values and the associated null hypothesis significance testing framework. Of course, much has been written about it, including potential alternatives. While change is needed, it is hard — and it will take time.

At present, I would advocate that the scientific community adopt the suggestion by John Carlin to remove the s-word:

  • “Eliminate the term ‘statistical significance’ from the scientific discourse”

In the past few years, I have been using something similar, but I think a stronger stance is needed by all of us (see also Carlin’s suggestions). In the research from my lab, we use more and more expressions such as “an interaction effect was detected (F value, p value)”.

As important, I suggest avoiding (like the plague) sentences saying that “there was no difference between conditions 1 and 2” — not to mention statements to the effect that conditions 1 and 2 are equivalent. A much better way of expressing this is to say that differences between conditions 1 and 2 were not detected (t value, p value)”.

This is a tiny step. By itself it won’t do much, but in conjunction with educating a new generation of researchers in alternative methods, it will help start changing scientific reporting — an hopefully improve science.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: