Don’t Have Time To Do a Replication? Have You Considered p-Curves?

So another study finds that X affects Y, and you are a sufficiently cynical TRN reader that you wonder if the authors have p-hacked their way to get their result.  Don’t have time (or the incentive) to do a replication?  You might consider using a “p-curve” analysis to determine whether the effect has “evidentiary value.”  How does one do that?  Let’s take as a given that most journals will not publish a result unless it is statisically significant.  Even if the journals only report significant results, one can examine the distribution of p-values to determine whether or not the effect is true.  Want to learn more about “p-curves?”  The original article by Simonsohn, Nelson, & Simmons (2011), “P-Curve: A Key to the File Drawer” can be found here. A straightforward explanation of the technique by Will Gervais can be found here.  A critique by Bruns & Ioannidis can be found here.  And an excellent response by the original authors can be found here.  P-curves are definitely worth a look!

One Comment on “Don’t Have Time To Do a Replication? Have You Considered p-Curves?

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: