{"id":606,"date":"2012-03-21T16:09:46","date_gmt":"2012-03-21T20:09:46","guid":{"rendered":"http:\/\/cedarsdigest.wordpress.com\/?p=606"},"modified":"2012-03-21T16:09:46","modified_gmt":"2012-03-21T20:09:46","slug":"put-your-head-up-to-the-meta-a-peer-reviews-post-post-publication-peer-review-a-bargh-full-of-links","status":"publish","type":"post","link":"https:\/\/riener.us\/WP\/uncategorized\/put-your-head-up-to-the-meta-a-peer-reviews-post-post-publication-peer-review-a-bargh-full-of-links\/","title":{"rendered":"Put your Head up to the Meta – A Peer Reviews Post-Post Publication Peer Review – A Bargh full of links"},"content":{"rendered":"

Attention conservation notice: This post dives into some inside baseball stuff on social psychology, how the science of psychology is practiced, and how science is communicated online. It is kind of long, but I intend it as sort of a reference for this event, but also something to point to for my colleagues who either aren’t aware of, or doubt whether real scientific discussion can occur online.<\/p>\n

There was a recent kerfuffle over a finding in social psychology.
\nNo, not that<\/a> one<\/a>.
\nNo, not that<\/a> one either.
\nNot that<\/a> one either.<\/a><\/p>\n

I’m talking about the non-replication of a famous study in psychology in which people who are unconsciously primed with words associated with the elderly end up walking more slowly toward the elevator after the experiment is over. I am going to give a brief summary of the situation, but then break it down into why I found it so interesting, not necessarily in terms of the psychological concepts themselves, but in terms of what this means for the modern practice and communication of science. In the past I have been skeptical of online post-publication peer review this serving as a bona fide replacement of peer review. Comparing my own comments when I have served as a reviewer (and recieved for my submitted manuscripts to those that I leave as a web commenter, the formal peer review process (anonymous or otherwise) has always come out on top. But this latest episode, and some related conversations, have convinced me that online post-publication peer review has made an amazing amount of progress.<\/p>\n

I’ll start with a sort of annotated bibliography of this “event.” (now updated with Bargh’s recent response<\/a>)<\/p>\n

1996: John Bargh, Mark Chen and Lana Burrows publish an article entitled “Automaticity of Social Behavior: Direct Effects of Trait Construct and Stereotype Activation on Action”<\/a> in one of the premier journals of social psychology, the Journal of Personality and Social Psychology. They find that one can unconsciously activate (“prime”) certain ideas (“trait concepts”) and that this unconscious priming can affect relevant social behavior. People knew priming existed, and that priming could influence things like how we interpret a social situation, but most people had nonetheless assumed that social behavior (action, not just thoughts or perception) was more under our conscious control, and less susceptible to unconscious priming. In other words, the paranoia of subliminal advertising was overblown. Flashing “BUY COKE” before the movie does not<\/a> cause hordes of brainwashed puppets would shuffle to the concession stand. Regular advertising, on the other hand<\/a>, remains amazingly effective.
\n

\"Gustav<\/a>
Gustav Fechner, one of the founders of psychology, and certified old dude.<\/figcaption><\/figure><\/p>\n

This article becomes very influential, inspiring a great amount of work on unconscious priming and the importance of unconscious attitudes on behavior. It fits into a spectrum of work on the modern unconscious (or what Tim Wilson<\/a> calls “non-conscious”<\/a> to separate from Freud’s psychoanalytic unconscious). Steele and Aronson’s stereotype threat research (beginning roughly in 1995<\/a>), Anthony Greenwald’s research on implicit social cognition<\/a> and many other social psychologists have studied how non-conscious stimuli and attitudes can affect our thoughts and behavior.<\/p>\n

Despite this influence, there doesn’t seem to be a record of anyone replicating this particular Bargh study. That is, no one has re-run it with the same method and found the same results. Given the way that publication bias works (there is a huge bias to only publish studies that “work” or find statistically significant differences between groups) it is likely that at least some people have tried, but failed (see some of the links below for more details on this point).<\/p>\n

2012 In the recent open access journal PLOS-One, a group published a non-replication of this study, apparently finding that even though they used the same methods (more on this in a second) they did not find the same results.<\/p>\n

Then, this is all within a week:<\/p>\n

1) Science writer extraordinaire Ed Yong writes up this failed replication attempt<\/a> on his blog at Discover magazine: “Not Exactly Rocket Science.”<\/p>\n

2) John Bargh responds<\/a> on his blog at Psychology Today.<\/p>\n

3) Ed Yong responds to the response<\/a> on his blog.<\/p>\n

4) The psychology blogosphere erupts with conversation and reaction. I am going to annotate a little here.
\n a) The comments on Ed Yong’s first post include someone who is a priming researcher (“Joe”), another research psychologist (“Chris”) as well as someone who I would assume is Bobbie Spellman (“Bobbie”), a well-respected social psychologist and one of the founders of a site devoted to rectifying the fact that unsuccessful studies often never see the light of day: PsychFileDrawer.org <\/a>
\n b) The initial response by Bargh (62 comments) includes comments by Ed Yong, a neuroscientist who goes by Neuroskeptic<\/a>, Publisher of PLoS One Peter Binfield, another founder of PsychFileDrawer.org Alex Holcombe, one Peter C who says he is an editor of a major journal in this field, in addition to a number of anonymous commenters.
\n c) Ed Yong’s response to the response has mostly named commenters, including: Well known cognitive (gorilla)<\/del> psychologist Dan Simons, social psychologist Dave Nussbaum<\/a>, social psychologist Michael Krauss<\/a>, neuroscientist Chris Chambers<\/a>, neuroscientist and editor at Nature Noah Gray<\/a>, cognitive neuroscientist Matt Craddock<\/a>, little old me<\/a>, and social psychologist (and someone who published with Bargh)Gordon Moskowitz<\/a>.
\n d) Dan Simons google+ post entitled “A primer for how not to respond when someone fails to replicate your work”<\/a>. This includes a long conversation between Dan and another well-known cognitive psychologist, and one of the few reasons I would still visit Psychology TodayArt Markman<\/a>.
\n e) Friend and social psychologist par excellence Sanjay Srinistava posts reflections on his blog<\/a>.
\n f) Conversations on twitter about the difference between conceptual replication and direct replication involving Ed Yong and quantitative molecular biologist, Professor of genomics and evolutionary biology at Princeton, and guy you should totally follow on twitter<\/a> Leonid Kruglyak<\/a>. And me. Here’s my first attempt at a storify of this convo.<\/p>\n

\n
\n
    \n
  1. \n
    Ed Yong tweeted about one commenters distinction between direct replication and conceptual replication:<\/div>\n<\/li>\n
  2. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    Comments on the Bargh post have veered into a discussion on “conceptual replication”, a concept that troubles me bit.ly\/yVLPxt<\/a><\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Ed Yong <\/a>\"edyong209\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:37:20<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  3. \n
    Leonid Kruglyak is also put off by the term “conceptual replication” which sounds alarmingly postmodern, as if it could have appeared in Sokal’s hoax paper. <\/div>\n<\/li>\n
  4. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    Conceptual replication: merely troubling or completely unscientific? @edyong209<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:42:27<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  5. \n
    He has a little fun with it…<\/div>\n<\/li>\n
  6. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    “We know chemicals kill cancer cells; who cares if we can reproduce this drug killing this cell line” #conceptualreplication<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:47:27<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  7. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    “I found a gene causes a disease, who cares if you don’t get the same gene” #conceptualreplication<\/div>\n<\/div>\n
    <\/p>\n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Joe Pickrell<\/a>\"joe_pickrell\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:14:35<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  8. \n
    One of the things that sucks me into conversations online is defending psychology as a science. Chip, meet shoulder. <\/div>\n<\/li>\n
  9. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @leonidkruglyak think u might be going bit too far here. Much of social psych is #conceptualreplication just bc prejudice isn’t chemical …<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:52:30<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  10. \n
    <\/p>\n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @leonidkruglyak @edyong209 Bystander studies too. Started with false premise (wrong story of Genovese murder). But repl. und dif conds.<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:48:30<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  11. \n
    JP McGinnis cites the classic Doug Mook paper “In Defense of External Invalidity” on why we should run more tightly controlled, unnatural, lab studies in psychology. I think these kinds of artificial situations that eschew ecological validity would likely be more easily replicable. Except in this case.<\/div>\n<\/li>\n
  12. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @leonidkruglyak @criener I’m way out of my depth on this, but thought this might be applicable to hard vs softer sci bit.ly\/ylpGfl<\/a><\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    JP McGinnis<\/a>\"jp_mcginnis\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:50:47<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  13. \n
    Kruglyak goes for the jugular (yes, he is a heavy weapons guy, see second tweet)<\/div>\n<\/li>\n
  14. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @criener either an experiment is directly replicable or it’s not science, no?<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:56:54<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  15. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    FYI: I’m a heavy weapons guy RT @WvSchaik: Awesome! The nine types of peer reviewers: goo.gl\/xSAzW<\/a> HT @edyong209<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Thu, Mar 15 2012 12:22:00<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  16. \n
    I try to defend non-replication still leaving the possibility of science<\/div>\n<\/li>\n
  17. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @leonidkruglyak If we don’t understand\/can’t control all relevant factors of phenomena, some unknown could cause to fail to replicate<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Sun, Mar 11 2012 23:59:57<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  18. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    .@criener I always understood replication to be central to scientific method. That’s why we write Methods sections. #conceptualreplication<\/div>\n<\/div>\n
    <\/p>\n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:00:20<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  19. \n
    Ed Yong refers to his comment at his blog, worrying that many conceptual replications would allow weak studies to appear to bolster each other, but without direct replication they would remain a house of cards.<\/div>\n<\/li>\n
  20. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @criener @leonidkruglyak My worry (see comment) is that using diff conditions w\/o direct replic’n allows weak studies to bolster each other<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Ed Yong <\/a>\"edyong209\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:01:16<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  21. \n
    I agree this can be a problem, especially from the outside, it can be hard to distinguish from well-worn replicated studies (like the bystander studies by Darley and Latane, or the obedience studies by Milgram) and sexy novel studies that turned out to be fraud like Diedrich Stapel.<\/div>\n<\/li>\n
  22. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @edyong209 @leonidkruglyak Agreed. In social psych sometimes hard to separate Darley&latane, Milgram, wheat from Stapel chaff<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:03:25<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  23. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @criener Exactly so. That’s my concern. How many Stapels are currently hidden because of this problem?<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Ed Yong <\/a>\"edyong209\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:04:38<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  24. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @edyong209 Don’t know. File Drawer journal trying to help. But to some degree, no shortcut to vetting – network of weak or strong results…<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:06:30<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  25. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    @edyong209 being evaluated by social psych’s within in the field. I don’t actually know what makes one priming study better than another<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Cedar Riener<\/a>\"criener\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 00:07:45<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  26. \n
    In conclusion (for now) Kruglyak boils it down…<\/div>\n<\/li>\n
  27. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    .@edyong209 @criener I think it comes down to, can you really have a solid concept if no individual instance of it is solid\/replicable?<\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 11:15:52<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  28. \n
    I replied to this in the comments at Ed Yong’s blog post, but I think this is right, a solid concept isn’t so solid if  no one of its legs is strong. My take on this for social psych, and this instance is that there may not have been a direct replication of this study, but priming in general is well-replicated, as is the concept of unconscious attitudes affecting behavior (for example Greenwald and colleagues work on the IAT). I guess I would say that different parts of the concept have different support. I am perfectly willing to be skeptical of the slow-walking based on elderly priming effect, but not that priming or unconscious drivers of behavior.<\/p>\n
    The next day, a few more follow up posts are relevant. <\/div>\n<\/div>\n<\/li>\n
  29. \n
    Kruglyak cites Roddy Roediger’s column about the value of replication (and citing his own highly replicated results with the DRM word lists).<\/div>\n<\/li>\n
  30. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    Psychology’s Woes and a Partial Cure: The Value of Replication bit.ly\/AEyA3K<\/a><\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Leonid Kruglyak<\/a>\"leonidkruglyak\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 12:17:08<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  31. \n
    Matt Lieberman posts an interesting blog post citing a recent paper by Scott Lilienfeld on some o the reasons for psychology’s reputation<\/div>\n<\/li>\n
  32. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    Why Are People So Skeptical About Psychology? @psychoBOBlogy bit.ly\/ywM1K6<\/a><\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Matt Lieberman<\/a>\"social_brains\"<\/a><\/div>\n
    <\/p>\n
    Tue, Mar 13 2012 10:46:16<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  33. \n
    This is Dan Simons’s epic google post.<\/div>\n<\/li>\n
  34. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    Nothing in Their Heads | Psychology Today<\/a><\/p>\n
    A primer for how not to respond when someone fails to replicate your work with a discussion of why replication failures happen In the lin…<\/div>\n
    \n
    <\/a><\/div>\n
    Daniel Simons<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n
  35. \n
    Sanjay Srivastava’s thoughtful follow-up post, including some social psych details and background<\/div>\n<\/li>\n
  36. \n
    \n
    <\/i>Share<\/span><\/div>\n<\/div>\n
    \n
    “<\/div>\n
    \n
    New post: some reflections on the Bargh-Doyen-@edyong209 elderly-walking nonreplicastravaganza wp.me\/pt9Wa-l2<\/a><\/div>\n<\/div>\n
    \n
    <\/p>\n
    <\/div>\n

    <\/a><\/div>\n

    Sanjay Srivastava<\/a>\"hardsci\"<\/a><\/div>\n
    <\/p>\n
    Mon, Mar 12 2012 17:00:34<\/div>\n

    <\/a><\/div>\n

    Reply<\/a>Retweet<\/a><\/div>\n
    <\/div>\n<\/div>\n<\/div>\n
    <\/div>\n<\/li>\n<\/ol>\n<\/div>\n<\/div>\n

    *********
    \nUpdate: Here is Bargh’s     recent response<\/a> from today (3\/25\/12)<\/p>\n

    Update2: Chris Chambers has a post up<\/a> about conceptual replication. I agree with him, and I think looking back up at the storify, I was defending the value converging evidence and just accepting that social psychology seems to be calling converging evidence “conceptual replication.” As Chris points out, this devalues direct replication, and leads critical thinking away from the converging evidence, which is necessary, but can sometimes be weak.
    \n*********<\/p>\n

    Post Publication Peer Review Online and the Future of Science?<\/strong><\/p>\n

    A few things struck me about these conversations.
    \nFirst, there is a generational divide here. Bargh’s response is defensive and snippy, but it made a certain kind of sense to me. He blew Ed Yong off in the first place, writing a very quick email in response to Ed’s request for his response to Doyen. Then, when the article came out, I think part of the reason he took offense was at being compared to William von Osten (Clever Hans owner, who apparently never gave up his faith in the amazing intellectual powers of his horse). As Dan Simons noted, Ed is one of the best young science writers today, with real integrity, and a desire to get things right, even if it involves going back and making corrections. Ed is also fun to read, which can involve creative comparisons and snarkiness. Bargh misread him in the first place (probably as a “blogger” instead of a well-respected science journalist). If Bargh had realized that talking to Yong in the first place would have been actually a much better forum to clarify his concerns than his own blog at Psychology Today, a lot of the heat of this debate wouldn’t have existed. Update:<\/strong><\/i> This generational divide still shows in his     most recent response<\/a>, which is more background on the article and studies, as well as more background on priming and social psychology. I happen to agree with most of what he says in this “Angry Birds” response, but I don’t think he is going to win any converts, partly because he is still treating his “blog” like a lectern (there are no external links and no one mentioned by name) and also because people’s attention span is likely over. That said, if he had given Ed Yong this lecture at the very beginning (when Ed asked for it), I think Ed would have been far more gracious.<\/p>\n

    Second, the distributed nature of the conversation makes it very hard to follow and participate. Trying to track comments on two different Ed Yong posts, Bargh’s psychology today post, as well as twitter and G+ made it overwhelming for me. I can only imagine psychologists not on twitter, or people loathe to sign up for a site to comment (like my advisor or many of my mentors) would be missing half of the conversation.<\/p>\n

    Third, it is fascinating to me how real scientific dialogue can now co-exist with science education and promotion of science. In a single blog comment feed, there can be someone who has the knowledge of an undergraduate asking a relatively basic question (and being answered), and someone who has done this research for ten years asking an advanced one (and having a dialogue with another expert for all to see). Daniel Bor’s recent blog post about the dilemma of weak neuroimaging results<\/a> has some great explaining, but at the same time, some practicing expert scientists referring to specific papers and specific methods that are sometimes over the heads of people who haven’t had graduate study in neuroscience.<\/p>\n

    Finally, the upshot of all of this to me is a balanced optimism about post-publication peer review online. To other scientists I say: Click on some of these links (especially that Bor post) and be amazed. There is real scientific discussion taking place in public, it is really not that watered down. It is totally worth it to blog (and tweet) about your papers, not just to inform “the public” of your results, but to possibly engage with a scientific audience. But second, to the public, I say, real post publication peer review looks kind of like regular peer review: experts often disagree, and “scientific consensus” isn’t always immediately apparent, or easily communicable at that moment. If we are expecting post-publication peer review to be that much better than regular peer review, I am still somewhat skeptical that the right “peers” will show up and apply due diligence to interpreting and evaluating a study.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Attention conservation notice: This post dives into some inside baseball stuff on social psychology, how the science of psychology is practiced, and how science is communicated online. It is kind of long, but I intend it as sort of a reference for this event, but also something to point to for my colleagues who either…<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[32,7,1,9],"tags":[42,43,44,142,137,45],"class_list":["post-606","post","type-post","status-publish","format-standard","hentry","category-psychology","category-science","category-uncategorized","category-web","tag-bargh","tag-ed-yong","tag-priming","tag-psychology","tag-science","tag-social-psychology"],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4Pqh9-9M","jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/posts\/606","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/comments?post=606"}],"version-history":[{"count":0,"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/posts\/606\/revisions"}],"wp:attachment":[{"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/media?parent=606"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/categories?post=606"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/riener.us\/WP\/wp-json\/wp\/v2\/tags?post=606"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}