Q: What if I want to contribute but am not a psychologist?
A: For the moment, we have decided to focus on loss-of-confidence statements from psychology. The main reason behind this decision is that we want to keep the project manageable and lack the expertise to properly evaluate the eligibility of statements from other fields. Of course, if you want to start a similar project for a different field, you are more than welcome to do so.
Q: If I admit to losing confidence in a finding based in part on methodological problems I acknowledge responsibility for, does that mean my paper will be retracted?
A: Participating in this project does not imply that any kind of official retraction, correction, or statement will be issued for the publication(s) in question. The collaborators on this project will take no collective stance at any point on whether or not a correction or retraction is appropriate for any of the participating studies. You are, of course, always free to pursue such an action on your own if you think one is warranted, but that decision remains entirely yours to make whether or not you participate in this project.
Q: What if my co-authors disagree with me about the status of the effect reported in our paper?
A: That’s up to you. To be eligible for inclusion in the project, you must be willing to take primary responsibility for any methodological or theoretical problems that have caused you to lose confidence in your findings since their publication. This means that, by definition, your co-authors are not also culpable. If you do not feel that you are primarily responsible for any problems with the original article, but the co-authors who you think share responsibility are not willing to participate in the project, then your study is not eligible for inclusion. Of course, even if you do take full responsibility for any problems with the study in question, it’s possible that your co-authors may disagree with your decision to publicly share your loss of confidence in the finding. We would encourage you to talk to your co-authors about these issues before making a decision to participate. However, provided you meet the inclusion criteria, we will accept your submission, and will not attempt to verify or otherwise contact your co-authors. Should they later express concerns about our inclusion of your article, we will direct them to contact you.
Q: What if I don’t think I did anything wrong, but have lost faith in my study due to repeated failures to replicate the original effect?
A: If you’ve lost confidence in any of your studies for any reason, we would encourage you to publicize that fact, as it could potentially help prevent other researchers from wasting resources conducting replications or extensions that may be unlikely to succeed, or it could highlight the need for further research on a topic since less is known about it than the published record would currently suggest. That said, the Loss-of-Confidence project currently only includes studies where the loss of confidence in the primary result occurred as a result of methodological or theoretical errors that one or more of the authors are willing to take responsibility for. The point of restricting the inclusion criteria in this way is to explicitly reward (with publication), and highlight the scientific character of, those researchers who are willing to publicly admit to making mistakes—since our primary goal is to help destigmatize such actions, which are critical for the continued health of the scientific enterprise. While admissions that one has lost faith in one’s findings for other reasons (e.g., because the findings cannot be replicated by others) are also important, they are surrounded by much less stigma, and in general, their informational value is much lower, because the original author generally has no privileged insight into the explanation for any replication failures.
Q: How will my colleagues receive my admission that I’ve lost confidence in one of my findings as a result of mistakes I acknowledge making?
A: While we obviously can’t state with any certainty how any person or group will receive either your individual admission or the project as a whole, our expectation is that the reception will be predominantly positive in nature. Previous surveys have provided very compelling evidence that many of the methodological, conceptual, and statistical problems that we expect collaborators on this project to admit to were nearly ubiquitous in many fields of psychology until recently. In most of the existing cases where individual researchers have spontaneously admitted to losing confidence in their findings as a result of p-hacking practices that they have since disavowed (e.g., Dana Carney’s admission of a loss of confidence in power posing), the reaction from other psychologists has been overwhelmingly one of praise and commendation. We suspect that a big part of the reason for this is that most of us recognize that we have also engaged in such practices in our own work, but are perhaps not quite brave enough to admit as much. We are optimistic that a large-scale, collaborative demonstration that everybody makes mistakes, and that acknowledging these mistakes is essential in order to move our field forward, will be met with an equally positive response.