Why is PCI Psychology evaluation focused on conceptualization and design?
Publication bias is due to many factors, but most prominently due to making publication selection dependent on the nature of study results. We have perhaps seen some improvements in some areas of psychology in recent years, but it remains a serious problem. Shifting the focus of selection (or in our case, recommendation) from results to conceptualization and design, can help move the field towards correcting for longstanding bias. The fact that this is the approach used in Registered Reports is not a reason to not also use it for traditional reports; indeed, adopting this approach has precedent at some journals in the field. We of course cannot ensure that recommenders/reviewers are not impacted by the results, but there is nevertheless value in making active statements about how recommendations will be made. Moreover, recommenders will be trained to not rely on reviewer assessments that are focused on the results. It will be a work in progress, but one that is worthwhile.
Why not have a standard submission format?
Free format submission is a cornerstone of journal workflow reform. The amount of time wasted by authors on custom formatting has been well-documented. We will not contribute to that. Authors can follow any reasonable manuscript formatting guidelines for their submissions, which includes referencing. Any submission that seems haphazard, disorganized, incomplete, or unintelligible will be returned to authors by the Managing Board.
Won’t the use of the term “preprints” reinforce a journal-centric model?
PCI is designed to move away from the traditional journal-centric approach to an author-focused approach, wherein authors can choose the eventual outcome of their manuscript. In that sense, yes, it is true that there may not ultimately be a “print” (i.e., publication in a traditional journal) for some manuscripts. However, that will not be the case for all submitted manuscripts. Additionally, the term preprint is widely recognized, and manuscripts are submitted to PCI Psychology via a preprint server (preferably PsyArXiv or PsychArxiv), and we believe using a different name would create confusion. The preprint itself is not “published” or housed in any way on the PCI site, only the recommendation is, with a link to the preprint. Thus, it is not a publication platform.
What do we do with rejected submissions? Can there be an anti-recommendation? Can PCI Psychology still post the reviews for manuscripts that are not recommended, or perhaps encourage posting of reviews on PubPeer/PREreview/similar platforms?
PCI’s general rule currently states that reviews for non-recommended manuscripts will not get published. Although we recognize the value in making reviews for rejected articles available, we feel like even encouraging authors to publish their reviews directly conflicts with this PCI rule, and so have decided not to add this for now. Such a policy change is also not without downsides, as it might create an entry barrier for authors if they feel like submitting to PCI might generate negative public reviews for their preprint. We do plan to look into alternatives that might be a better fit with existing PCI policies as PCI Psychology develops, and make a consistent plan to deploy any policy changes before we do so.
What about Verification Reports?
We chose not to include verification reports as one of the accepted formats for PCI Psychology at this time, as they require a two-step review process similar to registered reports, which the manuscript handling system for thematic PCIs (i.e., PCIs other than PCI-RR) is not equipped to handle. We suggest submitting manuscripts in the spirit of verification reports via commentaries (and/or lobbying PCI-RR to accept verification reports!)
I’m worried PCI Psychology’s policies do not apply to all types of articles and might disadvantage some kinds of scholarship.
These policies are meant to be guidelines and serve as general principles for recommenders, but we know there is no way to create one-size-fits-all policies in a field as broad as psychology. We fully expect recommenders to have some leeway in addressing the needs of different kinds of research on a case-by-case basis. Sample description is a good example of this; what we are providing are examples and recommendations of what to include, not mandates, as what is necessary/reasonable depends on the study topic, design, and context. We also generally avoid being overly specific about issues that could arise with particular types of research in favor of general principles. Our primary solution for handling all of these issues is to have well trained recommenders who understand the general guidelines and how to apply them.
Preregistration is good/bad/not enough/too much
We had a lot of excellent community feedback on this topic during the development of PCI Psych’s editorial policies! We know there are many ongoing debates about preregistration and not that much consensus, but we are adopting the broader view that preregistration is relevant for all research — this is why it is recommended and not required. We agree that preregistration has many issues with how it is currently implemented, but we don’t view this as a reason to remove it or downplay it; instead, a goal of PCI Psych is to help improve the implementation of preregistration and its value to the peer review process.
Policy on conflicts of interest
PCI has a very strict policy on conflicts of interest which is outlined in their code of conduct (bottom of this page). As one of many PCIs, PCI Psych is bound by the PCI code of conduct, which forbids financial COIs, and requires as little non-financial COI as possible for recommenders and reviewers. Checking that a recommender doesn’t have a COI with a given preprint and/or its authors is a shared responsibility between the recommender and the Managing Board; checking that reviewers do not have a COI is a shared responsibility between the recommender and the reviewer.
Some community feedback during policy development pointed out that this policy would prevent recommenders from seeking out a review from the author of an original study being replicated or commented on. We agree this should be an exception to the policy. After conversation with the PCI board, we have agreed that, in these cases, the original author can be invited to comment on the preprint, as long as they are not referred to as a “reviewer”. This exception does not require a modification to the Code of Conduct.
The manuscript length / word-count suggestions are too restrictive.
PCI Psychology doesn’t have manuscript length requirements (and neither does Peer Community Journal), but we believe the suggested lengths provided will help authors keep their manuscripts as concise as possible and ensure maximum flexibility for authors seeking to subsequently publish their recommended preprint in a traditional journal, many of which do have strict length requirements. Try to be as concise as possible, but use as much space as you need.
For questions related to Peer Community In more broadly, please see the general FAQs on the PCI website
