1. Introduction to PCI and PCI Psychology
2. The evaluation and recommendation process
3. Submission requirements, review policy, and workflow
    3.1 Submission requirements
    3.2 Transparent review
    3.3 Workflow
4. Tips for reviewers
    4.1 General advice
    4.2 What you should NOT do
    4.3 What you don't NEED to do (but you can if you want)
    4.4 What you SHOULD do
    4.5 Guidelines for evaluation of specific components of the article
    4.6 Sources & Further Resources

 

---

1. Introduction to PCI and PCI Psychology

Peer Community In (PCI) is a community of Recommenders, taking on the role of editors, who recommend unpublished articles based on peer reviews, thereby converting them into complete, reliable, and citable articles, without the need for publication in ‘traditional’ journals. Evaluation and recommendation by any PCI is free of charge. Peer Community In is an original idea of Denis Bourguet, Benoit Facon, and Thomas Guillemaud.

2. The evaluation and recommendation process

The logistics of the reviewing process for an article submitted to a PCI is very similar to that for an article submitted to a traditional journal, but with one key difference from the reviewer perspective: the peer review process for PCI is transparent, unlike that of (most) journals. The review history is, therefore, published by PCI if the article is recommended. Here is an overview of the process:

1. A PCI Psychology recommender may invite you to review a preprint. If so, you will receive an alert by email containing a link to the PCI Psychology website to allow you to decline or accept this invitation. If you are willing to review the preprint, you will first have to certify that you have no financial conflict of interest of any kind, and minimal or no non-financial conflict of interest with the content or with the authors of the preprint. Any existing conflicts, however small, must be disclosed.
2. Once you have agreed to review a preprint, you are expected to write your review within 4 weeks (with an automatic extension to 6 weeks, if necessary). The format of reviews is similar to that for traditional journals. Please bear in mind that the content of your reviews will be published if the article is recommended (although you may choose to stay anonymous, if preferred). 
3. Once you have completed your review, you can copy/paste or upload it onto the PCI Psychology website. At this stage, you can indicate whether or not you wish to remain anonymous. 
4. Based on the various reviews obtained, the recommender will come to a decision within about 10 days of receiving all completed reviews. This decision letter and all the reviews are then sent to the authors.
5. The three most likely outcomes, and their implication for your review(s), are:
• The recommender decides to ask the authors to revise and resubmit their article. In this case, the recommender may subsequently contact you again to ask you to evaluate a revised version of the article with the author’s replies to your comments and those of the other reviewers.
• After one or several rounds of review, the recommender decides to reject the article. In this case, your review(s) will not be published or publicly released by PCI Psychology. 
• After one or several rounds of review, the recommender decides to recommend the article. In this case, all the editorial correspondence (decisions, authors’ replies, all reviews, including yours) will be published by PCI Psychology, making the evaluation process transparent.

3. Submission requirements, review policy, and workflow

3.1 Submission requirements

PCI Psychology will review and recommend preprints in psychological science that are posted to any institutional or organizational repository and have not been previously published in a scholarly journal. Many types of articles may be reviewed and recommended by PCI Psychology; see here for more details. As a reviewer, the manuscript you are being asked to evaluate will already have been deemed to be within the scope of PCI Psychology by the managing board. 

PCI Psychology recommends only preprints of high scientific quality that are methodologically and ethically sound. Recommendation decisions are based on thorough assessments from the reviewers that indicate the preprint meets or exceeds these requirements, although the ultimate decision of whether to recommend a preprint is made by the recommender. The interest of the preprint, as determined by the recommender, may relate to its context, the scientific question addressed, the methodology, and/or the results. PCI Psychology has a large number of recommenders, ensuring a considerable diversity of interests. The recommendations published by PCI Psychology are designed to draw the attention of the research community to the qualities of the article, including the subjective reasons for the recommender’s interest.

For more information about what is taken into account during the evaluation process, you can read PCI Psychology's editorial policies, and refer to the Guide for Recommenders.

3.2 Transparent review

PCI Psychology publishes all reviews of recommended manuscripts, with reviewers retaining the right to choose whether to sign their reviews or remain anonymous. All reviews and recommender decision letters are published on the PCI Psychology platform on recommendation. Reviews of rejected submissions are sent to the authors, but are not published. Reviewers are expected to adhere to the PCI code of conduct, avoiding abusive or discriminatory language in their comments. Reviews considered to violate the code of conduct may be edited by recommenders or the Managing Board, returned to the reviewer for editing, or discarded.

3.3 Workflow

Handling of manuscript by recommenders

After validation from the Managing Board, preprints are available for recommenders to read. When a recommender finds a submitted preprint particularly interesting, they can decide to initiate the process of evaluation for that manuscript. As part of this process, the recommender invites reviewers to evaluate the manuscript so as to obtain at least two high-quality reviews. Note that the recommender and the reviewers must declare that they have no conflict of interest of any kind with the content or the authors of the preprint – see the code of conduct.

During the review process, reviewers will communicate directly with the recommender through the PCI system. Communications with the manuscript authors should always be directed through the recommender.

Word limits and formatting requirements

PCI Psychology imposes no word limits or specific formatting requirements on submissions. Manuscripts should be as concise as possible, but as long as necessary to ensure that the description of the rationale and methods is clear and comprehensive, and that all methods are reproducible. Authors should ensure that they follow established formatting conventions for articles in their discipline. Authors who intend to publish their article in a journal should also take note of any sectional or overall word limits or other formatting requirements that apply to specific journals.

4. Tips for reviewers

The following guidelines are derived from various sources included below, under Sources & Further Resources. They comply with the Ethical Guidelines for Peer Reviewers published by the Committee on Publication Ethics (COPE), to which PCI belongs.

4.1 General advice

• Be constructive, kind and respectful.
• Be precise and clear.
• Remain open to new approaches.     
• If you lack expertise on certain points, state it in your review.
• Only agree to review manuscripts that you can assess in a timely manner. PCI Psychology asks that reviewers return their reviews within 4 weeks (but extensions for up to a total of 6 weeks are allowable), so that manuscripts can be returned to authors within 60 days.

4.2 What you should NOT do

You should not have a conflict of interest with the authors or with the content of the article (cf the code of conduct). If you have a conflict of interest, you must decline the invitation. 

You should not discredit null findings or contradictory results.

You should not ask for new studies if the study is already well performed and the conclusions are clear enough. You can suggest that the authors perform further studies, but only if they are really necessary to boost confidence in the interpretation of the results. Instead of requesting more data collection, you can request that the authors calibrate their conclusions to the available evidence.

You should avoid unconstructive, ambiguous, and unsupported comments.

You must absolutely not use AI to generate your reviews. If you do so, you risk having your text rejected by the managing board of the thematic PCI and being banned from PCI. You can, however, use AI to translate, correct and improve your texts. 

4.3 What you don't NEED to do (but you can if you want)

You do not need to determine whether the article falls within the scope of the PCI. Once a submission has been validated by the managing board, its scope is considered suitable for that PCI.

You do not need to check for potential plagiarism. All the articles submitted to PCI are checked with the Ithenticate tool.

You do not need to check that scripts, code, and/or data are available to the reader (e.g. repository link/DOI or appendix); this is a prerequisite for submission to PCI. You can examine the work for computational reproducibility, but if the findings are not reproducible, you should contact the recommender to request that the authors make this available.

You do not need to comment extensively on typographical errors, spelling and grammar (but you can if you want). If language in the manuscript is unclear enough that it hinders comprehension of the study, this should be (politely) pointed out.

You do not need to make recommendations about acceptance or rejection – this is the role of the recommender handling this preprint.

4.4 What you SHOULD do

In general

Before accepting an invitation to review, check carefully that you do not have any financial conflict of interest with the content or with the authors of the article, and that you have no or only minimal non-financial conflict of interest. If you do not meet these criteria, you must decline to review the article. If you are unsure whether or not you have a conflict of interest, consult with the PCI Psychology Recommender and copy the Managing Board (cc: contact@psych.peercommunityin.org).

Promptly accept or decline review invitations, and try to keep to the deadlines; this shows respect for the authors and keeps the time-to-decision as short as possible. Post your review within four to six weeks after accepting the invitation. If you anticipate a delay, please inform the PCI recommender as soon as you know you will need more time.

Provide a detailed, well-supported report on the merits of the preprint.

Identify flaws (if any) in the design of the research, and in the analysis and interpretation of results.

Express your concerns (if any) about ethics or scientific misconduct.

State the preprint’s strengths as well as its weaknesses.

Try to consider both the technical merit and the scientific significance of the preprint.

If there is something critically missing, report it. 

Provide specific suggestions for improvements.

When reviewing a resubmitted preprint

Evaluate how the authors addressed your comments. If you disagree that the authors have sufficiently addressed your concerns, please state this in your review. If you have new requests for modifications, please state them clearly, but try not to start new lines of interrogation if the previous points were addressed adequately. 

If you disagree with the other reviewer(s) and agree with the authors' explanation and defense of their original article, help the recommender out by discussing these issues.

4.5 Guidelines for evaluation of specific components of the article

(Please note that not all of these sections will be present in every article and some or most of these suggestions may not apply to the article you are reviewing. These should not be taken as a checklist but as guidelines.)

Title

Check that the title clearly reflects the content of the article (e.g., by following these guidelines).

Abstract

Check that the abstract is concise and presents the main findings of the study.

Introduction

Check that the introduction clearly explains the motivation for the study.
Check that the research question/hypothesis/prediction is clearly presented.
Check that the introduction builds on relevant recent and past research performed in the field.

Materials and methods

More generally, check that sufficient details are provided for the methods and analysis to allow replication by other researchers.
Where applicable, check that there is a fully justified and elaborated statistical sampling plan (e.g. power analysis, Bayes Factor Design Analysis etc.)
Check that the statistical analyses are appropriate.
Where applicable, you may check whether analysis claiming to be preregistered match the preregistration.

Results

Focus on evaluating the rigor and transparency of the analytic approach rather than the perceived impact of the findings.
Confirm that the results are presented clearly, with appropriate use of tables, figures, and descriptive statistics that align with the study design and research questions.
Assess whether the authors provide sufficient detail about their statistical methods, including software, packages, and code, to allow replication and computational reproducibility. (Although this is not required, where feasible, reviewers may choose to attempt a reproducibility check by running provided code or scripts to verify that the reported results can be independently replicated.)
Ensure that the results are presented without selective reporting, with equal weight given to null findings and unexpected outcomes as part of a transparent and unbiased scientific process.

Tables and figures

Check that figures and tables are understandable without reference to the main body of the article.
Check that figures and tables have a proper, descriptive caption.

Discussion

Check that the conclusions and claims are adequately supported by the results (e.g., that the interpretation of the analysis is not overstated, that the claims do not generalize beyond the presented evidence).
Check that the discussion takes into account relevant recent and past research performed in the field.

References

Check that all references are appropriate and that the necessary references are present.
Report any reference cited in the text that does not appear in the reference list.

4.6 Sources & Further Resources

Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science

COPE Ethical Guidelines for Peer Reviewers

Career advice: how to peer review a paper

PLoS: how to write a peer review

Fixing scientific publishing and peer review