Submission to ARC consultation paper: Preprints – ERA

The ACDS has made a submission to the questions posed in the Consultation Paper: Preprints – ERA. In responding to the questions the ACDS based its submission on the principle that, ERA being an exercise in judging the quality of research, peer review is essential to that judgement.

In Summary the ACDS responded:

• Given ERA’s focus on evaluating research excellence, is it appropriate to consider inclusion of preprints in ERA noting the issues with the indexation of preprints?

No. ERA is focussed on research quality and this is usually demonstrated by citations or peer review. It does not mean that an article which has not been cited or peer reviewed is lacking in quality, but the judgement of quality becomes subjective and contestable. Given that at the moment (looking back over the ERA collection period of 2016-2021) preprints are mostly neither cited nor peer reviewed then they should not be included in ERA.

• Is it appropriate to include preprints as an eligible research output type for citation analysis disciplines? 

No. See answer to Q1.

• Is it appropriate to include preprints as an eligible research output type for peer review disciplines?
  No. See answer to Q1.

• What would be the material advantages and/or disadvantages of including preprints in ERA?

Advantages: None. In principle, they should not be included in the low volume threshold for either citation or peer review disciplines (because they have not been cited or peer reviewed).

Disadvantages: See Q5, plus it places a lot of subjectivity on the shoulders of the ERA assessors.

• What would be the impact on universities if required to include all preprints in ERA and undertake deduplication to ensure research outputs submitted to ERA are unique?

It is unlikely that universities could guarantee that all preprints (within the broad ARC definition) have been swept up into the ERA collection process. So it creates an unlevel playing field from the beginning. In addition, the extra checking required to determine whether a preprint has subsequently been published and to remove duplicate research outputs would be quite onerous and a huge financial burden on the sector. The effort required would not be commensurate with the desired outcome.

March 2022