Investigating Referee's Requirements in an Electronic Medium

Abstract

The performance of eight academics refereeing two scholarly journal articles is examined when the articles are presented either on paper or on a CRT screen. Three aspects of performance are analyzed: a) details of how the task was undertaken by each referee, b) individual assessments of the two presentation media, c) the effect of presentation medium on the evaluation of each article’s merit. The results showed that there was considerable variation among referees in the way they undertook the task and in their evaluation of each article. Referees tended to be slower with the CRT text and made several criticisms of the facilities available to them in the electronic medium. Suggestions were put forward concerning the support needed for refereeing academic papers presented in an electronic medium. In this study evaluative assessments of the merits of each article were not affected by the presentation medium.