RAE2001 logo


Another UoA

Section III: Panelsí Criteria and Working Methods

3.17 Statistics and Operational Research, UoA 24

UoA Descriptor

3.17.1 This UoA includes statistical methodology and applications (such as medical statistics, economic and social statistics, financial modelling), mathematical statistics, operational research, applied probability and probability theory.

UoA Boundaries

3.17.2 Statistics and operational research are all pervasive. In recognition of this the Panel members are anticipating a wide range of pure and applied submissions. Every effort will be taken to give credit, be the research theoretical, methodological or applied. The Panel recognises that good research in areas covered by this Unit of Assessment is often found in journals specialising in other subjects.

3.17.3 The Panel anticipates that some submissions will be referred to other panels and also that it will receive submissions from other panels. The Panel will decide which of the submissions it receives to cross-refer. In all cases of cross-referral, the whole submission will be made to all panels concerned.

3.17.4 Actuarial science, financial mathematics, demography and social and economic statistics may fall within UoA 24 or another UoA. When actuarial science is submitted to UoA 24 specialist advice will be obtained. In the case of demography, financial mathematics and social and economic statistics, judgement will be used and specialist advice may be sought.


3.17.5 A sub-panel with members chosen from outside the academic world will give advice on the practical importance of selected aspects of submissions referred from the main Panel.

Interdisciplinary Research

3.17.6 All panel members are involved in interdisciplinary research. If the panel members feel that submissions need to be referred, this will be arranged and carried out. Institutions also have the opportunity to request cross-referral.

Joint Submissions

3.17.7 If the Panel receives a joint submission, it will be treated by panel members as a unitary submission.

Assessment Criteria

3.17.8 Quality of submissions will be judged on the basis of the following four indicators:

  1. The quality of the listed research outputs(RA2);
  2. The extent of postgraduate research activity, as indicated by the numbers of research students, research studentships, and research degrees awarded (RA3a and RA3b);
  3. Evidence of esteem, from externally funded research support (RA4);
  4. Evidence of vitality of the department, prospects for continuing development, and national and international prestige (RA5 and RA6).

3.17.9 Of these, a) will be the leading indicator. The other three indicators are not placed in any order of priority.

Research Output (RA2)

3.17.10 The Panel expects mainly to receive the following types of research output:

  1. Journal papers (including those published in electronic journals)
  2. Books (excluding undergraduate texts)
  3. Chapters in edited books
  4. Research-related computer software

3.17.11 In RA2, for personnel who have been in post for the complete period and who expect to be regarded highly, the Panel would normally anticipate seeing a full submission of four items. Where people are not in post for the complete period, the panel members will use their judgement. The Panel is looking for quality and will assess new entrants into the profession with special care. The Panel recognises that researchers who are new to the profession may require time to establish a publication record. In assessing new researchers, the Panel will look for marks of excellence more widely.

3.17.12 The Panel will collectively examine in detail at least 20% of the items of research output cited in submissions. In selecting the items the Panel will pay particular attention to borderline cases.

For all cited works, whether read or not, the Panel will take note of the perceived editorial standards of journals, bearing in mind the character of research being described. The Panel will not use impact factors or citation indices. The Panel appreciates that the quality of material varies widely in some journals and edited books. All the submissions themselves will be read by every panel member individually.

3.17.13 In judging research quality, the value of authoritative and carefully crafted books and review articles will be recognised as will research that is innovative and has an impact on user and scientific communities.

3.17.14 In judging the quality of published conference papers, the Panel will also take account of such factors as the perceived status of the conference or workshop, the status of the contribution (e.g. invited or contributed), and whether or not the material underwent a rigorous refereeing process.

3.17.15 The Panel accepts pedagogic research in its subject area and will refer such research to the Education Panel or to specialist advisers where necessary.

3.17.16 It is expected that the author(s) submitting co-authored work will have made a substantial contribution to it. A co-authored paper may be cited more than once by the same department. However, departments may wish to submit such an article only once, as the submission of an additional paper will give a wider picture of the research in the department. RA6 can be used to emphasise the contribution made by the author(s) not submitting the paper.

3.17.17 In all cases quality will be measured by subjective assessment of actual intrinsic quality and potential impact.

3.17.18 The size of a submission should not in itself affect the quality rating. There is no reason in principle why a small submission cannot obtain a high rating.

Research Students and Research Studentships (RA3)

3.17.19 The numbers of research students, research studentships and research-related higher degrees awarded, taking into account the number and profile of submitted staff, will contribute to the assessment. The Panel may also take into consideration the source of funding for research studentships.

External Research Support (RA4)

3.17.20 External research support will be used as a measure of peer judgement of standing related to an individual's recent achievements and/or promise for the future. The perceived rigour of the funding body's reviewing process may be taken into account. However, the Panel does recognise that some researchers in the Unit of Assessment carry out excellent work independently of external research support.

3.17.21 The number of research assistants, relative to the number and profile of submitted staff, will contribute to the assessment.

Textual Description, General Observations and Additional Information


3.17.22 The Panel invites institutions to address the following in RA5:

  1. To comment generally on the research taking place within the UoA, highlighting its prime activities and main achievements.
  2. To list other UoAs to which related work has been submitted and detail any difficulties of fit between departmental structure and the UoA framework.
  3. To explain briefly the mechanisms and practices for promoting research, and sustaining and developing an active and vital research culture.
  4. To describe any arrangements which are in place for supporting interdisciplinary or collaborative research.
  5. To provide information on relationships with industry, commerce and government or other research users.
  6. To describe the arrangements for the development and support of research work of staff.
  7. To describe any arrangements which are in place for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research culture.
  8. Where appropriate, to explain the role and contribution of category A* staff who have joined the department.
  9. Where appropriate, to comment on how the departure of staff in categories A*, B and D has affected the strength, coherence and research culture of the department at the census date.
  10. To comment on the role of individuals returned as research active whose principal employment is elsewhere, whether UK or abroad, and whether or not in an academic institution.
  11. To provide a statement about the main objectives and activities in research over the next five years. The Panel's attention should be drawn to ongoing research work that is not producing immediate visible outcomes.
  12. The Panel will have the 1996 RA5 before it and institutions may wish to comment on their success in achieving their aims as described there, but the Panel will recognise the limitations of such long term planning.
  13. The Panel expects that submissions will contain a brief self-assessment of the success of measures promoting research, along with comments on any proposals for future changes.


3.17.23 In RA6 (general observations and additional information), the Panel would like institutions to concentrate on providing information on individuals.

3.17.24 Institutions should give profiles of all members of staff submitted for assessment, their prime activities and main achievements.

3.17.25 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.

3.17.26 Individual profiles allow institutions to provide further details/information if it is felt that the quality of research is not fully represented by the four named research outputs submitted. Individual profiles may also include information of the following kind:

Evidence of Esteem

3.17.27 To list indicators of peer esteem, such as awards, editorial positions, conference invitations and contributions to the scientific direction of major conferences.

Individual Staff Circumstances

3.17.28 To note any circumstances which have significantly affected a contribution to the submission (e.g. periods of sick leave, parental leave, career breaks, engagement on long term projects etc).

Additional Information

3.17.29 To provide details of major grants, identifying joint investigators where appropriate.

3.17.30 To give details of the number of research assistants supervised during the period of assessment.

Working Methods

3.17.31 In advance of the first meeting, all panel members will independently assess all submissions. The preliminary ratings will be collated by the office and used as a starting point for discussion, in particular to identify borderline cases. For each submission, two panel members will have been chosen by the Panel Chair to introduce the discussion on that case.

3.17.32 The overall process of reaching a set of ratings will be iterative, with particular care taken over borderline cases. Each submission, borderline or otherwise, will be revisited more than once during the process. It is expected that the final judgement for each submission will be reached by a consensus. As a last resort, if an agreement cannot be reached by consensus, a majority vote will determine the outcome.

3.17.33 The Panel will not use an algorithmic approach. Although individual panel members are free to devise their own methods of working, the final ratings will be based on the Panel's collective judgement and discussion of each submission.

3.17.34 Information on research students, studentships, research degrees awarded, research assistants and the value of external research support will have an influence in assessing the quality of submissions. The different aspects will be weighed up in the discussion of the individual submissions.

International Excellence

3.17.35 The distinction between national and international excellence is one of degree. Established staff working at the international level will typically be publishing high quality papers, be active participants in international conferences, and be supervising research. They will provide research leadership, and their work will influence the global research community and/or the practitioner communities.

3.17.36 The Panel will refer to non-UK based experts all 5*, 5 and 4 ranked submissions. The experts will be asked to comment on the definition of international excellence for the UoA.

Last updated 17 April 2000

[ Home | About the RAE2001 | Panels | Guidance for panel members | Guidance for institutions | Data collection | Publications | Contacts ]