Section III: Panelsí Criteria and Working Methods
3.17 Statistics and Operational Research, UoA 24
3.17.1 This UoA includes statistical methodology and applications (such as medical statistics, economic and social statistics, financial modelling), mathematical statistics, operational research, applied probability and probability theory.
3.17.2 Statistics and operational research are all pervasive. In recognition of this the Panel members are anticipating a wide range of pure and applied submissions. Every effort will be taken to give credit, be the research theoretical, methodological or applied. The Panel recognises that good research in areas covered by this Unit of Assessment is often found in journals specialising in other subjects.
3.17.3 The Panel anticipates that some submissions will be referred to other panels and also that it will receive submissions from other panels. The Panel will decide which of the submissions it receives to cross-refer. In all cases of cross-referral, the whole submission will be made to all panels concerned.
3.17.4 Actuarial science, financial mathematics, demography and social and economic statistics may fall within UoA 24 or another UoA. When actuarial science is submitted to UoA 24 specialist advice will be obtained. In the case of demography, financial mathematics and social and economic statistics, judgement will be used and specialist advice may be sought.
3.17.5 A sub-panel with members chosen from outside the academic world will give advice on the practical importance of selected aspects of submissions referred from the main Panel.
3.17.6 All panel members are involved in interdisciplinary research. If the panel members feel that submissions need to be referred, this will be arranged and carried out. Institutions also have the opportunity to request cross-referral.
3.17.7 If the Panel receives a joint submission, it will be treated by panel members as a unitary submission.
3.17.8 Quality of submissions will be judged on the basis of the following four indicators:
3.17.9 Of these, a) will be the leading indicator. The other three indicators are not placed in any order of priority.
Research Output (RA2)
3.17.10 The Panel expects mainly to receive the following types of research output:
3.17.11 In RA2, for personnel who have been in post for the complete period and who expect to be regarded highly, the Panel would normally anticipate seeing a full submission of four items. Where people are not in post for the complete period, the panel members will use their judgement. The Panel is looking for quality and will assess new entrants into the profession with special care. The Panel recognises that researchers who are new to the profession may require time to establish a publication record. In assessing new researchers, the Panel will look for marks of excellence more widely.
3.17.12 The Panel will collectively examine in detail at least 20% of the items of research output cited in submissions. In selecting the items the Panel will pay particular attention to borderline cases.
For all cited works, whether read or not, the Panel will take note of the perceived editorial standards of journals, bearing in mind the character of research being described. The Panel will not use impact factors or citation indices. The Panel appreciates that the quality of material varies widely in some journals and edited books. All the submissions themselves will be read by every panel member individually.
3.17.13 In judging research quality, the value of authoritative and carefully crafted books and review articles will be recognised as will research that is innovative and has an impact on user and scientific communities.
3.17.14 In judging the quality of published conference papers, the Panel will also take account of such factors as the perceived status of the conference or workshop, the status of the contribution (e.g. invited or contributed), and whether or not the material underwent a rigorous refereeing process.
3.17.15 The Panel accepts pedagogic research in its subject area and will refer such research to the Education Panel or to specialist advisers where necessary.
3.17.16 It is expected that the author(s) submitting co-authored work will have made a substantial contribution to it. A co-authored paper may be cited more than once by the same department. However, departments may wish to submit such an article only once, as the submission of an additional paper will give a wider picture of the research in the department. RA6 can be used to emphasise the contribution made by the author(s) not submitting the paper.
3.17.17 In all cases quality will be measured by subjective assessment of actual intrinsic quality and potential impact.
3.17.18 The size of a submission should not in itself affect the quality rating. There is no reason in principle why a small submission cannot obtain a high rating.
Research Students and Research Studentships (RA3)
3.17.19 The numbers of research students, research studentships and research-related higher degrees awarded, taking into account the number and profile of submitted staff, will contribute to the assessment. The Panel may also take into consideration the source of funding for research studentships.
External Research Support (RA4)
3.17.20 External research support will be used as a measure of peer judgement of standing related to an individual's recent achievements and/or promise for the future. The perceived rigour of the funding body's reviewing process may be taken into account. However, the Panel does recognise that some researchers in the Unit of Assessment carry out excellent work independently of external research support.
3.17.21 The number of research assistants, relative to the number and profile of submitted staff, will contribute to the assessment.
Textual Description, General Observations and Additional Information
3.17.22 The Panel invites institutions to address the following in RA5:
3.17.23 In RA6 (general observations and additional information), the Panel would like institutions to concentrate on providing information on individuals.
3.17.24 Institutions should give profiles of all members of staff submitted for assessment, their prime activities and main achievements.
3.17.25 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.17.26 Individual profiles allow institutions to provide further details/information if it is felt that the quality of research is not fully represented by the four named research outputs submitted. Individual profiles may also include information of the following kind:
Evidence of Esteem
3.17.27 To list indicators of peer esteem, such as awards, editorial positions, conference invitations and contributions to the scientific direction of major conferences.
Individual Staff Circumstances
3.17.28 To note any circumstances which have significantly affected a contribution to the submission (e.g. periods of sick leave, parental leave, career breaks, engagement on long term projects etc).
3.17.29 To provide details of major grants, identifying joint investigators where appropriate.
3.17.30 To give details of the number of research assistants supervised during the period of assessment.
3.17.31 In advance of the first meeting, all panel members will independently assess all submissions. The preliminary ratings will be collated by the office and used as a starting point for discussion, in particular to identify borderline cases. For each submission, two panel members will have been chosen by the Panel Chair to introduce the discussion on that case.
3.17.32 The overall process of reaching a set of ratings will be iterative, with particular care taken over borderline cases. Each submission, borderline or otherwise, will be revisited more than once during the process. It is expected that the final judgement for each submission will be reached by a consensus. As a last resort, if an agreement cannot be reached by consensus, a majority vote will determine the outcome.
3.17.33 The Panel will not use an algorithmic approach. Although individual panel members are free to devise their own methods of working, the final ratings will be based on the Panel's collective judgement and discussion of each submission.
3.17.34 Information on research students, studentships, research degrees awarded, research assistants and the value of external research support will have an influence in assessing the quality of submissions. The different aspects will be weighed up in the discussion of the individual submissions.
3.17.35 The distinction between national and international excellence is one of degree. Established staff working at the international level will typically be publishing high quality papers, be active participants in international conferences, and be supervising research. They will provide research leadership, and their work will influence the global research community and/or the practitioner communities.
3.17.36 The Panel will refer to non-UK based experts all 5*, 5 and 4 ranked submissions. The experts will be asked to comment on the definition of international excellence for the UoA.
Last updated 17 April 2000