Section III: Panelsí Criteria and Working Methods
3.19 General Engineering and Mineral and Mining Engineering, UoA 26, 31
3.19.1 The UoA covers research from departments or centres which include two or more of the main branches of engineering - chemical, civil, electrical and mechanical- and research in mineral and mining engineering.
UoA Boundaries and Interdisciplinary Research
3.19.2 Multi-disciplinary subjects such as offshore technology, industrial studies, medical engineering and bioengineering are considered to be relevant to this UoA. If necessary, the Panel will cross-refer or appoint specialist advisers for submissions which cross its boundary as defined in the UoA Descriptor. The Panel recognises the interdisciplinary nature of the research output it is assessing and its members have experience of interdisciplinary work.
3.19.3 The quality of submissions will be judged on the basis of the following measures:
Research Output (RA2)
3.19.4 All outputs including journal articles, monographs, conference papers, patents and software will be judged by the same criteria and treated in the same way.
3.19.5 The Panel will base its assessment of the quality of output on its judgement of the extent to which the research contributes to the advancement of knowledge or understanding, innovation, analytical techniques and products and processes, including design, production and management at the national and international levels. The Panel will also consider the novelty and originality of research outputs.
3.19.6 When considering journal articles, conference papers and other outputs the Panel will consider the editorial and refereeing standards as part of the indication of quality, but absence of these standards will not be taken to mean an automatic absence of quality.
3.19.7 The Panel collectively will examine in detail a minimum of 10% of the cited outputs. The Panel will examine work across the range of output media and especially from those outputs in media not subject to rigorous editorial and refereeing standards, as may be the case for new journals, books, chapters in books and other output in less familiar media.
3.19.8 Where output other than recognised publications is cited, including published reports of applied research work for industry, the 'Other relevant details' field on RA2 should be used to highlight the research component so enabling the Panel to give the work its due recognition.
3.19.9 The Panel will regard four items of research output per research active individual as the normal expectation. Where co-authored works are submitted by two or more members of the same department the Panel would expect each researcher to submit four different cited outputs.
3.19.10 Order of authorship will not be considered important.
Research Students (RA3)
3.19.11 The number of higher degrees awarded will be taken as an indicator of quality and the number of FTE students registered for a research degree will be taken as an indicator of vitality. Higher degrees, and doctorates in particular, will be considered more highly than research Masters.
3.19.12 No differentiation will be made regarding the source of support for students.
External Research Income (RA4)
3.19.13 External income for research will be used as an indicator of the standing of research or its promise. All income will be given the same weight. The Panel will not include income awarded under the Joint Infrastructure Initiative (JIF) or the Joint Research Equipment Initiative (JREI) in this indicator but will treat such income as an indicator of standing and vitality. Institutions may therefore wish to highlight the significance of JIF and/or JREI awards in RA5. Since income is considered to be primarily a facilitator, not an output measure, the effect of disproportionate research funding will be desensitised by placing a ceiling on average earnings per full-time equivalent active researcher in a submission, above which further income will not be included in the numeration.
Research Plans and General Observations (RA5 and RA6)
3.19.14 The research organisation and future plans stated in RA5 and RA6 will act as a framework for assessing the information provided in the rest of the submission, and will be tested against plans submitted to the Panel in the 1996 RAE, where available.
Research Structure and Environment
3.19.15 The Panel invites HEIs to submit the following information:
3.19.16 The Panel will also be concerned to identify evidence of vitality, in support of which a department may wish to refer to its volume of activity and its own view of the standing of its research groups. The number of research assistants will also be regarded as a measure of vitality. The Panel would welcome brief information on the following items, where appropriate:
3.19.17 The Panel would welcome the following information:
3.19.18 The Panel would welcome the following information:
3.19.19 The Panel invites HEIs to provide a self-assessment of performance within the textual commentary in relation to the issues detailed above.
Evidence of Esteem
3.19.20 The Panel will consider indicators of peer esteem which relate to the staff submitted. Indicators of peer esteem may include evidence of national and international visibility, collaboration with outside bodies, honours and awards given to individuals in recognition of their research and positions reflecting esteem and responsibility.
Individual Staff Circumstances
3.19.21 The Panel will consider any individual staff circumstances which have significantly affected their contribution to the submission (e.g. periods of sick leave, career breaks, engagement on long-term projects etc).
Contributions by Non Research-active Staff
3.19.22 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.19.23 All members of the Panel will review all submissions and make an initial assessment. The Panel will divide the examination of the research outputs among its members drawing on their areas of expertise as appropriate.
3.19.24 The Panel will take into account the various measures of a department's performance outlined above. It will use its judgement to numerate the various measures of performance, using the same criteria for each submission. All quantified elements of a submission will be normalised to allow for the full-time equivalent number of research active staff contributing to that element. A range of weightings will be applied to the average scores per FTE active researcher and aggregated to produce an overall score. The range of weightings in the assessment of measures will be as follows:
Quality of research output 30-50 per cent
Extent of postgraduate activity 10-20 per cent
Evidence of esteem by external funders 15-30 per cent
Evidence of standing and vitality of the department 15-40 per cent
3.19.25 The Panel will decide the specific weightings during the assessment process, and a sensitivity analysis will be carried out to test the influence of the range of weightings for each parameter. The specific weightings will be applied uniformly to all submissions in order to provide an initial guide to inform the assessment process.
3.19.26 Each submission will then be re-assessed with particular reference to research quality that equates to attainable levels of national and international excellence to ensure conformity with the definitions of the standard rating scale. The Panel will award the final rating on consideration of the balance of quality across the whole submission.
3.19.27 The Panel's concept of international excellence in research is work that will be widely regarded as making an appreciable contribution to the knowledge base within the field, and will influence, or have the potential to influence, the global research and/or the practitioner communities. Such work will help set the international research agenda in the field, or contribute significantly to its development, through, inter alia, leadership, impact, publication and/or collaboration. Research of national excellence is work that is not of such widely recognised significance, but is still substantive and, where appropriate, relevant to policy and practice.
3.19.28 The Panel will reach its decisions after consideration against the rating scale by consensus. In cases where consensus is not attained, final gradings will be decided by majority vote.
3.19.29 The Panel will seek the advice of up to five non-UK based experts on its identification of international excellence through referral of submissions graded 5* and 5 and a sample of those graded 4.
Last updated 17 April 2000