Section III: Panelsí Criteria and Working Methods
3.23 Mechanical, Aeronautical and Manufacturing Engineering, UoA 30
3.23.1 The UoA includes: Acoustics, noise and vibration; Aerodynamics and aeronautics; Automotive engineering; Biomedical engineering; Computational methods in engineering; Control, control of fluid power and fluidics; Dynamics; Engineering design; Heat transfer; Manufacturing, including manufacturing systems and manufacturing management; Materials, including polymers and composites; Mechatronics; Offshore engineering and related subjects where associated with mechanical engineering; Optical engineering; Process engineering; Product design; Solid mechanics; Structural integrity, fatigue, failure analysis and welding; Thermodynamics and fluid mechanics.
3.23.2 If the Panel should receive any submissions whose contents span the boundaries of its unit of assessment and others, it will refer such submissions to the panel(s) concerned for their advice. The Panel recognises that it may receive pedagogic research within mechanical, aeronautical and manufacturing engineering and, if necessary, it will draw on expert advice from the Education Panel or specialist advisers to assist it in its assessments of such research.
Sub-Panels and Specialist Advice
3.23.3 The Panel believes that it has the necessary expertise to assess the key subject areas of the unit of assessment and it does not anticipate that it will be necessary to establish any sub-panels. Submissions will be cross-referred to other panels as appropriate. The Panel recognises, however, that in the light of the submissions which it receives it might be necessary in exceptional circumstances to seek specialist advice.
3.23.4 The Panel will seek to ensure that interdisciplinary research is equitably and properly assessed by cross-referring submissions which cite work of an interdisciplinary nature to other cognate panels for their advice, if appropriate.
3.23.5 The Panel will make no distinction between joint submissions and single submissions and will assess both in the same way.
Treatment of Evidence
3.23.6 All the evidence in each submission will be taken into account but quality will be judged on the basis of the main sections of the submission which are:
A range of weightings will be applied to the individual measures.
Research Output (RA2)
3.23.7 The Panel anticipates that it might be required to assess the following research outputs. (The list does not indicate any hierarchical order.)
3.23.8 The Panel's assessment of the quality of outputs will be based upon their judgement of the outputs': originality; innovation; contribution to advancing knowledge and understanding; contribution to the field and, where appropriate, to policy and practice. In making their assessments the Panel will take account of any peer review process which the outputs have already undergone. Journal articles and conference papers which have undergone a rigorous refereeing or editorial process, invited or keynote papers for international conferences and other output which has been subject to peer review will be regarded as being of a certain quality, but output not in these categories will not automatically be regarded as being of lesser quality. Institutions will have the opportunity in the 'Other relevant details' field on RA2 to draw the Panel's attention to any particular circumstances associated with an output, e.g. the award of a prize, and the status of conference papers cited.
3.23.9 It would be helpful to the Panel if the submissions could indicate in 'Other relevant details' field on RA2 whether a conference paper was fully-refereed, abstract-refereed or non-refereed.
3.23.10 The Panel collectively will examine in detail a minimum of 10% of the cited outputs. The Panel will examine work across the range of output media and especially from those outputs in media not subject to rigorous editorial and refereeing standards, as may be the case for new journals, books, chapters in books and other output in less familiar media.
3.23.11 The Panel would normally expect research active staff in categories A and C to cite four items of research output and may regard failure to cite four such items of output as indicating lack of research depth. If there are particular circumstances relating to an individual for whom fewer than four outputs are cited (e.g. he/she is a new researcher) this should be indicated either in RA6.
3.23.12 The citation of the same output by two or more co-authors within the same submission may be regarded by the Panel as evidence of lack of research depth unless there are particular circumstances to justify it (which should be drawn to the Panel's attention in 'Other relevant details' field on RA2 ).
Research Students and Research Studentships (RA3)
3.23.13 The Panel expects to place a greater emphasis on research outputs, such as the number of degrees awarded, rather than on research studentships. The number of higher degrees awarded will be regarded as an indicator of quality, with doctorates being rated more highly than research masters.
External Research Income (RA4)
3.23.14 External income for research will be used as an indicator of the standing of research or its promise. All income will be given the same weight. The Panel will not include income awarded under the Joint Infrastructure Initiative (JIF) or the Joint Research Equipment Initiative (JREI) in this indicator but will treat such income as an indicator of standing and vitality. Institutions may therefore wish to highlight the significance of JIF and/or JREI awards in RA5. Since income is considered to be primarily a facilitator, not an output measure, the effect of disproportionate research funding will be desensitised by placing a ceiling on average earnings per full-time equivalent active researcher in a submission, above which further income will not be included in the numeration.
3.23.15 The textual commentary provided by institutions in RA5 & RA6 will act as a framework for assessing the information contained in the rest of the submission and may be tested against the plans submitted to the Panel in the 1996 RAE, where appropriate. This information will be assessed as a quality measure in terms of research vitality.
Research Structure and Environment
3.23.16 Institutions are invited to provide information on the research structure of the unit of assessment. If appropriate, they may structure RA5 according to the research groups within their department indicating for each research group their members, their prime activities, how they operate and their main achievements. If an institution structures its RA5 according to research groups, RA1 and RA2 should be similarly structured. Any difficulties of 'fit' between the departmental structure and the UoA framework may be highlighted.
3.23.17 The mechanisms and practices for promoting research and sustaining and developing an active and vital research culture should be explained and a description given of the nature and quality of the research infrastructure, including facilities for research students. Institutions should refer to any arrangements which are in place for supporting interdisciplinary or collaborative research and provide information on relationships with industry or other research users. Responses to national and international priorities and Government policy initiatives and objectives (e.g. Technology Foresight) should also be referred to.
3.23.18 A description of the arrangements for the development and support of the research work of staff should be provided. Any arrangements which are in place for developing the work of younger and/or new researchers and for integrating them into a wider supportive research culture should be highlighted. Where appropriate, institutions should explain the role and contribution of category A* staff who have joined the department and comment on how the departure of staff in categories A*, B and D has affected the strength, coherence and research culture of the department at the census date.
3.23.19 The Panel would welcome brief information on:
3.23.20 Institutions should provide a statement about the main objectives and activities of the UoA over the next five years. If appropriate, attention should be drawn to ongoing research work which is not producing immediately visible outcomes. Where relevant, an evaluation of the research plans put forward in the 1996 RAE should be provided.
3.23.21 Account will be taken of honest, self-critical and constructive self-assessments of performance in relation to the issues detailed in the textual commentary.
Evidence of Esteem
3.23.22 The Panel invites institutions to use this section to draw attention to items of note which might indicate the national or international standing or peer esteem afforded to the staff submitted. These might include:
Individual Staff Circumstances
3.23.23 Institutions are invited to indicate any individual staff circumstances which have significantly affected their contribution to the submission (e.g. periods of sick leave, career breaks, engagement on long-term projects etc.)
Contributions by Non Research-active Staff
3.23.24 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.23.25 All members of the Panel will review all submissions and make an initial assessment. The Panel will divide the examination of the research outputs among its members drawing on their areas of expertise as appropriate.
3.23.26 The Panel will take into account the various measures of a department's performance outlined above. It will use its judgement to numerate the various measures of performance, using the same criteria for each submission. All quantified elements of a submission will be normalised to allow for the full-time equivalent number of research active staff contributing to that element. A range of weightings will be applied to the average scores per FTE active researcher and aggregated to produce an overall score. The range of weightings in the assessment of measures will be as follows:
Quality of research output 30-50 per cent
Extent of postgraduate activity 10-20 per cent
Evidence of esteem by external funders 15-30 per cent
Evidence of standing and vitality of the department 15-40 per cent
3.23.27 The Panel will decide the specific weightings during the assessment process, and a sensitivity analysis will be carried out to test the influence of the range of weightings for each parameter. The specific weightings will be applied uniformly to all submissions in order to provide an initial guide to inform the assessment process.
3.23.28 Each submission will then be re-assessed with particular reference to research quality that equates to attainable levels of national and international excellence to ensure conformity with the definitions of the standard rating scale. The Panel will award the final rating on consideration of the balance of quality across the whole submission.
3.23.29 The Panel's concept of international excellence in research is work that will be widely regarded as making an appreciable contribution to the knowledge base within the field, and will influence, or have the potential to influence, the global research and/or the practitioner communities. Such work will help set the international research agenda in the field, or contribute significantly to its development, through, inter alia, leadership, impact, publication and/or collaboration. Research of national excellence is work that is not of such widely recognised significance, but is still substantive and, where appropriate, relevant to policy and practice.
3.23.30 The Panel will reach its decisions after consideration against the rating scale by consensus. In cases where consensus is not attained, final gradings will be decided by majority vote.
3.23.31 The Panel will seek the advice of up to five non-UK based experts on its identification of international excellence through referral of submissions graded 5* and 5 and a sample of those graded 4.
Last updated 17 April 2000