Section III: Panelsí Criteria and Working Methods
3.24 Metallurgy and Materials, UoA 32
3.24.1 The Unit of Assessment (UoA) encompasses research activities involving materials science and engineering in all structural and functional materials such as metals, glasses, ceramics, semiconductors and superconductors, polymers, composites, biomaterials, textiles and clothing, paper, leather and wood. Activity involving studies of the synthesis, structure, characterisation, processing, properties, applications, modelling, degradation and protection, and the recovery, recycling and re-use of these materials all come under the Panel's remit.
3.24.2 The Panel would not expect to receive for assessment work in disciplines which lie outside its terms of reference (for example, geology, fashion, art and design, the history of science and technology) which does not have a clearly identifiable relationship to the areas of study which come within the boundaries defined above. The Panel does, however, recognise the interdisciplinary and multidisciplinary nature of research and wishes to encourage submissions that include a clearly defined contribution by researchers in other disciplines to the advancement of knowledge within materials science and engineering.
3.24.3 In accordance with the rules of the RAE the Panel will cross-refer submissions to other panels where requested to do so. The Panel will also cross-refer submissions to other panels in cases where the work submitted requires additional expertise to ensure its proper evaluation. Where necessary the meetings of the umbrella group of Panel Chairs in physical sciences and engineering will be used during the assessment phase of the RAE. Pedagogic research will be treated equally and in order to ensure that it is assessed fairly the Panel will, as necessary, use cross referral, additional expertise and meetings with the umbrella group of Panel Chairs.
3.24.4 The membership of the Panel has been selected in a manner designed to ensure that its combined membership is, as far as is possible, representative of all the activities which fall within the UoA boundaries. The membership of the Panel has been expanded in the light of experience gained from the 1996 RAE to maximise the breadth of the representation. The Panel will also use cross-reference to other panels and, where necessary, will obtain specialist advice (see below) to ensure that equality of treatment is given to the assessment of all submissions received. The Panel feels that the measures taken are sufficient to ensure fairness and does not intend to convene sub-panels.
3.24.5 The Panel will seek external specialist advice in cases where the proper assessment of work submitted requires specialist expert knowledge which is not available within the Panel, or through reference to other RAE panels and sub-panels.
3.24.6 As already stated, the Panel recognises the importance of interdisciplinary research and the contribution that it makes to the advancement of knowledge, so wishes to ensure that the assessment process is fair and equitable. The Panel has noted and will be guided by the Circular RAE 1/99. In addition the Panel will cross-refer submissions containing interdisciplinary work to other panels where its proper assessment requires advice from members in other relevant disciplines, or, where necessary, will consult specialist advisers. The Panel, however, will make the final decision on how to use the advice.
3.24.7 The Panel will make the basic assumption that joint submissions are made because the partner institutions consider their collaborative activities to be of such a nature that it would be inappropriate to assess them separately. Consequently the Panel will give equal consideration to single and joint submissions. In the case of joint submissions, however, the Panel will pay close attention to the overall coherence of the submission, including what is stated and demonstrated in forms RA5 and RA6.
Treatment of Evidence
3.24.8 The Panel will rate output measures more highly than input measures. Submissions will be judged mainly on the basis of the following four criteria:-
3.24.9 Of the four criteria listed above a) will be the primary indicator: criteria b) to d) are not listed in any order of importance.
Research Output (RA2)
3.24.10 The Panel anticipates that most of the research output cited in form RA2 will comprise of articles published in journals. In addition patents awarded, authored books, chapters in books and conference papers may be submitted and will be given equal weight to journal articles. Outputs disseminated through media other than the printed word may be cited, but the Panel will request the institution concerned to submit a printed version of the output if it is not readily accessible (this will be especially relevant in cases where material published on the Internet is only accessible through subscription to the site on which the publication appears).
3.24.11 If software programmes are submitted for assessment the Panel may require to have access to the programme. The institution concerned will be responsible for providing the programme and any additional information that the Panel may require to ensure the proper assessment of the output. In cases where cited outputs take the form of a product the Panel will expect to have access to some written material which places the product in its proper research context to ensure the proper assessment of the work.
3.24.12 The Panel will examine research output selectively, but representatively, from all the submissions received. The Panel will collectively examine in detail a minimum of one item of research cited in Form RA2 for each researcher submitted. UoAs are encouraged to make full use of the field provided on the 'Other relevant details' field of form RA2 to draw attention to any significant factor that might assist the Panel's evaluation of the cited output.
3.24.13 In assessing the quality of the publications the Panel will not automatically regard work which has not been subjected to a review or refereeing process as being of lesser quality. However, the Panel will normally attach weight to articles published in journals with rigorous editorial and refereeing standards. It will not collectively rank journals, but individual panel members will be guided by their own views on the refereeing and editorial procedures of journals. Patents awarded will be assessed by applying the same criteria and will have equal standing with journal articles.
3.24.14 Other forms of output will be assessed by applying the same criteria and will also have equal standing with journal articles.
3.24.15 Authored books and chapters in books representing research scholarship will be assessed using the same criteria as for articles in journals.
3.24.16 Conference papers will normally be evaluated by applying the same criteria as those used to judge journal articles and books and chapters in books.
3.24.17 The Panel will not impose any automatic penalty where research active staff submit less than four publications. It does, however, believe that in this discipline a submission of less than four outputs per researcher may be indicative of a lack of vitality and where an individual researcher has less than four publications the Panel will wish to see an explanation for this in forms RA6.
3.24.18 When the same research output has been cited by different researchers in the same UoA, it will be taken as a lack of research depth unless an explanation of the significance of the output is given in forms RA6.
Research Students and Research Studentships (RA3)
3.24.19 The Panel will take into account the number of higher degrees awarded as an indicator of the vitality of the UoA. Doctorates will be rated more highly than research masters degrees. In making its evaluation the Panel will consider such factors as whether the data show a consistent throughput of students, and whether it is suggestive of a strengthening or a decline in postgraduate research student activity.
3.24.20 The Panel will place greater weight upon output rather than input factors; no distinction will be made so far as the source of studentships is concerned.
External Research Income (RA4)
3.24.21 The Panel recognises that it is the output generated rather than the amount of money earned that is indicative of the quality of the research activity, so it will use the data provided selectively to inform its judgements in relation to the quality of the outputs cited in Form RA2.
3.24.22 The Panel recognises that many departments are engaged in both basic and applied research. In such circumstances the Panel will look for the evidence of the existence of a distribution of research earnings from different sources, and the balance of the published works, patents and licenses.
3.24.23 The Panel will place greater weight upon output rather than input factors. No distinction will be made so far as the source of external research income is concerned.
Research Structure and Environment
3.24.24 Institutions should describe any coherent research groupings that exist within the unit. The structure of the overall research activity should be described and an explanation given as to how each group fits into this structure. Care should be taken to explain how these groups relate to each other, how they operate, and their main achievements. Any group structure described in form RA5 should clearly relate to the information given in form RA1 which should be set out in such a way that it shows to which group each active researcher belongs.
3.24.25 The Panel will make allowance for people who work in more than one area and thus overlap between groups. It is recognised that such people may often be the most creative and productive and their being allocated to one particular group would be inappropriate.
3.24.26 The Panel would be interested to know of any related work which has been submitted to another UoA. Institutions should also alert the Panel to any difficulties that have been experienced in fitting their activities comfortably into the boundaries defined in these criteria.
3.24.27 Institutions should explain the mechanisms and practices for promoting research and sustaining and developing an active and vital research culture. In assessing the vitality of each submission the Panel will pay particular attention to the staff profile presented in form RA1. The Panel will be particularly interested in knowing how new academic staff have been incorporated into the research culture of the department and how the vitality of the research activity is to be maintained in cases where key staff are approaching retirement.
3.24.28 A brief overview of the nature and quality of the research infrastructure should be given including facilities for research students. It will be the responsibility of the institution to decide whether there are matters to be included under this heading that should be highlighted as being of particular merit or significance.
3.24.29 Submissions should describe any arrangements that are in place to encourage and develop interdisciplinary research activities. It will be the responsibility of the institution to decide whether there are matters to be included under this heading that should be highlighted as being of particular merit or significance.
3.24.30 If research activity is in an applied area the Panel will be particularly interested to learn of any contribution that technology transfer has made to the advancement of knowledge in the discipline (institutions should take careful note of the definition of research which applies in the RAE to ensure that any information given is relevant). Institutions should comment, where appropriate, about the account they have taken of relevant Government policy initiatives and objectives.
3.24.31 Institutions should describe the arrangements and policies that are in place for the support of the research activities of staff.
3.24.32 The Panel will pay particular attention to newly appointed younger academic staff, especially those who are in their first post. Submissions should describe any arrangements that are in place for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research culture.
3.24.33 Submissions should explain the role and contribution being made to the research activity by experienced category A* staff who have joined the department.
3.24.34 Comment should be provided on how the departure of staff included in categories A*, B and D has affected the strength, coherence and research culture of the department at the census date. The Panel should be advised of any polices that have been implemented to ensure that the vitality of the research activity is maintained in the event of staff leaving.
3.24.35 The Panel requires all submissions to provide a statement describing the main objectives and activities of the department over the next five years.
3.24.36 Where relevant the Panel will use the information given in forms RA5 from the 1996 RAE to assist in its evaluation of the development of the research activity in the intervening period between the 1996 and 2001 RAEs. If a department is submitting to this Panel for the first time a brief description should be given of how the research activity has developed over the past five years.
3.24.37 The Panel encourages the submission of honest, self-critical and constructive self-assessment of the strengths and weaknesses of the department and of the value of the information given in form RA5.
Evidence of Esteem
3.24.38 The Panel will take account of evidence of peer esteem: examples are:-
3.24.39 The Panel does not rank the above examples in any particular order of importance, nor does it wish to exclude other indicators of peer esteem that are not included in the list.
3.24.40 The Panel will consider the information presented on form RA6 from an overall perspective and will expect to see evidence to show how these activities have contributed to the development of the research activity and the maintenance of its vitality.
Individual Staff Circumstances
3.24.41 Submissions should give details of any individuals circumstances which have significantly affected the content of the submission such as periods of sick leave, career breaks, engagement on long term projects, etc.
Contributions by Non Research-active Staff
3.24.42 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.24.43 Institutions are requested to highlight significant contributions relevant to the Technology Foresight programme. The Panel, does, however, recognise that there are distinct regional and local variations which influence the nature of these contributions. Where such variations are thought by the institution to be significant the Panel would like to be advised. Institutions are also invited to mention any regional or national facilities that come under its management which make a significant contribution to research in a national, or an international context. Institutions are also invited to highlight any other regional or local factors which they feel have a significant influence upon the research activity.
3.24.44 Institutions should mention in form RA5 and RA6 any significant factors which they feel the Panel should know and which are not self-evident from the data submitted.
3.24.45 The Panel will employ a quantitative approach in dealing with some of the data to determine the average number of publications, research students supervised, and amount of research grant income earned per research active member of staff returned. Such data will be used by the Panel to inform its assessment of submissions and to assist in making value judgements, but will not form the sole basis upon which final judgements of quality are made.
3.24.46 In general the quantifiable parts of the submission will be normalised to allow for the number of staff (FTE cat A + A*) contributing to that part. Thus the size of the department will not directly affect the outcome. The size of the department may, however, be taken into account in the subjective assessment process which will include an assessment of the department's ability to have available to it, or access to, the infrastructure and facilities to undertake high-quality research consistent with its stated plans.
3.24.47 The Panel will take into account all of the evidence submitted by each department. All Panel members will read all the submissions received and will be allocated a specific number of outputs to read in detail. The distribution of the submissions will be determined by the content of each submission. Where the Panel cannot assess the outputs properly in order to determine a rating it will seek specialist advice.
3.24.48 Individual research outputs in RA2 will be rated as 'international', 'national' or 'sub-national' based upon the following definitions:
3.24.49 The Panel will review the works listed in form RA2 by each member of staff. From the Panel's assessment of the quality of the research output data a preliminary ranking will first be established for the work of each member of staff. These preliminary rankings will then be used to determine an overall ranking for the research output submitted on form RA2. When considering the overall ranking the Panel's judgements will be informed not only by the individual rankings arrived at for the work of each member of staff, but also by taking into account the additional data provided by the department demonstrating how the work submitted by each person forms part of a larger cohesive group activity. The Panel will not, however, discriminate against individuals who are carrying out high quality research which, because of its nature, is not suited to group work.
3.24.50 As stated elsewhere in these criteria, departments are asked to describe clearly how the research activity is structured. When assessing the research outputs listed in form RA2 the Panel will bear in mind the descriptions given in form RA5 together with the groupings set out in form RA1 and consider whether the information given there is reflected in the nature and content of the work being reviewed. The Panel will look for consistency between forms RA1, RA2 and RA5.
3.24.51 Through the process described above provisional ratings will be reached on a consensual basis for all submissions through the Panel exercising its professional judgement on all the evidence submitted.
3.24.52 The Panel will ensure consistent and equitable treatment of all submissions made by employing the same assessment criteria consistently.
3.24.53 The Panel expects to agree its decisions through the achievement of consensus in preference to voting. Where a consensus cannot be reached the Panel will vote: if necessary the Chair will exercise a casting vote.
3.24.54 The Panel will interpret the rating points for the discipline by reference to the criteria set out in 3.24.48.
Non-UK based Experts
3.24.55 The Panel will seek the appointment of up to a maximum of five non-UK based experts. Those appointed will collectively be, so far as is possible, representative of the knowledge required to assess the quality of all the submissions which the Panel refers to them.
3.24.56 Once the Panel has assigned preliminary ratings to each submission it will ask the non-UK based experts to review a selection of submissions that are to be considered for the highest ratings. The experts will act in the capacity of advisers and will be asked to give advice to the Panel on the international relevance and quality of the research. The Panel will review the preliminary rating given to each individual submission upon which the non-UK based experts have been asked to comment in the light of the advice received and decide whether any adjustment should be made prior to agreeing its final ratings.
Last updated 17 April 2000