Section III: Panelsí Criteria and Working Methods
3.22 Electrical and Electronic Engineering, UoA 29
3.22.1 The Electric and Electronic Engineering Unit of Assessment includes the following key subject areas: communications (mobile, satellite, networks etc.); electronic materials and devices; electronic systems and circuits; optoelectronics and optical communications systems; multimedia, video and audio processing and coding; signal processing, modelling and estimation; radio frequency, microwave and millimeter wave techniques; measurement, instrumentation, sensors; control, robotics and systems engineering; electrical power, machines and drives; and computer and software engineering. Related subjects within electrical and electronic engineering will also be considered by the Panel.
3.22.2 The Panel recognises that some subject areas span the boundaries of this unit of assessment and others. The Panel intends to establish close lines of communication with panels in related areas, in particular Computer Science, Physics and other Engineering panels. It will seek advice from the appropriate panel where it considers that part of the submission falls out with its area of expertise, or where an institution has requested that its submission be referred to another panel.
Sub-Panels and Specialist Advisers
3.22.3 The Panel does not anticipate that any sub panels will be required.
3.22.4 Specialist advice may be required where the Panel considers that it does not have the necessary expertise. In the main the Panel will establish close links with panels in cognate areas and will seek advice from them when necessary. However, the Panel recognises that it may also be necessary in some circumstances to seek the advice of other special advisers.
3.22.5 The Panel considers that it has the necessary experience and expertise to asses areas of interdisciplinary research. However, the Panel may seek advice from other Panels if it considers that part of the submission falls out with its area of expertise, or where an institution has requested that its submission be referred to another panel using the cover sheet provided. Institutions are requested to draw the Panel's attention to work which they consider to be interdisciplinary and which may fall between this and another unit of assessment using the 'Other relevant details' field on RA2.
3.22.6 The Panel will treat joint submissions between two or more institutions in the same way as single submissions. The Panel does not wish to discourage institutions from making joint submission but will expect to see evidence of genuine collaboration between institutions.
Treatment of Evidence
3.22.7 The Panel will judge submissions on the basis of the following five measures:
Quality of cited research outputs(RA2).
Extent of postgraduate activity (RA3).
Evidence of esteem by external funders (RA4).
Evidence of standing and vitality of the department (RA5 and RA6).
3.22.8 A range of weightings will be applied to the individual measures.
3.22.9 The Panel requests that RA1, RA2, RA5 and RA6 are structured according to research groups. These groupings should reflect as closely as possible the natural working structure of the staff submitted, and single person groups will be permitted. Individuals belonging to more than one group should be flagged in RA1.
Research Output (RA2)
3.22.10 The Panel expects to receive the following types of research outputs. (This list does not indicate a hierarchical order).
3.22.11 The Panel will also consider other forms of research outputs, such as software, multimedia and video research outputs. Where outputs of this type are included in a submission, institutions should indicate to the Panel how it perceives these outputs to embody research. The 'Other relevant details' field on RA2 should be used for this purpose.
3.22.12 The Panel's normal expectation is that no item of output will be listed for more than one member of staff, but if this occurs then an explanatory comment should be included in the 'Other relevant details' field on RA2.
3.22.13 The Panel collectively will examine in detail a minimum of 10% of the cited outputs. The Panel will examine work across the range of output media and especially from those outputs in media not subject to rigorous editorial and refereeing standards, as may be the case for new journals, books, chapters in books and other output in less familiar media.
3.22.14 In assessing the quality of academic or professional journal papers, the Panel will use its expertise to judge the editorial and refereeing standard of the journal in which the article appears. Generally those which have undergone a rigorous refereeing and editorial process will be regarded as being of high quality. Similar criteria will be used to judge conference papers.
3.22.15 In assessing the quality of authored books and chapters in edited books, the Panel will take note of the novel research contribution of the author in the cited work or chapter.
3.22.16 In assessing awarded patents and other forms of research output, such as software, multimedia and video research outputs, the Panel will form a view of the research contribution of the output.
Research Students and Research Studentships (RA3)
3.22.17 The Panel expects to place higher emphasis on research outputs, such as the number of degrees awarded, rather than on research studentships. The number of higher degrees awarded will be regarded as an indicator of quality with doctorates being rated more highly than research masters.
External Research Income (RA4)
3.22.18 External income for research will be used as an indicator of the standing of research or its promise. All income will be given the same weight. The Panel will not include income awarded under the Joint Infrastructure Initiative (JIF) or the Joint Research Equipment Initiative (JREI) in this indicator but will treat such income as an indicator of standing and vitality. Institutions may therefore wish to highlight the significance of JIF and/or JREI awards in RA5. Since income is considered to be primarily a facilitator, not an output measure, the effect of disproportionate research funding will be desensitised by placing a ceiling on average earnings per full-time equivalent active researcher in a submission, above which further income will not be included in the numeration.
3.22.19 RA5 should be used to provide the Panel with information relating to the research organisation, achievements and vitality of the department. Institutions should provide an introductory statement indicating the department's overall research objectives and priorities.
3.22.20 The Panel requests that all other information provided in RA5 relates to research groups (see paragraph 3.22.9). Departments should define those research groups (referring to RA1), their prime activities, how they operate, their main research achievements and future research plans. The Panel expects to place particular emphasis on research achievements and research plans and will examine the mechanisms and practices for promoting research and sustaining and developing an active and vital research culture. In doing so, the Panel expects to refer to the research plans submitted in the 1996 Exercise.
3.22.21 The Panel also requests that departments consider the following list when making their returns (departments do not need to return information under every heading nor is this list exhaustive):
3.22.22 Departments are also invited to provide a brief self assessment of their research standing as they perceive it, without reference to the ratings scale.
3.22.23 The Panel expects departments to provide the following additional information in RA5 for each research group:
3.22.24 The Panel will regard this information as an indication of the level of activity for each research group. It will be used in a qualitative, not a quantitative manner. The Panel expects that outputs published jointly by more than one member of staff will only be counted once.
3.22.25 Departments are also encouraged to provide, in RA5, the average spend per research group per annum to the Panel as an indication of the breakdown of income reported in RA4.
Evidence of Esteem
3.22.26 The Panel expects institutions to use this section to draw attention to instances of external recognition or items of particular note that indicate the department's international and national standing. This section should also be structured by research group and evidence of esteem provided for each group. Examples of such evidence are given below Information need not be returned under every heading, nor is this list exhaustive.
Individual Staff Circumstances
3.22.27 In particular, the conclusion of form RA6 should be used to draw the Panel's attention to circumstances which may have affected a researcher's ability to contribute to the submission (e.g. periods of sick leave, career breaks, engagement on long term projects or newly appointed young staff.)
Contributions by Non Research-active Staff
3.22.28 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.22.29 All members of the Panel will review all submissions and make an initial assessment. The Panel will divide the examination of the research outputs among its members drawing on their areas of expertise as appropriate.
3.22.30 The Panel will take into account the various measures of a department's performance outlined above. It will use its judgement to numerate the various measures of performance, using the same criteria for each submission. All quantified elements of a submission will be normalised to allow for the full-time equivalent number of research active staff contributing to that element. A range of weightings will be applied to the average scores per FTE active researcher and aggregated to produce an overall score. The range of weightings in the assessment of measures will be as follows:
Quality of research output 30-50 per cent
Extent of postgraduate activity 10-20 per cent
Evidence of esteem by external funders 15-30 per cent
Evidence of standing and vitality of the department 15-40 per cent
3.22.31 The Panel will decide the specific weightings during the assessment process, and a sensitivity analysis will be carried out to test the influence of the range of weightings for each parameter. The specific weightings will be applied uniformly to all submissions in order to provide an initial guide to inform the assessment process.
3.22.32 Each submission will then be re-assessed with particular reference to research quality that equates to attainable levels of national and international excellence to ensure conformity with the definitions of the standard rating scale. The Panel will award the final rating on consideration of the balance of quality across the whole submission.
3.22.33 The Panel's concept of international excellence in research is work that will be widely regarded as making an appreciable contribution to the knowledge base within the field, and will influence, or have the potential to influence, the global research and/or the practitioner communities. Such work will help set the international research agenda in the field, or contribute significantly to its development, through, inter alia, leadership, impact, publication and/or collaboration. Research of national excellence is work that is not of such widely recognised significance, but is still substantive and, where appropriate, relevant to policy and practice.
3.22.34 The Panel will reach its decisions after consideration against the rating scale by consensus. In cases where consensus is not attained, final gradings will be decided by majority vote.
3.22.35 The Panel will seek the advice of up to five non-UK based experts on its identification of international excellence through referral of submissions graded 5* and 5 and a sample of those graded 4.
Last updated 17 April 2000