2001 Research Assessment Exercise: The Outcome
3 Description of the RAE results
Ratings of research quality are expressed in terms of a standard scale with common definitions of the points. The descriptors of the points on the rating scale were reviewed and amended for the 2001 RAE. The rating scale and definitions used in the 2001 RAE are:
5* (five star)
Quality that equates to attainable levels of international excellence in more than half of the research activity submitted and attainable levels of national excellence in the remainder.
Quality that equates to attainable levels of international excellence in up to half of the research activity submitted and to attainable levels of national excellence in virtually all of the remainder.
Quality that equates to attainable levels of national excellence in virtually all of the research activity submitted, showing some evidence of international excellence.
Quality that equates to attainable levels of national excellence in over two-thirds of the research activity submitted, possibly showing evidence of international excellence.
Quality that equates to attainable levels of national excellence in more than half of the research activity submitted.
Quality that equates to attainable levels of national excellence in up to half of the research activity submitted.
Quality that equates to attainable levels of national excellence in none, or virtually none, of the research activity submitted.
Panels used their judgement in applying the descriptions attached to points on the rating scale. They formed a view on the quality of all the research activity presented in each submission in the round.
In order to attain a point on the scale a submission had to exceed the requirements of the next lowest point. For example, in the description of 5, ’up to half’ implies more than the maximum for ‘some’ required for a grade 4.
Panels were permitted to form a view that the balance of quality justified the award of a particular grade even where the precise terms of the descriptions were not met. For example, a submission which considerably exceeded the required proportion of international excellence, but did not meet the requirement for national excellence in the remainder, has received the grade which the panel felt was justified on balance.
Attainable levels of excellence refer to absolute standards of quality in each Unit of Assessment, and are independent of the conditions for research within individual institutions.
The international criterion adopted equated to a level of excellence that it was reasonable to expect for the UoA, even though there may be no current examples of such a level in the UK or elsewhere. In the absence of current examples, standards were adopted from cognate research areas where international comparisons do exist. It should be noted that ‘national’ and ‘international’ refer to standards, not to the nature or scope of particular disciplines.
’Virtually all’ and ’virtually none’ should be understood as within the top and bottom 10 per cent respectively. ’Some’ should be understood as around 10 per cent.
For the RAE, ’national’ refers to the United Kingdom of Great Britain and Northern Ireland.
As in 1996, the published results contain three elements: grade, full-time equivalent (FTE) number of Category A and A* staff submitted, and proportion of staff submitted (A plus A*) as research-active. This last indicator takes the form of a letter denoting a band, using the following scale:
A 95-100 per cent of staff submitted
B 80-94.9 per cent
C 60-79.9 per cent
D 40-59.9 per cent
E 20-39.9 per cent
F below 20 per cent
The results are published as a series of tables, one for each UoA. Each table contains the grade awarded to each HEI making a submission to the UoA concerned. The web-based version of this document also contains tables of results ordered by institution.
In the case of Clinical Medicine (UoAs 1, 2 and 3), the tables also include the ratings awarded to any areas of work referred to the sub-panels of those UoAs at the request of the submitting HEI. These grades take the form: grade (using the standard 1 to 5* scale), FTE of Category A and A* referred to the sub-panel.
For all UoAs, the results may include flagged areas of research which the panel considered to be at a standard equivalent to at least two points above the grade awarded to the whole submission. Such flagging is at the discretion of the panel concerned.
Where the grade awarded is to a joint submission made by two or more HEIs, this is noted in the relevant UoA table.
A summary of the results awarded to each HEI is presented after the UoA tables. This shows, in graphic form, the number of each grade awarded to the HEI, and the proportion of all staff eligible for inclusion in Categories A and A* who were selected as research-active.
In calculating the proportion of staff selected as research-active, the number of non-selected staff in UoAs 10, 11 and 13 whose ultimate source of salary is the National Health Service has been omitted. This applies both to the individual UoA calculations and to those for each institution overall. This is a transitional arrangement for this RAE only: it recognises the fact that large numbers of staff in these areas have transferred from NHS to HEI employment in the period since the 1996 RAE. The calculation is presented in this way to enable more ready comparison with the equivalent calculation from the 1996 RAE.
In addition to the results published in this document, the funding bodies will produce written feedback to each submitting HEI on the reasons for the grades awarded to it. This feedback will be sent in confidence to the head of the HEI concerned. It is anticipated that this feedback will be available in early 2002.
The funding bodies will publish a short report by each UoA panel, based on its assessment work and commenting in general terms on the strengths and weaknesses in the field(s) of work assessed. These reports will be published in early 2002.
The rating scale used in 2001 maps directly onto the scale used in 1996. The definitions of the scale points are different but were designed to represent comparable levels of achievement with 1996. Nonetheless, comparisons with the ratings awarded in 1996 should be made with some caution. In some cases the definition of the UoAs has changed; also, an HEI may have chosen, as a result of staff changes or for other reasons, to submit different groups of researchers in the two exercises within the same UoA.
For the 2001 RAE, considerable stress was laid on consistency of approach and marking standards, particularly within the cognate subject groupings represented by the umbrella groups in order to ensure as far as possible that ratings are broadly comparable between subjects. However, these judgements have been made in different subjects by different panels against different criteria, and it would therefore be unsafe to assume that what a particular rating indicates about a submission is the same across all subjects.
Last updated 14 December 2001
| About the RAE2001
| Guidance for panel members
| Guidance for institutions