Section III: Panelsí Criteria and Working Methods
3.39 European Studies, UoA 48
3.39.1 The UoA embraces all of Europe including Eastern and Western Europe, Scandinavia and the Baltic region, southern Europe, and Russia. It covers the modern languages, literatures and cultures, including film and media studies; language teaching and learning, including the integration of new technologies; translation; linguistics; modern economic, social and political history; philosophy; religions; gender studies; also the business environment, economics, geography, international relations, politics, public policies, and sociology; as well as research into the teaching of European Studies. The UoA also deals with European integration and the external relations of Europe. Submissions may therefore legitimately include areas of research which are eligible for submission to other UoAs.
3.39.2 The Panel will consider submissions from any department that covers a range of European languages and associated literatures, linguistics and cultures and/or which studies modern Europe using a variety of social scientific, historical and literary methods.
3.39.3 Where a submission spans the boundary with one or more other UoAs, and the Panel decides that it lacks the appropriate expertise, the whole submission will be cross-referred to other relevant Panels with a request to advise on the relevant work in the context of the submission. Cross-referral will involve a meeting with other Panels at which at least one member of this Panel will be present. Responsibility for awarding the rating remains with the European Studies Panel. Where other Panels feel that they too lack the appropriate expertise, the European Studies Panel will establish ad hoc specialist advisory arrangements. It will appoint specialist advisers in for example Irish Studies, Scottish Studies and Catalan. The Panel reserves the right where appropriate to cross-refer to other panels submissions that focus entirely on single countries or languages.
3.39.4 The Panel represents a wide range of disciplines in recognition of the multidisciplinary character of the Unit of Assessment. Though its composition should enable it to deal with most of the interdisciplinary work that is submitted, cross-referral and specialist advice will have to be used in the manner outlined above.
3.39.5 Departments are urged to identify in their submissions aspects of their work that do not fit well within the framework of the European Studies Panel and to request cross-referral to one or more other panels. The Panel will always act on such requests. At the same time it will make cross-referrals at its own discretion where it judges that the assessment process would be improved.
3.39.6 The Panel is not expecting to use a sub-panel. It will appoint for example, specialist advisers in Catalan, Irish Studies and Scottish Studies.
3.39.7 The Panel will treat joint submissions similarly to submissions from individual units. Institutions are encouraged to include clear and tangible evidence of collaboration and demonstrate the positive rationale in making such submissions.
3.39.8 The Panel will base its assessment of submissions on its professionally informed judgement of the quality of the research presented, paying attention to all aspects of the submission. The Panel will not use a quantitative approach. Greatest weight is given to the cited outputs in RA2 as the primary indicator of research quality. The Panel will be concerned to identify the proportions of staff whose work falls into the categories of international excellence and national excellence. The Panel will assess the quality of the research cited for each individual researcher. All forms of output will be assessed on equal terms. Whilst collaborative research may be an important indicator of an active research culture, it is not regarded as intrinsically superior.
3.39.9 Full account will also be taken of the information contained in RA3-6 as evidence of the context and culture within which the research is conducted. The textual commentary contained in RA5 and RA6 will be regarded as
particularly significant in informing the Panel's understanding of the research context. Evidence from RA3-RA6 will be used to see whether it confirms or puts in question the provisional judgement arrived at from RA2.
3.39.10 Since the Panel will base its assessment primarily on the quality of the research, researchers who do not list four outputs will not be automatically penalised.
3.39.11 The Panel will avoid penalising small departments by, for instance, imposing on them a model of research activity that can only be met, or can much more easily be met, by larger departments.
3.39.12 The Panel will give due recognition to departments which have a record of bringing on new or younger researchers and/or of successfully encouraging staff to become research active. It will accept that these individuals may have a limited number of works to cite, and that they may not have yet reached national/international standing.
3.39.13 All submissions will be assessed using the methods outlined below. All Panel members will read all submissions.To ensure consistent and equitable treatment, the cited output of each member of staff will have been examined by the Panel.
Responsibility for Ratings
3.39.14 For each submission two members of the Panel will be appointed as rapporteurs. They will seek advice from other Panel members in relation to their areas of expertise and from cross-referral and specialist advisers as appropriate. They will ensure that the work of each member of staff is examined. They will initiate discussion within the Panel and they will also take responsibility for drafting feedback to individual HEI's. All Panel members remain responsible for reading all submissions, and the final decision rests with the Panel.
3.39.15 Panel members will make their own preliminary assessments based on their reading of the submissions. They will use an assessment form to record and make their own preliminary assessment of submissions in order to give provisional ratings. The Panel will then discuss submissions and give provisional ratings and identify cases presenting difficulties. Further work will be undertaken on these cases, and all provisional ratings reviewed at a later meeting.
3.39.16 The Panel will arrive at its decisions on the basis of professional consensus. Only in very exceptional circumstances would the Panel resort to majority voting.
3.39.17 The Panel's definition of excellence is not concerned with the publisher or journal or with the nature of the topic, but with the achievement of the highest standards attainable. The Panel's professional judgement of excellence is influenced by the quality, ambition and originality of the research design and methodology, by the depth and range of scholarship displayed, by the accuracy and clarity of the work and by the impact of findings on a particular field. Originality can be in the form of using new sources and in opening up new data or material or in rethinking interpretations and approaches. Impact is measured by the extent to which informed debate in the subject has been enhanced. In forming its judgement, the Panel will take particular note of recognition in the form of peer judgement as well as where appropriate referring to the non-UK experts.
3.39.18 Nationally excellent work will be at least satisfactory in all respects of research quality specified above, and may be outstanding in some. Internationally excellent work will be outstanding in respect of almost all these qualities. RA6 offers an opportunity to provide further evidence of international scholarship and debate.
3.39.19 In judging whether or not work meets a standard of national excellence, the Panel will take account of the standing which it has or is likely to have in the relevant community of scholars. Individuals' work which fails to satisfy the criteria as set out above will be judged as being below the threshold of national excellence.
3.39.20 The Panel will, as it judges appropriate, consult five non-UK based experts to assist with its judgements about international excellence. These experts will be expected to monitor the standards being applied in relation to their own expertise. They will examine the provisional 5 and 5* ratings as well as a sample of provisional 4 ratings. Each will receive the full submission from the relevant UoA along with any relevant briefing material and a summary of the rationale for the Panel's decision. The Panel will take due notice of the experts comments, though the final responsibility for the ratings given will be that of the Panel itself.
Research Output (RA2)
3.39.21 The Panel is particularly conscious that the use of the rating scale must be relevant to the range of outputs across the sub-fields of activity. The criteria developed by the Panel take account of the diversity of output rather than impose a narrow and prescriptive characterisation of research in the field of activities covered. The Panel is not operating with a single hierarchy of sources of output.
3.39.22 In assessing the quality of research output the Panel will be paying particular attention to originality, intellectual rigour and coherence, analytical insight, imaginative scope, the importance of the issues addressed, the extent of implications for other researchers, accuracy and depth of scholarship, and evidence of familiarity with other work in the field.
3.39.23 The principal categories within which it is envisaged that works will be cited are listed below. These works may appear in any medium. The Panel stresses that materials submitted under the headings of bibliographies, creative writing, short works, sound and video recordings, teaching materials and translations must clearly demonstrate that they embody original research as defined for the RAE. This might take the form of a succinct statement of not more than 300 words to be made in the 'Other relevant details' field on RA2.
Teaching materials - including:
3.39.24 The Panel collectively will examine in detail virtually all of the items of research output cited in submissions.
3.39.25 The Panel recognizes that there are differences of emphasis in forms of output, publishing norms and typical lead-times in sub-fields of activity, with books for instance of greater importance in some sub-fields than others. The Panel's sole preoccupation is with applying its definition of research excellence spelt out above.
3.39.26 The Panel normally expects four cited outputs, submissions must use RA5 to justify fewer publications, especially when major work is in progress or promising new and younger researchers have been recruited. The Panel will not penalise submissions for recruiting young and newer researchers without a track record. The submission must detail in RA5 their potential and what is being done to encourage them.
3.39.27 The Panel attaches great importance to full and accurate information about titles, pages in books and other articles. Edited works should normally be cited only once per individual member of staff. Where co-authored, edited and co-edited works have been cited, the nature and proportional extent of the contribution by each named researcher should be elucidated in the 'Other relevant details' field on RA2.
3.39.28 In cases where the published output is in a language not represented on the Panel an abstract in English is required so that the output may be referred to the appropriate specialist. The following languages are represented on the Panel - Catalan, Finnish, French, German, Italian, Polish, Portuguese, Russian, Slovak, Spanish and Swedish.
Research Students and Research Studentships (RA3)
3.39.29 Credit will be given to submissions showing evidence of the development of a postgraduate research culture; the strong recruitment of postgraduate research students; the existence of monitoring mechanisms; and evidence of progress and of successful completions. Acknowledgement will be given to the efforts of a department to contribute in these ways to the maintenance and strengthening of the national research base in the field of study.
3.39.30 Credit will be given to departments that have been successful in gaining research council recognition for doctoral training programmes and that have received research council studentships. Credit will also be given for studentships from other sources, depending on the quality of academic peer-group review involved. Different sources of studentship funding will not be regarded as particularly significant except in so far as these indicate a breadth of research activity. Allowance will be made for those sub-areas that attract fewer research studentships because of the nature of the research and for newer members of staff.
External Research Income (RA4)
3.39.31 Credit will be given where there is evidence of the generation of research income. The focus will be on the extent and nature of the peer-review process rather than on the size of grants. Allowance will be made for those disciplines that find it more difficult to attract research funding because of the nature of the research and where newer members of staff are involved. In such cases, the lack of such funding will not be considered a negative factor.
3.39.32 Proper consideration will be given to research and development grants and consultancy income as long as the activities to which they relate are consistent with the RAE definition of research and are clearly related to research outputs. RA6 should be used for this purpose and specific cases provided.
3.39.33 It is recognised that, in some instances, research income does not flow through HEIs' accounts and thus cannot be recorded on RA4. These cases should be referred to in RA6 (General Observations).
Current and Future Research Plans: RA5
3.39.34 Particular importance will be attached by the Panel to the way in which departments address in RA5 the issues detailed below. Departments are asked to structure their RA5 submissions around these issues. Well-defined objectives and clear planning and monitoring mechanisms will be viewed as more impressive than descriptions of intent.
3.39.35 Where appropriate the submission should analyse and evaluate the outcome of the research plans put forward in the 1996 RAE.
Research Structure, Strategy and Staffing Policy
3.39.36 The Panel requires clear statements under three main headings:
Research Structure and Environment
Evidence of Esteem
3.39.37 Indicators of peer esteem which the Panel will regard highly include inter alia:
3.39.38 Such evidence should support the case for excellence of research by individuals or groups. The Panel does not attach greater weight to some kinds of evidence than others, except that the evidence must relate to research.
Individual Staff Circumstances
3.39.39 Where fewer than four outputs are cited for an individual researcher, departments are invited to state on RA6 whether there are special reasons eg:
3.39.40 In such cases, the Panel will evaluate the information in conjunction with data on age and date of appointment in RA1.
Contributions by Non Research-active Staff
3.39.41 Departments may wish to identify the contribution to research by staff not returned as research active. This information will help provide a fuller picture of the research environment, culture and infrastructure.
3.39.42 The Panel expects an explanation in cases where staff may be submitting less than four cited outputs. The submission should indicate work in progress, with clear publication plans.
3.39.43 Units of Assessment are invited to identify in RA6 any more substantial body of research work of which RA2 forms a part.
Last updated 19 April 2000