RAE2001 logo

Contents

Another UoA

Section III: Panelsí Criteria and Working Methods


3.39 European Studies, UoA 48

UoA Descriptor

3.39.1 The UoA embraces all of Europe including Eastern and Western Europe, Scandinavia and the Baltic region, southern Europe, and Russia. It covers the modern languages, literatures and cultures, including film and media studies; language teaching and learning, including the integration of new technologies; translation; linguistics; modern economic, social and political history; philosophy; religions; gender studies; also the business environment, economics, geography, international relations, politics, public policies, and sociology; as well as research into the teaching of European Studies. The UoA also deals with European integration and the external relations of Europe. Submissions may therefore legitimately include areas of research which are eligible for submission to other UoAs.

UoA Boundaries

3.39.2 The Panel will consider submissions from any department that covers a range of European languages and associated literatures, linguistics and cultures and/or which studies modern Europe using a variety of social scientific, historical and literary methods.

3.39.3 Where a submission spans the boundary with one or more other UoAs, and the Panel decides that it lacks the appropriate expertise, the whole submission will be cross-referred to other relevant Panels with a request to advise on the relevant work in the context of the submission. Cross-referral will involve a meeting with other Panels at which at least one member of this Panel will be present. Responsibility for awarding the rating remains with the European Studies Panel. Where other Panels feel that they too lack the appropriate expertise, the European Studies Panel will establish ad hoc specialist advisory arrangements. It will appoint specialist advisers in for example Irish Studies, Scottish Studies and Catalan. The Panel reserves the right where appropriate to cross-refer to other panels submissions that focus entirely on single countries or languages.

Interdisciplinary Research

3.39.4 The Panel represents a wide range of disciplines in recognition of the multidisciplinary character of the Unit of Assessment. Though its composition should enable it to deal with most of the interdisciplinary work that is submitted, cross-referral and specialist advice will have to be used in the manner outlined above.

3.39.5 Departments are urged to identify in their submissions aspects of their work that do not fit well within the framework of the European Studies Panel and to request cross-referral to one or more other panels. The Panel will always act on such requests. At the same time it will make cross-referrals at its own discretion where it judges that the assessment process would be improved.

3.39.6 The Panel is not expecting to use a sub-panel. It will appoint for example, specialist advisers in Catalan, Irish Studies and Scottish Studies.

Joint Submissions

3.39.7 The Panel will treat joint submissions similarly to submissions from individual units. Institutions are encouraged to include clear and tangible evidence of collaboration and demonstrate the positive rationale in making such submissions.

Working Methods

3.39.8 The Panel will base its assessment of submissions on its professionally informed judgement of the quality of the research presented, paying attention to all aspects of the submission. The Panel will not use a quantitative approach. Greatest weight is given to the cited outputs in RA2 as the primary indicator of research quality. The Panel will be concerned to identify the proportions of staff whose work falls into the categories of international excellence and national excellence. The Panel will assess the quality of the research cited for each individual researcher. All forms of output will be assessed on equal terms. Whilst collaborative research may be an important indicator of an active research culture, it is not regarded as intrinsically superior.

3.39.9 Full account will also be taken of the information contained in RA3-6 as evidence of the context and culture within which the research is conducted. The textual commentary contained in RA5 and RA6 will be regarded as

particularly significant in informing the Panel's understanding of the research context. Evidence from RA3-RA6 will be used to see whether it confirms or puts in question the provisional judgement arrived at from RA2.

3.39.10 Since the Panel will base its assessment primarily on the quality of the research, researchers who do not list four outputs will not be automatically penalised.

3.39.11 The Panel will avoid penalising small departments by, for instance, imposing on them a model of research activity that can only be met, or can much more easily be met, by larger departments.

3.39.12 The Panel will give due recognition to departments which have a record of bringing on new or younger researchers and/or of successfully encouraging staff to become research active. It will accept that these individuals may have a limited number of works to cite, and that they may not have yet reached national/international standing.

3.39.13 All submissions will be assessed using the methods outlined below. All Panel members will read all submissions.To ensure consistent and equitable treatment, the cited output of each member of staff will have been examined by the Panel.

Responsibility for Ratings

3.39.14 For each submission two members of the Panel will be appointed as rapporteurs. They will seek advice from other Panel members in relation to their areas of expertise and from cross-referral and specialist advisers as appropriate. They will ensure that the work of each member of staff is examined. They will initiate discussion within the Panel and they will also take responsibility for drafting feedback to individual HEI's. All Panel members remain responsible for reading all submissions, and the final decision rests with the Panel.

3.39.15 Panel members will make their own preliminary assessments based on their reading of the submissions. They will use an assessment form to record and make their own preliminary assessment of submissions in order to give provisional ratings. The Panel will then discuss submissions and give provisional ratings and identify cases presenting difficulties. Further work will be undertaken on these cases, and all provisional ratings reviewed at a later meeting.

Decisions

3.39.16 The Panel will arrive at its decisions on the basis of professional consensus. Only in very exceptional circumstances would the Panel resort to majority voting.

Research Excellence

3.39.17 The Panel's definition of excellence is not concerned with the publisher or journal or with the nature of the topic, but with the achievement of the highest standards attainable. The Panel's professional judgement of excellence is influenced by the quality, ambition and originality of the research design and methodology, by the depth and range of scholarship displayed, by the accuracy and clarity of the work and by the impact of findings on a particular field. Originality can be in the form of using new sources and in opening up new data or material or in rethinking interpretations and approaches. Impact is measured by the extent to which informed debate in the subject has been enhanced. In forming its judgement, the Panel will take particular note of recognition in the form of peer judgement as well as where appropriate referring to the non-UK experts.

3.39.18 Nationally excellent work will be at least satisfactory in all respects of research quality specified above, and may be outstanding in some. Internationally excellent work will be outstanding in respect of almost all these qualities. RA6 offers an opportunity to provide further evidence of international scholarship and debate.

3.39.19 In judging whether or not work meets a standard of national excellence, the Panel will take account of the standing which it has or is likely to have in the relevant community of scholars. Individuals' work which fails to satisfy the criteria as set out above will be judged as being below the threshold of national excellence.

Non-UK Experts

3.39.20 The Panel will, as it judges appropriate, consult five non-UK based experts to assist with its judgements about international excellence. These experts will be expected to monitor the standards being applied in relation to their own expertise. They will examine the provisional 5 and 5* ratings as well as a sample of provisional 4 ratings. Each will receive the full submission from the relevant UoA along with any relevant briefing material and a summary of the rationale for the Panel's decision. The Panel will take due notice of the experts comments, though the final responsibility for the ratings given will be that of the Panel itself.

Research Output (RA2)

3.39.21 The Panel is particularly conscious that the use of the rating scale must be relevant to the range of outputs across the sub-fields of activity. The criteria developed by the Panel take account of the diversity of output rather than impose a narrow and prescriptive characterisation of research in the field of activities covered. The Panel is not operating with a single hierarchy of sources of output.

3.39.22 In assessing the quality of research output the Panel will be paying particular attention to originality, intellectual rigour and coherence, analytical insight, imaginative scope, the importance of the issues addressed, the extent of implications for other researchers, accuracy and depth of scholarship, and evidence of familiarity with other work in the field.

3.39.23 The principal categories within which it is envisaged that works will be cited are listed below. These works may appear in any medium. The Panel stresses that materials submitted under the headings of bibliographies, creative writing, short works, sound and video recordings, teaching materials and translations must clearly demonstrate that they embody original research as defined for the RAE. This might take the form of a succinct statement of not more than 300 words to be made in the 'Other relevant details' field on RA2.

  1. Academic journal articles
  2. Bibliographies
  3. Books - including:
    1. Co-authored works
    2. Edited books or special issues of journals with substantial research input on the part of the editor
    3. Scholarly editions
    4. Single-authored works
    5. Chapters in books - including:
    6. Contributions to conference proceedings
    7. Contributions to edited volumes
    8. Contributions to Festschriften
  4. Creative writing, with a clear research component
  5. Short works - including:
    1. Dictionary entries
    2. Encyclopaedia entries
    3. États présents
    4. Pamphlets
    5. Published lectures
    6. Research reports and consultancy based on research for public and private bodies
    7. Reviews
  6. Sound and video recordings, both sets for archival collections and published CD's and cassettes

  7. Teaching materials - including:
    1. Computer Assisted Language Learning (CALL) material
    2. Journals on the Web including recorded performances
    3. Language materials
    4. Readers
    5. Resource books
    6. Student guides
    7. Textbooks
  8. Translations

3.39.24 The Panel collectively will examine in detail virtually all of the items of research output cited in submissions.

3.39.25 The Panel recognizes that there are differences of emphasis in forms of output, publishing norms and typical lead-times in sub-fields of activity, with books for instance of greater importance in some sub-fields than others. The Panel's sole preoccupation is with applying its definition of research excellence spelt out above.

3.39.26 The Panel normally expects four cited outputs, submissions must use RA5 to justify fewer publications, especially when major work is in progress or promising new and younger researchers have been recruited. The Panel will not penalise submissions for recruiting young and newer researchers without a track record. The submission must detail in RA5 their potential and what is being done to encourage them.

3.39.27 The Panel attaches great importance to full and accurate information about titles, pages in books and other articles. Edited works should normally be cited only once per individual member of staff. Where co-authored, edited and co-edited works have been cited, the nature and proportional extent of the contribution by each named researcher should be elucidated in the 'Other relevant details' field on RA2.

3.39.28 In cases where the published output is in a language not represented on the Panel an abstract in English is required so that the output may be referred to the appropriate specialist. The following languages are represented on the Panel - Catalan, Finnish, French, German, Italian, Polish, Portuguese, Russian, Slovak, Spanish and Swedish.

Research Students and Research Studentships (RA3)

3.39.29 Credit will be given to submissions showing evidence of the development of a postgraduate research culture; the strong recruitment of postgraduate research students; the existence of monitoring mechanisms; and evidence of progress and of successful completions. Acknowledgement will be given to the efforts of a department to contribute in these ways to the maintenance and strengthening of the national research base in the field of study.

3.39.30 Credit will be given to departments that have been successful in gaining research council recognition for doctoral training programmes and that have received research council studentships. Credit will also be given for studentships from other sources, depending on the quality of academic peer-group review involved. Different sources of studentship funding will not be regarded as particularly significant except in so far as these indicate a breadth of research activity. Allowance will be made for those sub-areas that attract fewer research studentships because of the nature of the research and for newer members of staff.

External Research Income (RA4)

3.39.31 Credit will be given where there is evidence of the generation of research income. The focus will be on the extent and nature of the peer-review process rather than on the size of grants. Allowance will be made for those disciplines that find it more difficult to attract research funding because of the nature of the research and where newer members of staff are involved. In such cases, the lack of such funding will not be considered a negative factor.

3.39.32 Proper consideration will be given to research and development grants and consultancy income as long as the activities to which they relate are consistent with the RAE definition of research and are clearly related to research outputs. RA6 should be used for this purpose and specific cases provided.

3.39.33 It is recognised that, in some instances, research income does not flow through HEIs' accounts and thus cannot be recorded on RA4. These cases should be referred to in RA6 (General Observations).

Current and Future Research Plans: RA5

3.39.34 Particular importance will be attached by the Panel to the way in which departments address in RA5 the issues detailed below. Departments are asked to structure their RA5 submissions around these issues. Well-defined objectives and clear planning and monitoring mechanisms will be viewed as more impressive than descriptions of intent.

3.39.35 Where appropriate the submission should analyse and evaluate the outcome of the research plans put forward in the 1996 RAE.

Research Structure, Strategy and Staffing Policy

3.39.36 The Panel requires clear statements under three main headings:

Research Structure and Environment

  1. Explain

    the mechanisms and practices for initiating and promoting research and sustaining and developing an active research culture.
  2. Describe any

    research groups, who belongs to them, (referring to RA1), their prime activities, how they operate and their main achievements
  3. Describe the linkage between research income contained in RA4 and individual research projects and outcomes.
  4. Identify the role and relative contribution of individuals in collaborative research projects.
  5. Describe

    the nature and quality of the research infrastructure, including facilities for research students.
  6. Describe the

    arrangements for initiating and supporting interdisciplinary or collaborative research.
  7. List other UoAs to which

    related work has been submitted and detail any difficulties of fit between departmental structure and the UoA Framework.

Research Strategy

  1. Provide a clear statement about the aims, main objectives and activities in research over the next five years and about the problems that have to be overcome. The Panel will look for evidence of a 'rising or falling' curve in research activity and will expect an explanation to accompany any reference to research work which is not yet producing visible outcomes. Where appropriate the submission should analyse and evaluate the outcome of the research plans put forward in the 1996 RAE.
  2. The Panel requires that departments provide an honest, self-critical and constructive assessment of their performance.
  3. Describe the activities and results associated with the numerical data on research students and research income in RA3 and RA4.
  4. Comment on the progress of research plans submitted in the 1996 RAE.

Staffing Policy

  1. Outline policies for developing personal research by younger and newer researchers and for integrating them into a wider, supportive research culture
  2. When a UoA is only submitting a small number of staff and/or where staff entered are from different structures it is incumbent on it to demonstrate clearly an active research culture amongst the staff returned.
  3. Submissions should spell out the role and contribution of staff recruited in the year before the census date.
  4. The Panel wishes to be in a position to evaluate the contribution made by category C staff and does not regard them as carrying less weight. Evidence about their role should be given.
  5. Submissions should reflect on how the departure of staff in categories B and D has affected the strength, coherence and research culture of the UoA at the census date.

RA6

Evidence of Esteem

3.39.37 Indicators of peer esteem which the Panel will regard highly include inter alia:

  1. Editorship of journals and book series and participation in editorial boards
  2. Election to learned societies and involvement in their work and that of research councils
  3. Membership of relevant national and international committees or organizations
  4. Invited papers at international conferences, especially keynote papers
  5. Organization of international conferences
  6. Invited lectures at other HEIs or invited international lecture tours
  7. External funding
  8. Visiting posts
  1. Number of PhD external examiner appointments
  2. National and international honours
  3. Acting as readers for publishers and journals
  4. Research-based consultancies and evidence of successful collaboration with users of research
  5. Award of prestigious research fellowships or other honours, prizes or awards

3.39.38 Such evidence should support the case for excellence of research by individuals or groups. The Panel does not attach greater weight to some kinds of evidence than others, except that the evidence must relate to research.

Individual Staff Circumstances

3.39.39 Where fewer than four outputs are cited for an individual researcher, departments are invited to state on RA6 whether there are special reasons eg:

  1. New/young researcher: in a first full-time academic post, not cited in any previous RAE. In such cases, evidence of research performance will be put forward.
  2. Career change: recruited from outside the academic world, eg industry or the civil service, and entered for the RAE for the first time.
  3. Change of focus: employed by the department for some years, but previously in a predominantly administrative or teaching role, and entered for the RAE for the first time.
  4. Long-term project: established researcher participating in time-limited projects funded by the HE funding bodies, eg TLTP, with a consequently reduced research output. Specific information is required on these projects.
  5. Career breaks, leave in connection with family responsibilities, periods of sick leave.
  6. Part-time staff.

3.39.40 In such cases, the Panel will evaluate the information in conjunction with data on age and date of appointment in RA1.

Contributions by Non Research-active Staff

3.39.41 Departments may wish to identify the contribution to research by staff not returned as research active. This information will help provide a fuller picture of the research environment, culture and infrastructure.

Additional Information

3.39.42 The Panel expects an explanation in cases where staff may be submitting less than four cited outputs. The submission should indicate work in progress, with clear publication plans.

3.39.43 Units of Assessment are invited to identify in RA6 any more substantial body of research work of which RA2 forms a part.


Last updated 19 April 2000

[ Home | About the RAE2001 | Panels | Guidance for panel members | Guidance for institutions | Data collection | Publications | Contacts ]