Section III: Panelsí Criteria and Working Methods
3.41 English Language and Literature, UoA 50
3.41.1 The Unit of Assessment includes Old and Middle English Language and Literature; English Linguistic Studies, including Applied Linguistics; Old Norse/Icelandic; Renaissance Literature; 17th and 18th Century Literature; Romantic Literature; Victorian Literature; 20th Century Literature; American Literature; Colonial and Postcolonial Literature; Comparative Literature; Women's Writing; Creative Writing; Children's Literature; Critical and Cultural Theory and History; Gender and Gay Studies; Bibliography, Textual Criticism and History of the Book; Irish Literature in English; Scottish Literature in English and Scots; and Welsh Literature in English.
3.41.2 The Panel is aware that in some departments significant work will be done in areas such as the following: Theatre Studies; Film Studies; Translation Studies; Popular Culture; and Celtic. The Panel may refer work in these areas to a relevant Panel or specialist adviser.
3.41.3 The Panel recognises that English includes a very broad range of approaches and is by its nature frequently interdisciplinary, and it will take a broad view of what constitutes English. Where appropriate to the assessment process it will consult with other relevant Panels when submissions span the boundary between two or more Units of Assessment.
3.41.4 The Panel will not convene any Sub-Panels.
3.41.5 The constitution of the Panel is such that there may be no need for specialist advice but the Panel will be vigilant in identifying and seeking such advice if required.
3.41.6 The Panel will consult with other Panels as appropriate to ensure an inclusive and equitable assessment of any interdisciplinary research submissions.
3.41.7 The Panel will treat joint submissions in the same way as submissions from individual units. Institutions should give clear and tangible evidence of collaboration, along with the rationale for making such submissions.
Cross-referral of Submissions from Other Panels
3.41.8 These will be referred to an appropriate Panel member or members. Assessment of cross-referred work will include discussion between at least one member of each Panel involved.
Treatment of Evidence
3.41.9 The Panel collectively will examine in detail a minimum of one item of research output cited for each researcher, and will base its assessment of submissions on its professionally informed judgements of the quality, not quantity, of outputs cited in RA2. Evidence that research outputs have already been refereed for publication by peers may be used as a measure of quality but will be secondary to the Panel members' judgements of the intrinsic research merits of the items of research. The absence of such refereeing will not, in itself, be taken to imply lower quality.
3.41.10 Creative writing will be assessed in so far as it represents "the invention and generation of ideas, images, performances where these lead to new or substantially improved insights" (as defined in the Guidance on Submissions, RAE 2/99) and where it is in a publicly accessible form. Work in performance arts must be accessible, for instance, in audio or video form. Where such material is submitted it should be accompanied by a brief statement highlighting its research content and significance in the 'Other relevant details' field on RA2. Published materials relating to teaching are admissible if they can be shown to embody research. Publication will include conventional and electronic or digitised research outputs.
3.41.11 The assessment of the quality of cited research outputs (RA2) will be set in the context of quantitative information (RA1, RA3a-b and RA4), research structure and future plans (RA5), and general observations (RA6). When assessing the quality of submissions, the following criteria will be used as appropriate:
3.41.12 Where appropriate, the numbers of research degrees awarded and the number of nationally competitive studentships will be regarded as indicators of quality, with the number of higher degrees gained, and doctorates in particular, rated more highly than studentships gained.
Research Output (RA2)
3.41.13 The principal categories in which it is envisaged that works will be cited are listed below. No ranking or weighting should be inferred from the order in which they are listed. Professional expectations of quality will guide the Panel's judgements of the research contribution of individual items.
Note: Where an individual has edited a book, or a special issue of a journal, and has also contributed one or more items in the same publication, these may be submitted separately or together. Dictionary entries or encyclopaedia articles may, likewise, be entered separately or as groups.
3.41.14 The Panel will accept output in other media such as electronic, digitised, visual, audio etc. as published material and will judge it by the same criteria as sources in print.
3.41.15 The size of departments will not be regarded as an indicator of research quality, nor will breadth of research expertise across a wide range of topics be regarded as necessarily superior or inferior to a concentration of research within a narrower focus.
Research Students and Research Studentships (RA3)
3.41.16 The Panel will give credit to submissions that demonstrate the existence of successful research postgraduate activity in both full and part-time modes. It recognises, however, that the size of departments and demands for the efficient allocation of resources may make it inappropriate for some institutions to recruit and develop postgraduate students. Where appropriate, the Panel will take into account the proportion of staff engaged in supervision and overall numbers of research students. The Panel is aware that joint supervision of research students may affect these numbers. While, where appropriate, the Panel will regard success in winning AHRB or other peer-reviewed research studentships as a measure of peer esteem, those awarded by other external bodies will not necessarily be considered less valuable. The Panel is aware that the structure of the majority of externally awarded research studentships in Scotland does not follow the AHRB's model, and will not regard these studentships any less favourably than those awarded by the AHRB. Studentships funded by an institution itself will be considered an indication of the institution's provision for research infrastructure.
External Research Income (RA4)
3.41.17 The Panel recognises that research income is not necessarily an indicator or pre-condition of high quality research, but will give due credit where there is evidence of the generation of research income and subsequent output. The Panel recognises that different fields within the subject have different potentials for attracting external research funds.
3.41.18 The Panel would expect to see the following information in RA5. This is not a checklist, and institutions may depart from it as necessary to give a coherent account of their research activity.
Research Structure and Environment
3.41.19 Nature and quality of the research infrastructure, including, where appropriate, facilities for research students.
3.41.20 Explanation of the mechanisms and practices for promoting research and sustaining and developing an active and vital research culture.
3.41.21 Arrangements which are in place for supporting interdisciplinary and/or collaborative research.
3.41.22 Institutions should indicate, where relevant, how research bears on the teaching process.
3.41.23 Definition of research groups, where such groups exist, who belongs to them (referring to RA1), their prime activities and their main achievements. It is recognised that some scholars may be members of more than one research group whereas others may not be in a group at all. (Institutions may choose to identify staff with particular research groups using the fields on RA1.)
3.41.24 Institutions may wish to comment on research output within the census period which is not included amongst their researchers' four cited items in RA2.
3.41.25 Indication of the contributions of staff to scholarly reference works not returned in RA2, for instance, the New DNB and CBEL3.
3.41.26 Where relevant, institutions should list other Units of Assessment to which related work has been submitted and details of any difficulties of fit between departmental structure and the Unit of Assessment framework.
3.41.27 Information on relationships with industry and commerce or other research users and, where appropriate, on responses to public policy initiatives and objectives.
3.41.28 Description of the arrangements for the development of the research work of the staff. The Panel notes that the quality of a particular work cited may compensate for the absence of other works. The Panel may conclude that a lack of research depth in a department is indicated when a significant proportion of established active researchers cite fewer than four works each. The Panel will normally regard the submission of work by supervised students amongst the outputs of category A, A* or C staff as indicative of a lack of research depth. Institutions which choose to submit such work may wish to comment on their reasons for doing so in the 'Other relevant details' field on RA2.
3.41.29 The Panel will not penalise institutions which have appointed members of staff new to the profession without an established research track record, especially when there is evidence of an effective policy of developing research. The Panel would hope to see, where appropriate, a description of any arrangements for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research structure.
3.41.30 Where appropriate, an explanation of the role and contribution of category A* staff who have joined the department.
3.41.31 Where appropriate, comment on how the departure of staff in categories A*, B and D has affected the strength, coherence and research culture of the department at the census date.
3.41.32 Comment on the role of category C members of staff who are research-active within the department, and a description of the provision made to support their research.
3.41.33 Statement about the main objectives and activities in research over the next five years and ongoing research work that is not producing immediate visible outcomes. The Panel recognises that there are considerable differences in this context and will not penalise small departments by, for instance, imposing on them a model of research activity that can be met only, or much more easily, by larger departments.
3.41.34 The research organisation and future plans stated in this section will act as a framework for assessing information in the rest of the submission. Wherever possible, an assessment of how far the research plans put forward in the 1996 RAE have been achieved should be included. The Panel is aware that it may not be possible to carry out a direct assessment where, for example, new departments have been created or existing departments have merged.
3.41.35 An element of critical self-assessment should be included within RA5. This should include comments on plans and developments listed in 1996 submissions and significant structural or staff changes since 1996.
3.41.36 The Panel would expect to see the following information in RA6.
Evidence of Esteem
3.41.37 A list of indicators of peer esteem which relate to the staff submitted (for example, awards and honours, membership of relevant professional bodies, research fellowships awarded by national and charitable bodies, visiting professorships and fellowships, research grants awarded by the AHRB, editorial activities not cited in RA2, contributions to the academic and public good, invited lectures, conference organisation and participation of special note, and relevant other items). The Panel recognises that different research-related activities are appropriate to staff at different stages in their careers.
Individual Staff Circumstances
3.41.38 Note any circumstances which have significantly affected an individual's contribution to the submission (for example, periods of sick leave, maternity leave, career breaks, engagements on long-term projects such as the Teaching and Learning Technology Programme, etc.). Institutions may also note any recent staff PhDs gained.
Contributions by Non Research-active Staff
3.41.39 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.41.40 Institutions should draw attention to any grants or income that are not administered through an institution's own accounts and hence are not eligible for inclusion in form RA4.
3.41.41 All Panel members will read all the submissions. Members of the Panel will already have read a significant proportion of the cited works. The Panel collectively will examine in detail a minimum of one item of research output cited for each researcher and will review sufficient of the cited outputs of each researcher to arrive at secure judgements.
3.41.42 The Panel will begin by independently reading the submissions. The Panel will then be divided into pairs, and each pair will be allotted a group of institutions to start the process of examination and review set out above. Some items requiring advice from other Panels or the use of specialist advisers are likely to be identified in the course of this first stage.
3.41.43 Thereafter Panel members will arrive at preliminary gradings for all submitting institutions, and the pairs of Panel members will report to the Panel on their assigned institutions. After discussion, the Panel as a whole will arrive at a preliminary and provisional set of assessments. The chairman will then ask different pairs of Panel members, after further reading and review, to report back on clusters of units grouped round particular gradings or borderlines.
3.41.44 There will subsequently be further meetings to allow specific queries about particular submissions to be followed up; to allow Panel members to undertake further reading of what have been identified as key works; to seek the views of specialist advisers and other Panels; to consider the views of the non-UK based experts; and to allow further reflection before the final and definitive decisions on the gradings are taken. Decisions will be taken by consensus. If a decision cannot be reached by consensus, the Panel will vote. In the event of a split vote, the chairman will have an additional casting vote.
3.41.45 In interpreting the rating scale the Panel will use its professional judgement. Panel members have been chosen for their standing in, and knowledge of, the subject. The Panel's definition of international excellence is not concerned with the publisher or journal or with the nature of the topic but with the achievement of the highest standards attainable. In interpreting the rating scale, the Panel will judge submissions against the best work being done in the relevant fields internationally. In judging whether work meets the standard of "national excellence", the Panel will judge submissions against the best work being done in the relevant fields nationally. Factors which may affect the Panel's judgement include the influence, impact and recognition of submitted work in the UK and among international scholarly communities.
3.41.46 The Panel does not consider it appropriate to seek user input into the assessment of submissions in English studies.
Non-UK Based Experts
3.41.47 The Panel will make use of not more than five non-UK based experts from around the world as appropriate. These experts will each cover different areas of the subject. The non-UK based experts will be asked to comment on the appropriateness of all proposed 5* or 5 ratings, along with a small selection of 4 ratings. Each will receive the full submission from each institution, along with any relevant briefing material. The Panel will take due notice of the experts' comments, though the final responsibility for the ratings given will be that of the Panel itself.
Last updated 19 April 2000