Section III: Panelsí Criteria and Working Methods
3.27 Geography, UoA 35
3.27.1 The UoA includes all aspects of research conducted within the discipline of Geography, for example: Quaternary Science; Geomorphology; Environmental Science; Remote Sensing; Quantitative Geography; Environmental Geography; Economic Geography; Development Geography; Social Geography; Cultural Geography; and Historical Geography. (Since UoA 20/21 includes the term environmental sciences in its title, for the purposes of this UoA the term environmental science covers research in this field integral to research programmes in Geography)
3.27.2 The Panel recognises the considerable range of sub-disciplines and specialisms encompassed by research in geography and is aware of the need for flexibility of approach in applying criteria during the assessment process.
3.27.3 Reflecting the nature and breadth of the discipline, the Panel's members provide a wide range of expertise. Their expertise meets the Key Subject Areas most frequently identified by the subject community in its response to the funding bodies consultation exercise on panel membership.
3.27.4 The Panel does not believe that any additional sub-panels are necessary to meet the needs of the discipline. However, the Panel will convene a sub-panel of user representatives and will act as host panel to a sub-panel on Development Studies. The Panel will appoint advisers for particular specialisms, such as Climatology and Meteorology and pedagogical research as applied to geography. It will further review the need for additional advice in the light of the submissions received.
3.27.5 All research, whether applied, strategic or basic, will be considered and no type of research will be treated as superior to any other. The Panel will only be concerned with the quality of work submitted for review, and supporting information. The Panel will follow the funding bodies definition of research and will liaise with Panels for cognate disciplines to seek comparability of judgement.
3.27.6 The Panel will base its assessment of submissions on its professional judgement of the quality of research outputs, other research activities and indicators of esteem. It will not employ a mechanistic form of assessment based on numerical weights for different kinds of research input or output. The Panel will be guided by departments' decisions on how the work of any constituent research groups and of individual scholars is described, and on the role and contribution these make to the research output of the department. The final assessment and rating will be made for the department as a whole. As in previous exercises, the Panel will be able to identify research groups which are judged as having a quality rating significantly above the overall rating for a submission.
3.27.7 Given the diverse nature of the discipline, the benefits of multi-disciplinary and inter-disciplinary research will be given full recognition. Departments are encouraged to explain their approach to such work in RA5. They could, for example, clarify the role of their members in joint or team research. Where necessary, the Panel will seek specialist advice, or cross-refer material to another Panel, in order to reach an informed view of the quality of such work. In all cases where specialist advisers are used the Panel will remain the sole arbiter of the overall grade awarded to a submission.
3.27.8 In their submissions, departments are encouraged to explain changes that have occurred over the last five years (1996-2001), including staff appointments. Where appropriate, departments may wish to explain the contributions former staff have made to their research environment and how departures have affected the strength, coherence and research culture of the department at the census date. Departments are also encouraged to make a self-assessment of the period since 1996, but not to produce a rating, including the degree to which objectives presented in 1996 have been realised. The Panel will also take into account initiatives introduced or opportunities exploited. The Panel will expect departments to outline their main objectives for the next five years, but achievements since 1996 will be regarded more highly than statements of intent.
3.27.9 Submissions will be judged in the main on the two following dimensions:
3.27.10 The Panel will attach importance to three other factors:
3.27.11 The rating for each department will be in accordance with the funding bodies standard rating scale. The Panel understands research of international excellence to be that which is judged to shape the development of the field with respect to new empirical findings, or new concepts, or new methodologies, or new applications, and is expected to steer research agendas in the foreseeable future. We understand research of national excellence to be that which contributes high quality work within existing research agendas.
Research Outputs (RA2)
3.27.12 Full consideration will be given to the existing wide range of forms of research output including: papers in refereed journals; authored books; chapters in books; research monographs; edited books; refereed conference papers; software; electronic and web-based publications; published maps; and other research-based contributions to debates on major scientific and policy issues. All research outputs cited in RA2 must satisfy the definition of research for the RAE. Single or multi-authored work are equally acceptable. The Panel would normally expect to receive four outputs per researcher. Where this expectation is not met explanations should be provided in RA6.
3.27.13 In reviewing all submissions, the Panel will attach greatest weight to research outputs which demonstrate originality; and have made, or are expected to make, a considerable contribution to the discipline, one of its sub-fields, or to an inter-disciplinary area of research enquiry. Demonstrable impacts of geographical research beyond the discipline and the academy will also be taken into account where they are evidence of the quality of the research.
Record of Research Vitality (RA5)
3.27.14 The Panel will look for evidence of a thriving research culture based upon a clearly stated policy. It is important that the department's research organisation and plans are clearly explained. The Panel will seek evidence of vitality within each area of research activity identified. Size and range are not in themselves criteria relevant to assessment. The relevance of critical mass in any one area varies across the discipline and will be considered in the context of the department's wider research culture and strategy, including links beyond the department. With regard to research structure, submissions should:
3.27.15 With regard to staffing policy, submissions should:
3.27.16 The role and contribution of post-doctoral researchers and research fellows should be assessed within the context of the department's research culture and strategy.
Postgraduate Activity (RA3)
3.27.17 The emphasis will be on output rather than input. Higher degrees awarded per FTE research active staff (Category A/A*) will be regarded as the primary indicator. Award of doctorates will be rated more highly than research-based masters degrees. If it is claimed that other masters degree programmes contribute to the research profile of the department, information about these should be given in RA5.
3.27.18 Note will be taken of the number and source of postgraduate studentships funded by external sources. Departments are invited to comment on their strategies to achieve the desired level of postgraduate activity.
External Research Funding (RA4)
3.27.19 An indicator of research quality recognised by the Panel is the award of funding in a competitive environment. The size of research contracts and grants is not necessarily the major criterion. The nature of the refereeing process and level of competition for contracts and grants will be taken into account.
3.27.20 If appropriate, the Panel welcomes information on department's strategies towards external grant and contract income, and the distribution of that income within the department.
Evidence of Esteem
3.27.21 The Panel welcomes evidence which reflects upon the impact and standing of the research of individual staff. This might include for example, appointment to national or international organisations, research councils, the editorship of international journals, awards from learned societies, prestigious fellowships, leadership of major research projects, substantial contributions to the peer review process, and other markers of research esteem.
Individual Staff Circumstances
3.27.22 The Panel also welcomes information on individual staff circumstances which have significantly affected their contribution to the submission (eg periods of sick leave, maternity and parental leave, career breaks, engagement on long-term projects).
Contributions by Non Research-active Staff
3.27.23 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
The Assessment Process
3.27.24 At this stage, the Panel anticipates a similar assessment process to 1996 consisting of approximately five meetings between June and November 2001. This pattern could be affected by the number of submissions and the amount of work cross-referred to the Panel. In outline, the first meeting will, inter alia, review the pattern of preliminary assessments, and decide upon cross-referrals and what material should be sent to specialist advisers and to the user sub-panel. At the second meeting, usually the longest, a provisional rating will be agreed for each submission. Where this is not possible, decisions will be taken on what additional advice is required. Decisions on which submissions to forward to the group of non-UK based experts will be taken as far as possible. In the third and fourth meetings, additional information from advisers, non-UK based experts, cross-referrals and panel members will be collated and, where appropriate, used to reach provisional ratings for those submissions for which no decision could be reached at the second meeting. At the final meeting, all the submissions will be reviewed again, ratings agreed, the contents of reports to be submitted to the heads of HEIs agreed, and a report for the subject community prepared.
3.27.25 On receipt, all panel members will independently review the submissions, conduct their own preliminary assessments, and provide preliminary ratings. This will be done purely as a means of gaining an overview of the range of submissions, stimulating discussion, and identifying advice to be requested of non-Panel members.
3.27.26 In reaching its initial judgements, the Panel will make use of the quantitative information made available by the funding bodies, such as the level of external funding and number of completed doctorates per FTE of research active staff, but this information will not be used in a mechanistic way. It will be treated as contextual information to inform the assessment of the quality of research outputs.
3.27.27 The research outputs from each submission will then be reviewed independently and in detail by at least two members of the Panel, selected to match as far as possible the research profile of the department. At a minimum, the Panel collectively will examine in detail 50% of the cited publications from any one department and at least two items from any one researcher. At the Panel's second meeting, the two members will lead a full Panel discussion resulting in a provisional rating for that department. If they request additional assistance in their evaluation, or if the Panel feels unable to reach a provisional rating, further Panel members or non-Panel advisers will review additional material and the case will be reconsidered fully at a later meeting.
3.27.28 Through this procedure, the Panel will employ an iterative process of discussion and rating over a number of meetings. Only when provisional ratings have been agreed for all submissions will a final rating decision be taken on any individual submission, and up to that time any Panel member can re-open the discussion on any submission. The Panel will base its decisions upon the material brought to its attention in the submissions and the requirements laid down by the RAE rating scale. The Panel may refer to research plans included in the 1996 RAE. The overriding objective of the Panel will be to reach consensus on the ratings awarded. Where this cannot be achieved the rating assessment decision will be based on a simple majority vote.
3.27.29 Joint submissions from more than one HEI will be treated as a single submission and the Panel will, in such cases, seek evidence of intellectual coherence and a history of collaboration.
3.27.30 Where the Panel feels that its expertise does not cover the field(s) being assessed or where there is overlap with another unit of assessment, then either supplementary advice from a specialist or, where appropriate, the view of another panel (cross-referral), will be sought. In all cases, the whole of the submission will be forwarded, even where assistance is only requested on part of it, making the full research context available to the adviser. Cross-referrals will involve at least one meeting between a member of each of the relevant panels. The Geography Panel will retain sole responsibility for the award of the rating for the whole department concerned.
3.27.31 In principle, the Panel endorses the participation of user members. However, the identification of appropriate user members within such a broad discipline has proved difficult, and the Panel believes that no one user member could represent adequately all areas of applied research in geography. The Panel will appoint a sub-panel of users with varied geographical expertise who will be asked to advise on the quality and impact of applied research contained in the submissions.
3.27.32 A small group of non-UK based experts will be consulted by the Panel to confirm standards of international excellence in research in geography. The experts will be sent a sample of submissions containing areas of research provisionally rated 4, 5 and 5*. As in the case of specialist advisers, they will be asked to comment, in the context of the entire submission, on the quality of research in those areas of the submission provisionally rated 4, 5 and 5* which fall into their area of expertise.
3.27.33 The experts will not be asked to adjudicate provisional ratings for whole, or even part, submissions. Ratings are the responsibility of the Panel. However, should the burden of evidence from the non-UK based experts indicate that the Panel's initial judgement of what constitutes international excellence is misplaced, then all submissions provisionally rated 4, 5 or 5* will be reviewed again by the Panel.
3.27.34 The funding bodies have agreed to establish a separate sub-panel for Development Studies. They recognise Development Studies as a growing area of multi-disciplinary research, lying very largely within social sciences, with strong links to users.
3.27.35 The Geography Panel has been asked to act as the host panel. As the host panel, it will be responsible for agreeing the ratings awarded by the sub-panel. The sub-panel will be chaired by the Chair of the Geography Panel.
3.27.36 The remainder of the sub-panel's membership, and its specific remit and working methods, will be published separately. It is agreed, however, that submissions made to the Geography Panel will not be referred to the Development Studies sub-panel unless this is specifically requested by the institution concerned.
Last updated 17 April 2000