Section III: Panelsí Criteria and Working Methods
3.31 Politics and International Studies, UoA 39
3.31.1 The UoA includes: comparative, area, national and sub-national politics; public administration and policy studies, including science and technology policy; political behaviour and political sociology, including gender; political theory and philosophy, including history of political thought; international relations, including strategic, war and peace studies, international political economy and foreign policy analysis; methods in political studies, and HE pedagogic research.
3.31.2 The Panel recognises that some of the work submitted might cross disciplines. Where work clearly falls within the remit of another panel that work will be referred to the relevant panel. Principally, the identified cognate disciplines include: public management; social policy; political economy; political philosophy; history and law.
3.31.3 The Panel will base its assessment of submissions on the basis of the three following measures with the most weight being given to the first:
3.31.4 Publication and other output data in submissions will be assessed on an individual by individual basis, but departments will be invited to specify research groups if they so choose. Absence of research groups will not be taken to imply absence of research culture.
3.31.5 The mission, degree of specialisation, percentage of total staff returned as research active, or the size of a unit will not be grounds for allotting any particular ranking to a unit.
3.31.6 The Panel would normally expect researchers to cite four publications that reflect their best quality work over the period covered by the RAE, but recognises there may be less than four in certain circumstances, such as:
Research Output (RA2)
3.31.7 The Panel will read sufficient work to form a reliable judgement of each submission. This will mean that a higher proportion of submitted outputs will be examined in detail where the submission is on the borderline between particular grades. The Panel collectively will examine in detail at least 25% of the submitted output of any department. Where the Panel judges that it needs further information to make an informed assessment it will seek the advice of expert, specialist advisers or other panels.
3.31.8 The Panel will seek, above all, to consider the intrinsic research merits of an item. Whether the item has been peer-reviewed or not, the Panel will always seek to judge work by the standards of research that are employed in a peer-review process. In the absence of contrary evidence the Panel will ascribe equal merit to all authors for work that is jointly authored. Where a significant number of jointly authored works from researchers in the same department are cited the Panel will be looking for evidence of either research synergy or [that these have been] necessitated by the work involved.
3.31.9 Bearing in mind, that some research groups appropriately use some forms of dissemination over others, the Panel will generally observe the following rules:
3.31.10 Because they generally require greater effort the Panel will tend to place great weight on research monographs, depending, of course, on their quality.
3.31.11 Articles published in journals which have high editorial standards, will be taken as a strong indicator of research excellence. Review articles that make a major contribution to the field will be treated on a par with journal articles. The Panel recognises that it is appropriate for some types of research to be published in less prominent or more specialist journals and will take this factor into account.
Chapters in Edited Books
3.31.12 The Panel will judge chapters in edited books by the same standards of research excellence as articles in journals.
3.31.13 Whilst the Panel will normally attribute greater significance to an authored research monograph, it is recognised that, on occasion, an edited book may involve a high degree of research effort. Where the editor has also written chapter(s) of the book, he or she is free to submit both chapter and edited book for assessment, although it may be more beneficial to submit a separate item in place of one of these.
3.31.14 Other works may include textbooks, CD-roms, web-sites, conference papers and proceedings, annotated bibliographies, information guides, research guides and commentaries on, or introductions to, translated versions of existing works, if these can be shown to contain significant research effort. Book reviews (as distinct from review articles) and translations without research input will not regarded as demonstrating research activity.
3.31.15 The Panel wishes to make clear its attitude to the following types of evidence of research quality:
3.31.16 Whilst the Panel recognises that, in many cases, it is possible to distinguish clearly textbooks from research monographs, it also notes that textbooks frequently report the writer's research or provide an innovative synthesis of existing scholarship. The Panel is also aware that publishers often market a research monograph as a textbook to increase sales. The Panel will strive to judge the research merits of such works.
Teaching Materials and Subject-Related Pedagogy
3.31.17 These will be treated in the same way as textbooks. Higher education pedagogic research will be judged by the standard of research.
3.31.18 Full credit will be given to inter-disciplinary works and, where necessary, such items will be referred to outside assessors or other relevant panels for consideration.
Guest Editorship of Journals
3.31.19 Guest editorship of journals will be regarded in the same way as the editorship of books.
Applied and Commissioned Research
3.31.20 The Panel will accord the same significance to applied research and commissioned research as it will to pure research works and will judge their quality according to the same standards of rigour with due attention to their distinguishing characteristics.
3.31.21 It should be noted that collections of essays or papers originally published before 1996, and reprinted in unmodified form in the period 1996-2000, will not be regarded as displaying new research. Unmodified reprinting or reproduction of other work published before 1996 will be treated similarly.
Post-Graduate Research Students (RA3)
3.31.22 The panel will consider the numbers of post-graduate research students in relation to the number of research active staff, and, where appropriate, the number of higher research degrees per research active staff successfully completed in the review period. The Panel will also consider evidence supplied in RA5 concerning the quality of research post-graduate work such as related publications.
External Research Income (RA4)
3.31.23 The Panel is aware that the likelihood of funding may vary considerably across sub-fields within a unit of assessment. The Panel will place greater weight upon the output from grants rather than on their value.
Research Structure and Environment
3.31.24 In making its judgement on research structures and culture, the Panel will examine the following elements:
3.31.25 The research activities that contribute to and maintain the unit's research culture and ethos.
3.31.26 The quality of the research infrastructure.
3.31.27 The relationship with any work carried out in the unit that is submitted to other panels.
3.31.28 The Panel recognises the value of both individual and collaborative research activity. It recognises that collaborative research can take place between individuals working inside and outside of a department. In assessing the contribution of groups to a research culture it will consider well-defined research programmes or other collective research activity at group level that contributes to the overall research culture and ethos of the department.
3.31.29 Recent entrants to the profession will be treated according to the expectations that are appropriate to someone at that stage in his or her career and in the light of the department's research culture and research support as disclosed in forms RA5 and RA6. The Panel will be looking to the arrangements for supporting and developing the research of newly appointed staff.
3.31.30 The research record of staff in categories B and D will be considered as evidence of a research culture. The Panel will look at the research and recruitment strategies of departments in assessing the contribution of recently appointed and transferred staff.
3.31.31 The Panel will examine the research strategies of the unit. The Panel will especially consider the extent to which the goals set out in the 1996 plans have been achieved or successfully adapted over the period of the current research assessment exercise in the form of a self assessment by the department.
Taught Masters' Programmes
3.31.32 Taught masters' programmes may provide indirect evidence of research culture. Information relating to these may be included in RA5.
Academic Public Goods
3.31.33 The Panel will ensure that contributions to academic public goods and esteem indicators (e.g. fellowships of learned societies, editorships of journals, editorial boards of journals, convenors of specialist research groups, senior office holders in professional bodies, refereeing of journals) awarded to staff in departments are recognised.
3.31.34 Contributions to public service (e.g. NGOs, Royal Commissions), wealth creation and the quality of life will be taken into account and may be included in this section.
Staff PhD Theses
3.31.35 Departments are invited to note staff PhD theses.
Individual Staff Circumstances
3.31.36 Departments may wish to mention those staff whose research has been seriously affected during the assessment period by unusual or particular circumstances (eg periods of sick leave, career breaks, maternity leave, engagement on long term projects).
Contributions by Non Research-active Staff
3.31.37 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.31.38 The Panel will make all its decisions according to the principle of collective responsibility under which the panel as a whole takes responsibility for any judgement that is made.
3.31.39 The Panel intends to calibrate its marking using a sample of submissions before the full exercise. Members will then read through all submissions and will award indicative gradings, in strict isolation from each other. These gradings will be circulated amongst panel members for the first assessment meeting. Items that require the views of other panels, will be identified. Two rapporteurs will be assigned to each submission to lead the discussion.
3.31.40 The Panel will read, or otherwise examine in detail, selectively from the publications and other output cited.
3.31.41 In order to avoid any one panel member having excessive impact on the grading of any one unit, cited output of particular individuals may be read, in borderline cases, by several members. This will also be the procedure if there is doubt or disagreement about a submission amongst the Panel, and where appropriate, specialist advisers or other RAE panels will also be asked to examine research output in detail.
3.31.42 The Panel will strive to form its judgement through deliberation and consensus but where differences remain, decisions will be reached by a simple majority vote of the Panel.
3.31.43 The pattern of gradings will be reviewed to ensure consistency and gradings will not be confirmed until the Panel's last meeting.
Interpretation of Rating Points
3.31.44 The Panel will assess the quality of research outputs by reference to the depth of material, marshalling of evidence, clarity and accessibility, accuracy, theoretical advance, reliability, persuasiveness, rigour of analysis or conceptual or other methodological innovation. It will interpret the rating points as follows:
3.31.45 International excellence equates to work of highest quality. It denotes work with which any researcher in the field or sub-field ought to engage.
3.31.46 National excellence equates to work of good quality with which other researchers in the field or sub-field should be expected to be familiar.
3.31.47 Where a research group or specific area of activity has made an outstanding contribution to the overall grading of a submission, this will be communicated to the funding bodies along with the gradings.
3.31.48 The Panel will establish one sub-panel for Science Policy, comprised of two members of the panel and two specialist advisers. Departments may wish to identify submissions for assessment by this sub-panel.
3.31.49 The Panel will establish a user-group, comprised of two panel members and a number of specialist advisers. The user-group will be used to advise on the Panel's assessment of accessibility, and where appropriate the policy relevance, of the work in question.
3.31.50 The Panel will consult a number of non-UK experts on all prospective 5*and 5 submissions and a selection of prospective 4 submissions.
Last updated 17 April 2000