RAE2001 logo

Contents

Another UoA

Section III: Panelsí Criteria and Working Methods


3.2 Community-based Clinical Subjects, UoA 2

UoA Descriptor and Boundaries

3.2.1 The UoA includes Epidemiology, Public Health Research, Health Services Research, Primary Care and Psychiatry.

3.2.2 Related subjects which may be included, and which overlap with other units of assessment, are: Medical Sociology, Medical Statistics and Biostatistics, Population Genetics, Health Psychology, Clinical Psychology, Health Economics, Community Paediatrics and Child Health, Tropical Medicine, Operational Research and Modelling (in the health context), Clinical Trials Methodology and Organisation, Community Based Clinical Trials, Medical Anthropology, Occupational Medicine, Medical Demography, Medical Ethics, and community-based health research from any health care discipline. The Panel will also consider pedagogic research related to the subject areas covered by this Panel.

3.2.3 The Panel expects to receive submissions from all areas of Community-based Clinical Subjects. Submissions which are primarily concerned with laboratory based subjects (as defined in UoA 1) or hospital based subjects (as defined in UoA 3) should be submitted to those UoAs. However, the UoA 2 Panel is well aware of the breadth of its field and the probability of overlap. It will work closely with other panels where their work may overlap.

3.2.4 Submissions which span the boundary between two or more UoAs will be cross-referred, using the Panelís judgement or where requested by the submitting HEI, to the most appropriate UoA for advice; this could include cross-referral to non-medical panels. The final grade decision will remain with the Panel for the UoA to which the submission was originally directed by the submitting HEI. It is the responsibility of the submitting HEI to determine which would be the most appropriate UoA to assess its profile of research activity.

Sub-Panels

3.2.5 There will be three sub-panels within the panel structure of UoA 2. These will cover Epidemiology, Public Health Research and Health Services Research; Primary Care; and Psychiatry. The sub-panels will also be available to provide advice, if required, to the panels for UoAs 1 and 3. There will be cross-representation in the membership of the Psychiatry sub-panel and the Neurosciences sub-panel of UoAs 1 and 3. Each sub-panel will be chaired by a main panel member and will have an approximately equal number of panel and outside expert members. The membership of sub-panels will be published in advance of the submission date.

3.2.6 The role of the sub-panels is to provide a wider range of expertise and viewpoints, particularly in relation to the research output cited in submissions. Sub-panels will also further the accurate and consistent assessment of the large quantity of medical research which is of an interdisciplinary nature by allowing panel members from other clinical UoAs to offer their perspectives in a common forum. Sub-panels will examine in detail the research output referred to them. In reading submissions, examining items of output and reaching decisions sub-panels will work within the procedures established by main panels.

3.2.7 As part of their advice to the UoA 2 Panel, sub-panels will suggest a grade, using the RAE rating scale, for work which they have examined. The grade, when ratified by the UoA 2 Panel, will be published, together with the FTE number of staff referred to the sub-panel. This will apply only when HEIs request referral of part of their submission to a sub-panel. The Panel may additionally refer items of output submitted in order to utilise the expertise of a sub-panel. In this instance, the advice given will not include a proposed grade.

3.2.8 It is the responsibility of the submitting HEI to determine whether specifically to request sub-panel assessment. There is no advantage or disadvantage to HEIs requesting the involvement of sub-panels, nor is it expected that submissions should mirror sub-panel areas. The final grade decision remains with the main Panel at all times and the Panel will ensure that all activity submitted is taken fully into account in arriving at an overall grade. Sub-panels will receive the relevant part of form RA2 and all of forms RA5 and RA6 from the submission. The main panel will consider the advice of the sub-panels on these sections in the context of the entire submission and decide on the final grade. HEIs, therefore, should indicate whether, and if so, which outputs they wish to be considered by which sub-panel(s). No part of a submission may be directed to more than one sub panel. When requesting referral to a sub-panelPanel the normal expectation is that an institution will request referral of all of its work in the appropriate area (e.g. neurosciences, oncology) contained in the submission.

Specialist Advice

3.2.9 The Panel recognises that it may have to seek specialist advice in areas where despite the use of sub-panels, some fields may be outside panel membersí expertise. The use of specialist advisers will be decided once submissions have been received and any gaps in expertise identified.

Interdisciplinary Research

3.2.10 The Panel recognises the interdisciplinary nature of its field, which is reflected in the experience its members have as inter-disciplinary researchers. Cross-UoA consistency and equity will be maintained by regular meetings between panel chairs in cognate fields to maintain common standards. There will be meetings between panel chairs in UoA 1, 2 and 3 to address the particular issues of the medical disciplines, in addition to the umbrella meetings involving a wider range of disciplines. HEIs should be reassured that interdisciplinary research will not be disadvantaged and should seek to submit their research in groupings which most accurately reflect activity in the institution.

Joint Submissions

3.2.11 Joint submissions shared between two HEIs will be considered in the normal way, as if they had originated as a single submission from one HEI.

Treatment of Evidence

3.2.12 Quality of submissions will be judged by the Panel on the basis of the following measures:

a The quality of research output (RA2).
b Extent of research activity as indicated, for example, by the number of research assistants (RA1), research students (RA3a), and research studentships (RA3b).
c Evidence of esteem by external funders as indicated by research income (RA4).
d Evidence of vitality and prospects for continuing development (RA5 and RA6).

Of these measures greatest weight will be afforded to quality of research output.

Research Output (RA2)

3.2.13 The Panel expects the research output it has to review to consist very largely of papers in scientific journals, monographs and books, but other forms of output, including patents and other applied research, will also be accepted and judged on their individual merits. The evaluation of research papers will not use quantitative bibliometric indices. Panel members will be guided by their professional judgement on the quality of the research output, bearing in mind the widespread differences in the specialist areas included within the UoA.

3.2.14 The Panel collectively will review all submissions and examine in detail a minimum of 25% of research output cited in submissions.

3.2.15 The main Panel and, where so required, sub-panels and other sources of expertise will use their combined expertise in undertaking review and establishing the merit of the work submitted. Those contributing to the assessment will be asked to use their expertise and judgement in assessing the work, using criteria including the originality, scientific rigour, potential and actual implementation and impact in health care, contribution to knowledge and conceptual framework of the field.

3.2.16 Systematic overviews and meta-analyses and research into methodology will be considered equally with papers describing primary research. Papers presenting a new hypothesis will be given equal weight, assessed in terms of their originality and power to stimulate new work. More traditional reviews or editorials will need to display originality. The Panel will give full recognition to the needs of the public and voluntary sectors, as well as to commerce and industry, including patents and other applied research of direct relevance to health care.

3.2.17 The evaluation of research papers will not use quantitative bibliometric indices or an algorithmic approach, but be based on the professional and scientific judgement of the Panel. The Panel will assess the international and national impact of publications in the relevant field of interest. Order of authorship will not be considered important; brief comment on relative contributions may be added in the 'Other relevant details' field on RA2. The Panel considers group authorship appropriate and often desirable for the publication of much multi-centre collaborative research, and will assess group authored work accordingly.

3.2.18 A submission of less than four research outputs per active researcher may be indicative of lack of depth within the subject area or of low productivity, but there will be no automatic penalty for citing fewer than four items of research . Where different individuals have nominated the same publication among their cited works an explanation should be given in RA5.

Research Students and Studentships (RA3)

3.2.19 The number of research-based higher degrees awarded and the number of studentships will be regarded as indicators of quality, with higher degrees, and doctorates in particular, rated more highly than studentships.

3.2.20 Greater credit will be given to studentships subject to rigorous competition.

External Research Income (RA4)

3.2.21 The Panel will not adopt an algorithmic approach when assessing the research income per head of research active staff. In general terms, the Panel would expect to see a dynamic research culture reflected in substantial research income, while accepting that different areas of research may require different levels of funding. Some forms of research may require little external funding. The Panel will expect to see the distribution of research income over the different sub-groups described in the textual commentary. The Panel would expect to see research funding at a level commensurate with the research strategy of each sub-group.

3.2.22 Amongst academic funding sources, greater weighting will be given to those forms of support known to be the result of a rigorous peer-reviewed assessment process and open to broad competition. Due account will be taken of long term support from major funding agencies known to require the very highest application standards, such as support provided by charities in, or governed by the rules of, the AMRC. Institutions are requested to identify separately charities income derived from AMRC sources. Industrial income and awards will also be considered as indicators of usersí esteem, and in cases of sizeable and sustained support, may be regarded as equivalent to peer reviewed funding.

Textual Commentary

RA5

Research Structure and Environment

3.2.23 HEIs should define research groups, including who belongs to them (referring to RA1), their prime activities, how they operate and their main achievements. To facilitate analysis of submissions HEIs will be expected to identify research groups in both the staff record (RA1) and research outputs form (RA2). HEIs may also wish to comment on the operation of major research groups under RA5. The submissionís research groups need not be related to sub-panelsí structures. Research groups selected and identified by the HEI should be consistent between RA1, RA2, RA5 and RA6.

3.2.24 Difficulties of fit between departmental structure and the UoA framework should be described, and other UoAs to which related work has been submitted should be listed.

3.2.25 The mechanisms and practices for promoting research and sustaining and developing an active and vital research culture should be explained. Some explanation would be helpful if a substantial number of staff are not returned as research active in any or all of the research areas identified by the HEI.

3.2.26 The research infrastructure should be described. This might include availability of computing and statistical support, efficient access to libraries and relevant databases and support for development of collaborative links; the procedures for internal review of research proposals; the facilities for research students; the means and extent to which interdisciplinary research is facilitated, given that much research in the UoA should have input from multiple disciplines.

3.2.27 Information should be given on research relationships with the NHS and other service providers, with international research or health organisations, non-Governmental organisations, industry and other research users. Account taken of Government policy initiatives and objectives should be noted where appropriate. The impact of the research on strategic development of the area, on practice, or on policy which draws on the research should be noted.

3.2.28 One or more of the research groups may have been involved in a major collaborative research programme with a number of other institutions. Where the role of the research group is not fully evident from other parts of the submission this should be explained.

Staffing Policy

3.2.29 The arrangements for the development and support of the research work of staff should be described. Any arrangements which are in place for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research culture should be described. HEIs should where appropriate, explain the role and contribution of Category A* staff who have joined the department. The arrangements for the research career development of both non-clinical and clinical scientists should be included. Particular mention of the ole of clinical lecturers would be helpful. Where appropriate, comments n how the departure of staff in categories A* B and D has affected the strength, coherence and research culture of the department at the census date should be included.

Research Strategy

3.2.30 The main objectives and activities in research over the next five years should be stated. This should include the balance sought between long-term and short-term research, the development of infrastructure to facilitate research and ongoing work that is not producing immediate visible outcomes. The procedures for developing research strategy should be described.

Self-Assessment

3.2.31 HEIs should provide a self-assessment of performance in relation to the issues detailed above in the textual commentary. HEIs should be aware that the Panel will have access to RA5 and RA6 statements from the 1996 RAE. Any changes since 1996 which affect the research capacity of the HEI will be of interest to the Panel. The self-assessment should be drafted in light of this fact and tied into sub-groups as defined within the submission for 2001. If no submission was made to the Community-based Clinical Subjects UoA in 1996 institutions should comment on the development of research in this area since 1996.

RA6

Evidence of Esteem

3.2.32 Indicators of peer esteem of the staff submitted, relating to the relevant time period should be listed. This could include for example Fellowships of the Royal Society, of the Academy of Medical Sciences, named lectures and invitations to give key-note addresses, senior awards from Research Councils, the Medical Charities or similar bodies in other countries, membership of national or international review boards and funding bodies, editorship of international journals and election to international bodies.

Individual Staff Circumstances

3.2.33 Any individual staff circumstances which have significantly affected their contribution to the submission should be noted.

Contributions by Non Research-active Staff

3.2.34 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.

Working Methods

3.2.35 The assessment process will be one of peer review based on professional and scientific judgement. It will not be mechanistic.

3.2.36 The Panel will divide the research outputs of each submission into the components to be assessed by the Panel and the sub-panels. The Panel or relevant sub-panel will assign two lead reviewers for each part submission. The sub-panels will only assess the research output referred to them, seen within the context of the information provided in RA5 and RA6. Each sub-panel will assign a rating to the work it has assessed.

3.2.37 A minimum of 25% of all research outputs (i.e. items of research output) will be examined in detail by either main or sub-panel members or specialist advisers. The remaining submitted items will be reviewed. Therefore, all submitted items within each overall submission will have been taken into account in the main Panelís final assessment.

3.2.38 The final step in the assignment of ratings will be semi-quantitative, in that a weighted average will be taken of the assessment attached to the general research areas covered by each sub-panel. The weightings will be obtained from an evaluation of the contribution of the research referred in these general research areas, seen in the context of the whole submission.

3.2.39 The Panel, at its final meeting, will review all the ratings, to ensure consistency and minimise any variations over the period of assessment that may have occurred.

3.2.40 The initial UoA 2 Panel rating will be based on a ballot of panel members. The results of the ballots for individual HEIs, particularly their spread over two or more grades, will be reviewed at the end of the meeting. After further discussion a second ballot will then be held focusing on two grades which received the highest support in the first ballot. The result of the second ballot will be final, and in the event of a tie the higher grade will apply.

3.2.41 The Panel confirms that the terms "international" and "national" refer to a quality standard, and not to the nature or scope of any particular research. The Panel will define international excellence as on a par with the best work in its area anywhere in the world. Such work will normally be disseminated internationally, will most probably be rigorously peer or user group reviewed and will display a substantive knowledge of developments in theory and/or practice internationally. The international impact will be assessed by the professional and scientific judgement of the Panel, in consultation with international experts. Work addressing a local problem may have international significance, where it reflects best international standards or would be seen as a methodological benchmark which others aim for.

3.2.42 In contrast, national excellence will be taken to refer to work which is judged to be nationally competitive and makes a valuable contribution to knowledge, theory or practice relevant to the discipline.

3.2.43 The Panel will consult a group of non-UK experts for all submissions that are provisionally ranked as 5* and 5 and a sample of those provisionally ranked 4, to test its application of the standard of international excellence. At its final meeting the Panel will discuss the comments made by the non-UK based experts. The final responsibility for the ratings to be awarded rests with the Panel itself.


Last updated 30 March 2000

[ Home | About the RAE2001 | Panels | Guidance for panel members | Guidance for institutions | Data collection | Publications | Contacts ]