Section III: Panelsí Criteria and Working Methods
3.25 Built Environment, UoA 33
3.25.1 The UoA includes: Architecture; Design; History of Architecture; Theory of Architecture; Landscape Architecture; Quantity Surveying and Building Economics; Legal Systems; Construction Management; Construction Technology and Construction Materials; Information Technology; Building Science and Services; Buildings and Energy; Building Surveying; Sustainability; Education in the Built Environment; Process Organisation (including the supply chain) and Management; Building Engineering; Facilities Management; Ecology in the Built Environment; Social and Human Factors of the Built Environment; and other research in which the built environment could either form a major field for application or provide the context for development.
3.25.2 The Panel will take a holistic view in assessing submissions made to Unit of Assessment 33, Built Environment encompassing all evidence presented and will interpret data and information according to the group profile of each submission received.
3.25.3 The remit of the Panel covers a wide range of disciplines from building science to architectural history and from services engineering to construction management. The Panel recognises the diverse nature of the areas of work which fall within the UoA. The Panel also acknowledges the emerging and maturing nature of research in some areas of its discipline and is sensitive to the problems posed by the range of dissemination outlets available across the whole range of disciplines incorporated within UoA 33.
3.25.4 Where submissions clearly span the boundary between two or more units of assessment (UoA) the Panel will automatically refer the entire submission to the appropriate Panel(s) for advice, identifying the particular items of research output on which views are sought.
3.25.5 The Panel will work closely with the Town and Country Planning Assessment Panel(UoA 34), referring for advice submissions which span the boundary between the two units of assessment. In particular, the Panel will refer submissions which include research work in housing, transport, planning and property markets and will receive work related to construction, architecture and landscape architecture.
3.25.6 The Panel does not intend to establish sub-panels to advise on particular areas. Whilst the Panel has not currently identified specific areas of research for which it would appoint specialist advisers, it will review this position in the light of submissions received and consideration of interdisciplinary research.
3.25.7 The Panel has wide ranging experience in this area and will welcome interdisciplinary submissions.
3.25.8 For each submission, a lead Panel member will be identified Among other things, the lead member will consider the level of interdisciplinarity, advising the Panel on issues such as the contribution which interdisciplinarity has made to the quality of the submission and the extent to which interdisciplinarity has enhanced vision and leadership where appropriate.
3.25.9 The Panel recognises that interdisciplinary research may cover disciplines which fall outside of the multi-disciplinary nature of UoA 33. Where the lead member feels that the Panel's expertise does not adequately cover the research being assessed then the advice of another panel or panels or specialist advisers will be taken. However, responsibility for awarding the rating will remain with the Built Environment Assessment Panel.
3.25.10 The Panel will assess joint submissions against the same criteria, and on the same basis, as all other submissions. It will, however, look to the common textual element of the joint submission (RA5 and RA6) for evidence of genuine collaboration between the institutions concerned and will also expect to see a description of the nature of, and arrangements for, joint working.
Treatment of Evidence
3.25.11 The overall quality of the research output is the key factor. Each submission will be treated on its own merits. In recognition of the different contexts in which different research groups operate, no fixed weightings will be used. The overall quality of research will be judged on the basis of the following:
Research Output (RA2)
3.25.12 The Panel expects to receive a wide range of research output including papers in refereed and professional journals, articles, data on built artefacts, books, edited works, chapters in books, papers to national and international conferences, design, software, demonstrators, patents, toolkits, government and regulatory output. Teaching materials will be considered where they are shown to embody research outputs within the RAE definition but the preparation of teaching material in itself is not accepted as a research activity for the purposes of the RAE. All research outputs submitted should comply with the RAE definition of research, and will be assessed for their contribution to knowledge and innovation. In order to support its assessment of cited public outputs other than publications, the Panel wishes textual evidence of the research content and context of the cited research output to be entered into the 'Other relevant details' field of RA2.
3.25.13 The Panel expects to be already familiar with some of the work submitted. Each submission will be assigned to at least three Panel members who reflect the subject content of the submission for the overall assessment of the cited research. One of the three Panel members will act as the lead assessor for each submission. The lead role will incorporate the collation of any specialist advice. The assigned Panel members will examine in detail at least one cited item per member of staff from the submission. The Panel will aim collectively to review the majority of the cited work, the lead assessor examining in detail approximately 25% of this work. The reading will be targeted in the following ways:
3.25.14 Research output which has not been subject to peer review will not automatically be regarded as of lower quality.
3.25.15 The Panel will assess all forms of output against the same range of criteria and will base its judgements of quality on the extent to which output has contributed to the advancement of knowledge. In regard to design and industrial output, the Panel will be informed by their investigation of the standing of various competitions, awards, journals and exhibitions involved. The Panel considers group authorship appropriate and even desirable on occasion for the publication of much collaborative research, and will assess group-authored work accordingly. A single item of research output with co-authors, however, should not be submitted more than once within a department's submission. The Panel will assess all types of substantive research output against one or more of the following criteria:
Research Students and Research Studentships (RA3)
3.25.16 The number of higher degrees awarded and the number of postgraduates registered will be regarded as indicators of quality. Doctorates will be rated more highly than research masters degrees.
3.25.17 Studentships funded by external sources will be will be considered as evidence of esteem.
External Research Income (RA4)
3.25.18 The Panel recognises that the level of external research income will differ between the various research areas covered by (or included within) UoA 33 and the total for any submission will therefore be largely dependant upon the particular research groups included. External research income will be used as a measure of peer judgement of standing related to a group or an individual researcher's previous achievements, or promise.
3.25.19 More weight will be attached to funding provided by the research councils where peer review and policy relevance are rigorously applied. Equal recognition will also be give to research funds provided by European and National Governments and agencies, elite charities and industry which are also subject to peer review and strong competition.
3.25.20 Other research funds will be assessed on the basis of the support they provide to the research groups and their outputs at large. It will consider whether the funds are in the form of a grant, consultancy or contract, and whether they are part of a larger research programme or research consortium. The RA5 statement should identify clearly the source of the these funds and the output in research terms that has been achieved.
3.25.21 The RA5 statement should include an assessment of research performance over the RA5 period, reflecting on the 1996 RAE Research Strategy and its objectives. The statement should explain the progress made over the assessment period, explain anticipated and unanticipated events and the strengths and weaknesses of the research culture and its outputs. This self-assessment should encompass all key aspects of research inputs and outputs, structure, staffing and research strategy.
3.25.22 In the textual commentary (RA5 and RA6) the Panel will seek to identify significant coherence in research strategy and evidence of vision and originality. The Panel will be concerned to seek evidence of vitality and research leadership within each research group identified. Clear objectives and plans for implementation will be viewed as more impressive than description of intent. Future potential is important and the inclusion of recent recruits who are active but still maturing in terms of research performance is acceptable providing the leadership pertaining to the research group is demonstrated.
3.25.23 The Panel will take note of work which has direct relevance to the needs of industry, commerce and government via its regulatory policy. Within this, the extent to which the output has, or has the potential to, contribute to the discussion and possible implementation of various reports/initiatives will be important. In this context, reports/initiatives include those emanating from external agencies, such as government and other 'construction industry' bodies set up to comment upon research strategies.
3.25.24 It is important that the management and organisation of research is clearly stated and as such, institutions are asked to address the following matters in RA5 and RA6:
Research Structure and Environment
3.25.25 Clearly describe the research group's department's place within the institutional structure and its support arrangements.
3.25.26 Clearly define the research groups in the UoA and list the staff belonging to each group. A research group consisting of one research active member of staff will be accepted. Staff can be included in more than one research group. Younger and/or new researchers should be identified. The inclusion of young staff who are active but still maturing in terms of research performance within a strong research group will be viewed favourably within the wider context of the department's research activity, especially where there is evidence of an effective policy to support their development. Allowance for a lower quantity of output will be made.
3.25.27 Define the prime activities of the research groups, how they operate and their main achievements. Information should be provided on the organisation and management of each research group, including details of how research is undertaken and developed and the process by which the research agenda is set. Where a department encompasses staff of disparate disciplines, please present a reasoned case why the research group has been included in this submission to UoA 33.
3.25.28 List other UoAs to which related work has been submitted and detail any difficulties of fit between departmental structure and the UoA framework. Please indicate if it is believed that the submission to this UoA spans the boundary between more than one UoA and state which other discipline(s) is/are involved.
3.25.29 Describe the nature and quality of the research infrastructure, including the facilities for research students.
3.25.30 Explain the mechanisms and practices for promoting research and sustaining and developing an active and vital research culture.
3.25.31 Describe any arrangements which are in place for supporting interdisciplinary or collaborative research. If the submission is interdisciplinary, please indicate this and state other discipline(s) involved.
3.25.32 Describe existing networks, collaborative arrangements and linkages with industry, commerce and other users of research, including inter- and intra-institution work. Please give details of technology transfers and industry monetary support both direct and indirect (including instances where in-kind funding has been received).
3.25.33 Where appropriate, information should be provided on the account taken of Government policy initiatives and objectives. This should include information on any regional, national and international initiatives, where national is understood, in this context, to refer to England, Scotland, Wales and/or Northern Ireland.
3.25.34 Describe the arrangements for the development and support of the research work of staff.
3.25.35 Describe any arrangements which are in place for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research culture. Please provide an explanation of the support mechanisms, together with information regarding their actual and potential career paths.
3.25.36 Where appropriate explain the role and contribution of category A* staff who have joined the department.
3.25.37 Where appropriate, comment on how the departure or movement (for example, a secondment to industry) of staff in categories A*, B and D has affected the strength, coherence and research culture of the department at the census date.
3.25.38 For joint submissions only - define the role of staff and the policy related to such staff across the joint submission. Information should also be provided on the management of the departments involved.
3.25.39 Provide a statement about the main objectives and activities in research over the next five years. The Panel's attention should be drawn to new and developing initiatives that are not yet producing immediate outcomes or which may not yet be performing at national or international level.
3.25.40 Where research plans have been put forward in the 1996 RAE, they will be considered by the Panel. Comment briefly on the extent to which the plans have been achieved. Where objectives have been changed, or have not been met, please give reasons. Clearly describe any planned changes to objectives, research groups, structure (including management, staffing and studentships), funding (including expected sources), networking/collaboration (including inter-and intra-institution, industry, commerce and other users of research).
Evidence of Esteem
3.25.41 The Panel will seek evidence of peer esteem and could include the following in terms of both research groups and individuals:
3.25.42 Specifically to the area of design, this evidence could include:
Individual Staff Circumstances
3.25.43 Note any individual staff circumstances which have significantly affected their contribution to the submission (eg periods of sick leave, career breaks, engagement on long term projects etc).
Contributions by Non Research-active Staff
3.25.44 Institutions may also wish to discuss the relative contribution made to research by staff who have not been returned as research active.
3.25.45 The Panel will attach the greatest importance to the quality of research output, this being assessed by research groups only. Judgements made by the Panel will not be affected directly by the size of a research group or department. It is recognised that small groups can produce work of highest quality. A preliminary grading for the whole department will then be made, taking into account the indicators relating to the submission as a whole. Account will be taken of activities which span two or more research groups where this leads to enhanced quality. Note will be taken of quantitative indicators that relate to the submission as a whole.
3.25.46 The Panel will assess submissions on the basis of its informed judgement. Quantitative measures will be used, as background, to inform views of ratings where these exist (for example, research income, research students and studentships): all quantifiable parts of a submission (for example, external research income and numbers of research students) will be adjusted to allow for the number of staff contributing to that part. Thus, the size of the department per se will not directly affect the outcome, and there is no minimum number of staff required to be submitted.
3.25.47 All members of the Panel will read all submissions and make an overall assessment of quality without consulting fellow Panellists. These 'blind' evaluations will then be collated and compared as the starting point for deriving ratings. For each submission at least three Panel Members (two academics and one practitioner) will be assigned to read in detail and review each cited publication. One academic will act as lead assessor, the other as a secondary assessor and the practitioner will make a particular assessment of the quality of work from the users' perspective of the work submitted. Their quality rating will be based upon the criteria cited in paragraph 3.25.15. Their judgements will be informed by fellow panellists who will have specialist knowledge of the range and quality of work in each research area of the Unit of Assessment.
3.25.48 In order to both focus consideration of the submissions on the criteria as agreed by the Panel and to guarantee the integrity of the ratings process in procedural terms, the Panel will structure its approach to assessment using a proforma. The Panel will develop this proforma based on the Panel's working methods cited above. It will function as an aide memoire in discussion of the rating of submissions to help to ensure consistent and equitable treatment of all submissions made. In order that the Chair is able to discharge his responsibilities effectively, ensuring the integrity of proceedings, he will not be identified in either a lead or supporting role. The Chair will, however, be familiar with all submissions and make his own assessment.
3.25.49 All submissions will subsequently be considered in greater detail by the Panel and there will be an iterative process of rating and discussion until a consensus is reached by the Panel. Where this is not possible members will vote on the rating to be awarded submissions, with the Chair holding a casting vote in the event of a tied vote
3.25.50 The Panel will classify submissions according to the standard rating scale. Evidence for international research excellence can include that which contributes to setting the international research agenda for the research topic, or contributes significantly to its development, through leadership, impact, publication and/or collaboration. International excellence is interpreted as being "as good as the leading research in those countries where there is a significant body of work in the field." The Panel does not in general confine the international standard of comparison to any particular geographical area. The Panel confirms that the terms 'international' and 'national' refer to a quality standard, and not to the nature or scope of any particular research.
3.25.51 Having made provisional ratings, the Panel will forward a sample of submissions to a group of up to six individuals with expertise to cover a range of research disciplines and from a range of countries outside the UK. The sample will include submissions provisionally rated 5* and 5. Experts will be chosen to cover the range of activity covered by the Panel. The reports of these non-UK based experts will be considered by the Panel, and will contribute to the final grading. However, the final grade decision will remain with the main Panel.
Last updated 17 April 2000