Panelsí Criteria and Working Methods
3.58 Music, UoA 67
3.58.1 The UoA includes composition and performance (including classical, commercial, and popular); history and criticism of music; ethnomusicology; theory and analysis, including empirical approaches; technology and computer applications.
UoA Boundaries and Interdisciplinary Research
3.58.2 The Panel understands its remit as encompassing academic and practice-based research across the widest range of musical fields.
3.58.3 The Panel is conscious of the multidisciplinary nature of much music research, and will refer to other panels or specialist advisers wherever it is appropriate to do so (for example in relation to research in music education, audio engineering, popular and commercial music practice, or music therapy); final decisions on ratings will rest with the Panel for Music and will be reached against its criteria. The formation of sub-panels is not anticipated.
3.58.4 Joint submissions between institutions, if received, will be evaluated as a whole, with supporting data being aggregated and a single rating awarded to the submission.
Treatment of Evidence
3.58.5 Assessment will be based primarily on work listed in RA2, interpreted within the context of the relevant research environment and strategy. Whereas input measures such as external research funding and studentships will be considered as evidence of the research environment, the Panel's overriding concern will be with the quality of research output. Where reference is made to quantitative measures, these will be normalized as appropriate on the basis of the number of research active staff (including category C staff). In the case of fractional and category C staff, the Panel will wish to be satisfied as to the extent and value of the contribution made to the overall research environment, and the extent to which they in turn draw upon it.
3.58.6 The size of departments will not be regarded as an indicator of research quality, nor will breadth of research expertise across a wide range of topics be regarded as necessarily superior or inferior to a concentration of research within a narrower focus.
3.58.7 The Panel recognizes the importance of practice-based research to the research cultures and activities of many departments and individual researchers. Practical output which demonstrably embodies original research as defined for the RAE will be assessed on an equal basis with other outputs.
3.58.8 Those submitting practice as research may include (in the 'other relevant details' field of form RA2) a succinct statement of not more than 300 words for each item in this category listed under RA2; for this purpose practice is defined as all outputs listed in paragraph 3.58.12 b) to e) below. These statements should make clear how the practice embodies research as defined in the RAE. They may, where appropriate, include an indication of the aims, methods, procedures, innovation, significance, and context of the practice. It should be noted that the submission of such statements is not a requirement; the Panel will not expect them when the status of the practice as research is self-evident.
3.58.9 Where practice results in multiple outcomes (eg scores, recordings, and performances of the same composition or musical edition), one 'lead' outcome should be identified for inclusion in RA2; other outcomes may be identified in the 'Other relevant details' field of form RA2. Practice-based research originally disseminated prior to 1994 and submitted as such for a previous research assessment exercise may not be submitted in another form for the present exercise (except in the case of revisions, in which case credit will be given only for new elements).
Research Outputs (RA2)
3.58.10 Research in music results in assessable public output of various kinds including, for example, live performances, broadcasts, recordings, films, videos, and computer media as well as conventional published materials. Where appropriate, as in the case of live performances, evidence may be sought that the output was in fact accessible to the public (for instance by virtue of being announced as such). Electronically-disseminated publications (such as internet journal contributions) will be assessed on the same basis as printed materials; where applicable the full URL must be stated in RA2.
3.58.11 In all cases reference will be made to the general definition of research applicable to the RAE, with the following being regarded where relevant as positive qualities: originality or innovation in method, interpretation, or insight; rigour, imagination, scope, substance, or significance; evidence of national or international impact. In reaching a view on the quality of submitted material the Panel will, where appropriate, take account of factors associated with the publication or other dissemination of the work, for example the extent to which it has involved rigorous peer review, but work not subject to such review will not be automatically assumed to be of lower quality; the Panel will be aware of relevant developments affecting, for instance, the music publishing, broadcasting, and recording industries, as well as variations in public support for practice-based research, and will treat such indicators with appropriate sensitivity.
3.58.12 The following listing of research output should be understood as indicative rather than exclusive:
Research Students and Research Studentships (RA3)
3.58.13 In assessing the overall research environment, the Panel will take account of established patterns of successful research supervision and other indications of high-quality research preparation and support. While institutionally funded studentships will be seen as evidence of commitment to the research infrastructure, studentships awarded on the basis of external specialist review will be regarded as indicators of esteem. The Panel recognizes the uneven patterns of funding across the discipline (for instance in relation to composition and performance) and will not adopt a simply quantitative approach.
External Research Income (RA4)
3.58.14 The Panel will be primarily concerned with the quality of output resulting from externally funded research rather than with quantitative measures of funding. In addition, the Panel recognizes the uneven patterns of access to funding across the discipline. It will however consider external research income, particularly when resulting from a process of specialist peer review, as one of a range of positive indicators relating to the departmental research environment. Institutions may wish to comment in RA5 or RA6 on income which supported research during the assessment period but does not appear in RA4.
3.58.15 The Panel will wish to understand submitted work in the context of the departmental research strategy and arrangements for its support (RA5) and indications of esteem (RA6). These sections should be organized according to the following sub-headings. The suggested content is indicative and should be supplemented by other material to which institutions wish to draw the Panel's attention. Commentaries should incorporate an element of self-assessment (i.e. should be critical rather than merely descriptive, identifying measures taken to address perceived problems), but no rating may be proposed.
Research Structure and Environment
3.58.16 Institutions should provide an explanation of the context within which research is conducted. This should include the nature and quality of the research infrastructure, and the means by which research is promoted and an active research culture maintained. Where appropriate information should be provided concerning any mismatches between departmental and UoA structure; research themes or groupings where these play a significant role in the management of research; any arrangements for supporting interdisciplinary research; and relationships with any external users of research.
3.58.17 Information should be provided on staff development and support, including arrangements for developing the research of younger and/or new researchers and for integrating them into a wider, supportive research culture. The Panel will form a view as to what the balance of staff on permanent and short-term contracts tells them about the research culture of the department; institutions are invited to comment on the role and contribution of staff on short-term contracts, as well as of category A* staff who have joined the department. Comment may also be made on how the departure of staff in categories A*, B and D has affected the strength, coherence, and research culture of the department at the census date.
3.58.18 Institutions should provide an overview of research activities and achievements during the assessment period (organized as appropriate by individuals, themes, or research groups) and explain the department's research plan and priorities. While assessment will be based on the items listed in RA2, institutions may list other significant public output within the assessment period where they consider that this contributes to an understanding of the overall pattern of activity. Where appropriate, comment should be made on the relationship of externally funded research to the departmental research strategy, and on any major deviations from research plans put forward in the 1996 RAE.
3.58.19 Institutions should provide a statement about the main objectives and anticipated activities in research over the next five years; well defined objectives and measures for their realization will be viewed as more impressive than generalized descriptions of intent. Mention should be made of significant investment expected to result in research outcomes during this period, and of ongoing research which has not yielded major publication or other outputs within the period of assessment; anticipated modes and dates of dissemination should be indicated.
Evidence of Esteem
3.58.20 Evidence of esteem relating to submitted staff may include, for example, visiting or research appointments, journal or book series editorships, membership of editorial boards and national or international committees, translations of published research into other languages, keynote addresses and invited papers at international conferences, direction of major conferences or festivals, commissions, recording contracts, prizes and other marks of recognition.
Individual Staff Circumstances
3.58.21 The Panel recognises that there are circumstances under which researchers may reasonably submit less than four items of output but will wish to be informed of these circumstances. Information should be provided (with dates and concise details) concerning staff who have completed doctorates or other professional training and/or initiated a research-active career during the assessment period; where such staff are unable to submit significant output, the Panel will expect to see alternative evidence of their research activity, including for instance details of doctoral dissertations (which are recognised as representing significant research), conference presentations, and forthcoming publications. Information should also be provided concerning any individual staff circumstances which have significantly affected their contribution to the submission (e.g. periods of sick leave, career breaks, engagement on long term projects, or participation in projects initiated by the HE funding bodies).
Contributions by Non Research-active Staff
3.58.22 Comment is invited on the role within the department of staff not submitted as research active, including any contribution made towards overall departmental research activity.
3.58.23 Assessment will begin with the evaluation of the work of individual researchers, each of whom will be assigned to two panel members primarily on the basis of subject expertise. The Panel collectively will examine at least 25% of submitted items (one by each individual researcher). Further detailed examination will be carried out selectively and it is anticipated that the overall proportion examined will be in the order of 50%. The Panel will concentrate in particular on those which have not been subject to rigorous peer review. Ratings will be presented to and agreed by the whole Panel.
3.58.24 A preliminary rating of departments will be derived from the ratings of individuals through a process of calculation involving reference to the RAE rating descriptions. The contribution of researchers on fractional contracts will be normalized on the basis of FTE value. In the case of departments including in their submission staff members who have begun research-active careers within the assessment period, this exercise will be carried out twice: once including, and once excluding such staff. This is to ensure that departments are not penalized for appointing and developing entry-level staff; any discrepancy between the resultant preliminary ratings will be viewed in the light of individual circumstances, including details provided of research activity by such staff, and the department's arrangements for and track record in the development of junior staff.
3.58.25 Following preliminary rating, each department will be assigned to two panel members, who will review the preliminary rating(s) in light of the contribution of category C staff, departmental and institutional research environment and strategy, and other information provided in RA5 and 6. Their recommendations will be presented to the entire Panel, all members of which will read all submissions and will be collectively responsible for all ratings awarded. Agreed ratings will be subsequently reviewed in order to moderate between submissions rated at different times. It is anticipated that final decisions will normally be made by consensus, but where this is not possible ratings will be determined by voting.
3.58.26 The Panel will devote particular attention to the boundaries between different ratings. RAE profiling criteria for these turn upon the distinction between research of national and international quality; it should be noted that 'national' and 'international' do not refer to the nature or scope of particular disciplinary areas, nor necessarily to quantitative measures of dissemination. In general, individual work assessed as of international standard will be equivalent to the best work in the same field undertaken anywhere in the world, and will where appropriate demonstrate most or all of the positive qualities mentioned in paragraph 3.58.11 above; work assessed as of national standard will be at least satisfactory in all relevant respects and may be outstanding in some. Again, internationally excellent work is likely to contribute to the formation of new paradigms or the significant extension of existing ones, as compared with work of national standard which may be more limited in its significance. At both the individual and departmental level, the highest levels of achievement will be associated with disciplinary leadership.
3.58.27 The entire submissions (RA1-6) of all departments provisionally rated 5 and 5*, together with submissions from a sample of departments provisionally rated 4, will be forwarded to a group of non-UK based experts who will be requested to comment on the extent to which the departments in question meet international standards of excellence. Final decisions on ratings will however remain the sole responsibility of the Panel.