Background Challenging behaviour, especially in intellectual disability, covers a wide range that is in need of further evaluation.
Aims To develop a short but comprehensive instrument for all aspects of challenging behaviour.
Method In the first part of a two-stage enquiry, a 28-item scale was constructed to examine the components of challenging behaviour. Following a simple factor analysis this was developed further to create a new short scale, the Problem Behaviour Checklist (PBCL). The scale was subsequently used in a randomised controlled trial and tested for interrater reliability. Scores were also compared with a standard scale, the Modified Overt Aggression Scale (MOAS).
Results Seven identified factors – personal violence, violence against property, self-harm, sexually inappropriate, contrary, demanding and disappearing behaviour – were scored on a 5-point scale. A subsequent factor analysis with the second population showed demanding, violent and contrary behaviour to account for most of the variance. Interrater reliability using weighted kappa showed good agreement (0.91; 95% CI 0.83–0.99). Good agreement was also shown with scores on the MOAS and a score of 1 on the PBCL showed high sensitivity (97%) and specificity (85%) for a threshold MOASscore of 4.
Conclusions The PBCL appears to be a suitable and practical scale for assessing all aspects of challenging behaviour.
Declaration of interest None.
Copyright and usage © 2016 The Royal College of Psychiatrists. This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) licence.
There is increasing concern about problem behaviours in many forms of psychiatric care, and research has been handicapped by the absence of formal incorporation of these behaviours into diagnostic systems. Although there are several instruments that record these features,1 many have poor internal consistency and reliability or only assess one component of challenging behaviour, and others with much better psychometric properties such as the Aberrant Behaviour Checklist2 are a little long and not ideal for repeat assessments. We describe the development of a seven-item 5-point scale, the Problem Behaviour Checklist (PBCL), and tested its reliability and utility in practice.
Problem behaviours, mainly in people with intellectual disability, cover a wide range of disturbance, are a source of considerable distress to hospital staff and carers3,4 and are often expensive to manage in practice, especially at more severe levels.5 Assessment is handicapped by the absence of a satisfactory diagnostic system for recording challenging behaviour and the overlap with existing diagnoses such as personality and mood disorders.6 Thus, although the prevalence of challenging behaviours is higher than any formal diagnostic group,7 the fact that it is not recognised diagnostically makes it even more important for it to be reliably and consistently assessed. In the course of research into interventions for these problems, we recognised the need to examine the full range of behaviours reported as challenging and felt that these could be condensed into a much shorter instrument.
During a randomised trial on the management of aggressive challenging behaviour8 it was observed that some forms of challenging behaviour apparently independent of aggression were not identified and these were noted. We subsequently attempted to encompass the range of all potential behaviours suitable for inclusion by close examination of two international comprehensive descriptions.9,10 Two studies were then involved in testing the scale. In the first, a field study was carried out for the World Health Organization on personality status and aggressive challenging behaviour in patients with intellectual disability in Jamaica.11 The participant population was selected from a specialist intellectual disability high school (School of Hope), a supported care home and an adult day centre for people with intellectual disabilities, all operated by the Jamaican Association on Intellectual Disabilities (JAID) in Kingston, Jamaica.
In the second study, linked to a trial of nidotherapy (details available from the author on request) in the treatment of challenging behaviour in 200 residents in care homes, the same list of behaviours was recorded for all residents at monthly intervals over the course of at least 1 year. The projects were ethically approved by the JAID and North West Wales Research Ethics Committee (10/WNo01/1).
Each of the possible challenging behaviours were scored using a 4-point scale. A key aim of the analysis was to understand the associations between the different behaviours, and factor analysis was the main tool of investigation, using a standard Varimax rotation. An individual's behaviour was considered to be associated with each factor if the factor loadings for that variable were greater than 0.5. Separate factor analyses were performed for each of the two data-sets. The scale, called the PBCL, was created after the first analysis.
A secondary objective was to compare the levels of agreement in the total scores of the scale in a subset of the data where the scores were determined by two different observers. During the course of the randomised trial, each of the two independent observers (A.T. and R.E.) visited at different times. A large proportion of the scores on the PBCL were zero and to avoid spurious agreement the scores were divided into five categorical groups (0, 1–3, 4–6, 7–11 and ≥12). Weighted kappa was used to examine the level of agreement between observers.
Concurrent validity was also assessed by examining scores on a well-established scale for aggression, the Modified Overt Aggression Scale (MOAS).12 Receiver operating characteristic curves were used to identify the optimum cut-off point for the total score in the prediction of aggressive challenging behaviour, defined as an MOAS score of 4. The sensitivity and specificity at this cut-off point were calculated.
Thirty-seven potential types of challenging behaviour were identified from study of the literature (Table 1) but because several of these appeared to be very similar the number was reduced to 28 in the final analysis (Table 1). Factor analysis revealed seven discrete factors, personal and property violence, self-harm, sexually inappropriate behaviour, contrary behaviour, demanding and difficult behaviour, and wandering. Several of these made only a small contribution to the total variance, but at this stage it was felt they were sufficiently distinct to be included. Together these factors accounted for 91% of the variance in the Jamaican study with a Cronbach's alpha of 0.7. The first factor explained around a third of the variation in the data, with components 2 and 3 explaining more than 10% of the variation in the data. Components 4–10 explained a minor proportion of the variation. Each element was scored in terms of severity, with degree of disturbance and risk being the main driving force leading to higher scores (Table 2). The final PBCL (Appendix) therefore comprised seven problem behaviour groups with five levels of severity. Aggressive behaviour was by far the most common of these. In the second factor analysis, threatening, violent, demanding and contrary (oppositional) behaviour clustered with the aggressive factor, with sexually inappropriate and self-harming behaviour accounting for much less variance (Table 3).
Comparison with scores on the MOAS
In the randomised trial, scores for aggressive challenging behaviour were assessed using the MOAS at the same assessment using the PBCL. The MOAS is a well-established and reliable instrument for assessing aggressive behaviour in this population13 and a common threshold for aggression is a MOAS score of 4 or greater.8 The PBCL was strongly associated with this outcome, giving an area under the receiver operating characteristic curve of 0.95 (Fig. 1). A PBCL score of 1 was found to give the best prediction of this outcome, which yielded high levels of sensitivity (97%) and specificity (85%).
In the randomised trial, two raters (A.T. and R.E.) assessed data from 38 participants in 7 care homes over an extended period up to 1 year, providing a total of 407 monthly repeat assessments. In this study, a large proportion of scores on the PBCL (62%) was zero and to avoid spurious agreement the scores were divided into five categorical groups (0, 1–3, 4–6, 7–11 and ≥12). Using weighted kappa, the level of agreement was 0.91, with the 95% confidence interval ranging from 0.83 to 0.99. This high value indicates very good agreement between the two observers.14
The results suggest that the PBCL is a useful measure of challenging behaviour in people with intellectual disability. It has the advantages of simplicity, shortness and repeatability, and may be of particular use in longitudinal studies. It also appears to be a comprehensive measure even though its main use in these studies has been to assess aggressive challenging behaviour and so many of the factors have correlates with aggression. The high agreement between the MOAS and PBCL scores also adds construct validity to the scale as the MOAS is a frequently used measure in the assessment of challenging behaviour.15,16
Although the current work has been confined to people with intellectual disability it might well be extended to other populations with challenging behaviour (e.g. dementia, head injury), where direct questioning of participants may yield limited information. Its weaknesses are the relative absence of personal input by people with intellectual disability in scoring the scale. Although the high correlation between the PBCL and MOAS scales suggests that both scales are equivalent in recording challenging behaviour, the PBCL covers a broader range of items than the MOAS and so is more comprehensive. It needs further testing before the preferred populations for assessment can be chosen.
This research was funded by the National Coordinating Centre for Health Technology Assessment (NCCHTA) programme (project number 01/07/02), the Nicola Pigott Memorial Fund, and the National Institute for Health Research: Imperial Biomedical Research Centre. The views expressed in this publication are those of the authors and do not necessarily reflect those of the HTA programme, NIHR, NHS, or the Department of Health.
We thank Sandra O'Sullivan for her support and coordination during this study.
This scale is published under the CC BY-NC-ND licence. It may be freely used for non-commercial purposes; for example, it may be copied and used by individual clinicians. For commercial uses (including, but not restricted to, pharma studies), please contact.
Please assess each behaviour over the past week/month
Patient Code/Identifier: ________________ Date: _________________ Assessor: ______________
This is a hierarchical scale – the higher levels are assumed to contain all the elements beneath it, so if a person scores 4 on an item, the scores below that are disregarded. However, when the frequency or intensity of a behaviour becomes very great (e.g. repeated threatened violence or minor assault) to the point where it leads to major concerns to others the score may be raised by 1 (but no more). In deciding this please note the general requirements for the problem behaviour score at the top of the scale.
When making the assessment use as many informants as possible to cover all settings and observations.
- Received October 22, 2015.
- Accepted December 8, 2015.
- © 2016 The Royal College of Psychiatrists
This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) licence (http://creativecommons.org/licenses/by-nc-nd/4.0/).