The typical pattern for intervention outcome studies for conduct problems has been for effect sizes to dissipate over time with decreasing effects across subsequent follow-ups.
To establish whether the short-term positive effects of a parenting programme are sustained longer term. To observe trends, and costs, in health and social service use after intervention.
Parents with children aged 36–59 months at risk of developing conduct disorder (n = 104) received intervention between baseline and first follow-up (6 months after baseline n = 86) in 11 Sure Start areas in North Wales. Follow-ups two (n = 82) and three (n = 79) occurred 12 and 18 months after baseline. Child problem behaviour and parenting skills were assessed via parent self-report and direct observation in the home.
The significant parent-reported improvements in primary measures of child behaviour, parent behaviour, parental stress and depression gained at follow-up one were maintained to follow-up three, as were improved observed child and parent behaviours. Overall, 63% of children made a minimum significant change (0.3 standard deviations) on the Eyberg Child Behavior Inventory problem scale between baseline and follow-up (using intention-to-treat data), 54% made a large change (0.8 standard deviations) and 39% made a very large change (1.5 standard deviations). Child contact with health and social services had reduced at follow-up three.
Early parent-based intervention reduced child antisocial behaviour and benefits were maintained, with reduced reliance on health and social service provision, over time.
Approximately 10% of children in the UK aged 5–15 years have a mental disorder, half of whom present with clinically significant conduct disorders.1 A UK study of children in disadvantaged areas estimated that 20% had a conduct disorder,2 and a study in the USA3 found that 35% of 500 preschool children from a deprived Head Start community were above the clinical cut-off point for conduct problems. These studies indicate that conduct disorder is more prevalent in children living in socially disadvantaged areas where there are higher frequencies of single-parent families, families with frequent changes of parental figures, parental substance misuse, psychopathology and marital problems.3 These factors enhance children’s risk of developing antisocial and criminal behaviour in adolescence and adulthood; however, their effects are mediated through their impact on the parent’s ability to parent effectively.5 Up to 40% of untreated children diagnosed with conduct disorder develop later problem behaviours including drug misuse, criminal and violent behaviour.4 There are severe financial implications for the individual, family and society through increased use of health, social, criminal justice and special education services if such childhood disorders are not prevented.6
The British government is now recognising the need to deal with antisocial behaviour by tackling the causes and is funding the delivery of evidence-based parent programmes. Behaviour-based parenting programmes, developed to improve parenting skills, are the most effective interventions for childhood conduct disorders with very little else having been shown to work.7 The Incredible Years BASIC parenting programme8 is empirically validated for both the prevention and treatment of conduct disorders, incorporating all factors identified as improving outcomes from parent training, particularly with high-risk children traditionally considered least likely to benefit.9 The programme improves parenting skills, child behaviour and parent–child interaction by promoting positive parenting.10 Furthermore, the programme addresses barriers to attendance for socially disadvantaged, hard to engage, families.11
Hutchings et al10 conducted a replication of the US Head Start study3 with disadvantaged parents of preschool children at risk of developing conduct disorder in Welsh Sure Start areas. Considerable effort was made to ensure that the parent programme was delivered with fidelity including the provision of all materials and regular, weekly supervision for group leaders. The statistically significant short-term results demonstrated enhanced parental reports of positive parenting behaviour, reduced parental stress and depression, less child problem behaviour and enhanced child social competence. Parental reports were corroborated by objective observations of parent–child interaction. In the field of parent training, short-term follow-ups are typical, and, on balance, parent-training interventions with families of children with conduct disorder appear positive.12 However, it is necessary to establish whether initial gains achieved by parent training are maintained over time, thereby representing real gains for families and society. Research has demonstrated that parent-training interventions are not always successful, especially in long-term follow-ups.12 Despite their importance, longer-term follow-up studies are rare with a median follow-up time across a large number of studies of 5 months.13 Long-term follow-ups are more likely to detect the more time-sensitive nature of behaviour change,14 demonstrate the real world value of interventions15 and establish for whom interventions are more effective over time.16 The typical pattern for intervention outcome studies has been for effect sizes to dissipate over time with maximum effects immediately post-intervention and decreasing effects across subsequent follow-ups.14 Having shown significant short-term effects we subsequently completed follow-ups to 18 months after baseline to test the stability of intervention outcome in the longer term. Based on past Incredible Years long-term outcomes from the USA3 we hypothesised that post-intervention: improvements in parenting skills, parental stress and depression will be maintained on a long-term basis; improvements in child problem behaviour and social competence will be maintained on a long-term basis; and that use of health, social and special education services will decline over time.
The study ran from 2002 to 2006 with follow-up three completed by April 2006. Eligible families were recruited over a period of 21 months between January 2003 and September 2004. This study reports on the 104 intervention families from the previously reported randomised controlled trial, which compared intervention v. waiting list control at baseline and follow-up one. Intervention took place between these two time points. The trial took place in 11 Sure Start areas across North and Mid-Wales.10 Ethical approval was granted by North West Wales research ethics committee (reference number 02/12). The trial registration number is: ISRCTN46984318. Families were eligible if they were disadvantaged, the primary carer was able to attend group times, the index child resided with the primary carer within a Sure Start area, was aged 3–4 years and scored above the clinical cut-off as rated by the parent on either the Eyberg Child Behavior Inventory17 (ECBI) problem (11) or intensity (127) scale (norms 6.9 (s.d. = 7.8) for problem and 103.8 (s.d. = 34.6) for intensity). For non-clinical children aged 2–7 (based on a cohort of 1526), 10.4% of children scored in the clinical range on the ECBI.18
Online Table DS1 presents demographic data highlighting the ‘at risk’ nature of the children for developing conduct disorders, for example, young/single parents, low income.
There were no significant differences between intervention and control groups at baseline. The waiting list control group were not followed past follow-up one as they were then offered the intervention. This paper reports on the long-term findings for the intervention condition only. The 18, 22 and 25 participants lost to follow-up one, two and three were included in an intention-to-treat analysis. There were no differences between the families retained in the study and those lost to follow-up on any measure (online Tables DS1 and DS2).
The Incredible Years BASIC parent programme was used. This was run by two trained leaders for one 2 to 2.5-hour session per week for 12 weeks, introducing a structured sequence of topics using a collaborative approach.10 All leaders had run at least one group previous to the study, received 3 h of weekly supervision from a certified trainer, and attained group leader accreditation during, or shortly after, completion of the study.11
Measures and procedures
The evaluation examined parent-completed questionnaires and ‘ masked’ observation of parent–child interactions during two separate home visits at each time point, baseline and three follow-ups (6, 12 and 18 months after baseline). The parent-reported measures of child problem behaviour include the primary outcome measure, the ECBI and secondary outcome measures: Strengths and Difficulties Questionnaire (SDQ),19 Conners Abbreviated Parent Rating Scale (CAPRS) 20 and Kendall Self-Control Rating Scale (SCRS).21
Parent-reported measures of parenting competencies, mood and demographics include the Parenting Stress Index – Short Form (PSI–SF),22 Arnold et al’s Parenting Scale23 and Beck Depression Inventory (BDI).24
In addition to the above parent-completed questionnaires, an objective observational measure was used, Dyadic Parent–Child Interaction Coding System (DPICS).25 The summary variables reported here are positive parenting (the sum of positive affect, physical positive, labelled and unlabelled praise and problem-solving behaviours) and critical parenting (the sum of negative commands and critical behaviours) and deviant child behaviours (the sum of cry, whine, yell, physical negative, smart talk and destructive behaviours). Observer interrater reliability levels were maintained above 70% through weekly training. Twenty per cent of all home observation visits were interrated with agreement levels exceeding 70%. Parents also completed a Client Service Inventory (CSI)26 for their child’s previous 6 months’ service use.
The study utilised a pragmatic randomised controlled trial design with a sample size calculation.10 After baseline assessment the first author masked and randomly block-randomised participants by area at a 2:1 ratio, intervention to control (waiting list) after stratification by gender and age of the index child, using a random number generator.10 A 2:1 design allows evaluation of a larger intervention sample than a 1:1 ratio, with only a small loss of statistical power and is the favoured design in this type of research.27 This paper examines intervention families only across all data points.
The researchers were initially masked to allocation.10 Only intervention families were followed after follow-up one. A fifth of all observation visits were carried out by two researchers to ensure interrater reliability. Researchers had to maintain 70% interrater reliability, calculated at weekly coder reliability meetings.
All families allocated to intervention (n = 104) were included in the analysis, irrespective of uptake of intervention; a further analyses was performed to only include outputs from parents and children for whom complete data-sets across all time points were available (a per-protocol analyses). The intention-to-treat analysis, in which those lost to follow-up were included (n = 25 from the intervention condition by follow-up three) assumed no change since the last available measurement for those lost to follow-up.27 Use of the last-observation-carried-forward rather than expectation-maximisation method of estimating missing values resulted in mean values at the three follow-up time points that were conservative estimates of the effect of the intervention. The fact that there was no difference in child behaviour at baseline between completers and those lost to the study by follow-up three also suggests this is the appropriate analysis. A linear mixed model command in SPSS version 13 for Mac was used to perform repeated measures analysis of variance (ANOVA) over time with no assumption of equal variances, using an unstructured covariance matrix, within-participants for time. All time points were compared with scores at follow-up one. A significant difference existed between baseline and follow-up one.10 If positive outcomes have been maintained there should not be a significant difference between follow-ups one and three. Effect sizes were calculated using Cohen’s guidelines.28
As previously analysed,29 service use was costed using national costs measured from a multi-agency public sector perspective30 (n = 72). Costs from 2003/04 were inflated to 2006/07 using hospital and community health service inflation indices.31 Costs at follow-up three were discounted at 3.5%.32 No cost-effectiveness analysis was carried out as a result of a lack of a control condition after follow-up one.
Of the 104 parents allocated to the intervention condition, 79 completed all three post-intervention assessments (Fig. 1). Mean session attendance did not differ for the intervention sample with complete data at follow-up one, two, and three (9.2 (s.d. = 3.2), 9.32 (s.d. = 3.04) and 9.4 (s.d. = 2.97) respectively). Eighty-five per cent of those who completed all follow-ups attended 7 or more of the 12 sessions, which is comparable to the 88% attending 6 or more sessions quoted by Webster-Stratton.3
Long-term effectiveness findings
Tables 1 and 2 and online Tables DS3 and DS4 show no statistical differences between follow-up one, two and three on the vast majority of measures in both the intention-to-treat and complete data per-protocol analyses. The improvements were maintained for the primary child outcome measure (ECBI) and secondary child outcome measures: CAPRS, SCRS, SDQ and observed child deviance. Similar findings were found for the parent outcome measures: observed positive and critical parenting, Parenting Scale, BDI and PSI. Very small effect sizes reflect these findings. The long-term findings were identical for both the intention-to-treat group and the per-protocol group, therefore demonstrating that the results are robust. However, there was a small, yet significant, negative change in scores for one measure, the Parenting Scale, between follow-up one and follow-up three whereby the mean scores worsened from 2.81 to 2.9 in the intention-to-treat analyses (Table 1).
Indices of change
Indices of significant change are given in Table 3 and online Table DS5. Using the value of 0.3 of one standard deviation as a minimum index of significant change, from baseline to follow-up three, two-thirds of the intention-to-treat sample (63%) made at least this modest change in ECBI problem behaviour. Using a stricter criterion of at least 0.8 standard deviation indicating a large effect size,28 just over half of the sample (54%) made at least this large change and over a third (39%) made a very large change (1.5 standard deviations). For those families who had complete data at every time point the findings are even more marked, with 77% demonstrating modest change, 68% large and 41% very large change. The effects were even greater for those over the clinical cut-off at baseline (n = 65/79) for the ECBI problem scale where 82%, 75% and 56% made moderate, large and very large improvements (eligible children were over the cut-off on either the problem or intensity scale). These findings are also reflected in all other child and parent outcomes (Table 3 and online Table DS5).
Long-term child health, social and special educational service use
Tables 4 and 5 show the frequency and associated cost of service use over time. This data is categorised as primary and secondary healthcare, social and special education services. Table 4 shows that by follow-up three total mean costs per child for all services, with the exception of special education, had fallen.
Improvements in child behaviour were maintained over time with no loss of effectiveness, as shown by non-significant differences between follow-up one and subsequent follow-ups, 12 and 18 months after baseline. Furthermore, there was a steady but modest decline in the frequency and associated costs of contacts with primary care and social services. This perhaps reflects a reduction in reliance on formal services. Parent intervention studies for children with conduct disorder offer long-term savings for health, social care, voluntary and education services;33 the vast majority of the cost savings accrue to the health service (37%). Scott et al34 calculated the cost of children with and without conduct problems as £ 5350 and £568 respectively (to compare with our study, these costs 1997/1998 were inflated to 2006/07 using hospital and community health services inflation indices); in our sample, costs for children with conduct problems were reduced to £1034 by follow-up three, showing that they may be on the path to becoming less frequent service users. The reduction in service use of our sample may be because of fewer child problem behaviours.
The increase in frequency and associated costs of contacts with special education services (e.g. extra classroom support) may reflect that over the follow-up period (even by follow-up one) children were entering and progressing through primary education. Children in the control group did not improve between baseline and follow-up one suggesting that there is no reason to expect improvement without intervention or that the intervention condition had simply regressed to the mean of the general population. These results therefore show that the positive benefits of intervention reported by parents were maintained in the longer term without additional public sector service support, and that these benefits were corroborated by direct observations. Parents were observed to retain the increased use of positive, and reduced negative, parenting behaviours that they demonstrated immediately post-intervention. These changes in parenting skills were accompanied by reduced child deviance and parental stress and depression. The one exception was the small but significant decrease in self-reported parenting competence between follow-up one and follow-up three. However, this was not substantiated by objective observations of parenting skills, or reported or observed child behaviour.
Strengths and weaknesses
This long-term study has demonstrated the stability of positive intervention outcome for families within this disadvantaged, at-risk, population, using both self-report and objective observational data. The relatively low drop-out rate and high mean attendance reflects the effectiveness of the intervention in engaging typically hard-to-reach families. Similar findings in both the intention-to-treat and per-protocol analysis offer reassurance about the real world significance of the results. The fact that only 18% of the children in the intervention group above the ECBI problem scale cut-off at baseline did not demonstrate a small improvement at follow-up three highlights that the positive effects of the Incredible Years programme are maintained over time for many children and families who are in greatest need. Up to 40% of untreated children diagnosed with conduct disorders develop later problem behaviours including drug misuse, criminal and violent behaviour.4 The lack of long-term control data is a weakness of this, and most longitudinal intervention studies, but given the wealth of evidence for the effectiveness of such programmes, and the long-term risks for untreated children, it is unethical to withhold intervention from families in need. An additional weakness of this study is that observers were not masked to condition after follow-up one, although the results of this trial show remarkable stability between findings at follow-up one through to three. Despite the long follow-up period in this trial, further follow-up is needed to examine the important influence of transition into school on children’s behaviour.
Comparison with other studies
This early preventive study demonstrates comparable results to samples of referred children in USA and UK long-term parenting outcome trials,16,34 highlighting the stability of this evidence-based parenting programme in the longer term with disadvantaged, at-risk, multicentre community samples.
Meaning and implications of the study
The stability of the intervention in the longer term, with reduced reliance on health and social services, holds important lessons for UK and other governments. The use of an evidence-based programme delivered with serious attention to fidelity11 achieves remarkable long-term outcomes with a high-risk population. These results are in stark contrast to the Sure Start initiative in England, whereby services initially used a variety of programmes (some without an evidence-base), delivered without fidelity checks, without rigorous evaluation and that were generally ineffective and detrimental for those particularly disadvantaged children.35
The early results of the Welsh Sure Start study10 prompted the Welsh Assembly Government to fund training in the Incredible Years programme as part of its Parenting Action Plan36 and it is hoped that other national governments will follow suit. It is important for governments to commission effective programmes for high-risk children with conduct problems that have been demonstrated to work longer term and to ensure that the resources are provided for them to be delivered with fidelity making the programmes likely to be good value for money.
Questions and future research
Future research should focus on examining the impact of early intervention using the Incredible Years programme on subsequent longer-term child development, including tracking progress through early school years, middle childhood and adolescence. The combination of the Incredible Years parent intervention with the Incredible Years child and classroom intervention programmes delivered to nursery nurses, teachers or directly to the child could potentially further enhance the long-term benefits.37
Research grant from the Health Foundation, grant number 1583/1566. The funders had no involvement in the writing of this paper.
We thank all participating families, Sure Start staff, group leaders, and research and intervention staff for their help.
- Received July 3, 2008.
- Revision received March 14, 2009.
- Accepted March 24, 2009.
- © 2009 Royal College of Psychiatrists