Considerable work has been done over 2015-16 to develop an Outcomes Framework for measuring the outcomes of the support experienced by participants and their carers. The development of the Outcomes Framework considers how outcomes can be measured at a broader level of the Scheme, as well as at the individual level.
The development of this framework has involved consideration of a number of domestic and international frameworks, and has included consultations with a range of experts and stakeholders, including the Independent Advisory Council (IAC or the Council).
Once finalised, the Outcomes Framework will include eight specific areas of participant experience (known as participant domains) as well as outcomes related specifically to families.
The Outcomes Framework was piloted in the first three months of 2015. The results of the pilot, along with feedback from consultation with the disability sector, are being used to improve the framework.
Measuring outcomes of the Scheme for people with disability
A key aim of the Scheme is to assist people with disability to live ‘an ordinary life’. That is to fully realise their potential, to participate in and contribute to society, and to have a say in their own future—similar to other members of Australian society. The Scheme also involves families and carers, respecting their role whilst supporting them to achieve their goals by providing certainty of support for people with disability. These aims are embedded in the Scheme’s legislation. The Agency is responsible for measuring and reporting on the Scheme’s success against these aims.
The Scheme’s new Outcomes Framework has been developed to measure the medium and long-term benefits of the Scheme to participants, families and carers.
The Outcomes Framework is based on information from participants and families and carers on how they are progressing in different areas of their lives through a series of questions, or surveys.
The Outcomes Framework will allow tracking of participant and Scheme progress over time, and demonstrate how participants are faring relative to other Australians and other OECD countries. It will also contribute to an understanding of what types of supports lead to good outcomes for people with disability, their families and carers.
While the primary purpose of the pilot study of the Outcomes Framework was to assess the validity of the questions used in the surveys, some early trends were observed.
For example, the study showed that the Scheme is making a difference to the lives of participants and their families and carers, particularly in the areas of choice and control and daily living activities.
The Agency will also use the Outcomes Framework to identify areas that need targeted strategies to ensure the Scheme continues to deliver for people with disability.
Development of the Outcomes Framework
Development of the Outcomes Framework was guided by some basic principles, including that the indicators should be meaningful, informative, and feasible to collect and report.
A comprehensive review of other national and international frameworks was undertaken, together with an extensive consultation and co-design process that involved people with disability, IAC and CEO forum (CEOs of major providers, peak bodies and advocacy groups), disability researchers, experts working with specific cohorts (such as Aboriginal and Torres Strait Islanders and Culturally and Linguistically Diverse (CALD) communities), and intellectual disability experts.
Almost 400 interviews were conducted during the pilot study, and the information collected was used to refine the questionnaires and inform implementation strategies. Once refined, the questionnaires were called short form versions, and these were used for the pilot study of the Outcomes Framework.
An example of key results from the pilot study of the Outcomes Framework
Collection of the short form Outcomes Framework questionnaire for all participants commenced in November 2015 and by 26 June 2016, 13 290 questionnaires had been received for participants and 10 566 for family members or carers. These questionnaires are currently being analysed.
The graph following details key results from the 25 to 55 year age group analysis. The graph summarises responses to the final question from each domain. For example, for the adult choice and control domain, the final question was ‘Has the Scheme helped you have more choices and more control over your life?’ Possible responses were ‘It’s my first plan’, ‘Yes’ or ‘No’. Since the Scheme would not have had a real opportunity to help participants in their first plan, results show the percentage of respondents who thought the Scheme had helped, after excluding those who said they were in their first plan.
Participants aged 25 to 55
Responses to this question were most positive for Domain 1 (choice and control), where 76 per cent believed the Scheme had helped, followed by Domain 2 (daily living) and Domain 5 (health and wellbeing).
Figure 7: Example key result form pilot study Outcomes Framework. Participants aged 25-55 answering ‘Has the NDIS helped?’
A full report has been prepared detailing analysis of the responses to each question in the Outcomes Framework questionnaires and this can be viewed on the NDIS website (www.ndis.gov.au)