It should never be assumed a program is working, no matter how great it seems. Programs should commit to measurable outcomes for the people they serve and social impact targets for the communities in which they operate. The purpose of programs is not to conduct activities. It’s to close the gap.
by Nyunggai Warren Mundine, Executive Director of the Australian Indigenous Chamber of Commerce
“I know this great program that’s making a difference”. It’s a phrase you hear a lot in Indigenous affairs. When people say this to me what I say is – how do you know it’s a great program? If they answer by telling me about the first rate activities and resources the program delivers, for me that’s a red flag.
Here’s a fictional example to illustrate. Imagine a program where experienced teachers from major cities are seconded to work in Aboriginal communities to teach children to read using a proven literacy method. The teachers are committed and passionate and use state of the art learning aids. This certainly sounds great. But the only measure of whether it works is whether children do in fact learn to read. If literacy rates don’t improve, if only a small number of children learn to read or if the reading levels attained are still below where they should be, then the program isn’t working.
There are various reasons why programs are ineffective when it feels like they should work. In my example, perhaps children didn’t turn up consistently. Or perhaps the method didn’t work for these children. Whatever the reason, the program in its original design shouldn’t be re-funded.
But just imagine if government cut funding to this imaginary literacy program. There’d be a chorus of complaint about funding cuts to this great program that sends top teachers to the bush to teach poor Indigenous kids. Listen carefully though – a complaint like this refers to the activities and resources being withdrawn, not the results the program has achieved.
When programs that sound great lose funding there are howls of outrage. Twitter is flooded with dire warnings of future crises in Indigenous communities. But in reality the crisis already exists, even amidst these wonderful programs.
I once saw a copy of a grant application for an impoverished remote Aboriginal community. The applicant provided accredited training in things like home care, food preparation and hygiene. As evidence of past successes, the applicant said its training completion rate was higher than the national average (but it was still less than 50%), it had won industry awards and had provided training in this community before. None of these are evidence of success. It doesn’t matter how many people complete training. What matters is whether what they learned improves health and wellbeing. Given that this community was still living in squalor with major environmental health problems, the fact the applicant had previously provided training in this community was, if anything, a sign the program made no difference whatsoever.
There are thousands of Indigenous programs and initiatives throughout Australia funded by government, charities and the corporate sector. Some have been operating for years at huge expense.
Many undertake great activities and provide exceptional resources. But what’s important is the difference they’ve made. What does the data say about the communities in which they operate? Have there been improvements in school attendance, retention rates and NAPLAN results? What are the statistics on incarceration rates, recidivism, police callouts and crime? What percentage of adults are now employed in a real job (not work for the dole)? Have a good look at the data and how it has moved during the operation of the program. Is the gap actually closing?
I’m not interested in the activities or resources a program delivers. I’m interested in the outcomes it achieves. Real outcomes like people getting a job, children and adults earning to read and communities becoming demonstrably safer.
I also care about cost effectiveness. Programs that are very expensive but only deliver results for a small number of people need to be reviewed. If the same result can be delivered for less money in a different way that frees up funding to reach more people.
It’s not only governments and charities that should ask these questions. The corporate sector is extremely generous in funding initiatives to break disadvantage and close the gap. They too should be looking at whether they are funding outcomes or just activities. Big companies do this in their own business operations. They measure project delivery against pre-agreed success criteria and audit their operations and spending. They should require the same for the initiatives they fund under the banner of corporate social responsibility.
It should never be taken as a given that a program is making a positive social impact, no matter how great it seems. Programs should commit to measurable outcomes for the people they serve and social impact targets for the communities in which they operate. And be measured against those. Because the purpose of programs is not to deliver activities and resources. The purpose is to close the gap.
An edited version of this article appeared in The Australian on 20th October 2014.