In fact both the Minister for the NDIS Stuart Robert and the National Disability Insurance Agency (NDIA) have been pretty keen to point out just how much information has been released. On the NDIS website you can now find more frequently asked questions, an evaluation report and an open letter to all NDIS participants from the CEO of the NDIA Martin Hoffman.

There are also some technical documents about the assessment tools themselves. The documents look at the tools chosen in a bit more detail. A tick for transparency but both are pretty long and tough to plough through. Not for the faint hearted.

So lots of pages and lots of words. But do we really know anything more than we did before?

We’ve had a look at what has been released so far … and what we think it all means.

When is an evaluation report not an evaluation report?

Before we dive into the detail – just a quick recap.

Back in September the Minister for the NDIS Stuart Robert announced the introduction of compulsory assessments for all new and existing NDIS participants.

From February next year everyone applying for the NDIS will have to undergo one of these assessments.

Then from July existing participants will be required to have an assessment whenever they have a plan review.

The NDIA will pay an organisation (or organisations) to do the assessments. A tender went out back in March – well before the Minister made his announcement. The assessors will be allied health professionals such as speech therapists, occupational therapists or psychologists. They will use a set of standard tools chosen by the NDIA (more on the tools a bit later).

There was a very small pilot (or trial) of these assessments back at the end of 2018. A second pilot had just kicked off when COVID19 hit and had to be called off.

Up until now there has been very little information available about that first pilot. One of the questions we have all been emailing the Minister is why the results of that first trial had not been released.

Well, seems all those emails did make a difference. A couple of weeks ago the NDIA released a new report about the pilot.

We hesitate to call the report an “evaluation”. Because as becomes very clear when you flick through the pages, there actually hasn’t been a proper evaluation of how this is all going to work. In fact it’s clear this report has only been written recently – and not when the pilot took place.

So what does the “NDIS Independent Assessment Pilot” report say?

The first pilot was voluntary. About 500 people who lived in New South Wales volunteered to take part. Most (95%) were already participants in the scheme.

The trial was limited to three disability types – only people with an intellectual disability, psychosocial disability or autism were eligible to take part.

It was also heavily weighted to young people. Of the people who took part, 63% were aged 7-14 years old. About 24% were aged between 15 – 25 years old. That means only 14% were over 25 years old – that’s about 71 people.

The group was also not very diverse. More young men (71%) than women (29%) volunteered. Only 7% indicated they were from a Culturally or Linguistically Diverse background – and even less were from an Aboriginal or Torres Strait Islander background (1%).

According to the report, the objective of the pilot was to:

“demonstrate the potential benefit of independently sourcing standardised functional assessments for NDIS applicants and participants, to improve the consistency, accuracy and reliability of NDIA decisions”

So the purpose of the pilot was not to work out IF the use of assessments improved consistency. It was just assumed they did.

The assessors also only used four of the proposed tools. They used the Vineland, the Pediatric Evaluation of Disability Inventory Computer Adaptive test (PEDICAT), the Life Skills Profile and World Health Organisation Disability Assessment Schedule (WHODAS). They did not use the other proposed tools.

And most importantly – the final assessment reports were NOT used in the planning process. They were not used to create people’s plans or determine their funds. All that just happened the regular way.

After it was all over, the people who volunteered to take part were given a survey to complete. We don’t know what questions they were asked because a copy of the survey has not been released. But the few answers that were included in the report suggest it was a relatively simple satisfaction survey. For example – how satisfied were you with the length of the appointment? How satisfied were you that the assessor understood your disability?

Most importantly, only 145 out of 513 people completed the survey. In other words,  only 28% of people who took part said – or were willing to say – what they thought about the whole thing.

So in summary…

The vast majority of people in the pilot (95%) were existing participants. And yet the first cabs off the rank for this new process are prospective participants – people applying for the scheme. They are going to have to go through this assessment process from February next year.

So just to be clear – this enormous change is being made after it has been tested with about 25 newbies to the scheme.

The pilot only included three disability types. And of the three types the vast majority of participants were autistic (66%). Only 27% had an intellectual disability and even fewer (7%) had a psychosocial disability.

So just to be clear – this enormous change is being made after it has been tested with about 138 people with an intellectual disability. And about 36 people with a psychosocial disability.

And no one with any other type of disability.

The pilot was heavily weighted to young people. Again to be clear – this enormous change is being made after it has been tested with about 70 people over the age of 25 years old.

This trial was not evaluated. The most you could say is that a small group of the volunteers appear to have filled out a satisfaction survey. And even those that did fill out the survey were not happy with all aspects of the process.

The NDIA and the Minister have been quick to point out that many people were happy with the fact that the NDIA did the work for them and organised the appointments. What they haven’t been so quick to highlight is that only 72% felt the assessor understood their disability. That means 28% did not.

That’s a big group of people who didn’t think their assessor “got” their disability. And it’s these assessors whose opinions will ultimately determine what plan and budget people will receive. Whose future will be determined by a stroke of their pen.

And finally, and most importantly, we don’t know if these assessments did result in more consistent plans and budgets. Because that information has not been released.

All the report says is:

“Estimates on the impact of independent assessments (IAs) on plan budgets were also calculated where a participant had a completed plan by the end of the pilot.”

What this sentence suggests is that behind the scenes the NDIA came up with a likely plan and budget based on what the assessment report said.

We assume the NDIA then compared that pretend plan and budget with the real plan and budget the participant actually received.

But we say “assume” because no information about that part of the process has been released.

So in summary – the NDIA are saying the whole process did result in more accurate plans and budgets.

But haven’t released any evidence to show that. Or how they defined what “accurate” was. Accurate according to who? Based on what?

So again just to be clear – this enormous change is being introduced on a “trust us it made things better” basis.

Kickstarting a second pilot

So that’s what we know about the first pilot. This week the NDIA announced they are restarting the second pilot.

This pilot will be much bigger – 4,000 people. Again it will be voluntary. But this time the NDIA will be selecting people to participate. They are directly contacting people and asking them if they want to take part.

This pilot is clearly designed to try and fill in some of the gaps from the first. The NDIA have said, for example, that they are going to target people from a more diverse background, as well as different ages and disability types.

But once again there are many unanswered questions. What exactly is being tested? And will the results actually change anything that is being planned?

Or is it all just rolling out regardless?

So where does this leave us?

Let’s start with what we all agree on. The scheme does have a problem with both fairness and consistency.

It’s not fair that people have to pay gobsmacking amounts of money for evidence and reports to try and gain access to the scheme. Or wait for a long time on public waiting lists to try and get assessments done – and live without much needed support in the meantime.

And it is not fair that people who are better able to explain their circumstances and the impact their disability has on their daily lives are more likely to get an NDIS plan and budget that meets their needs.

The Quarterly Reports show, for example, that people from higher socio-economic areas do tend to get better plans and more funds.

But the $25 billion question is – will these new assessments actually fix the problem?

Because none of the material released actually answers that question. It just says – trust us, it did.

Nothing that has been released explains the all important HOW the assessments will fix the problem. Or WHY they were selected as the way to fix the problem – rather than other solutions.

Take applying for the scheme. People will still have to get other evidence of their disability as part of their application.  They will, for example, have to show that their disability is permanent. The new assessments won’t help with that.

And people who are better able to advocate for themselves or who have family members who can speak up for them are still likely to do better in the assessment process.

The NDIA have said, for example, that people with disability can have family members and others present during the assessments. They can even have their own health professionals present.

So once again that means that people who are better able to explain their circumstances and the impact of their disability on their daily life are more likely to get an assessment that provides a more complete picture of their lives. And if you have someone in your corner like a family member who can help you explain all the better for you.

And that’s before you even hit your planning meeting. Where, once again, if you have more reports, more evidence and know the magic words to say, you are more likely to get a plan and funds that meet your needs.

The proposed assessment process changes none of that.

But what it will do is create more hoops for you to jump through before you get to the same place you were before.

Is it any wonder that people with disability and their families are anxious/stressed/exhausted/just plain pissed off?

When is an insurance scheme not an insurance scheme?

The NDIS is supposed to be an insurance scheme. One of the features of the scheme is meant to be that its operation is supposed to guided by data and evidence.

But what the pilot report tells us is that the biggest change to the scheme is being introduced after one small voluntary trial that has not been rigorously evaluated.

A short satisfaction survey completed by a handful of people is not an evaluation. A proper evaluation would identify what problem you were trying to fix – and then measure how well you did in fixing the problem. And if you need to make any changes to make sure you really did fix the problem once and for all.

But that wasn’t what was done in the first pilot. And it is not what is being done in the second.

This is going ahead regardless of what these trials find.

And despite the mountain of words that have been released lately there are still so many important questions about how it is all going to work that remain unanswered.

This is the biggest change to the scheme since it began. And it is being rushed through without an evaluation of whether it fixes the problem – and over the concerns and objections of people with disability and their families.

Want to read more?

There is more information about the assessments on the NDIS website.

You can find information in Easy Read.

There is also a question and answer page.

You can read more about the Independent Assessment: Pilot learnings and ongoing evaluation plan here.

You can read more about the Independent Assessment Framework here.

You can also read more about the second pilot here.

You can read our take about what has been announced here.

And finally don’t miss your chance to have your say about this enormous change to the NDIS. It’s our scheme – we fought for it. And we should have a say in how it is being run.