Evaluation of CQC's local authority pilot assessments

The local authorities were asked to complete a local authority information return within a set timeframe ahead of the fieldwork. They were also given the option of completing a self-assessment that they could share with CQC.

Challenges in completing the local authority information return

There were a number of points of feedback about the information return from local authorities. These were mainly in relation to the time it took to complete the return and a lack of clarity about what was required. We heard that most local authorities requested more time in which to complete the return as it became a very time-consuming process. It was often described as intense and more work than they expected. One local authority described the nature of the process they undertook in completing the return:

It did take us quite a bit of resource and time to pull together and also putting in those governance steps at our end to make sure you know that senior managers were happy with what we suggested. So we'd have regular meetings really, to review what information we've got, whether that met the brief and then have that sign-off procedure in there.

Even the local authorities that said they had existing good systems for collating documentation found the information return process a lot of work. However, most were quick to add that they will now be much better prepared for the next time.

Although certain asks in the return were very clear, such as policies that all local authorities hold, the time it took local authorities to complete the return was compounded by other asks that were open to interpretation. One example included ‘feedback people have sent the local authority’, which some felt was a very broad description. In these instances, some local authorities described providing a wide range of documents in the hope that something would be right, others spoke of asking CQC for more clarity.

Other issues included unclear or outdated language used to describe certain topics or documents, acknowledging that this is made more challenging by local authorities not always using consistent terminology themselves.

Some local authorities spoke of how certain documents would not ‘stand up’ alone, and it was therefore necessary to include contextual information to support them, adding to the volume of documents and time taken to complete it. One local authority described how some policies tended to be underpinned by lower-level practice guidance, using their governance framework as an example, and not knowing whether to include these additional layers.

Local authorities also found it difficult because they had not been given a clear sense of how much evidence was enough, with one local authority adding that they:

wanted to shout loud and proud and had no real sense of what the optimum volume would look like.

One local authority described how it tackled this by submitting what it called ‘document bank A’ and holding back ‘document bank B’ that contained further information if CQC requested it.

Some local authorities also reported some technical issues with uploading the information to the portal. Not all reported these issues, but where it was a problem, this added a lot of additional time to the process. Some of this was thought to be glitches with a new system, but some of it may have been alleviated if there had been more guidance on using the system.

A positive exercise for local authorities

Despite these issues, we heard that the local authorities found the process of completing the information return helpful for their local authority. One explained that as a result it meant "we definitely know ourselves better." We also know that it was a learning exercise for some local authorities as it spotlighted areas for improvements. Some also described how they will be maintaining documents in this way from now on to make compiling this in the future much easier.

Some local authorities also described how other similar work helped prepare them for the information return and the self-assessment process, including sector-led improvement work that was already in progress and having already undertaken other types of self-assessments recently.

The impact of the local authority information return on the assessment team

All returns and self-assessments were received in advance of fieldwork, although some local authorities had requested additional time to complete them. For the assessment team, the information return process became unwieldly due to the volume of the returns. This needed more time to be reviewed, which reduced time for the information to inform the direction of fieldwork as much as was intended. Some of the team described the onerous nature of the process in having to review excessively long documents to summarise them into a paragraph for the evidence grid. This was compounded by the vast volumes of documentation from most of the local authorities, some of which included blank templates and duplicative documentation. One of the assessment leads described using the return much more when report writing than in the planning, partly because they’d not had time to fully use it earlier, but also because it proved just as helpful at this stage too.

Improving the local authority information return

CQC team members were clear that the current return process would not be sustainable, and that the guidance needs to go much further in providing detail of what is needed, as well as why and how that is best supplied.

For the local authorities involved in the pilots, a good starting point for some would be to receive feedback from CQC about whether they had supplied the information needed, at the right level and volume. Some were also keen to understand more about how the information was used in the assessment, although they anticipate some of this may become apparent in the assessment report.

Suggestions from CQC and the local authorities to develop the guidance and improve the overall process in future included:

• Guidance on ‘what a good local authority information return looks like’. This should give a sense of volume of the return, whether policies should be adult social care or organisation-wide, and technical matters such as whether links or embedded documents are acceptable.
• More clarity on the purpose of documentation requests to enable local authorities to be clearer about the level of information to provide, and whether to add context.
• Requesting a very core set of information initially, and only requesting wider information if necessary.

Self-assessments

Self-assessments were an optional part of the return, but we understand all the local authorities had undertaken some form of self-assessment in preparation. They all reported this as a broadly helpful exercise. One local authority described being surprised that it was an option to complete a self-assessment. They felt it should be a requirement of the process as it was both helpful for them to complete and for CQC in setting the scene about their local authority. One local authority commented on the range of self-assessment tools available to them and how this had been confusing at the time. They felt there could be some guidance for local authorities on the different tools available to support them and which to use.