Evidence used in this report
This report sets out the Care Quality Commission’s (CQC’s) assessment of the state of health care and adult social care in England in 2023/24.
In this report, we use evidence from our inspection and registration activities, along with other information and personal experiences, including from people who use services, their families and carers, to inform our view of the quality of care.
We have collected bespoke qualitative evidence to supplement our findings. This has enabled us to gather views from our inspection and assessment teams and subject matter experts on quality issues for specific types of health and social care services.
This report is also based on our wider horizon scanning activity. We have reviewed reports published by our stakeholders, drawn on findings from national surveys, and analysed publicly available datasets to supplement our understanding of the challenges facing health and social care today and the experiences of people using services. Where we have used data from other sources, these are referenced within the report. To aid readability, we have rounded many of these figures so they may not match exactly with the published source.
To ensure that the report represents what we are seeing in our regulatory activity, analytical findings have been corroborated, and in some cases supplemented, with expert input from our Chief Inspectors, colleagues in our Regulatory Leadership directorate, specialist advisers, analysts and subject matter experts. Our analysis has also been supplemented with expert input from our Clinical Fellows, Katherine Bowman, Zahra Yasen, Saadiq Moledina and Martha Martin.
Where we have used other data, we reference this in the report.
Here, we provide further detail relating to the evidence used in this report.
People’s experiences and what they have told us
Our view of quality and safety has been informed by information that people have shared with us through our online Give feedback on care service, phone calls to our Customer Service Centre and social media. People’s experiences and comments submitted through Give feedback on care enabled us to focus on the following areas:
- To deepen our understanding of the experiences of people who have accessed services for cancer, we carried out a thematic analysis of a sample of comments spanning the period 1 April 2023 to 31 March 2024. The sample was selected by searching all submissions in this period using 45 key words relating to cancer illnesses and treatments. This resulted in a total of 2,169 submissions. From this total, we analysed the data until no further significant themes emerged, a process called saturation. In total, we fully analysed 417 comments, coming from people using services, their family, friends and carers, and people working in services.
- To explore how people are accessing GP services and the quality of the care they receive, among other issues, we analysed 343 comments. These experiences were all shared between 1 April 2023 and 31 March 2024. A total of 3,420 comments were selected from this period, and all related to our programme of inspections looking at the responsiveness of NHS GP services, which ran from 30 October 2023 to 31 March 2024. We selected a sample of 10% of these comments (343) to represent all geographical regions in England in a proportionate way.
- To understand the experiences of people who are entitled to free NHS dental care, we thematically analysed 563 comments from the period 1 April 2023 to 31 March 2024. For the purposes of this research, we defined people who are entitled to free NHS dental care as: children and young people aged under 18 years or under 19 years and in full-time education, people on low income or receiving benefits, and women and birthing people who were pregnant or had given birth within the past year. We selected comments by searching the data for 12 key terms that would allow us to identify relevant respondents.
- To represent the experiences of autistic people when engaging with mental health services, we thematically analysed 212 comments using a key search term method to identify those that were relevant. As there were only a small number of relevant comments in the year from 1 April 2023 to 31 March 2024, the timeframe was expanded to include the previous year, meaning the analysis covers the period from 1 April 2022 to 31 March 2024.
When people share their experiences through our Give feedback on care service they are consenting to giving us their feedback. This process means that we are told about the many different issues that people face as they interact with health and social care services in England. We tend to receive more negative comments than positive comments from the public. Positive comments are often less detailed than negative comments, which can mean that these experiences are more generic. People often include more detail in their negative feedback to us, which means we can say more about these experiences.
As in previous years, we have used findings from our published surveys to better understand what people think about the NHS services they use. Furthermore, we have used evidence from bespoke surveys and research projects that we commissioned or developed ourselves.
We used free text responses to the 2022 NHS adult inpatient survey to better understand the impact of waiting for care on people’s health and wellbeing. We selected 945 respondents and analysed their responses using thematic analysis. We only sampled respondents who had said their health deteriorated while waiting for care. Approximately half of the respondents were from an ethnic minority background. This was to ensure that the sample highlighted the voices of people in these groups.
We used free text responses to the 2023 NHS community mental health survey to gather insights into the experiences of people accessing these services. We focused on 3 different groups: the general population, children and young people aged 16 and 17, and Black men (including men from a mixed Black ethnic background). For all 3 groups, we used thematic analysis to understand their concerns and issues, which involved:
- a random sample of 667 respondents from the general population
- all 575 respondents aged 16 and 17 who had answered at least one free text question
- 195 Black male respondents who fitted our selection criteria.
To more fully understand people’s experiences across a range of sectors, we commissioned market research company OnePoll and worked with them to produce 7 surveys:
- 3 related surveys explored people’s experiences of medical and dental services, as well as specific issues while trying to access services. These were sent to:
- 2,000 adults from the general population
- 1,000 people with caring responsibilities
- 1,000 parents.
- A survey focused solely on dental services was sent to 1,000 adults to look further at issues with access to dental services, and the impact on people if they couldn’t access a dentist.
- With OnePoll, we surveyed 1,000 parents and carers of children and young people. This was to learn more about the experiences of parents and carers who raised a concern about their child to a healthcare provider during the period April 2023 to March 2024. We analysed open-ended responses from 99 of these participants using thematic analysis.
- To understand the impact of closures of community and retail pharmacy on people’s health and wellbeing, we asked 575 adults about their experiences of using community pharmacies.
- We asked 720 adults about their experiences of intermediate care.
Where we highlight different responses to the same question from different groups, these are presented as descriptive statistics, without statistical significance testing.
We also commissioned Aqua, an NHS organisation that promotes improvement in health and quality of care, to gather case studies for us. Participants were selected from people who had responded to the NHS Community mental health and Adult inpatient surveys and who confirmed they were willing to participate in further research. The case studies focused on the experiences of children and young people, and of Black men using mental health services throughout their care pathway. Case studies were also developed for adult inpatients who had reported that their health deteriorated while waiting for care.
Providers of health and care services
We have used the data and insight gained through our routine monitoring of and engagement with providers, for example information collected through our surveys and our data from statutory notifications received.
To identify common themes in urgent and emergency care services, we analysed a sample of 10 inspection reports that had been completed during 2023/2024.
To identify indicative common themes and examples of what good quality care in adult social care looks like, we analysed a sample of 10 inspection reports across different types of adult social care locations that had been rated as outstanding in 2023/24.
To help us understand how adult social care providers are responding to challenges and improving services, we qualitatively analysed information submitted through provider information returns (PIRs). We ask adult social care providers to submit a PIR form every year. As well as helping to monitor the quality of care, the PIR gathers qualitative data through a set of 12 open-ended questions, asking the provider about any changes that have been made in the past 12 months and how they are ensuring their service is safe, effective, caring, responsive and well-led. In this analysis, we used PIRs submitted between 1 April 2023 and 31 March 2024. We analysed a random sample of 110 PIR documents, using a mix of adult social care providers, including both residential and community services.
Quantitative data from PIRs were also used to estimate staff vacancy, turnover and bed occupancy rates in residential adult social care providers between 1 April 2021 and 30 April 2024. Additionally, this year we have included experimental analysis on the use of restraint and restriction in residential care homes, with a particular focus on residents who are autistic or have a learning disability. As providers are required to submit one return every 12 months, each monthly cut of data represents only a subset of providers and is not necessarily representative of the whole sector. Data are collected through an online self-completion questionnaire and, while we have applied cleaning rules as part of our analysis, the returns are not validated and may be subject to data quality issues.
This report also provides an analysis of data submitted to us by providers in our Market Oversight scheme, as well as information and insight gained from our engagement with providers that participate in the scheme. The scheme covers providers with a large local or regional presence which, if they were to fail, could disrupt continuity of care in a local authority area.
We have conducted quantitative analysis of our inspection ratings of more than 32,000 services and providers. Aggregated ratings for the main sectors and services we regulate are provided in the data appendix of this report. These are a snapshot of the latest ratings for all services that were active or registered on 1 August 2024.
In March 2020, we paused routine inspections and focused our activity where there was a risk to people’s safety. Last year, we noted that we had also started to carry out inspections in low-risk services to quality-assure our risk identification process. From January 2024, we started to implement a new approach to assessment using the single assessment framework. The number of published assessments and ratings using the single framework is still small, so we only present ratings awarded under our previous frameworks this year. As we begin to publish more ratings under the new approach, and continue to improve and develop this, we will begin to include these ratings in our analysis of aggregated ratings.
As we are in a transition period with ratings, we will be unable to make comparisons with those from previous years.
Quantitative analysis of inspection ratings includes information on the proportion of services that are categorised as having ‘insufficient evidence to rate’. This rating can be used when, on inspection, we have not been able to collect enough information to give a rating. Charts in our data appendix visualise the proportion of all active services with a current rating of either outstanding, good, requires improvement or inadequate. We indicate the proportion of services where there was insufficient evidence to rate in a note below the chart, where applicable.
Earlier this year, we convened an event for senior leaders and specialists from across England to understand their local experiences around congestion in urgent and emergency care services – and the impact for providers and people using these services. The findings are summarised in the report.
We worked with the Strategy Unit (hosted by NHS Midlands and Lancashire), a specialist NHS analytical team, to understand the scale of people attending emergency departments (A&E) because of a mental health crisis and whether this highlighted any inequality.
To do this, the Strategy Unit analysed patient-level data in the Emergency Care Dataset (ECDS), NHS 111 dataset and the Mental Health Services Data Set (MHSDS). They looked at attendance and call rates, and how these varied by characteristics such as age, gender, ethnicity and deprivation. They also examined the characteristics of people’s contact with the service, such as the time and mode of arrival, their presenting mental health condition and whether they were already known to mental health services.
(For our own analysis of attendances in emergency (A&E) departments, we used published data from NHS England. These use different data sources (MSiteAE and ECDS), which leads to some differences in figures.)
Local health and care systems
Working with the Nuffield Trust, we surveyed integrated care systems (ICSs) to understand what integrated care boards (ICBs) perceive as the main challenges or barriers to addressing inequalities in health care.
The Nuffield survey was for people with responsibilities for addressing health inequalities in their ICSs. Responses covered 23 individual ICSs (55% of all 42 in England). Several ICSs submitted responses from people across a range of different roles, including chief executives, chief medical officers, directors of population health, public health, and strategy, and from health inequalities leads, GP and clinical leads, pharmacists, and one patient.
With such small numbers involved, it is difficult to tell whether these answers represent a broader picture. Therefore, the results should be interpreted with caution as small changes to answers will significantly change the proportions. More in-depth work with people involved in different health inequalities roles would be important to further understand these early perspectives.
To help us understand approaches of regional and local health systems to service improvement and its impact on health, a group of CQC operational colleagues carried out a desktop review of key documents. They reviewed:
- joint forward plans for 2023/24 and 2024/25
- ICS strategy documents
- Health and Wellbeing board strategies, plans and case studies.
Last year, we formed an initial view of the performance of local authorities regarding their adult social care duties by carrying out a desktop assessment of selected publicly available information for all 153 local authorities in England. This year, we have begun baseline assessments of local authorities and have used findings from 11 such assessments to inform our view of local authorities’ performance against their statutory duties.
Statutory responsibilities
We report on our own data for notifications of the outcome of an application to deprive a person of their liberty under Deprivation of Liberty Safeguards (DoLS) received between 1 April 2023 and 31 March 2024. This excludes Court of Protection applications and notifications from primary medical services, but due to changes in our systems, we cannot exclude Court of Protection from other DoLS notifications for the last 6 months of the reporting year (this number is small, typically less than 1,000 in a 6-month period). We also report on the annual data publication for DoLS from NHS England published on 22 August 2024.
To gather insight, we conducted a survey and a focus group with our inspectors.
In June and July 2024, we conducted a survey of the National DoLS Leads Network. This was to learn more about local authorities’ experiences as supervisory bodies and the ongoing challenges they face in this area. A thematic analysis of the 51 responses aimed to identify common experiences and trends across local authorities.
During June 2024, we conducted a survey of an external stakeholder group comprising:
- people with professional experience in caring for and supporting people who have been deprived of their liberty under DoLS
- people with a special interest in DoLS in a personal or professional capacity
- people who had cared for or supported friends or family who have been deprived of their liberty under DoLS.
Participants answered questions relating to their experience of the operation of DoLS across the period 1 April 2023 to 31 March 2024. We analysed responses from 46 people to understand themes and trends.
Evidence in this report, alongside our annual report and accounts, enables us to fulfil our legal duties to report on equality issues and on the operation of DoLS.