Health Survey England Additional Analyses, Ethnicity and Health, 2011-2019 Experimental statistics

This document provides information about the data quality of the Health Survey for England: Health and Ethnicity, 2011-2019 report.  The report contains new analysis using data from the Health Surveys for England that were carried out during 2011- 2019.   

The Health Survey for England series was designed to monitor trends in the health, and health related behaviours, of adults and children in England. The survey is used to estimate the proportion of people in England who have health conditions, and the prevalence of risk factors and behaviours associated with certain health conditions. The surveys provide regular information that cannot be obtained from other sources. The surveys have been carried out since 1994 by the Joint Health Surveys Unit of NatCen Social Research and the Research Department of Epidemiology and Public Health at UCL.

Data collection comprised an interview, followed by a visit from a specially trained nurse for all those who agreed. The nurse visit included additional questions, measurements, and collection of blood samples from adults.

Reports about the Health Surveys for England from 2004 onwards are available.  The annual reports contain statistics on frequent and ad-hoc topics from the surveys, methodological information and the questions used in the survey in that year included. Not all questions included in the survey are reported on in the annual reports because the survey is large.  A summary of the topics included within the survey questionnaires  in each year is available. 

This document aims to provide users with an assessment of the quality of the statistical output included in the Health Survey for England: Health and Ethnicity, 2011-2019 report. The report is classified as experimental statistics because they are new and under evaluation for how well they meet users’ needs. Feedback is welcomed.

This document reports against the nine European Statistical System (ESS) quality dimensions and principles appropriate to this output.  The original quality dimensions are: relevance, accuracy and reliability, timeliness and punctuality, accessibility and clarity, and coherence and comparability; these are set out in Eurostat Statistical Law.

However more recent quality guidance from Eurostat includes some additional quality principles on: output, quality trade-offs, user needs and perceptions, performance cost and respondent burden, and confidentiality, transparency and security. In doing so, this meets NHS Digital’s obligation to comply with the United Kingdom Statistics Authority (UKSA)  Code of Practice for Statistics, and the following principles in particular:

• Trustworthiness pillar, principle 3: Orderly release which states “Organisations should commit to releasing their statistics in an open and transparent manner that promotes public confidence.”
• Trustworthiness pillar, principle 6 (Data governance) which states “Organisations should look after people’s information securely and manage data in ways that are consistent with relevant legislation and serve the public good.”
• Quality pillar, principle 3 (Assured Quality) which states “Producers of statistics and data should explain clearly how they assure themselves that statistics and data are accurate, reliable, coherent and timely.”
• Value pillar, principle 1 (Relevance to Users) which states “Users of statistics and data should be at the centre of statistical production; their needs should be understood, their views sought and acted upon, and their use of statistics supported.”
• Value pillar, principle 2 (Accessibility) which states “Statistics and data should be equally available to all, not given to some people before others. They should be published at a sufficient level of detail and remain publicly available.”

This dimension covers the degree to which the statistical product meets user needs in both coverage and content.

In the last decade, only limited analysis of the Health Survey for England data by ethnicity has been published by NHS Digital, covering a few topics such as obesity, Body Mass Index and diabetes risk, and alcohol has been done. These looked at five higher level ethnic groups, (white, black, Asian, mixed and Other) and merged three years of survey data together to get sufficient numbers for analysis.  Prior to that, there have been two boosted samples to get larger numbers from those in ethnic minority groups, however the last of these was in 2004 Health Survey for England - 2004, Health of Ethnic Minorities, Main report - NHS Digital which looked at seven groups.

There is interest in whether health varies by ethnicity and NHS Digital have been asked by users if our Health Survey for England reports contain data for different ethnic groups. NHS Digital commissioned NatCen and UCL to produce a report combining data sets from recent years that would examine the health status and health-related behaviours of adults by ethnicity and so provide data for minority ethnic groups.

The Health Survey for England contains question on ethnicity at the first interview, using the ONS harmonised question and 2011 Census categories. Respondents self-classify their ethnicity. There are 18 groups:

White British

White Irish

Gypsy, Irish traveller

Other White backgrounds

White and Black Caribbean

White and Black African

White and Asian

Any other Mixed or Multiple ethnic background

Black African

Black Caribbean

Other Black backgrounds

Bangladeshi

Indian

Pakistani

Chinese

Other Asian backgrounds

Arab

Any other background

The groups used in the report analysis were chosen to reflect two principles:

• As far as possible, specific ethnicity categories should be shown, rather than the combined five higher level groupings.
• The tables and report should use the same categories throughout.

There were insufficient responses from individuals in some groups to produce adequately robust analyses. Confidence intervals around the estimates would have been much too wide. Some of the 2011 census categories were combined and in this report eleven groups are used for analysis:

1. white British
2. white Irish
3. Other white backgrounds (including Gypsy or Irish Traveller)
4. Mixed and multiple backgrounds (including white and black Caribbean, white and black African, white and Asian, and any other mixed or multiple backgrounds)
5. Indian
6. Pakistani
7. Bangladeshi
8. Chinese
9. black African
10. black Caribbean
11. Other backgrounds (including any other Asian background, any other black background, Arab, and any other ethnic group).

This is almost the same as the groups in  Coronavirus (COVID-19) case rates by socio-demographic characteristics, England - Office for National Statistics (ons.gov.uk) which had ten groups. The difference being that ONS included white Irish within the ‘Any other white group’ whereas in this report they have been identified separately because the number in the sample was not too small and because the Health Survey for England 2004, Health of Ethnic Minorities report included white Irish as a distinct group.

Combining groups in this way has also simplified the charts.

The Health Survey for England series includes core questions and measurements (such as blood pressure, height and weight and analysis of blood and saliva samples), as well as some modules of questions that are on specific topics that vary from year to year. Comparability over time is maintained by using the same questions. If changes are made, these are carefully assessed for possible implications on time series data.

Age, sex and ethnicity questions are asked every year. Many question topics have been included each year since 2011 and some others have been asked less frequently each every two, three or four years. Topics included within this report focused on those questions asked every year or in enough years to provide adequate estimates and of interest to the steering group. The topics covered are

• height, weight, BMI (body mass index)
• smoking
• alcohol
• fruit and vegetable consumption
• self-assessed general health
• long-standing illness
• prescribed medicine use
• blood pressure and hypertension
• diabetes
• Mental well-being using the Warwick Edinburgh Mental Well-being Scale

This dimension covers, with respect to the statistics, their proximity between an estimate and the unknown true value.

Accuracy of the statistics: estimating and reporting uncertainty

The combined sample for whom there is ethnicity information available between 2011 and 2019 is 96,036 people aged 16 and over. The majority classified themselves as White British. The household response rates for the surveys over the period 2011-2019 have ranged from 66% to 59%. Within these households, between 84% and 87% of adults took part each year.  Information on the response rates for each year is in the appendices of this report. More details of the sample design and response rates for each year can be found in Methods reports in the annual survey reports via Health Survey for England - NHS Digital.

The figures in the report come from surveys, which gather information from a sample rather than from the whole population. The sample is designed to be as accurate as possible given practical limitations such as time and cost constraints. Results from sample surveys are always estimates, not precise figures. As the number of people available in the sample gets smaller, the variability of the estimates that we can make from that sample size gets larger. Estimates for small groups are less reliable and tend to be more volatile than for larger aggregated groups. This issue applies to the analysis in this report and the estimates are subject to sampling error. The Health Surveys for England use a clustered, stratified multi-stage sample design and weights were applied when obtaining survey estimates. The calculation of standard errors, confidence intervals and statistical significance testing take into account the clustering, stratification, and weighting of the data.

Age-standardised data is provided as well as observed prevalence. Comparisons between groups within the text are based on age-standardised estimates to allow for the differences in age profiles of individual groups. This is because Health and health-related behaviours are strongly related to age and there are markedly different age profiles across the ethnic groups. The profile of the sample by ethnicity, age, household income, area deprivation and whether they live in urban or rural areas is described in the report. 

Age-standardised estimates for individual groups, including the confidence intervals, are shown in the charts. Despite aggregating up to nine years of data for the data tables the number of people surveyed for some ethnic groups reported here are small. This means that the confidence intervals around some estimates are relatively wide.

Where the text states that estimates of prevalence vary between groups, this means that there is variation by ethnicity overall, and that this is statistically significant at the 95% confidence level. In other words, there is a 95% certainty that the variation across all groups is real, and not just within the margins of sampling error. This does not describe differences between individual groups. Where individual groups are identified within the text as having high or low levels of prevalence, this reflects the overall pattern of difference, but the estimate for a single group may not be significantly different from estimates for other groups if the confidence intervals overlap. The p-values for the statistical significance testing are reported. The report includes tables showing confidence intervals and standard errors for measures of age-standardised prevalence for each ethnic group where statistically significant differences are commented on in the report. This enables readers to do their own statistical comparison tests between specific groups.

The samples were designed to be representative of the population living in private households in England. People living in institutional settings such as residential care homes, offender institutions, prisons, in temporary housing (such as hostels or bed and breakfasts) or sleeping rough are outside the scope of the survey. This should be borne in mind when considering survey findings, especially those for older people, since the institutional population in care homes is likely to be older and, on average, less healthy than those living in private households. The health of people not covered by the survey might vary from that of people in private households in some ways.  However, the proportion of people not living in private households in the England population is very small and so is likely to have little impact on the prevalence estimates in this report.   

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

This publication covers data over a long time period, 2011 – 2019 to get sufficient numbers of participants to analyse health and health related behaviours for more than five broad ethnic groups with adequate precision. Consequently, the estimates may not reflect the most current position, especially where behaviours or health status have been changing within the population over time.

This publication was pre-announced in the upcoming publications part of the Data and information - NHS Digital page and was published on that day.

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

The report is accessible on the NHS Digital website free of charge that include charts to illustrate the survey findings.   Tables in the publication are provided in Excel format. 

Information about the questions and health measures used in the survey are within the report.  More detailed metadata are in the annual publications and available: 

• a Methods report, giving a full account of the technical aspects of the survey including sampling, protocols for conducting survey measures such as height, weight and blood pressure and topics covered.
• Survey Documentation, including questionnaires and fieldwork materials.

Coherence is the degree to which data which have been derived from different sources or methods but refer to the same topic are similar. Comparability is the degree to which data can be compared over time and domain. 

The questions and variables used in this report are consistent across the time period 2011-2019 and consistent with harmonised questions for surveys as recommended by the Office of National Statistics.

This dimension describes the extent to which different aspects of quality are balanced against each other. 

When asking questions about smoking and drinking in a survey there is potential for the methodology to have an impact on how people answer. The findings are based on participants’ reports of their behaviour.

Adults aged 25 and over were asked about their smoking and drinking behaviour within the face-to-face interview. For those aged 16 to 17, information about smoking and drinking was collected through a self-completion questionnaire, to offer participants more privacy by allowing them to reply without disclosing their smoking or drinking behaviour to other household members. At the interviewer’s discretion, those aged 18 to 24 could answer the smoking and drinking questions either through the face-to-face interview or through the self-completion questionnaire. This makes it easier for respondents to answer honestly.

It is also possible that some question topics in this survey (e.g., smoking, drinking and fruit and vegetable consumption) may be susceptible to social desirability bias, where the individual is tempted to give an answer which is more socially acceptable. Respondents are assured that their answers will be kept private to reduce this temptation.

This dimension covers the processes for finding out about users and uses and their views on the statistical products. 

From our engagement with customers, we know that there are many users of the annual Health Survey for England publications. They are used by the Department of Health and Social Care, NHS England and NHS Improvement, Local Government, people working on public health policy and planning, or implementation of public health policy, the NHS, charities, academics, professional groups, the public, and the media. Uses of those publications include:

• informing, monitoring, and evaluating policy;
• monitoring changes in health or health related behaviours e.g., smoking;
• comparing local indicators with national figures;
• informing the planning of services;
• and writing media articles.

The content of the survey questionnaires is discussed and agreed with a steering group which contains representatives from NHS Digital, Department of Health and Social Care (including the Office for Health Improvement and Disparities), NHS England and NHS Improvement, academia, Local Government Public Health, other government departments as well as the contractor carrying out the survey.

This report was produced to meet requests from users of the annual publications and the steering group to examine the health and health-related behaviours of the population by ethnicity in more detail than has been done by NHS Digital during the past decade and to see if there are disparities across ethnic backgrounds.

We hope that a wide range of users, including will find the information in the publication useful and welcome feedback. NHS Digital is keen to gain a better understanding of the users of this publication and of their needs; please send any feedback to [email protected] (please include ‘Health Survey for England - Ethnicity’ in the subject line).

This dimension describes the effectiveness, efficiency and economy of the statistical output.

Data for this report comes from the annual Health Survey for England. For individual years, the number of respondents in the sample in some ethnic groups was too small to provide robust analysis. Combining the samples from annual surveys together was a cost-effective way to provide estimates: costing less than increasing the sample size in one or more annual surveys. However, combining years does extend the time period that the statistics relate to and make the estimates less current.  

The procedures and policy used to ensure sound confidentiality, security and transparent practices. 

The Code of Practice for Statistics is adhered to from collecting the data to publishing.

The addresses and names of people who take part are held securely by the survey contractor.

As for all NHS Digital publications the risk of disclosing an individual’s identity has been assessed, a risk assessment is undertaken prior to publication which addresses any potential issues around disclosure.  Information is presented at a high level of aggregation in the reports and tables and data are not presented in a form that can reveal any personal information that could be used to identify individuals.

Health Survey for England datasets for 2011 to 2019 are available via the UK Data Service. The datasets do not include information which directly identifies an individual.  Copies of the anonymised and disclosure-controlled datasets can be made available for specific research projects. For the latest information about dissemination of data please see the Health Survey for England  Health, social care and lifestyles 

Please see the links below to relevant policies:

UK Statistics Authority

NHS Digital Statistical Governance Policy [Archive Content]

NHS Digital Freedom of Information Process

NHS Digital Statement of Compliance with Pre-Release Order [Archive Content]

NHS Digital Disclosure Control Procedure [Archive Content]