This documentation is intended to support the use and interpretation of the accompanying report: 30-day mortality post-SACT case-mix adjusted rates, bowel cancer patients treated between 1 January 2021 and 31 December 2022. Each section consists of a set of tabs, please click on each tab to see more information. To navigate to a specific section of the documentation, please use the contents list to the left-hand side of the page.
The 30-day mortality post-Systemic Anti-Cancer Therapy (SACT) Case-Mix Adjusted Rates (CMAR) report is produced by the National Disease Registration Service (NDRS). It presents 30-day mortality metrics for patients treated with SACT in England.
The 30-day mortality post-SACT CMAR report includes patients diagnosed with cancer in England and treated with SACT, with each report focusing on a defined cancer type and treatment period. This report considers patients diagnosed with bowel cancer who received treatment between 1 January 2021 and 31 December 2022.
Metrics include:
This report may identify a trust data submission issue or provide an opportunity to review and improve clinical practice.
Since August 2020, the SACT team have released several workbooks containing case-mix adjusted 30-day post-SACT mortality rates for a range of cancer types. Cancer types and treatment periods covered in previous workbooks include:
Each workbook was based on routine data reported by NHS trusts in England through their monthly routine SACT data uploads. In advance of these publications the adult workbooks were sent to the NHS trusts and the CTYA Acute lymphoblastic leukaemia (ALL) workbook was sent to Principal Treatment Centres (PTCs) for review, giving them the opportunity to provide a statement to accompany their data. These statements have been included in the companion reports released with the CMAR workbooks.
The data for these workbooks are now available as an interactive web application. This presents all CMAR cancer types published up to February 2022.
The National Cancer Registration and Analysis Service (NCRAS), part of the NDRS within NHSE, is the population-based cancer registry for England. NCRAS receives data from across the National Health Service (NHS) and produces the National Cancer Registration Dataset (NCRD) for England.
NCRAS also maintains a Rapid Cancer Registration Dataset (RCRD) for England, which contains proxy tumour registrations and some associated events on the cancer patient pathway (e.g. surgery, radiotherapy and chemotherapy) from January 2018 to the most recently available data on cancer diagnoses. This rapid dataset provides a quicker, indicative source of cancer data compared to the gold standard registration process, which relies on additional data sources, enhanced follow-up with trusts and expert processing by cancer registration officers.
For this report, both the NCRD and the RCRD were used to collate the initial cohort of patients with a diagnosis of the cancer of interest.
The patient cohort was then linked to the SACT Dataset, which collects systemic anti-cancer therapy activity from all NHS providers in England, using NHS number to identify the latest treatment records for the patient cohort. Only patients with activity in the treatment period of interest were included in the analysis. Certain restrictions were applied to the data to ensure appropriate patients and treatments were selected for each cancer site. These restrictions mean that some trusts may have no data included in the report because their patients and patients’ treatment activity did not fit the criteria applied.
A patient’s Charlson co-morbidity score was derived using the Quan methodology and the inpatient Hospital Episode Statistics (HES) dataset.
The full, 7-category Index of Multiple Deprivation (IMD) has been used, in quintiles, to report deprivation.
The date of death for each patient comes from the Personal Demographics Service via the Demographics Batch Service.
The analysis cohort contains patients with a relevant diagnosis and SACT activity in the treatment period of interest.
Patient and tumour criteria:
Treatment criteria:
Trust criteria:
It is important to note that in some cases, the cancer diagnosis recorded in the NCRD or RCRD may differ from that recorded by the trust.
*Performance status describes a patient’s level of functioning in terms of their ability to care for themself, daily activity, and physical ability (walking, working, etc.). This measure is used to determine whether a patient can receive chemotherapy, whether dose adjustment is necessary, and as a measure for the required intensity of palliative care. Thus, performance status is an impactful variable to adjust for in the statistical model, and so we have applied a completeness requirement of 70% to be included in the analysis. We use performance status recorded at the last (most recent) cycle if it is present. If this is missing, we will use the performance status recorded for that regimen. We do not look at other cycles from within the regimen.
If you would like support in improving the quality and completeness of your trust’s data feed into the NCRAS then please contact [email protected]
1. Restricting to relevant, eligible SACT treatments
After linking the NCRD, RCRD, and SACT dataset to identify a patient cohort with a relevant diagnosis and SACT activity in the treatment period of interest, the treatment records are restricted to meet one of the following criteria:
Restrictions are then applied to remove all treatment records that are non-harmful, supportive treatments, hormones, non-chemotherapy treatments, and treatments not commonly associated with the cancer type of interest. This restriction is done at drug level wherever SACT data submissions provide this detail, defaulting to regimen level where drug level information is not available.
2. Identifying the most recent treatment date in the period of interest
Of the relevant, eligible SACT treatments, the most recent recorded treatment date in the SACT dataset is identified as the treatment date for use in the analysis. In most cases this is the most recent administration date (date of the SACT drug administration or the date an oral drug was initially dispensed to the patient). However, if this is missing, then the most recent cycle start date for that patient is taken, and failing that, the most recent regimen start date is used.
The most recent treatment date identified in SACT depends on the detail of data submitted by trusts:
All independent variables for the statistical model adjustment are taken from as close to the most recent treatment date as data allows.
3. Calculating 30-day mortalities
For every patient, the interval between the most recent treatment date and date of death is calculated. From this, a 30-day mortality flag is then created to identify all patients who have died within the first 30 days following their most recent SACT treatment.
It should be noted that this calculation includes all deaths. This is because the cause of death can be misclassified, especially those relating to deaths from cancer treatment, and so the decision was made to take an inclusive approach to cause of death for this report.
4. Calculating rates and statistical modelling
Crude rates were calculated using the observed number of deaths within 30-days and the total number of patients treated.
For each overall 30-day mortality post-SACT CMAR analysis, the best-fit model is sought. Case-mix adjusted rates for this analysis were calculated via a mixed effects logistic regression model, which estimates the odds of a death within 30 days for each patient. The model includes independent variables in its adjustments and so the case-mix adjusted rates are based on an average group of patients as opposed to the trust’s own group of patients. For more details on the case-mix adjustment and the mortality rate calculations please see the ‘Methodology - Definitions’ section.
Stratified rates were calculated for both the unadjusted and case-mix adjusted rates. For any particular stratification, the case-mix adjustment includes all independent variables from the best-fit overall analysis model, except for the variable used for stratification. For example, the case-mix adjusted rates stratified by age group include gender, deprivation quintile, ethnicity group, performance status, and co-morbidity score in the model adjustments but omit the age group variable.
5. Output production
R Shiny was used to create an interactive web-based report.
*Chemotherapy timeframes are defined using the NDRS ‘Linking treatment data for cancer diagnoses CAS SOP v4.9’ which can be downloaded here.
Trusts
An NHS trust is an organisational unit within the NHS in England, generally serving either a geographical area or a specialised function. In this report, these are referred to as a ‘trust’. Not all trusts have been included in the analysis. Trusts were not included if they did not meet the data completeness criteria or if there were an insufficient number of patients treated at the trust for the cancer of interest during the treatment period.
To allow uniformity in analysis in the context of trust merges, trusts were grouped as per their status at the end of the treatment period considered in the analysis. This means that if trusts merged after the end of the treatment period, the trusts will remain separate in the report. If the trusts merged prior to or during the treatment period, the trusts will appear under their single merged trust name in the report.
Case-mix adjustment
For each overall 30-day mortality post-SACT CMAR analysis, the best-fit model is sought. Case-mix adjusted rates for this analysis were calculated via mixed effects logistic regression models, which estimate the odds of a death within 30 days for each patient. A mixed effects logistic regression model adjusts for a set of specified independent variables and assumes these variables are constant over time. Therefore, case-mix adjusted rates are an estimate of a trust’s mortality rate if all trusts had the same sample of patients. In this work, differences in the variables listed below for patients within each trust have been accounted for. This allows for comparisons to be made between trusts and within trusts over time.
Fixed effects:
Random effect:
These variables were chosen because they are known to influence 30-day post-SACT mortality and have sufficient completeness. It is important to note that due to the absence of any information on factors such as patient choice and clinical factors such as liver function tests, the case-mix adjustment may not fully account for the differences in caseload between trusts.
Intent of treatment was not included in the model adjustments, however a stratification of results has been provided by intent of treatment. With this, ‘Adjuvant’ and ‘Neo-adjuvant’ intents were grouped to ‘Curative’.
Calculations
The unadjusted 30-day mortality rates are calculated, as percentages, as follows:
Unadjusted rate (%) = (Observed Deaths in the trust / Number of Patients Treated in the trust) x 100
The case-mix adjusted 30-day mortality rates (CMAR) are calculated, as percentages, as follows:
Case-mix adjusted rate (%) = ( (Observed Deaths in the trust / Predicted Deaths in the trust) x Population rate ) x 100
Where:
The confidence intervals presented in this report reflect the distribution of trust mortality rates around the national average. This work highlights the mortality post-SACT rates within 2 standard deviations (SDs) (roughly equivalent to 95% confidence intervals) and within 3 SDs (roughly equivalent to 99.8% confidence intervals). Trusts with mortality post-SACT rates above the upper, +3SD have been identified as outliers.
Each patient was allocated to the trust where they received their most recent treatment, regardless of any prior treatment received at other trusts. A total of 128 trusts in England were submitting SACT data at the end of the analysis treatment period, however not all trusts have been included in the analysis.
Trusts were not included in the overall mortality analysis if:
Trusts were not included in the stratified mortality analysis if:
Trust coverage:
For detailed information on specific trust exclusions, please see the Exclusions - Excluded trusts section.
*This decrease was caused by the inability to link the SACT treatment records to the qualifying cancer diagnosis.
This was mainly due to the registration team being unable to assign the SACT regimen to the bowel cancer diagnosis of interest during the registration process. The assignment of a SACT regimen to a cancer diagnosis first fails when the primary diagnosis 3-character ICD-10 code in the SACT dataset does not match that in the NDRS cancer registration dataset. If this criterion is not met, then we look at the gap between date of diagnosis and the SACT regimen start date. If this gap is over 12 months, then the treatment is not considered to be a tumour match. If the gap is within 12 months, then the regimen is tumour linked even if the SACT primary diagnosis (ICD-10) code doesn’t match the cancer registration diagnosis (ICD-10) code.
Please see the tumour linkage algorithm consort diagram in the Appendix for more information.
**Data criteria includes the removal of patients who could not be found in the death tracing process, patients who received treatment at more than one trust on their most recent treatment date, and patients treated at a trust with less than 5 patients in total.
Interpreting my trust rate
While there is no ‘target’ mortality post-SACT rate, generally, lower rates are better as they reflect that in curative settings, complications are being identified and managed rapidly. In palliative settings they reflect that the clinician is aware when the patient is in the final stages of their disease and discusses with the patient whether other non-SACT treatment options are more appropriate.
However, persistently very low mortality post-SACT or zero mortality post-SACT rates may be indicative of a risk averse approach to prescribing, whereby patients who may potentially have benefited from SACT are not receiving this treatment. Trusts who have persistently very low mortality post-SACT or zero mortality post-SACT rates may want to consider auditing their clinical practice for this reason.
If a trust only treats a small number of patients and one patient dies within 30 days, the trust’s mortality post-SACT rate will be high. However, the funnel plot structure may account for this as the confidence limits will be wider for trusts treating a smaller number of patients.
A trust can request the NHS numbers for their patients who died within 30 days of receiving SACT. To request this information please contact the SACT Helpdesk at [email protected].
Case-mix adjusted rates
Case-mix adjusted mortality post-SACT rates are presented in this report.
Each trust will see and treat a wide variety of patients depending on the population they serve and the services they provide. Thus, the predicted probability is calculated from a case-mix adjusted logistic regression model. It takes into account (where these variables are appropriate or available) differences in the gender, age, deprivation quintile, ethnicity, performance status, and co-morbidity score of patients within each trust. These characteristics are known as the ‘independent variables’. The mixed effects logistic regression model uses all the data to estimate the odds of a death within 30 days for each patient. As a result, the case-mix adjusted rates are based on an average group of patients as opposed to the trust’s own group of patients. These rates can then be compared between trusts and within a trust over time.
It should be noted that the absence of any information on critical factors such as frailty and patient choice, as well as clinical factors such as liver function tests, mean that the case-mix adjustment may not fully account for the differences in caseload between trusts.
Unadjusted rates
In extension to previous CMAR workbooks, this report supplements all case-mix adjusted rates with the unadjusted, crude 30-day mortality post-SACT rates.
Unadjusted rates do not take into account any independent variables of the treated population. Instead, the unadjusted rates represent the real, observed 30-day mortality post-SACT rate within a trust as per the data submitted to SACT.
Unadjusted rates should not be used to compare between trusts or within a trust over time.
Stratified rates
This report also provides an extension to previous CMAR workbooks by stratifying 30-day mortality post-SACT metrics by key independent variables. Stratification has been applied to both case-mix adjusted and unadjusted rates.
For any particular stratification, the case-mix adjustment includes all independent variables used in the overall analysis model except for the variable used for stratification. For example, the case-mix adjusted rates stratified by age group include gender, deprivation quintile, ethnicity group, performance status, and co-morbidity score in the model adjustments. With this, trusts are able to consider their adjusted mortality rate for certain patient groups, for example patients aged under 40, and compare this to other trusts.
Due to limitations on releasing small patient counts (less than 5), not all trusts included in the overall mortality analysis could be included in the stratified mortality analyses.
It should also be noted that if a trust appears in the plot for some but not all groups, e.g. deprivation quintiles 1 to 4, but not 5, this is due to the trust not treating any patients for that particular group.
To explore the charts in more detail, consider the following tips:
In the web-based report, data tables will auto-update depending upon selections made on the left-hand side.
Funnel plot content
The funnel plot is used to visualise 30-day mortality post-SACT rates for included trusts in England.
Confidence intervals - identifying outliers
The confidence intervals presented reflect the distribution of trust mortality post-SACT rates around the national average. The plot highlights the mortality post-SACT rates for ±2 SDs (roughly equivalent to 95% confidence intervals) and ±3 SDs (roughly equivalent to 99.8% confidence intervals). Trusts with rates above the upper, +3SD will be identified as outliers.
Trusts that are found to be outliers by having 30-day mortality post-SACT rates that are above the upper, +3SD limit will be informed of this. All trusts are given the opportunity to respond to the SACT team prior to this information being made available to the public, and we particularly encourage responses from outlying trusts.
If a trust has been identified as an outlier but the trust caseload (number of patients) is fewer than 10 patients, this is unlikely to be statistically robust. Where a trust is >3SD and has fewer than 10 patients, outlier status is likely to be caused by the small cohort size rather than trust treatment practice.
It should be noted that outlier status can occur due to data quality issues, problems with linking the SACT dataset to the National Cancer Registration Dataset and Rapid Cancer Registration Dataset and other factors.
Reasons as to why a trust identifies their caseload to be higher than that of the funnel plot vary depending on the cancer type, however some possible explanations include:
Certain treatments have been excluded from this analysis:
Cancer mortality-specific exclusions are applied when the regimen or drug identified in SACT would not be expected for the cancer type of interest. This may be due to an incorrect primary diagnosis submitted to SACT accidentally, or if the patient has had multiple diagnoses (and treatment for a different cancer type has been submitted under the incorrect diagnosis code).
The full regimen and drug exclusion list was agreed via consultation with clinicians.
Regimens excluded from the analysis
Certain treatments have been excluded from this analysis:
Cancer mortality-specific exclusions are applied when the regimen or drug identified in SACT would not be expected for the cancer type of interest. This may be due to an incorrect primary diagnosis submitted to SACT accidentally, or if the patient has had multiple diagnoses (and treatment for a different cancer type has been submitted under the incorrect diagnosis code).
The full regimen and drug exclusion list was agreed via consultation with clinicians.
Drugs excluded from the analysis
Some trusts have been excluded from this report, as summarised by the table below, due to:
Please note, a trust being identified as having ‘No data’ does not necessarily indicate lacking submissions. The trust may not have any SACT activity to submit to NDRS, for example because the trust may not treat this condition.
If you have been excluded due to low performance status completeness and would like support in improving the quality and completeness of your trust’s data feed into the NCRAS then please contact [email protected]
Trusts excluded from the analysis
The SACT helpdesk at NHS England compiled a list of named contacts at each NHS trust including the SACT uploader, lead cancer clinician, the regimen mapper, CEO and medical director, unless we were otherwise notified. Please contact the helpdesk at [email protected] if you would like to make changes to your distribution list.
The report is sent to these named contacts. There is no patient identifiable data included in the report, trusts are invited to request the NHS numbers of patients in their data who have died within 30 days of receiving SACT. NHS numbers will only be provided via secure means following a request from a trust. This aims to restrict the circulation of this sensitive information to individuals who intend to use it for clinical audit and local governance processes. NDRS are unable to share the NHS number of a patient treated by a trust if they have not died within 30 days of receiving SACT.
The report is made publicly available on the NDRS website within two months following release to the NHS trusts.
Trusts identified as outliers were invited to provide a statement for inclusion in the report.
Each release will be publicised through a range of channels, including the SACT and the NDRS newsletters.
Sign up to the SACT newsletter here.
For more information on NDRS treatments data, and to review ‘Treatment data CAS SOP’, please click here.