MBRRACE-UK perinatal mortality surveillance

2. Statistical methods

2.1 Calculation of stabilised & adjusted mortality rates

The stabilised & adjusted mortality rate for each organisation (m_j) is calculated by multiplying the appropriate 'comparator' mortality rate (M) by an organisation-specific standardised mortality ratio (SMR_j) calculated from the data, i.e.:

where

m_j is the estimated stabilised & adjusted mortality rate for organisation j

M is the appropriate comparator mortality rate

SMR_j is the estimated SMR for organisation j:

Currently, for all organisations, except for the Trusts and Health Boards of birth, the comparator mortality rate is the overall mortality rate for the whole of the UK and Crown Dependencies. For the Trusts and Health Boards of birth the comparator mortality rate is the overall rate for Trusts and Health Boards in the same comparator group (described below).

The SMR is estimated using a multilevel logistic regression model. A multilevel model is used as it can accommodate the hierarchical structure of the data through the random term; that is, births clustered within organisations. These models also allow the calculation of stabilised (also known as ‘shrunken’ or ‘smoothed’) estimates of the organisation-specific terms, which reduce the likelihood of organisations being falsely identified as outliers by chance alone.

Various approaches to calculating a SMR from a multilevel logistic model have been proposed. The method used for the MBRRACE-UK report “… is determined by dividing the smoothed, risk-adjusted, provider-specific estimate of mortality by the estimate of expected mortality obtained using the average intercept for all … providers”. In this approach, the observed number of deaths is replaced by a model-based predicted number reflecting sampling variation in the observed deaths; that is, a stabilised observed number of deaths is estimated for each organisation. Hence, the SMR is the ratio of the stabilised number of deaths to the deaths that would be expected if the organisation’s patients were from an ‘average’ organisation.

2.2. Statistical models

Two multilevel logistic regression models are used, one for the stillbirths as outcome and the other model for neonatal deaths. The reference group for the both models is the births surviving at least 28 days from birth. The multilevel logistic regression model for stillbirth compared to survival to the end of the neonatal period is:

where

Y_(SB)ij is the indicator variable of stillbirth for the i^th baby in the j^th organisation:

Y_(SB)ij = 1 if stillbirth; 0 if survivor to end of neonatal period; missing if neonatal death;

x_ij is the vector of risk-adjustment factors for the i^th baby in the j^th organisation

z_ij is the vector of risk-adjustment factors for the j^th organisation

δ_(SB)j is the random term representing organisation j: δ ~ Normal(0,σ²)

A similar model is estimated for neonatal deaths compared to survival to the end of the neonatal period:

where

Y_(NND)ij is the indicator variable of neonatal death for the i^th baby in the j^th organisation:

Y_(NND)ij = 1 if neonatal death; 0 if survivor to end of neonatal period; missing if stillbirth;

x_ij is the vector of risk-adjustment factors for the i^th baby in the j^th organisation

z_ij is the vector of risk-adjustment factors for the j^th organisation

δ_(NND)j is the random term representing organisation j: δ ~ Normal(0,σ²)

The SMR for stillbirth is then given by combining these two models:

The SMR for neonatal deaths is derived directly from the second multilevel logistic regression model, since stillbirths are not included in the calculation of neonatal death rates:

The SMR for the extended perinatal deaths is obtained by combining the results of both models:

2.3. Risk-adjustment factors

For service delivery organisations the multilevel logistic regression model outlined in the previous section includes patient-level and organisation–level factors to adjust for differences in key factors which are known to increase the risk of stillbirth and neonatal mortality. The factors which can be included in the model are limited to those that are routinely collected for all births across the whole UK. For this report the patient-level risk-adjustment factors included in the statistical model were:

• mother’s age (under 20 years, 20 to 24 years, 25 to 29 years, 30 to 34 years, 35 to 39 years, 40 years or older);
• child poverty (measured by the Children in Low-Income Families Local Measure based on mother’s residence (quintiles with approximately equal number of total births));
• baby’s ethnicity (White, mixed or multiple ethnicity, Asian or Asian British, Black or Black British, other);
• baby’s sex (male, non-male);
• multiple birth (singleton, multiple);
• interaction between child poverty and baby’s ethnicity;
• interaction between child poverty and mother’s age;
• gestational age at birth for neonatal death rates only (24 to 27 completed weeks, 28 to 31 completed weeks, 32 to 33 completed weeks, 34 to 36 completed weeks, 37 to 41 completed weeks, 42 completed weeks or greater).

The only organisation-level factor (z_ij) currently included in the MBRRACE-UK analysis is a marker for the ‘comparator group’ of each organisation responsible for delivering maternity care. In the absence of detailed clinical data, to help account for the variation between organisations due to their differences in risk profile, all of the Trusts and Health Boards have been classified hierarchically into five mutually exclusive comparator groups based on their level of service provision. They are then compared to the average mortality rate within their comparator group. See section 1.7 for full details of the groups.

2.4. 95% confidence intervals

The reported 95% confidence intervals for the stabilised & adjusted mortality rate are obtained through bootstrap methods:

1. J organisations are sampled with replacement (where J is the total number of organisations).
2. The multilevel model is estimated for the sample, keeping each appearance of an organisation distinct if it is sampled more than once.
3. The estimated value, and prediction error, of the random term is obtained for each organisation: δ̂j and err̂or(δ_j) - if an organisation is sampled more than once then a single set of values is selected at random.
4. The bootstrap estimates for the fixed terms are noted (α*, β* and Γ*).
5. A new value (δj*) for the organisation-specific random term is sampled, where δj* ~ N(δ̂j , err̂or[δ_j]).
6. The bootstrap stabilised & adjusted mortality rate (mj) is obtained by substituting α*, β*, Γ* and δj* for α, β, Γ and δj as appropriate.
7. This is repeated 1,500 times, giving approximately 1,000 values for the bootstrap stabilised & adjusted mortality rate for each organisation since organisations are not necessarily included in each bootstrap sample.
8. The lower and upper limits of the 95% confidence interval are obtained for each organisation from the 2.5th and 97.5th percentiles respectively of the distribution the bootstrap stabilised & adjusted mortality rates.

2.5. Probability of falling above a benchmark

The statistical methodology used allows the calculation of empirical Bayes posterior probabilities to estimate the probability that the underlying mortality rate for an organisation falls above (or below) a specified benchmark; for example, it would be possible to report the probability that the underlying stabilised & adjusted mortality rate for organisation j is greater than 6 per 1,000 births (mj > 6.0). In MBRRACE-UK reporting, organisations have been identified when the probability that they fall above, or below, a specified benchmark is greater than 0.5; that is, ‘it is more likely than not’ that their underlying mortality rate falls outside the benchmark.

2.6. Missing data

Where information is unavailable for the risk-adjustment factors because it is missing from the routine data source, in order to allow all appropriate births to be included in the analyses the missing values are assumed to fall into the following categories:

• mother’s age - 30 to 34 years;
• socio-economic deprivation - middle quintile;
• baby’s sex - male;
• baby's ethnicity - White;
• multiplicity - singleton;
• gestational age at birth - 37 to 41 completed weeks.

Since missing observations are imputed with values generally representing low risk groups, stabilised & adjusted mortality rates are potentially overestimated for those organisations with missing data. However, as the proportion of missing data is low, and the effect of adjustment is relatively small, any overestimation will be small and unlikely to change any conclusions inferred from the reported rates.

3. Further details of MBRRACE-UK data collection

3.1 Approvals for collection of patient identifiable data

The necessary approvals obtained by the MNI-CORP programme prior to the start of the data collection process are listed below. These are applied for in order to collect patient identifiable data and access information collected by statutory organisations without consent.

England and Wales

• The Confidentiality Advisory Group of the Health Research Authority: ECC 5-05 (f)/2012 (from 10 October 2012); 15/CAG/0119 (from 1 May 2015)
• Health & Social Care Information Centre, Data Access Advisory Group: IC604DS

Scotland

• The NHS Scotland Caldicott Guardian: 2014-62 MBRRACE-UK Programme – Update (2013-05)
• The Privacy Advisory Committee, ISD, NHS National Services Scotland: PAC16/14

Northern Ireland

Due to the different data privacy arrangements in Northern Ireland, only de-identified data is provided to the MNI-CORP programme. This data is supplied by the NIMACH office.

3.2. The system for online data submission

Security

Access to the MBRRACE-UK website is via the internet using the secure HTTPS protocol. The web and database servers are housed in a secure data centre with firewall protection. All staff requesting online access must be approved by their Trust or Health Board and log-in is only possible with either an NHS or UK university email address. When an approved reporter first accesses the website they are required to request an activation code. This is used as a one-time password which must be changed on first access. All passwords must meet a set of criteria which ensures all passwords accepted are ‘strong’. Reporters are assigned a profile which restricts their access to only the appropriate parts of the website for their role (the system is used to report deaths, to review deaths with the Perinatal Mortality Review Tool (PMRT), and to provide access to anonymised medical case notes for assessors taking part in MBRRACE-UK confidential enquiries).

All patient identifiers are encrypted before they are stored. Access to identifiable data is only allowed under very limited circumstances. Reporters may view the data from their own Trust or Health Board (subject to the use of a valid password) while access to identifiable data by MBRRACE-UK staff is subject to NHS information governance, security and confidentiality regulation.

Data integrity and validation

Reporters wishing to report a new death or edit an existing death record are required to confirm the mother’s details (NHS or CHI number, name, date of birth) on each occasion. The nationally defined algorithm for checking NHS and CHI numbers is used to ensure only valid numbers are entered.

Where appropriate, the information reported is checked against a range of acceptable values during the data entry process. For each such data item there is a range of expected values and an absolute range. If a value is outside the expected range the reporter is warned and informed of the range. If it is outside the absolute range then the value cannot be entered and, additionally, the record cannot be closed. Before the reporter can close a record additional checks are carried out; for example, date values across the whole record are validated against each other to test for consistency.

In some circumstances there may be a small number of data items that are unavailable. In these situations reporters may indicate that an item is ‘not known’, with an opportunity to add the missing data at a later date.

For a significant number of deaths some of the data required will be held in more than one hospital, e.g. some aspects of maternal data after the death of a baby following postnatal transfer. If the additional information is held within the same Trust or Health Board but on a different site then reporters can access all the information they need in collaboration with obstetric, midwifery, neonatal or nursing colleagues. However, if the missing information is held by a different Trust or Health Board, then the MBRRACE-UK system allows the reporter to temporarily assign ownership of the MBRRACE-UK record to the other Trust or Health Board who can then return it once the missing information has been provided.

Online help

On every data entry screen many of the variables have specific help available by clicking on the ‘Help’ icon. Also, on every screen of the website there is a function to allow the reporter to enter a help request. This is sent via email to the MBRRACE-UK office for attention by the technical, clinical or administrative staff, as appropriate. A detailed user manual is also available to download from the MBRRACE-UK system.

Reports

The MBRRACE-UK online reporting system allows reporters access to information relating to local deaths:

• the Trust/Health Board Case Summary list provides abbreviated details of all deaths reported in an individual birth year together with the current status of each case’s surveillance form and Perinatal Mortality Review Tool review;
• the Trust/Health Board Reported Cases list provides abbreviated details of all deaths reported;
• the Trust/Health Board Summary provides the number of deaths by year, case-type and unit;
• the Trust/Health Board Case Review list provides the opportunity for local reporters to check the accuracy (within a fixed time frame) of the data reported by their organisation prior to the analysis for the report.

In addition, users can access a real-time data monitoring tool which allows them to view, filter and summarise live surveillance data for their Trust.

Web browser compatibility

The security requirements of the NHS in relation to electronic data flows mandate that the highest levels of security be employed. In order for this to be achieved, those accessing the MBRRACE-UK reporting system need access to an up-to-date web browser compatible with these security specifications. Appropriate browsers are available to download free of charge, although the installation of such software may require the co-operation of local NHS IT departments.

3.3. Ensuring all births and extended perinatal deaths are identified

Data sources

The sources of data used to ensure complete data collection of births and extended perinatal deaths for this cohort are listed below:

England and Wales

• Birth registration data – ONS
• Death registration data – ONS
• PDS data on all births – NHS Digital (PDS)

Scotland

• Birth registration data – NRS
• Death registration data – NRS
• Maternity Inpatient and Day Case Dataset (SMR02) inpatient data - PHS, NHS National Statistics Scotland

Northern Ireland

• Birth registration data - NIMACH, Health and Social Care Public Health Agency – derived from NIMATS
• Death registration data - NIMACH, Health and Social Care Public Health Agency – derived from NIMATS
• Inpatient data - NIMACH, Health and Social Care Public Health Agency – derived from NIMATS

Crown Dependencies

• Birth registration data - Health Intelligence Unit, Public Health Services, States of Jersey
• Death registration data - Health Intelligence Unit, Public Health Services, States of Jersey
• PDS data on all births, Isle of Man – NHS Digital (PDS)

Identifying all extended perinatal deaths

Statutorily registered deaths which meet the MBRRACE-UK reporting criteria are matched to the deaths reported to MBRRACE-UK in order to identify any stillbirths or neonatal deaths which have not been reported to MBRRACE-UK. Due to the different system of implementation in Northern Ireland, the NIMACH office staff ensured full ascertainment of their data on our behalf.

For England, Wales and Scotland the matching is performed using a combination of deterministic and probabilistic matching methods based on the mother’s given name, mother’s family name, postcode of residence at time of delivery, Trust or Health Board of birth, baby’s NHS number (England – where available), CHI number (Scotland), gestational age at delivery, birthweight, date of delivery and date of death.

Once the checking is complete the MBRRACE-UK Lead Reporters are notified of any known deaths that have occurred in their Trust or Health Board which could not be identified on the MBRRACE-UK system and asked to confirm that this was a death in their organisation and provide the missing information.

This checking for deaths missing from the MBRRACE-UK database cannot start until information on statutorily registered deaths are provided to MBRRACE-UK by ONS (England and Wales) and NRS (Scotland), meaning that we cannot inform MBRRACE-UK Lead Reporters of missing deaths until some months after the event. Although most missing deaths can be identified in this way, not all deaths to be reported to MBRRACE-UK are available from statutory sources in a timely manner:

1. A small percentage of statutorily registered deaths are registered only after considerable delay, most often because an inquest was being held;
2. Late fetal losses delivered at 22 to 23 completed weeks’ gestational age are not officially registered;
3. RCOG guidance is that stillbirths delivered at 24 completed weeks’ gestational age or greater where the death was confirmed before 24 completed weeks’ gestational age should not be registered as stillbirths; however, in order to investigate variations in the reporting of stillbirths occurring at around 24 completed weeks’ gestational age, these deaths should all be reported to MBRRACE-UK.

There are no timely and easily accessible data sources for deaths that have not been officially registered and, therefore, it is not possible to ensure that all of these deaths have been reported to MBRRACE-UK.

Identifying all births

Individual information on all births in the UK and Crown Dependencies is obtained in order to generate mortality rates adjusted for maternal, baby, and socio-demographic risk factors. Information for England, Wales and the Isle of Man (PDS and ONS birth registration data), Scotland (NRS and PHS), Northern Ireland (NIMATS) and the Bailiwick of Jersey (Health Intelligence Unit) is combined to give a single dataset of births for the whole UK and Crown Dependencies. This data is then combined with the information on the deaths to obtain the final data for analysis.

The allocation of births to an organisation is complex, given the wide variation in the recording of the notifying organisation, and it is not always possible to easily identify the place of birth from the data reported. In many cases this either requires further detailed enquiry or correction of the place of birth, where an incorrect organisation has inadvertently been recorded. Complete and accurate recording is vital to enable MBRRACE-UK to allocate births to the appropriate Trust or Health Board for analysis and reporting.

Home births are allocated to the Trust or Health Board responsible for this service, whenever this is recorded, in order for the correct denominator(s) to be calculated. All Trusts and Health Boards in England, Wales and the Isle of Man completing information for the PDS should ensure that they are identified as the notifying organisation for all births related to their service.

3.4. Generating the births dataset

The births and extended perinatal deaths identified using the sources and methods described above are combined to generate a single dataset for analysis. Due to the variations in the data sources from the different countries, this is undertaken separately for each set of data sources as described below. Once the datasets have been generated for each country these are combined into a final, single dataset for analysis.

England and Wales

The complete dataset of births and extended perinatal deaths for England and Wales is generated using birth registration data (ONS), death registration data (ONS), PDS records, and MBRRACE-UK death notification records. The process undertaken by MBRRACE-UK is:

1. All datasets are restricted to births in the reported year.

2. All records of births are combined into a single dataset (Figure 1): i.e. livebirth registrations (ONS); stillbirth registrations (ONS); PDS birth records; MBRRACE-UK notifications of late fetal losses. All of these datasets are used in order to obtain complete ascertainment of all births in England and Wales:

   • late fetal losses are only recorded in the MBRRACE-UK death records;
   • late birth registrations are captured by the PDS records;

3. Births at less than 22 completed weeks’ gestational age and pregnancies ended by a termination of pregnancy are removed from the dataset of births, as these are not reported by MBRRACE-UK.

4. Births at 22 to 23 completed weeks’ gestational age are removed from the dataset of births for the main tables and maps as these births are currently reported separately by MBRRACE-UK.

5. All records of late fetal losses, stillbirths, and neonatal deaths are combined into a single dataset (Figure 2): i.e. death registrations (ONS); MBRRACE-UK death notifications. Both of these datasets are used in order to obtain complete ascertainment of all extended perinatal deaths in England and Wales.

6. All deaths where the births occurred at less than 22 completed weeks’ gestational age and pregnancies ended by a termination of pregnancy are removed from the dataset of deaths as these are not reported by MBRRACE-UK.

7. All deaths where the births occurred at less than 24 completed weeks’ gestational age are removed from the dataset of deaths for the main tables and maps as these deaths are currently reported separately by MBRRACE-UK.

8. The dataset of deaths are merged into the dataset of births in order to create a single dataset for analysis.

Description of Figure 1: Flow chart showing how datasets from multiple sources are combined in order to generate a complete births dataset for England and Wales.

Description of Figure 2: Flow chart showing how datasets from multiple sources are combined in order to generate a complete extended perinatal deaths dataset for England and Wales.

Scotland

The complete dataset of births and extended perinatal deaths for Scotland is generated using a similar approach to that used for England and Wales. For Scotland, data is obtained from birth registration data (NRS), death registration data (NRS), SMR02 Maternity Inpatient and Day Care Case records (PHS), and MBRRACE-UK death notification records. The birth registration data and the SMR02 data are merged before being released to MBRRACE-UK (PHS). The process undertaken by MBRRACE-UK is:

Step 1: All datasets are restricted to births in the reported year.

Step 2: All records of births are combined into a single dataset (Figure 3): i.e. birth registrations/SMR02 (PHS); PHS notifications of late fetal losses; MBRRACE-UK notifications of late fetal losses. These datasets are used in order to obtain complete ascertainment of all births in Scotland.

Step 3: Births at less than 22 completed weeks’ gestational age and pregnancies ended by a termination of pregnancy are removed from the dataset of births as these are not reported by MBRRACE-UK.

Step 4: Births at 22 to 23 completed weeks’ gestational age are removed from the dataset of births for the main tables and maps as these births are currently reported separately by MBRRACE-UK.

Step 5: All records of late fetal losses, stillbirths, and neonatal deaths are combined into a single dataset (Figure 4): i.e. death registrations and SMR02 (PHS); MBRRACE-UK death notifications. Both of these datasets are used in order to obtain complete ascertainment of all extended perinatal deaths in Scotland.

Step 6: All deaths where the births occurred at less than 22 completed weeks’ gestational age and pregnancies ended by a termination of pregnancy are removed from the dataset of deaths as these are not reported by MBRRACE-UK.

Step 7: All deaths where the births occurred at less than 24 completed weeks’ gestational age are removed from the dataset of deaths for the main analyses, as these deaths are currently reported separately by MBRRACE-UK.

Step 8: The dataset of deaths is merged into the dataset of births in order to create a single dataset for analysis.

Description of Figure 3: Flow chart showing how datasets from multiple sources are combined in order to generate a complete births dataset for Scotland.

Description of Figure 4: Flow chart showing how datasets from multiple sources are combined in order to generate a complete extended perinatal deaths dataset for Scotland.

Northern Ireland and the Crown Dependencies

Datasets of births and extended perinatal deaths for Northern Ireland and the Bailiwick for Jersey are supplied to MBRRACE-UK as complete datasets from the appropriate national data providers. The birth records for the Isle of Man are obtained from the PDS records. In each case the birth and death records are then linked to the MBRRACE-UK records.

Data cleaning, linking and derived variables

Where information on a variable is available from more than one source a ‘best value’ algorithm is applied in order to obtain the value to be included in the analyses. The algorithm chosen is:

• where available, the value recorded in the MBRRACE-UK death record is used as the prime source;
• if unavailable (e.g. the baby survived the neonatal period) the value recorded in the statutory birth or death registration record is taken as the secondary source;
• for England and Wales, the value recorded in the PDS record is used as the third source: the gestational age at delivery is only available from the PDS records.

3.5. Location of mother’s residence

The postcode of the mother’s residence at the time of delivery is used to identify the country, CCG (England), Health Board (Scotland and Wales), Health and Social Care Trust (Northern Ireland), Crown Dependency, and Local Authority of reporting using postcode linked data supplied as part of GridLink. In addition, it is used to obtain the appropriate value for the child poverty index.

The Trust or Health Board of birth is derived using the most appropriate source from all available datasets. For England and Wales the recorded Communal Establishment Code in the ONS birth records is used as the primary source the location of the birth. When the place of birth cannot not be located from the ONS records (e.g. births at home and in-transit) the location is derived from the PDS record. If neither record provided a clear Trust or Health Board of birth then an estimate is made based on the postcode of birth.

The Trust or Health Board of death is obtained directly from the MBRRACE-UK death record.

4. Data items reported to MBRRACE-UK

5. Definitions and abbreviations

5.1 Definitions used by MBRRACE-UK

5.2. Abbreviations

6. Additional information

6.1. Funding

The Maternal, Newborn and Infant Clinical Outcome Review Programme, delivered by MBRRACE-UK, is commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP). HQIP is led by a consortium of the Academy of Medical Royal Colleges, the Royal College of Nursing, and National Voices. Its aim is to promote quality improvement in patient outcomes. The Clinical Outcome Review Programmes, which encompass confidential enquiries, are designed to help assess the quality of healthcare, and stimulate improvement in safety and effectiveness by systematically enabling clinicians, managers, and policy makers to learn from adverse events and other relevant data. HQIP holds the contract to commission, manage, and develop the National Clinical Audit and Patient Outcomes Programme (NCAPOP), comprising around 40 projects covering care provided to people with a wide range of medical, surgical and mental health conditions. The programme is funded by NHS England, the Welsh Government and, with some individual projects, other devolved administrations and Crown Dependencies.

More details can be found on the HQIP website.

6.2. Stakeholder involvement

Organisations representing parents and families are involved in the MBRRACE-UK programme as part of the ‘Third Sector’ stakeholder group, identifying possible areas for future research and helping to communicate key findings and messages from the programme to parents, families, the public and policy makers, including through the development of lay summary reports. A full list of organisations can be found in the acknowledgements.

6.3. Attribution

This manual should be cited as:

Draper ES, Gallimore ID, Smith LK, Matthews RJ, Fenton AC, Kurinczuk JJ, Smith PW, Manktelow BN, on behalf of the MBRRACE-UK Collaboration. MBRRACE-UK Perinatal Mortality Surveillance: Technical Manual. Leicester: The Infant Mortality and Morbidity Studies, Department of Population Health Sciences, University of Leicester. 2023.

Published by:

The Infant Mortality and Morbidity Studies
Department of Population Health Sciences
University of Leicester
George Davies Centre
University Road
Leicester LE1 7RH

7. Version history