Using AI-enabled ambient scribing products in health and care settings guidance for IG professionals

It is important to be aware that the use of AI tools within health and care settings, particularly those classified as a medical device, has implications and regulations beyond IG. Organisations looking to develop or implement an ambient scribe should ensure they are familiar with the wider regulatory framework. For more guidance, see AI and Digital Regulations Service (AIDRS) guidance on regulations and NHS England guidance on adopting ambient scribing products.

It is important that organisations independently assess their chosen tools for compliance with data protection legislation and the common law duty of confidentiality (CLDC), taking account of the following, which includes recommendations based on both legal requirements and best practice in IG.

You should involve the right decision makers in your organisation in your selection of an ambient scribe. This may include Chief Information Officers (CIOs) and Chief Clinical Information Officers (CCIOs), your Senior Information Risk Officer (SIRO), your board members, your Caldicott Guardian and your Data Protection Officer (DPO).  

You will need to establish a lawful basis under both the UK GDPR and CLDC for the processing of data by an ambient scribe.

The Information Commissioner’s Office (ICO) has also produced guidance on ensuring lawfulness in AI.

Where you are using the tool and the data to assist you with providing care to individuals, for example, producing a letter for a patient following a consultation, the most likely lawful bases under UK GDPR are:

• article 6 (1) (e) Public task
• article 9 (2) (h) Health and care

The most likely lawful basis under CLDC is:

• implied consent

You do not need to ask for explicit consent under either UK GDPR or CLDC to process data for individual care, but you still need to be transparent with patients and service users regarding the use of AI and provide them with an opportunity to object and dissent (see the transparency section for further information). If an individual dissents, you cannot then use the tool.

The appropriate transparency arrangements when using an ambient scribe need to be carefully considered, taking into account the setting as well as the expectations and needs of the individuals who are being recorded.

As the use of ambient scribes in health and care is a new technology, individuals must be told at the beginning of a session when these technologies are being used as part of their individual care. This will help to ensure that there are no surprises to patients and service users about the recording and other elements of the data processing. As familiarity and understanding of this technology increases, it may not be necessary for individuals already familiar with the technology to be verbally notified every time the tool is being used for their individual care. It is important to keep your approach to transparency under review.

As well as verbal notification, you should:

• update your privacy notice with sufficient detail about how the data is processed and used and how individuals can request that ambient scribes are not used 
• consider other ways to promote understanding of the use of ambient scribes in the context of providing individual care, such as by providing information in public areas (for example, through posters and TV screens in waiting areas) or consultation with 
• patients and service users. This is to create expectations of the use of ambient scribes in your setting
• consider publishing further information about the tool so that patients and service users can learn more about these. For example, publishing information on your website or in newsletters or in leaflets for those lacking digital access
• ensure that your transparency materials are appropriate for different audiences

• The ICO has produced further guidance on transparency in health and social care, as well as guidance on ensuring transparency in AI.

Retaining audio recordings and transcripts

Where a recording and its transcript is summarised by an ambient scribe, and this summary is signed off as accurate and adequate for its purposes by a health or care professional, the original recording and transcript could be deleted and the summary only retained as part of the health and care record, unless there is a need to retain further in order to monitor the safety and accuracy of the tool.

You should ensure that your chosen tool has functionality to comply with retention periods, using automatic deletion where appropriate.

Outputs are likely to be subject to separate and distinct retention periods depending on their use and final storage location. For example, where a summary is added to a health and care record, it should be retained in line with the retention of the health and care record.

See the section on storage for further information and the NHS Records Management Code of Practice for guidance on retention periods.

You should check where inputs and outputs from your chosen tool are stored (whether by the provider or in an internal location) at each stage.

Once records have been transferred to the relevant (final) storage location (for example, the health and care record), they should be deleted from any other storage locations that they may have been held in while the information was processed. This will ensure that the information is not duplicated across multiple systems and is not at risk of being retained for longer than necessary. 

If any data is stored with the provider of the tool at any time, you must give them clear instructions on storage and retention and ensure that they are able to comply with these.

The term ‘accuracy’ in the context of data protection has a different meaning from the term in the context of AI performance, though they are closely linked and must both be given sufficient consideration.

In a data protection context, accuracy refers to:

• the accuracy principle, requiring you to ensure that personal data is correct, not misleading and kept up to date

In an AI performance context, accuracy refers to:

• statistical accuracy, a measure of how often an AI produces an output that is correct

The output from the ambient scribe will not always be accurate for a number of different reasons. For example, the technology may misinterpret a word or misunderstand the context of a discussion. The output therefore must always be checked and verified by the ambient scribe user to ensure that any inaccurate output is corrected before it may impact the patient or service user. The checking of the output ensures that the information being used is both accurate and fair in compliance with the principles under the UK GDPR.

You should:

• ensure that you review your chosen tool’s performance, including statistical accuracy, to ensure that there is no significant risk of inaccurate outputs
• ensure that users of the tool are aware of and adhere to the requirement to thoroughly check outputs, for example through policies, training and audits where needed

The ICO has produced further guidance on ensuring fairness in AI.

You must carefully assess and document who is controller and processor for each processing activity relating to the use of the ambient scribe.

When an ambient scribe is used for individual care, the health or care organisation should be the controller of the information processed for this purpose and data processing arrangements should be built with this in mind. To ensure this, the health or care organisation should:

• determine the means and purpose of the processing
• provide clear instruction to the provider of the AI tool about any data processing, including storage and retention
• ensure that instructions and responsibilities are clearly documented in agreements (see the Data Sharing and Processing Agreements (DSPAs) section)

If an AI supplier determines some or a significant portion of the means and the purposes of processing, it may be considered a joint controller and will have joint responsibility for compliance with data protection legislation. Organisations must carefully assess processing arrangements to ensure they are happy with the controller arrangement and appropriately document these in agreements. In particular, organisations should ensure that patient data is not used by the technology provider for purposes beyond the care of the individual.

Ambient scribes may introduce security vulnerabilities through their own processing as well as through their integration with your systems. It is critical to engage with technical and security specialists to fully assess and mitigate the security risks prior to using an ambient scribe.

NHS England has produced further general guidance on the use of ambient scribes, and the ICO also has guidance on assessing security in AI.

You should ensure that your chosen tool will allow you to comply with individuals’ rights under UK GDPR, where they apply. Further information on the rights and key considerations relating to ambient scribes are provided below.

The right to object

If using public task as the lawful basis for processing under the UK GDPR, you must also give individuals the opportunity to object.

Under the UK GDPR, individuals can object to the processing of their data including processing by an ambient scribe. It is important therefore to be able to comply with the extent of their specific objection when this is appropriate.

To uphold this right, you should:

• provide sufficient information to individuals before the data capture or use takes place (before the ambient scribe begins ‘listening’), so that individuals have sufficient time to understand the processing and raise an objection if they wish
• ensure that individuals are made aware of their right to object, either through appropriate written materials or verbal notification
• ensure during procurement that your chosen tool has functionality that will allow for an objection to be upheld, e.g. by preventing it from processing data once someone has objected
• ensure that your processes factor in the potential for individuals to object, for example by making sure that health and care professionals are prepared to create their own summaries when the tool cannot be used

Objections can be overridden where there are ‘compelling legitimate grounds’ to continue to process data in this way which outweigh the rights of the individual. However, this should be assessed on a case-by-case basis. 

Separately, individuals can dissent to the use of their data under common law. In practice, this means that if an individual does not want the tool to be used, you cannot then use it. 

Where you have already used the tool on the basis of implied consent, dissent can only be applied to its use moving forward (not the use that has already happened). Individuals can still object to the processing, for example the ongoing storage and use of the data in the tool.

The right of access

Individuals have a right to receive copies of information you hold about them. In the case of an ambient scribe, this would apply to data inputted (like audio recordings) and any output (like transcripts, notes and summaries).

To uphold this right, you should:

• ensure that your chosen tool has functionality that enables you to extract relevant information in a format that is accessible to the general public, for example, ensuring that when audio recordings or transcripts are retained and a request is received, they can be downloaded in a shareable format from their storage location
• ensure that your Subject Access Request (SAR) processes take into account information held in your chosen tool and that relevant staff are able to access these to facilitate a SAR
• ensure that, once a SAR has been received, any deletion of the requestor’s data is paused
• keep in mind that your process will need to allow you to do this within one month

In certain circumstances individuals may also have the right to access information about deceased people. In these cases, the Access to Health Records Act will apply to the inputs and outputs of an ambient scribe.

The right to rectification

Individuals have the right to request that inaccurate or incomplete data you hold about them is corrected.

An audio recording can be seen as a factual capture of a conversation. However, an individual may still contest the accuracy of its content. For example, if a health or care professional makes a statement during the recorded conversation about the individual having a certain diagnosis, when the individual does not have that diagnosis.

A transcript that is generated by the AI may not accurately record a conversation, in the same way that the AI output generated from an audio recording may not be accurate.

If the audio recording or transcript is only used to produce an output (such as a summary or a letter) and not retained or used beyond this, if there is a mistake or an inaccuracy, when the output is corrected and the audio recording and transcript are deleted, this right will have been upheld.

Audio recordings and transcripts may be retained for a more significant amount of time or used for other purposes (e.g. beyond the initial production of a summary or letter). In this case, while you are not expected to amend the audio recording or transcript itself, which may remain an accurate capture of the conversation (including the mistake), you would have to ensure that records reflect the fact that information in the recording or transcript is inaccurate. This would ensure that decisions are not based on it, and any onward use of the recording or transcript, or summaries based on that recording or transcript, do not inadvertently repeat the inaccurate information onto other records.

It is also possible that, while the content of a recording is accurate, an ambient scribe may still produce inaccurate outputs. Outputs must always be checked by the responsible professional to ensure that the outputs are correct before they are added to a health and care record. If incorrect information is saved into other records, it is important that those records are corrected.

To uphold this right you should:

• ensure that it is possible within your chosen tool or processes to highlight inaccuracies in a way that sufficiently protects against the risk of this information being taken or used as fact in decision-making processes
• ensure that your chosen tool and storage of outputs allows for editing or updating of incomplete or inaccurate information

The right to erasure

There are a number of circumstances where the right to erasure may not apply in health and care settings. This includes where you process data for one of the following reasons:

• to perform a task carried out in the public interest
• for public health purposes
• for the purposes of preventative or occupational medicine
• for medical diagnosis
• for the provision of health or social care
• for the management of health or social care systems or services

Therefore, in the case of an erasure request it is important to assess your lawful basis for continued retention on a case-by-case basis depending on the reasons for processing and retaining the information.

The right to restrict processing

When an individual has challenged the accuracy of the data you hold about them (right to rectification) or they have objected to the processing (right to object), they also have the right to ask you to restrict the processing of their data until their request has been considered. This means that you must not do anything with the data, including deleting it, until the restriction is lifted.

When you are using automated systems, technical measures may be required to ensure that processing does not take place while a restriction is active.

In order to uphold this right, you should:

• ensure that you have a way of marking data as “restricted”, either within your chosen tool or in a separate place such as the health and care record. This should prevent staff from continuing to use the data while a restriction is active
• ensure that your chosen tool has functionality to retain records (recordings, transcripts and outputs) for longer when they are subject to a restriction
• ensure that your processes factor in the potential for individuals to request restriction. For example, you could make sure that health and care professionals are prepared to create their own summaries when the tool cannot be used

Rights related to automated decision-making

Automated decision-making means decisions made without any human involvement. The purpose of the ambient scribe is to act as a tool to aid health and care professionals with creating notes and other documents. They do not to make decisions about an individual’s care. Therefore this right does not apply.

The national data opt-out will not apply when you are using an ambient scribe for individual care.

A DPIA is highly likely to be a legal requirement for the use of an ambient scribe in a health or care setting because:

• they usually require large-scale processing of special category data
• they usually involve systematic evaluation of personal aspects about individuals based on automated processing
• the technology and its use in health and care is new and not widely understood by the public

NHS England has produced a tailored DPIA template for the use of an ambient scribe for individua care which can help you fulfil this requirement.

As a possible addition to your DPIA, the ICO has produced an AI and data protection risk toolkit which provides practical pointers on potential risks and how to mitigate these. The ICO also provides further guidance on the accountability and governance implications of AI.

It is important to have appropriate documentation in place which details your arrangements with any parties involved in sending, receiving or using data as part of your use of an ambient scribe. Your agreements should clearly document the processing as well as the roles and responsibilities of each party.

NHS England provides a template DSPA which can be used to fulfil this requirement.

It is important that staff are aware of their obligations and the requirements associated with the use of an ambient scribe. These obligations include its permitted use, undertaking appropriate checks and knowing how to comply with individual rights. You should consider the best way to socialise these requirements in your setting. This may include:

• providing training to users of the tool including assessments of competence
• writing policies or guidance specific to your setting, including an AI policy, and monitoring compliance