It is widely recognised that AI technologies have the potential to positively transform healthcare by improving patient outcomes, supporting clinical decision-making and helping health services work more efficiently. However, AI systems can behave differently once they move from controlled testing environments into real-world clinical settings. The data, users, workflows and patient populations they encounter may also change over time, and these changes can affect performance.
Manufacturers of medical devices are required to monitor how their devices perform once they are in use. This is achieved through post-market surveillance (PMS), the ongoing process of collecting and reviewing information about a device's safety, performance and effectiveness throughout its lifetime. PMS helps manufacturers identify emerging issues, understand how devices are performing in real-world settings and take action where needed.
However, the importance of PMS extends beyond regulatory compliance. The National Commission into the Regulation of AI in Healthcare highlighted that AI regulation should be lifecycle-based, recognising that assurance does not end at market entry but continues throughout deployment and use. The Commission also emphasised that the safe adoption of AI is a system-wide responsibility, requiring manufacturers, healthcare organisations, professionals, regulators and others to play a role in identifying, managing and responding to risks as evidence emerges in real-world settings. Finally, as we move into a world where device performance is increasingly data sensitive and may rapidly change, proactively monitoring for and responding to emerging trends will be an important part of safe and responsible device deployment. This blog looks at what the AI Airlock has learned about applying PMS to AI-enabled medical devices, why practical challenges remain, and how the next phase of the programme will build on this work.
What earlier Airlock work told us
Over the past two years, during the pilot and Phase 2, the AI Airlock has explored PMS across a range of candidate technologies through workshops and engagement with regulatory, clinical and technical stakeholders. This work has helped the programme examine how existing PMS requirements apply to AI-enabled medical devices in practice and identify the areas where further evidence and testing are needed.
Phase 1 – FAMOS and continuous monitoring
The pilot phase first explored this challenge through Newton’s Tree’s FAMOS platform, one of the programme’s first four case studies, which examined whether continuous, near-real-time monitoring could strengthen PMS for AI-enabled medical devices. Building on the case study’s real-world testing across 2 deployment sites, a simulation workshop considered how monitoring could detect changes arising from the model, input data and interactions between users and the device, and how the earlier identification of emerging risks and performance changes might support risk management. The case study also explored monitoring measures related to data quality, performance drift and automation bias. Together, the work highlighted the potential for earlier identification of performance trends that could affect safety, while also raising practical questions about access to suitable real-world data, clinically meaningful thresholds, false alarms, stakeholder responsibilities and the action required when a signal is detected (AI Airlock Sandbox Pilot Programme Report; PMS simulation workshop report).
Phase 2: PMS and predetermined change control plans (PCCPs)
Phase 2 tested these questions further across several AI technologies. The Phase 2 findings, captured in the Programme Report, reinforced a central lesson: point-in-time, pre-market validation alone may not fully reflect how an AI medical device will perform in practice. Real-world performance can be difficult to reproduce or to address entirely or accurately in a controlled setting. That is why post-market monitoring needs to be considered early and prospectively designed as complementary to the device’s pre-market evidence, intended use and lifecycle plans.
Phase 2 also examined the relationship between PMS and predetermined change control plans (PCCPs), which describe planned modifications to software and how those modifications will be assessed, across several AI technologies, including through a dedicated simulation workshop. The work showed that managing change in AI-enabled medical devices requires a joined-up lifecycle approach: pre-market evidence should reflect the intended use, deployment environment and anticipated changes, while PMS should monitor real-world performance and identify changes that may affect safety, performance or the device’s intended purpose. The workshop also explored how manufacturers might assess whether a change is significant and whether it falls within an agreed PCCP or requires further regulatory action. Together, the findings highlighted the importance of defining monitoring measures, decision thresholds and responsibilities early, as well as proactively capturing usage and performance data post-market, while retaining sufficient flexibility to respond to evidence generated once the device is in use (AI Airlock Sandbox Phase 2 Programme Report and Insights from the AI Airlock Simulation Workshop: Post Market Surveillance and Predetermined Change Control Plans).
Why PMS remains a practical challenge
While the principles of PMS remain the same, applying them to AI medical devices can present different challenges compared with more traditional or relatively static devices. Unlike fixed devices, AI systems may be particularly sensitive to changes in input data, the clinical environment and the way people use and rely on the device, creating additional complexity when monitoring real-world performance.
The importance of PMS has also been reinforced by the National Commission into the Regulation of AI in Healthcare, which highlighted the need for stronger post-market assurance, including monitoring and the use of real-world evidence. The Commission recognised the Airlock’s findings that assurance for AI does not end at market entry, but must continue throughout the product lifecycle, with manufacturers, healthcare organisations, professionals, regulators and others all playing a role in the safe deployment and oversight of AI technologies.
Manufacturers may therefore need further support to translate existing PMS requirements into monitoring arrangements that are effective and appropriate for AI. The practical questions include what to monitor, how to set clinically meaningful thresholds, how to identify and investigate drift, how to obtain suitable real-world data, and when a detected change should trigger action.
Continuous or near-real-time monitoring may enable emerging trends to be identified earlier, but may not be appropriate in every case. The approach should be proportionate to the device, its risk and the context in which it is used, with a clear purpose for the data collected and agreed responsibilities for reviewing signals and responding to them.
Further clarity may also be needed on the respective roles of manufacturers, healthcare organisations and other actors. The aim is to provide regulatory confidence without imposing impractical monitoring arrangements or generating large volumes of data that do not support meaningful decisions.
What the next phase will explore
Building on these findings, the Airlock has selected post-market surveillance and lifecycle oversight as the first regulatory focus area for Phase 3, recognising the need for further evidence on how these approaches can be applied effectively and proportionately in practice (AI Airlock Phase 3 application).
To help shape this phase, the Airlock held a series of expert stakeholder workshops over the summer with colleagues contributing regulatory, clinical and technical perspectives. The discussions examined how PMS expectations apply to AI-enabled medical devices, including:
- proactive signal and trend detection, thresholds for action and rollback decisions;
- deployment context and stakeholder responsibilities;
- lifecycle and change management;
- communication; and
- additional challenges presented by remote and direct-to-consumer use or other non-clinical use environments.
These discussions informed the PMS challenge areas and the questions that applicants are now invited to explore.
Supported by multi-year funding from DHSC, Phase 3 will allow the programme to move beyond annual funding cycles, be more flexible in project design and pursue more ambitious, longer-term testing. Working with innovators and stakeholders, the Airlock will explore and test practical approaches to capturing deployment-context factors more proactively; defining meaningful thresholds and proportionate actions when signals arise; maintaining effective monitoring in remote, at-home or direct-to-consumer settings with varying levels of oversight; and ensuring that the right information on safety, performance and change effectively reaches the right people at the right time.
The aim is not to predetermine a single solution or policy position, but to generate evidence about which approaches work in practice, how they can be tailored to a device, its risk and deployment context, and where existing requirements are sufficient or further guidance, tools or regulatory support may be needed. Taking this approach, the Airlock hopes to identify practical, proportionate methodologies and/or processes that support innovation while protecting patients and the public as AI-enabled medical devices are deployed across the health system.
The Phase 3 call for applications is now open and applications can be submitted on a rolling basis. We welcome proposals from AI-enabled medical device developers and manufacturers, providers of solutions to regulatory challenges, and other stakeholders who can help test practical approaches to PMS and lifecycle oversight. If your organisation could contribute to these challenge areas, find out more and apply through the AI Airlock Phase 3 application page. AI Airlock Phase 3 application - GOV.UK.
Recent Comments