Shadow AI is creeping in to healthcare - leaders need to recognize the risk before it’s too late | Tech Radar
Overview
News, deals, reviews, guides and more on the newest computing gadgets
Start exploring exclusive deals, expert advice and more
Details
Unlock and manage exclusive Techradar member rewards.
Unlock instant access to exclusive member features.
Get full access to premium articles, exclusive features and a growing list of member rewards.
Shadow AI is creeping in to healthcare - leaders need to recognize the risk before it’s too late
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Given the pressure that NHS doctors are under, it’s no surprise that many trusts are turning to AI for help. On the whole, this is a good thing.
AI tools can play a valuable role in managing workflows, automating documentation, coordinating care rotas and streamlining communications.
However, the pace of AI development, and generative AI in particular, has accelerated dramatically.
As a result, governance frameworks are failing to keep pace with frontline demand and healthcare practitioners have begun to use tools that have not been authorized by their organisation.
This is causing problems at all levels - particularly when those tools are used to support clinical decision-making.
Why health AI needs a new approach, not just smarter algorithms
‘You fix it by making the secure option just as fast and frictionless as the risky one’: Practical advice on addressing shadow AI
“Shadow AI” is a term commonly used to describe the unauthorized use of AI within organizations. While the phenomenon exists across all industries, it has been growing rapidly in healthcare. One recent US survey revealed that 17% of healthcare workers admitted to using unauthorized AI tools in the workplace.
A another survey found that 39% of frontline healthcare staff say they use generic, free AI tools weekly or more. These are not isolated experiments. AI is already becoming embedded in day-to-day operations, often without formal governance or oversight, despite well-documented risks including hallucinations, inconsistencies and biases.
The reasons for this are understandable. Healthcare professionals are dealing with staff shortages, rising patient demand and overwhelming administrative workloads. Clinicians are expected to document more, communicate faster and process growing volumes of information - all while maintaining high standards of patient care.
It’s not surprising that they are reaching for the nearest AI tool. Particularly when that AI tool is extremely accessible, free or low-cost and easy to use. Shadow AI usage in the healthcare sector currently ranges from frontline clinicians right through to back office functions such as report drafting and recruitment support.
Most health professionals using “shadow” tools are not deliberately flouting rules, therefore. They are simply accessing some much-needed help from a convenient source. And their employer is not providing them with a robust alternative.
Shadow AI is a business design problem, but it can be overcome
Google's future for healthcare has AI for both the clinician and the patient
If everyone is rushing to board the AI ship why are so few workflows secure?
Medical organizations, and the NHS in particular, have often been cautious to roll out new technologies, though frequently with very good reason. Clinical systems require extensive testing, strong evidence bases and rigorous governance before deployment and decisions supported by AI can have direct consequences for patient outcomes.
Any clinical decision-making tool must therefore be thoroughly tested, based on solid medical research and exist under constant review and improvement. Yet delays allow shadow AI to fill the gap.
AI tools of the right quality are now reaching the market but the problem remains as to how leaders can rein in the use of shadow AI. It’s a problem that must be addressed quickly because the stakes are high.
The risks of unsanctioned AI usage include security breaches and privacy violations as well as the top concern - patient care.
In terms of data security and privacy, generic AI solutions are effectively “black boxes”. Ask Chat GPT whether sensitive health data will be safe in its hands, for example, and it will advise that it does not comply with health information regulations.
This is not surprising - patient information is “special category data” under GDPR and frequently requires additional protection - but it illustrates the danger of submitting too much information into an unauthorized app.
When it comes to patient care, generic AI solutions used to support clinical decision making carry an inherent risk: even when outputs appear credible, errors may be difficult to detect and assess. They may pull information from a broad range of sources which may be outdated, unaccredited or inaccurate.
A clinician in a hurry, referring to an unauthorized tool, is taking a big chance. If an unauthorized tool provides incorrect guidance relating to symptoms, medications, diagnoses or treatment pathways, the consequences could be severe.
Guidance and leadership, not heavy-handed authority
The answer is not a blanket ban, which is likely to send unauthorized AI further into the shadows. The focus for healthcare leaders must be to establish clear guidelines on the use of AI and to invest in education and training regarding its risks and benefits.
At the same time, organizations need to provide clinically validated, secure and easy to use alternative tools. If approved systems are too slow to deploy or fail to meet operational needs, clinicians and administrators will inevitably continue to turn to unauthorized tools instead.
Governance structures are also an important part of the process. These should include IT teams, security specialists and operational management because AI deployment simultaneously affects patient care, compliance, workforce efficiency and organizational risk.
AI will undoubtedly become an increasingly important part of modern healthcare delivery. The organizations that succeed will not necessarily be those that adopt AI fastest, but which create trusted frameworks on which clinicians can access secure, medically robust and properly governed tools.
We feature the best Electronic Health Records software.
This article was produced as part of Tech Radar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of Tech Radar Pro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit
You must confirm your public display name before commenting
1 Don't buy a new work PC right now — memory crunch and price rises lead to global shipments falling for the first time in two years
2'Not for everyone': Sony announces its most premium wired earbuds yet, but their ideal use case is nothing if not niche
4‘Lies about driver support’: Valve now lets you install Windows 11 on a Steam Machine, but many people don’t want to be fooled twice
5 Microsoft is moving to annual price adjustments for its commercial cloud services — could more bill shock be on the way?
Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.
© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.
Key Takeaways
- News, deals, reviews, guides and more on the newest computing gadgets
- Start exploring exclusive deals, expert advice and more
- Unlock and manage exclusive Techradar member rewards
- Unlock instant access to exclusive member features
- Get full access to premium articles, exclusive features and a growing list of member rewards
