- There is increased urgency as autonomous AI takes on increasingly important medical tasks with less direct oversight.
- The question is who is responsible if something goes wrong.
Driving the news: The American Medical Association this week called for federal and state legislation to protect doctors from liability when AI products harm patients, even if doctors use the tools as intended.
- The group is also pushing for new laws to require AI companies to carry adequate liability insurance and disclose product limitations and performance data.
- Hospitals, health systems, insurers and health technology providers should be responsible if required use of specific AI products results in harmful errors, the AMA policy says.
Between the lines: The group’s push comes as AI clinical tools flood the healthcare system, unlocking new efficiencies but giving algorithms increasing power to make decisions, like ordering a diagnostic test or prescribing an expensive drug.
- Legal and medical experts point to the possibility of triage errors, in which AI systems that process clinical histories, vital signs and imaging may misjudge the severity of a case without the treating physician’s knowledge.
- The AMA also raises the possibility that an AI system could make a plausible treatment recommendation, but with incomplete lab results or outdated diagnoses. In such cases, a doctor may not be able to verify the entries.
- In cases where a patient suffers harm, liability should fall on AI manufacturers and providers — and potentially health systems that require the use of AI systems — rather than filing a medical malpractice claim against the doctor, the AMA asserts.
Context: There have already been lawsuits over medical decisions based on algorithms, including a wrongful death case in Alabama involving a heart abnormality screening tool.
- The software developer was not held responsible because its licensing agreement left final medical decisions to doctors.
The big picture: Courts across the country are finding that companies that design, train and deploy AI systems are responsible for what those systems do, said Martin Hilbert, a professor at the University of California, Davis, who studies the role of digital information and algorithms in society.
- Accountability depends on factors such as who shares responsibility when multiple parties build an AI system, how to prove errors in an opaque model, and what to do. when damage only surfaces after system deployment.
- “Different parties are responsible for different cases, and you can’t just hold the end user responsible,” he told Axios.
- WADA wants to address situations where the use of an AI system is mandatory.
What we’re looking at: The Food and Drug Administration is considering whether a new framework is needed to regulate AI-based medical devices.
- But the agency oversees software that meets the definition of a medical device, not all AI in health care.
- Hospitals generally argue that it is the responsibility of developers and vendors to monitor system performance, saying the “black box” nature of many systems can make it difficult to identify flaws.
- AdvaMed, the trade group representing device manufacturers, urges the FDA to continue using its current approaches to evaluating AI-based generative medical devices. He also wants the agency to avoid post-market surveillance that imposes additional burdens regardless of a product’s actual risk.
Gn bussni
