What risks should healthcare institutions
know when using AI?
Hospitals, insurers and laboratories use AI for many tasks, from summarising discharge reports to assessing provision requests. In this article we describe six risks that an institution processing health data should know, using real incidents from recent years.

The Economist published “The World Ahead 2027”, its annual outlook issue for the coming year, on 17 November 2026. According to news reports, the magazine lists AI-related risks among the main themes of the year alongside wars, debt and climate events, and writes that cybersecurity threats linked to AI are attracting greater attention.
In the six sections below, we first define each risk briefly and then explain it through incidents that have taken place.
1. Supply chain: data can leave your institution even when the breach is elsewhere
Every digital service an institution uses depends on other services. The weakest link in this chain also determines the security of the institution at its end.
In February 2024, Change Healthcare, which provides clearinghouse services for healthcare payments and claims in the United States, was hit by a ransomware attack. The attackers got in through a remote access service that did not use multi-factor authentication. Payment and claims processing was disrupted for about two months, and pharmacies and hospitals across the country faced serious problems during that time. The company paid a ransom of 22 million dollars. The number of affected individuals grew over time and reached about 193 million in 2025.
In June 2024, Synnovis, which provides laboratory services to hospitals in London, was hit by a ransomware attack. Blood matching tests were disrupted and blood testing was reserved for the most urgent patients. According to NHS England, more than 800 planned operations and 700 outpatient appointments at the two most affected hospital trusts had to be rearranged in the first week. Type O blood was used for patients who could not wait for blood matching.
The same risk applies to AI services. Fireworks AI, which runs open-weight AI models for its customers, disclosed a new breach in November 2026. According to the company, the attacker reached its development environment through a third-party cloud provider that hosted its research and development machines, and accessed datasets and incomplete model weights belonging to some customers.
The effects of the breaches at the company reached its customers. Notion said that some chat transcripts used in one of its AI features were affected. Uber reported that driver's licence information of about 150,000 drivers was affected in a breach at a service provider, and news reports identified that provider as Fireworks AI.
2. Everything typed into AI may be stored somewhere
A question that a physician or an insurance specialist types into an AI tool often contains patient information. These questions and answers may be stored on the service provider's servers.
In early 2023, it emerged that Samsung employees had entered confidential information, including the company's source code, into a general-purpose chatbot. In May 2023, Samsung temporarily restricted the use of generative AI tools on company devices.
In January 2025, security researchers found that a database belonging to a China-based AI company was accessible on the internet without a password. The database held more than one million log lines, including plain-text chat histories and API keys. The company closed the exposure shortly after it was notified.
An institution therefore needs to know not only where its data is processed, but also whether conversations are recorded, how long they are kept and whether they are used to train the model.
3. The model changes silently
A general-purpose language model is updated by its provider from time to time, and users are not always told about these updates in detail. A study by researchers from Stanford and UC Berkeley, published in 2024, showed that the behaviour of a language model service offered under the same name changed markedly on some tasks within a few months.
In healthcare, this means that a drug interaction check or a coding suggestion that worked correctly yesterday may give a different result today without the institution knowing. An institution needs to know whether it can pin the model version it uses and whether it can measure results on its own cases before every change.
4. Access keys and permissions
In the Change Healthcare attack, the attackers got in through a single remote access service without multi-factor authentication. The exposed database of the China-based AI company also contained API keys. In its October 2026 statement, Fireworks AI said that the attackers were linked to a financially motivated group whose aim was to sell access to large language models obtained through unauthorised access. In security circles, such attacks are called “LLMjacking”.
In applications developed at speed, leaving access keys in code or in open configuration files is a known type of error. A single leaked key creates both a cost risk and a data access risk.
5. An AI agent oversteps its limits
AI no longer only answers questions. AI working as an agent can also take actions in systems on its own, and this creates a new risk.
In June 2026, an AI agent run by a major AI company during an evaluation accessed non-public files on Australia's Medicare statistics portal. The portal had blocked the agent's access, and the agent tried other ways to reach the information. The portal held non-sensitive data such as spending statistics, and the company said it had found no evidence that patient records were accessed. The company reported the incident to the government about three months later. The Australian Prime Minister criticised the delay in notification and conveyed his concern directly to the company's leadership.
This incident shows that an institution needs to define clearly the limits of the permissions given to an AI agent and to monitor them.
6. Legal framework: health data is special category personal data
Turkey's Law No. 6698 on the Protection of Personal Data (KVKK) classifies health data as special category personal data and subjects its processing to strict conditions. Processing data on a server abroad is subject to the law's rules on cross-border transfer. Which country an AI service processes data in is therefore a direct legal question.
Under the law, the institution is the data controller. When a vendor processes data on the institution's behalf, the institution is jointly responsible with that vendor for taking data security measures.
Questions to ask a vendor
Before choosing an AI solution, an institution that processes health data should obtain written answers to the following questions.
- In which country and on which server is the data processed and stored?
- Which subcontractors does the service depend on? Are the model provider, cloud and logging services on this list?
- Are conversations recorded, how long are they kept, and are they used to train the model?
- Can the model version be pinned, and is the institution informed before a change is made?
- Is multi-factor authentication used for all remote access, and how are access keys stored?
- Is it restricted which actions AI agents can take in which systems?
- Are access logs shared with the institution, and how quickly is the institution informed if a breach occurs?
- Can the solution run on the institution's own servers or in an isolated environment?
- Do the AI's answers come with the sources they are based on, and can they be audited afterwards?
These questions do not suggest staying away from AI. They help the institution know who operates the AI, where the data is kept and under whose control the model changes.
Our approach
At Opinion AI, we are building MINA to answer these questions from the start. On our clinical platform TIS, model and tool calls run without going to an external AI service, and patient data is not sent to an external AI service. MINA's suggestions come with their reasoning and the sources they are based on.
We measure MINA's versions on fixed test sets, and a new version is put into use only after it passes this measurement. The physician always makes the decision.
You can read how MINA was built on the MINA page and our approach to data protection on the Ethics and compliance page. If you would like to see it in your own institution, you can request a POC.
Related: The FBI's ARTEX alert and healthcare institutions · TIS for hospitals
Sources: The World Ahead 2027: Türkiye Today and Azernews, 17 November 2026; Change Healthcare: HIPAA Journal; Synnovis: NHS England statement, Security Affairs, June 2024; Fireworks AI: BleepingComputer, 10 November 2026; Notion: BleepingComputer, 11 November 2026; Uber: BleepingComputer, 17 November 2026; Samsung: TechRadar, 2 May 2023; exposed database: Wiz Research, 29 January 2025; model drift: study by researchers from Stanford and UC Berkeley, 2024; Medicare portal: CNBC and iTnews, 24 September 2026; Law No. 6698 on the Protection of Personal Data. Information about companies is given as reported in these sources.
SSS
What risks should healthcare institutions know when using AI?
The article covers six risks. These are the supply chain, the storage of information typed into AI, silent model changes, access keys, the permission limits of AI agents and legal responsibility under KVKK.
Which questions should be asked of an AI vendor?
An institution should ask in which country and on which server data is processed, who the subcontractors are, whether conversations are stored, whether the model version can be pinned, how multi-factor authentication and agent permissions are handled, how quickly breaches are reported and whether answers come with their sources. The answers should be obtained in writing.
What does KVKK provide for health data?
Law No. 6698 classifies health data as special category personal data and subjects its processing to strict conditions. Processing data abroad is subject to the rules on cross-border transfer. The institution is the data controller, and when a vendor processes the data, the institution is jointly responsible with the vendor for security measures.