Starting a hospital AI project with a proof of concept

The purpose of a POC in an AI project

A POC is a limited piece of work that shows whether a solution does the expected job on the hospital's own data and workflow; a pilot, by contrast, puts the solution into daily use with real users in a limited unit.

In AI projects this step matters even more. The model results a vendor shares are based on a particular test set and should be read as a model evaluation; the hospital's documentation style, coding habits and the structure of its HBYS, the hospital information system used in Türkiye, may differ from that test set. Language models can also present information that is not grounded in fact or in any source, and do so in convincing language. How often this error, known as hallucination, occurs on an institution's own records can only be measured by an evaluation that compares each output with the document and source it relies on. General-purpose chat tools such as ChatGPT or Gemini are not designed around a specific hospital's regulations, data and workflow, so general trials with such tools cannot replace a POC run inside the institution.

Setting the scope and data access

A POC should start with a single process and, where possible, a single module; testing several processes at once makes it hard to tell which result came from where. A suitable process usually has these features:

• A frequently repeated process builds up enough cases in a short time.
• A process whose outcome can be measured from records makes comparison with the earlier situation possible; billing denials, coding corrections and audit findings are already on record.
• A process with a clear owner settles from the outset who reviews the outputs and who decides on the result.
• A process that can run in parallel with the existing workflow can be tested without putting patient safety at risk; in the first stage, the AI's output is compared with the current process without influencing any decision.

Suggesting ICD-10 codes from discharge summaries, checking SUT compliance before billing, preparing pre-authorization files and monitoring inventory thresholds are typical starting points. SUT, the Healthcare Implementation Communiqué published by SGK, the Social Security Institution, sets the rules for public reimbursement in Türkiye; hospital groups in other countries work with their own reimbursement and coding rules. The scope document should state clearly which unit, which record types and which date range are included, and what is left out. Starting with a retrospective evaluation on past records whose outcome is known lets the system be tested without affecting patient care.

Data access is planned together with the scope. Health data is special category personal data under both KVKK, Türkiye's personal data protection law, and the GDPR. Because a POC works with real records, the system should process them without taking them outside the institution, on the hospital's own servers or in an isolated cloud environment dedicated to the hospital. The hospital remains the data controller; whether the vendor has access to the data, and if so in what role, should be set out in the contract. Only the fields in scope should be used, identifying details should be masked, access should be granted by role, and every access should be logged.

Defining success criteria before the POC starts

Success criteria are written before a POC begins; with a criterion set afterwards, any result stays open to interpretation. Thresholds differ between institutions, so the steps below give no ready-made figures and describe how to define a criterion instead:

1. Define the output to be measured in one sentence; for example, measure whether the suggested ICD-10 codes match the coding team's final decision.
2. Before the POC starts, build a baseline by taking the current performance of the same process, in the same unit and over a comparable period, from the records.
3. Agree in advance which physicians or specialists will review the outputs and how disagreements will be settled.
4. Count the cases the system flags unnecessarily and the cases it misses separately, and define an acceptable level for each.
5. Add safety criteria specific to AI; check whether each recommendation shows its rationale and source, whether that source actually contains the information, and whether what the document says is shown separately from what the model infers.
6. Record whether the output adds work to the workflow and why users accept or reject a recommendation.
7. Agree the thresholds and the number of cases needed with the vendor, and write them into a document approved by the stakeholders.

The criteria should also cover auditability: it should be possible to see later which data, rule and source each recommendation was based on.

Stakeholders and moving to rollout

The POC should have a single project owner, and each stakeholder's role should be defined from the start:

• The chief physician or medical director takes clinical ownership and secures physicians' participation.
• The IT department manages the deployment environment, the data flow from the HBYS and access rights.
• Finance and medical accounting assess the effect on billing, denials and collections.
• The quality department tracks how the process relates to audit indicators.
• Physicians and the coding team review the outputs and determine the right answer.
• The data protection and legal teams approve the data processing framework and the contract before the POC starts.

At the end of the POC, the result is assessed against the criteria written at the start. If the criteria are met, rollout proceeds in stages:

1. Extend the same process to similar units and specialties first, and compare the first results in each new unit with the baseline.
2. Add a new module once the first process runs steadily, and write new criteria for each new scope.
3. Keep monitoring in live use, and revalidate the results whenever the model or rule version changes.

If the criteria are not met, the team examines whether the problem lies in the data, the scope or the solution; the decision is made before rollout.

Running a POC with Opinion AI

Opinion AI is a clinical decision support platform for hospitals and health insurers, built on MINA, a clinical AI adapted to Turkish. Its hospital platform, TIS, does not replace the existing HBYS; it is added on top as an intelligent layer and can be deployed on the hospital's own servers or in an isolated cloud assigned to the hospital. Deployment starts with a single module and grows module by module as needed, which fits the single-process POC approach.

A POC runs on the hospital's own applications and indicators with one chosen module; this might be, for example, the pre-authorization panel, the clinical assistant that works on ICD-10 coding, SUT checks and discharge summaries, or the audit cockpit that tracks BH and ADSH indicators. We define the scope together with the hospital, and the timeline is planned together according to the institution's infrastructure and the chosen module.

The data stays inside the hospital, the hospital is the data controller, and the data the model sees passes through KVKK-compliant masking; Opinion AI does not collect personal data. People make the decision. MINA shows the rationale and source of its recommendation, keeps what the document says separate from its own inference, and every output can be audited. This allows the AI-specific safety criteria of a POC to be checked directly.

MINA's open-weight base model has been adapted to Turkish and the clinical language of 14 specialties through continued pre-training (CPT) and specialized by specialty, institution and task with LoRA adapters; knowledge retrieval runs through agentic orchestration over a clinical knowledge graph (GraphRAG). The Physician Ethics and Advisory Board, with physicians from 15 specialties, regularly reviews the model's clinical accuracy and ethical boundaries.

In Türkiye, we work as an intelligent layer on top of the HBYS in more than one large private hospital group, and we have started POCs with a number of hospital groups in Northern Europe. The details are on the Success Stories page.

Frequently asked questions

What is an AI proof of concept (POC) in healthcare, and how does it differ from a pilot?

A POC (proof of concept) is a limited piece of work that tests, against criteria written in advance, whether an AI solution does the expected job on a hospital's own data and workflow. A pilot puts the solution into daily use with real users in a limited unit. The common sequence is POC, pilot, then rollout.

Which process should a hospital AI POC start with?

A POC should start with a single process that is frequently repeated, has an outcome measurable from records, has a clear owner and can run in parallel with the existing workflow. Suggesting ICD-10 codes from discharge summaries, checking compliance with reimbursement rules such as Türkiye's SUT before billing, preparing pre-authorization files and monitoring inventory thresholds are typical starting points.

How are success criteria set for an AI POC?

Success criteria are written before the POC starts. The output to be measured is defined in one sentence, a baseline is taken from the records of the current process, and who determines the right answer and which thresholds apply are agreed in advance. Specific to AI, it is also measured whether recommendations show their rationale and source, whether the source actually contains the information, and which cases the system misses.

Does an AI POC require patient data to leave the hospital?

No. Because health data is special category personal data under KVKK, Türkiye's personal data protection law, and under the GDPR, it matters that the POC runs on the hospital's own servers or in an isolated cloud environment and that the hospital remains the data controller. Opinion AI's POCs are set up this way: the data stays inside the hospital, the data the model sees passes through KVKK-compliant masking, and Opinion AI does not collect personal data.

How long does a hospital AI POC take?

The duration should be set less by the calendar than by the number of cases needed to assess the criteria reliably. In a frequently repeated process that number is reached sooner; preparing the deployment environment and the data flow, and the time of the reviewing physicians, also affect the timeline.

How does a hospital move from an AI POC to wider use?

The result is assessed against the criteria written at the start of the POC; if they are met, use is extended in stages. The same process is first taken to similar units, then new modules are added, with new criteria written for each new scope. Monitoring continues in live use, and results are revalidated whenever the model or rule version changes.

To scope a POC in your own hospital that starts with a single process, briefly describe your institution and the process you want to evaluate in the POC Request form. For evaluation criteria, see the guide on criteria for choosing a hospital clinical decision support system; for the data framework, see the guide on using AI without patient data leaving the institution.