Criteria for choosing a hospital clinical decision support system

What a clinical decision support system does and why choosing one is hard

A clinical decision support system is software that brings patient data and clinical knowledge together at the point of decision and gives the physician recommendations, alerts and checks. A good system does not decide in the physician's place; it puts the information behind the decision in front of the physician, together with its rationale.

The first difficulty is that systems look alike in presentations; the real differences appear when they work with the hospital's actual records. The second is scattered data. Information about the same patient sits in the hospital information system (HBYS in Türkiye), in Medula records and in discharge summaries written as free text, each in a different format. Across the health system, the rate of unnecessary tests is put at 50%, and the root of that problem is data quality; 35% of physicians' time goes to documentation rather than to patients.

The third difficulty is local context. In Türkiye, the context of a decision is set by discharge summaries written in Turkish, by SUT, the Healthcare Implementation Communiqué published by the Social Security Institution (SGK), and by Medula, SGK's pre-authorization and billing system. The fourth is data protection. Under KVKK, Türkiye's personal data protection law, health data is a special category of personal data, and where that data will be processed is the first thing to settle.

Six criteria to check

The criteria below are not tied to any product; each one can be put to a vendor directly during procurement.

1. Confirm that the system understands the local clinical language and regulations. It should correctly read discharge summaries in the local language, including specialty-specific terminology, and check ICD-10 coding, SUT rules and Medula pre-authorization rules within the workflow. Ask how the model was adapted to the language and which specialties it covers; hospital groups planning a deployment in another country should ask the same question for their own language and reimbursement rules.
2. Make sure patient data stays inside the institution. Ask whether the system can run on the hospital's own servers (on-premise) or in an isolated cloud dedicated to the hospital, and whether the institution's data is used for any other purpose, such as training the model. Health data is a special category of data under the EU's GDPR, as it is under KVKK. Because the hospital is the data controller, request the architecture documentation and data flow diagram in writing.
3. Check that every recommendation comes with its rationale and source. The answer should show which guideline or literature a recommendation relies on, which information comes from the patient's documents and which is the model's own inference. A system that does not cite its sources cannot be audited, even when its answer is right.
4. Require integration without replacing the existing hospital information system. Decision support should be added on top of the hospital information system, the institution's record backbone, not replace it. Clarify at the outset how integration will work (file export, a database connection or an application interface) and what will change in the physician's workflow. A system that adds extra screens and data entry for physicians runs a high risk of falling out of use.
5. Ask who validates the model clinically, and how. Find out which specialties the reviewing physicians come from and how often they review the model. Ask which model version, how many protocols and how many cases the published accuracy figures are based on. These figures are a model evaluation; only a study on your own data shows the result in your hospital.
6. Verify that outputs can be audited and the decision stays with the physician. Each recommendation should be logged with who saw it, whether it was accepted or rejected, and the rationale it relied on. For rule-based checks such as reimbursement and coding, the result should come from a rule engine, with AI explaining it. Critical decisions such as approvals, rejections and prescriptions should always remain with people.

Running the evaluation step by step

Once the criteria are clear, the selection should move from comparison on paper to testing in the field.

1. Define in writing the problem to be solved, for example discharge summary coding, pre-authorization or a SUT check before billing.
2. Bring physicians, the IT and information security team, the unit responsible for personal data protection and the billing or insurance unit into the evaluation team from the start, and have this team review the vendor's architecture documentation and data processing terms.
3. Write down the success criteria before the POC begins: the task, the sample of records and the result that will count as acceptable should be set in advance.
4. Run the POC with a single module on the hospital's own records, and have physicians review the outputs against existing decisions.
5. Assess the result for accuracy, physician acceptance, effect on workflow and auditability. Decide on a wider rollout only then, and add further modules one at a time.

The role of AI in clinical decision support

Large language models have widened the scope of clinical decision support for tasks such as reading free-text discharge summaries, summarizing scattered records and answering questions physicians ask in their own words. A model that is not tied to sources, however, can produce answers that are fluent but unsupported. Clinical use therefore calls for three things together: a retrieval layer that ties answers to peer-reviewed literature and clinical guidelines, a presentation that keeps information from documents apart from the model's inference, and a rule engine that applies rules the same way every time. The right place for AI is to put a reasoned summary and recommendation in front of the physician; the decision stays with the physician and the relevant specialist.

How Opinion AI meets these criteria

TIS, Opinion AI's clinical decision platform for hospitals, runs on the clinical AI model MINA and meets the criteria above as follows.

• MINA is adapted to Turkish clinical language. An open-weight base model was adapted to Turkish and the clinical language of 14 specialties through continued pre-training (CPT), and LoRA adapters add specialty, institution and task expertise. TIS checks ICD-10 coding, SUT compliance and Medula pre-authorization rules in the same flow.
• Data stays inside the hospital. TIS is installed on the hospital's own servers or in an isolated cloud dedicated to the hospital. The hospital is the data controller; personal data is masked in line with KVKK, and Opinion AI does not collect personal data.
• Every answer comes with its source. Through GraphRAG over a clinical knowledge graph, MINA draws on 30M+ sources, including PubMed, The Lancet, JAMA, BMJ, UK Biobank, MIMIC-IV and ClinicalTrials. Information from the patient's documents and MINA's own inference are shown separately in every answer.
• The HBYS stays as it is. TIS is added on top of the existing HBYS as an intelligent layer and receives records through XML/CSV export or a direct database connection.
• Physicians oversee the model. The Physician Ethics and Advisory Board, with physicians from 15 specialties, regularly reviews the model's clinical accuracy and ethical boundaries. In MINA's latest release evaluation, model results on hospital tasks were 87% for ICD-10 coding, 89% for discharge summary field extraction and 91% for SUT decision accuracy (MINA v3.7 · 8 protocols · ~9,300 cases). Details are on the MINA v3.7 page.
• People make the decision. The rule engine produces the result of rule-based checks, and AI only explains it. The physician confirms, corrects or rejects each recommendation, and every output can be audited later.

Opinion AI works as an intelligent layer on top of the HBYS in more than one large private hospital group in Türkiye and has started POCs with a number of hospital groups in Northern Europe; the details are on the Success Stories page. For the physician side, see clinical decision support for physicians; for data protection, see ethics and compliance.

Frequently asked questions

What is a clinical decision support system?

A clinical decision support system is software that brings patient data together with current clinical knowledge at the point of decision and gives the physician recommendations, alerts and checks. Typical outputs include diagnosis and treatment suggestions, drug interaction and allergy alerts, ICD-10 code suggestions and checks against reimbursement rules. The system supports the decision; the physician makes it.

Does a hospital need to replace its hospital information system to use clinical decision support?

No. A well-designed clinical decision support system does not replace the existing hospital information system (in Türkiye, the HBYS); it is added on top as a layer, reads records through file export or a database connection, and leaves physicians' workflows as they are. Opinion AI's hospital platform, TIS, works this way.

Can AI-based decision support be used without patient data leaving the hospital?

Yes. When the system runs on the hospital's own servers (on-premise) or in an isolated cloud dedicated to the hospital, data does not leave the institution and the hospital remains the data controller. Health data is a special category of personal data under both KVKK, Türkiye's personal data protection law, and the EU's GDPR, so where the data is processed is the first question to settle.

Is a general-purpose AI chatbot enough for clinical decision support?

Not on its own. General-purpose chatbots are not connected to the hospital's records, are not designed to apply local reimbursement rules such as Türkiye's SUT and Medula rules consistently within the workflow, and often require data to be sent to a service outside the institution. Clinical use calls for a system that cites its sources, separates documents from inference and runs inside the institution.

How should the accuracy of a clinical decision support system be evaluated?

Ask which model version, how many protocols and how many cases the vendor's figures are based on, and read those figures as a model evaluation. The real result in a hospital is shown by a POC run on the hospital's own records against success criteria written down in advance, with physicians reviewing the outputs.

How should a POC for a clinical decision support system be set up?

A POC (proof of concept) should start with a single use case, such as discharge summary coding or a SUT check before billing. The scope, the records to be used and the success criteria are written down in advance, and physicians and the relevant administrative units review the outputs. If the result is positive, the system is extended one module at a time.

To evaluate TIS in your own HBYS environment through a POC that starts with a single module on your own records, use the POC Request form. You can read how data stays inside the institution in the guide on using AI without patient data leaving the institution, and how the patient profile is used at the point of decision in the guide on the patient digital twin.