Skip to Content
AI 6 min read

AI in clinical workflows: where a person must sign off

AI can remove repetitive work from clinical teams. It should never be the last step in a clinical decision.

Where AI helps, and where it must stop

Clinical teams spend hours on work that is repetitive and rule-bound: reading referral letters, sorting worklists, drafting summaries, suggesting codes. AI can take some of that weight. It becomes a risk when it is allowed to finish a clinical decision that a qualified person has not actually made.

The useful question is not whether a feature uses AI. It is what the feature is allowed to do, who decides, and how that decision is recorded.

Classify every AI task by what it is allowed to do

LevelWhat the system doesWho decidesExample
AssistPrepares information: sorts, extracts, summarisesThe person reads and decidesOrdering a worklist so urgent items come first
SuggestProposes a specific value or actionThe person accepts, edits or rejectsA suggested code or drafted note text
Act with sign-offCompletes an action only after approvalA named person, recordedReleasing a report after review
Act aloneCompletes an action with no person involvedNobody at that momentNot appropriate for clinical decisions
Ladder of four levels of AI in clinical workflows: assist, suggest, act with sign-off, and act alone, which is not appropriate for clinical decisions.
Classify every AI task by level, and name who signs off at each level that needs it.

Define sign-off in the workflow, not in a policy document

A policy that says "a clinician reviews AI output" protects nobody if the screen makes it easier to accept than to read. Design the sign-off into the workflow itself.

  • Each AI task has a named role that signs off, not "the user".
  • The sign-off records who approved, when, and what they were shown.
  • The person sees the source data, not only the suggestion.
  • Rejecting a suggestion is as quick as accepting it, and can carry a reason.
  • Nothing is accepted by default, and clinical content cannot be approved in bulk.

Log what the AI saw, suggested and what the person did

  • Input. A reference to the data the AI worked on.
  • Version. Which model or rule set produced the output.
  • Output. The suggestion exactly as it was shown.
  • Decision. Accepted, edited or rejected, by whom and when.

These records let you audit a decision, review patterns of error, and answer questions from clinicians, patients and regulators with evidence rather than recollection.

Test failure before you rely on success

  1. Cases with no right suggestion. Check that the system can say nothing rather than guess.
  2. Poor or incomplete data. Use scanned, partial and inconsistent inputs, because real records look like that.
  3. Arabic, English and mixed text. Test each language and the mix, including names and units.
  4. Unavailability. Switch the AI service off and confirm that the clinical workflow still completes.
  5. Review over time. Sample outputs regularly to catch drift after data, models or workflows change.

Questions to ask any vendor about AI in a clinical system

  • Which tasks use AI, and at which level for each?
  • Who signs off, and where is the sign-off recorded?
  • What data does it use, and where is it processed and hosted?
  • How is it evaluated, on what data, and how often?
  • What happens when it is wrong, and when it is unavailable?
  • Can it be switched off by facility or by task?
  • How are changes to the model validated and communicated?
  • Do the same access controls and audit trail apply to AI use as to any other access to the record?

Data and privacy

Patient data used by an AI feature is still patient data. The same access rules and audit trail should apply, and the location of processing should be agreed in writing, for each deployment, before go-live. Read this alongside the integration readiness guide, because AI often depends on the same interfaces.

What a clinician should see on screen

The screen decides whether sign-off is real. A good design shows the suggestion, the source it came from and the action required, in that order. It never hides the source behind a click, and it never makes the suggestion look like a fact.

  • Label. Mark AI-generated content clearly, in the same place every time.
  • Source. Link each suggestion to the data it used.
  • Action. Make accept, edit and reject equally easy.
  • Uncertainty. Show when the system has little to go on, instead of presenting every output with the same confidence.

Roles and accountability

Name an owner for the AI feature itself, separate from the clinicians who use it. That person is accountable for evaluation, change control and incident review, and works with the clinical, privacy and IT leads. Without an owner, nobody notices when behaviour drifts.

Where this fits in a procurement

Add AI questions to your requirements list, and ask for a demonstration of failure and downtime behaviour, not only the successful path. The HIS procurement checklist shows where these questions belong.

Frequently asked questions

Should AI make clinical decisions?

No. In clinical workflows AI can prepare, sort or suggest. A qualified person decides and signs off, and the sign-off is recorded.

What should be logged when AI is used?

The input reference, the model or rule version, the output as shown, and the person's decision with time and identity.

Can AI help with result triage?

It can help order a worklist so urgent items are seen first, provided the order never hides an item and a person still reviews each result.

How do we evaluate AI features during procurement?

Ask for the task classification, the sign-off design, evaluation evidence on comparable data, and a live demonstration of failure and downtime behaviour.

Planning a deployment?

Evaluating AI in a clinical system? Tell us about your workflows and we can map where a person must stay in the decision.

Plan a discovery workshop

More insights