Let AI read the record before the clinician has to

Not another chatbot

The easiest version of AI in an EHR is a box. Ask a question. Get an answer. Useful, maybe. But it still leaves the clinician responsible for deciding what to ask.

I was more interested in something proactive. What if the first thing the EHR did when you opened a patient was work?

Not diagnose. Not prescribe. Not make a decision. Work.

Read the longitudinal record. Compare results and assessments. Check medications. Look for incomplete follow-ups. Notice recurring themes in sleep, mood or function. Connect events sitting months apart in completely different parts of the system. Then hand the clinician a much better place to start.

That became Patient State.

AI takes the first pass

The prototype begins when a fictional patient record opens. Instead of immediately throwing another dashboard at the clinician, the system starts preparing a brief.

Reviewing longitudinal history
Comparing recent results and assessments
Checking medications and follow-ups
Reviewing sleep, mood and function
Looking for unresolved care
Connecting related events
Prioritising recent changes

Those lines are not there to make AI look clever. They are there to make its job explicit. The AI is taking on the reconstruction work. The clinician still owns the meaning.

I started with a body. That was too small.

My first visual idea was a person made from thousands of particles. As AI read the record, the particles would assemble into a human form and areas requiring attention would begin behaving differently. A knee could pulse. A systemic change could move through the torso. A physical signal could become visible before the clinician had opened six tabs to find it.

It looked great in my head. There was one problem. A patient is not an anatomy diagram.

Some of the most important things in a record do not belong to an elbow, liver or left ankle. Sleep does not live in one convenient coordinate. Neither does anxiety. Neither does mood, fatigue, cognition, general function, bereavement, a missed follow-up or the fact that somebody has been struggling for three months.

If the interface could only visualize physical location, I had designed a very pretty way of making the EHR's existing problem worse. So the idea changed.

From body map to whole-person state

The constellation now has three conceptual layers.

The body. Used when physical location genuinely helps. A knee injury can be a knee. That is allowed.

The whole person. Used for systemic health and wellbeing. Mood. Sleep. Fatigue. Cognition. General function. Rather than making somebody's brain glow red because their PHQ-9 changed — which feels like a particularly efficient way to design something awful — the behaviour of the whole constellation changes subtly. The person is the unit, not the organ.

The space around the person. Used for relevant recorded context and care pathways. A missed review. An unresolved referral. A medication follow-up. Potentially relevant life context, but only when it is actually recorded and useful to the current conversation.

Suddenly the visualization stopped being a body map. It became an attempt to represent a person in context. That is much closer to what I wanted AI to understand in the first place.

Emma

For the prototype I created a fictional patient called Emma Richardson. She is 42 and has presented with fatigue, poor sleep and feeling increasingly overwhelmed. Her record contains several things worth noticing.

Her haemoglobin has changed across three tests: 132 → 119 → 108 g/L. Her PHQ-9 assessments have also changed: 6 → 11 → 16. Difficulty sleeping appears in three recent encounters. Sertraline was started in May. A six-week medication review was planned, but the EHR cannot find a completed review in the record.

None of those facts, individually, require AI to pretend it is a doctor. But finding all of them, comparing them, remembering them and placing them into one chronology is exactly the sort of work software should be very good at.

This is where AI actually takes over

The interesting bit is not generating a paragraph about Emma. It is letting AI take responsibility for the first pass across the fragmentation.

• 01 — Blood results changed. Haemoglobin has fallen across the last three recorded results.
• 02 — Wellbeing pattern worth reviewing. Recorded wellbeing scores have increased across three assessments, alongside repeated reports of poor sleep.
• 03 — Follow-up appears unresolved. A planned medication follow-up does not appear to have been completed.
• 04 — Chronology worth reviewing. Several physical, wellbeing and care-pathway changes overlap in time.

And then one very important sentence: No causal relationship is implied.

That is the handoff I am interested in. AI can surface. AI can compare. AI can remember. AI can group. AI can say these things may be worth looking at together. Then it stops pretending to be the cleverest person in the room.

Try the prototype

This is where I want the actual prototype to live. Not hidden behind a portfolio link at the bottom. Right here, immediately after explaining what I am asking the AI to do.

Press Prepare AI patient brief and the article briefly stops being an article. The AI begins reconstructing Emma's record. Particles assemble into a person. The blood-result signal emerges. Then the behaviour of the whole constellation shifts as the system surfaces the wellbeing pattern. A separate contextual signal appears for the unresolved follow-up. Finally the AI connects the chronology and settles on 4 areas worth reviewing.

The reader can select any one of them and inspect the evidence that caused it to appear. That interaction demonstrates the thesis better than another 800 words from me possibly could.

AI does the first pass. The clinician keeps the judgement.

The AI has to show its receipts

The moment AI decides what deserves attention, trust becomes part of the interface. So nothing in Patient State gets to be magical. Click the wellbeing signal and the interface shows why it exists.

PHQ-9
Feb       6
May       11
Aug       16

Sleep difficulty mentioned in 3 recent encounters

Why am I seeing this? This signal is based on three recorded wellbeing assessments and recent encounters that mention difficulty sleeping. The system is grouping recorded changes for review. It is not making a diagnosis or claiming that one caused another.

The AI gets to do the searching. It does not get to hide the source material.

Mental health makes the idea more important, not less

This is where the project became much more interesting to me. Mental health information is often longitudinal by nature. A score changes. Sleep changes. A medication changes. Something is mentioned in one appointment and again six weeks later in another. A follow-up is planned. Life happens.

The useful thing AI can do is not announce what somebody "has." It can make sure the clinician does not have to manually rediscover the pattern every time.

There is also something important about refusing to represent mental health as a red flashing brain. If the product is meant to help clinicians see the whole patient, then the visual language should treat mental wellbeing as part of the person's state rather than a broken component in their head. That feels both more humane and more useful.

Help the helpers

This became the bit of the idea I cared about most. AI in healthcare is often framed around doing more. More automation. More documentation. More throughput. More output.

But there is another version I find much more interesting. Use AI to remove some of the cognitive archaeology. Let it find the blood result from January. Let it remember the medication change from May. Let it notice that a review does not appear to have happened. Let it compare three wellbeing assessments without somebody opening three different encounters. Let it notice sleep has appeared repeatedly. Let it prepare the chronology. Then give that attention back to the clinician.

Not so they can process more records. So they can spend more of the visit helping the person sitting in front of them. Not AI replacing the helper. AI helping the helper.

Then turn the screen around

Once I had that, the next question was fairly obvious. What happens when the clinician wants to show Emma? Because the same AI-prepared understanding could be useful on the other side of the desk too.

So Patient State has a Share with patient mode. The clinician can turn an iPad or laptop toward Emma. Clinical metadata disappears. Speculation disappears. The constellation stays. The evidence stays.

Recorded wellbeing scores have increased across three assessments, alongside repeated reports of poor sleep — becomes: your recent wellbeing check-ins and sleep reports suggest things have been getting harder over the last few months.

Planned medication follow-up does not appear to have been completed — becomes: a planned follow-up after a medication change does not appear in the record yet.

Same evidence. Different explanation. Then the clinician can point at the chronology and ask something no model should answer for Emma: Does this match how the last few months have felt to you?

Now the visualization is not just displaying information. It is helping create a conversation.

One AI process, two conversations

That gave me a stronger mental model for the product. The AI does not sit in a chat drawer waiting to be summoned. It runs through the workflow. First it helps the clinician reconstruct the patient state. Then, if useful, it helps the clinician explain that state. The clinician remains between the AI and the patient throughout.

It feels less like replacing expertise and more like increasing the leverage of it.

How it would actually run

None of that requires science fiction. In a real clinic, Patient State would sit beside the EHR via FHIR R4 — read-only pulls of encounters, labs, medications and documents. A nightly batch for active panels, plus a pre-visit run so the brief is ready when the clinician opens the chart. Processing stays inside the organisation: on-prem or a VPC boundary, with an audit log for every read and every signal shown.

The AI layer is deliberately narrow. Rules and small models handle trend detection, follow-up gaps and temporal grouping. NLP extracts reported symptoms from notes — neck pain, poor sleep — without interpreting them. An LLM writes the one-line summaries, but only with citations back to source entries. No chatbot replacing the chart. A structured brief the clinician can verify before the patient arrives.

Workflow: brief prepared → review the signals → open the evidence drawer → consult → share approved items on a shared screen or iPad → document in the EHR as usual. AI does the reconstruction. The clinician keeps the judgement, and nothing patient-facing is a diagnosis.

Building the illusion first

For the first prototype, I am deliberately not using a real model. The AI process is deterministic. I already know which pieces of Emma's fictional record should be surfaced, which means I can design the interaction properly.

record opens → AI reconstructs → patient assembles → physical + wellbeing + context signals emerge → clinician investigates → evidence appears → patient mode translates

There is very little value in plugging an LLM into the prototype early and occasionally having it discover a fifth condition because it was feeling entrepreneurial that afternoon. The interaction is the experiment. The model can come later.

Underneath it, the AI engine is abstracted so the deterministic version can eventually be replaced by a real model returning structured, evidence-linked results.

The prototype is part of the argument

I normally think of prototypes as something that sits beside the writing. This one works better inside it. The article explains why I think AI should take on the reconstruction work. The embedded prototype lets you feel that handoff.

There is not a blog version of Patient State and a product version. There is one concept being explained in two ways: through prose, and through interaction.

Where I want to take it

• How much should AI proactively surface before it becomes distracting?
• How should uncertainty behave visually?
• When should the system deliberately stay quiet?
• How should clinicians approve what enters patient mode?
• How do you represent a mental-health pattern without stigmatizing it?
• How do you surface relevant context without becoming invasive?
• How do you distinguish "not found in this record" from "did not happen"?
• How do you make AI feel genuinely useful without making it feel more authoritative than it is?

Those are much more interesting design problems than deciding what colour the chatbot button should be.

The actual experiment

The project started with a visual idea. A person made of dots. Then I realised a body was not enough. The more interesting opportunity was to let AI reconstruct something closer to the whole patient state: physical health, wellbeing, care history and relevant context, all tied back to evidence.

The real idea is therefore much simpler than the visualization: The clinician should not have to be the first person to read everything.

Let AI do the first pass. Let it compare. Let it remember. Let it find what is unfinished. Let it bring together information that the software previously scattered apart. Let it prepare the evidence. Then let the clinician do the part we actually need clinicians for.

Understand. Decide. Ask. Listen. Explain. Reassure. Help.

If AI is going to earn its place inside healthcare software, helping the helpers feels like a pretty good place to start.