Writing / A 4B model in a synthetic hospital: the agent is more than the weights

A 4B model in a synthetic hospital: the agent is more than the weights

October 16, 2026 · 33 min read

TL;DR

Bar chart of test F1: Qwen3-4B untrained 0.17, SFT 3 epochs 0.39, plus GRPO 0.40, SFT 8 epochs 0.53, tool-SFT 0.74. GPT-6 Luna 0.69, Kimi K2.5 0.73, GPT-6 Luna plus one paragraph 0.81. Copy-and-code ceiling 0.82

How I got here

A few weeks back I saw a twitter post announcing the Synthetic Hospital benchmark from some Carnegie Mellon researchers. Their work was focused on building a dataset of over a thousand fictitious patients with years of synthetic medical history, served through a standard electronic health record system interface built for AI agents. (Park, Chen, Dettmers. Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark)

A closer look

I dug into the repo and the paper, and one line caught my attention: the training split is labelled to be “used as a verifiable-reward environment.”

This means there’s a brand new realistic agent environment with a built-in reward, a perfect place for a reinforcement learning (RL) project. So I committed some time over the weekend to rent a GPU, pick one of the paper’s defined tasks, and see whether I could post-train a small open model on it. I wanted to answer one question:

Could I use RL to teach a small open model to outperform larger models?

A week and about $57 later, the answer was: not really.

But why led me down a path where the smallest model ended up outperforming the largest model from the paper!. The better questions were: what is the agent missing, and what’s the cheapest and most practical way to provide it?

Spoiler: the answer depends a lot on the model. I ended up comparing my small model with a frontier one, and they were missing completely different things.

Disclaimer: I’m no expert!

Quick disclaimer before we go further: I’m not a researcher. In fact, my research experience started and ended as an undergraduate intern at the Cardiovascular Research Center back at UVA. But I have trained a model or two, and I’ve done some very basic RL on Atari game environments with the excellent PufferLib library.

This is a time-boxed weekend project where I rented a GPU for the weekend and let my curiosity lead. Do not take any of my conclusions or opinions as any more than a curious enthusiast’s. Corrections very welcome.

While I’m not a researcher, I tried to work with some rigor:

How I worked. I relied on coding agents to expedite much of the engineering work. I used omp as my agent harness, using mostly Claude Opus 5.5 with a little GPT-6.1 Astra. Every decision and prediction was recorded in a timestamped lab log file (including the incorrect predictions).

About “Synthetic Hospital”

The authors started with medical board exam questions and worked backwards to create ~1200 synthetic patients with ~5600 hospital visits. They took patient diagnoses and synthesized hospital visits including notes and full charts. They did such a good job that in a blind test real physicians couldn’t reliably tell the synthetic charts from real ones.

The benchmark has four tasks, but I focused on the first one: patient diagnosis. In this task, the agent gets a patient ID and tools to do things like list visits, read notes, view labs, and search the patient’s chart. The goal is to submit every active diagnosis and chronic condition, each with its standard ICD-10 code (the medical classification system, here the US version, ICD-10-CM).

Scoring is built into the benchmark, and uses F1 score, which balances two things: did you submit all of the correct diagnoses (recall) and were the ones you submitted actually right (precision)? The scorer weights serious conditions more heavily, and only checks the first three characters of each code, which in the ICD-10 system maps to the broader category (e.g. E11.9 and E11.65 are both in the “type 2 diabetes” E11 category, but with differing details).

The paper’s best model was Kimi K2.5, which scored 0.73 on this task. There’s my target.

Target acquired meme

My plan

This needed to be cheap and small enough to complete over the weekend without breaking the bank on GPU resources. Beyond the economics, I am generally interested in the use of small open models for specific tasks, especially in regulated or sensitive industries. I’ve seen first-hand at pWin.ai how much compliance requirements can limit certain organizations’ usage of frontier model APIs.

Looking at the data

All of the benchmark’s data ships as one SQLite file, which was convenient because before training anything I needed to know what’s actually in the dataset, so I used a few lines of pandas for exploratory data analysis:

import json, sqlite3
import pandas as pd

con = sqlite3.connect("benchmark_v1.3.db")
patients = pd.read_sql("SELECT * FROM longitudinal_patients", con)
visits = pd.read_sql("SELECT * FROM longitudinal_encounters", con)

# how many visits do we see per patient?
visits.groupby("patient_id").size().value_counts()

# how many diagnoses do we observe per patient?
key = pd.read_sql("""SELECT patient_id, split, ground_truth FROM benchmark_ground_truth
                     WHERE task = 'patient_diagnosis' AND split != 'heldout'""", con)

answers = key.ground_truth.map(json.loads)  # JSON columns arrive as plain strings
answers.map(lambda g: len(g["active_diagnoses"]) + len(g["chronic_conditions"])).describe()
visits per patient    2: 350   3: 355   4: 58   5: 116   6: 29   7: 29   8: 331
diagnoses per patient mean 3.6   median 3   min 1   max 8

A few things stood out:

Ok, let’s try to break it

The best way to understand an eval is to be a bit adversarial and try to break it. Before training anything, I wanted a feel for what I’d be training against, and what “gotchas” or tricks might be hiding. The biggest one: each chart’s note has an “Active Problem List” of every earlier diagnosis.

Here’s an example:

...
Active Problem List:
- Exclusive breastfeeding (diagnosed 2020-01-15)
- Peripartum cardiomyopathy (diagnosed 2020-02-15)
- Postpartum thyroiditis, hyperthyroid phase (diagnosed 2020-06-15)
- Dichorionic diamniotic twin pregnancy (diagnosed 2020-12-15)
- Vasa previa (diagnosed 2021-03-15)
- Impaired glucose tolerance (diagnosed 2021-05-15)
- History of gestational diabetes mellitus
- History of postpartum thyroiditis
- Generalized anxiety disorder
...

Turns out, 6 of this patient’s 7 answers are in the chart notes! The seventh comes from the final visit, so it’s the only one that really has to be “figured out”.

Notice the “History of” lines being left out of the 7 “correct answers”. Past conditions aren’t considered active diagnoses, and neither are ongoing conditions like generalized anxiety disorder according to the answer key. (There’s that pickiness I mentioned)

Okay so that’s a good learning. Now to understand the scoring, I threw a bunch of answers at the scorer to see how it behaves:

So this task is mostly read Active Problem List and code it correctly using the ICD-10 system, with a little bit of diagnose the patient based on the final visit notes.

And the coding is the hard part. As one more test, I took the diagnosis names straight from the problem lists and asked a much bigger off-the-shelf model, Qwen3-235B, to just code them, no chart reading needed. It scored 0.69, versus 0.86 when the same names get the benchmark’s own codes. So even a bigger model gets a lot of the codes wrong.

Step 1: establish a baseline, do some SFT

First, the baseline: I ran the 4B model through the standard setup, and it scored 0.17.

Reading the transcripts, it was a mess! About half of the episodes failed because they ran out of actions (meaning it used up all 40 tool calls!). Also, the answer submission was difficult: it kept responding directly in the chat instead of actually submitting it.

As a sanity check, I tried a stronger prompt that spelled out the procedure step by step. It barely helped (+0.005).

Okay, there’s certainly room for improvement.

So the first problem wasn’t even medicine-related: the model just couldn’t operate the tools. So my plan was simple: create training data from using the correct diagnoses of the training patients and SFT a standard “routine” to go from chart to diagnosis for the model to learn:

  1. List the visits, and find the latest one.
  2. Open the latest note, because as we saw, the problem list holds the most recent visit information as well as all of the previous diagnoses.
  3. Submit the answer properly using the submit_diagnosis tool, not the chat!

At first I thought I could use a smarter model to generate training examples, but that was really overthinking it. The training patients already had their answers, and the procedure was actually quite simple, so it just took a few lines of scripting to produce the perfect “oracle” demonstrations for behavior cloning.

I trained for three epochs (37 minutes) and the F1 improved more than 100% from 0.17 to 0.39! By the end, every episode did follow the three step routine, and chart reading was basically solved: the model named ~99.5% of the documented diagnoses.

There was a real problem though: it was horrible at coding them. Hypomagnesemia became E53.1 (vitamin B6 deficiency). Interstitial lung disease became J44.9 (COPD). If I kept the diagnosis names from the model and manually swpaped in the correct codes, its score jumps from 0.38 to 0.78 (!), so the real gap was knowing how to take the diagnosis and map it to the correct ICD-10 billing code.

Two limitations with the oracle demonstrations we trained with:

  1. It kind of locks the model into exactly three steps, so it never tried anything else. Fine for this simple task, but could limit creativity if necessary.
  2. For the final visit, the one diagnosis that has to be figured out, it trained the model to answer after a single look at that note, where a real doctor would then check labs and pore over earlier visits to make connections.

Step 2: let’s try some RL

I started this project to try RL in the provided environment, but before I’d trained anything, I noted in the log file that the SFT already “gets everything RL would.” The reality is, the oracle demonstrations already cover everything the reward checks, so I predicted RL would add nothing. This is also about when I tweeted “you don’t need RL you just need good old fashioned SFT” lol)

But I still gave it a shot.

I used prime-rl to run GRPO starting from the 3-epoch SFT checkpoint. The hope with GRPO is that across a group of attempts (rollouts) at the same patient, the model will sometimes get the right answer, and we can update the policy (in this case the LoRA weights) to nudge it toward those right answers over time.

I ran 85 policy updates over ~4 GPU-hours with 4 attempts per patient.

And the result? F1 improved by +0.01. Four GPU-hours to confirm my own prediction that it wouldn’t really help. 🙃

But this makes sense. If none of the attempts ever produce the right code, there’s nothing to nudge toward. So I counted, for every diagnosis, how many of the four attempts got it right:

Stacked bars for three windows of the RL run: never right stays at 33 to 35 percent. Sometimes right falls from 38 to 25 percent. Always right rises from 29 to 40 percent

A third of the codes were never right, and that third never budged. RL was able to turn “sometimes right” into “always right.” It sharpened the abilities of the model, getting it more consistent at things it could already do. But it could never teach it to use a new diagnostic code.

The model is playing Marco Polo, shouting “Marco!” over and over, but it never hears “Polo”, so all it can do it swim at random.

Turns out this is a known result: researchers at Tsinghua’s LEAP Lab showed that RL with verifiable rewards concentrates a model on answers it can already produce, but doesn’t expand what it can produce. So let’s just say I was trying to reproduce their research, yeah!

Takeaway: So yes, the environment supports RL, but this task’s gap was mostly a knowledge gap, which RL can’t reliably add. The one part of this task that involves reasoning (diagnosing the final visit) would potentially be a candidate to aim RL at once we hit the 0.88 ceiling (more on that below).

So okay, now with the RL experiment out of the way, it was time to really focus on improving this model by first squeezing more juice out of the SFT, as after the three epochs, our model still only scored 0.48 on its own training patients. It hadn’t even finished learning from its homework yet!

Step 3: more SFT!

I reran the same fine-tuning recipe for 10 total epochs and scored every epoch on validation. Validation climbed steadily until epoch 8 (0.58) and then flattened, while the score on its own training patients kept going up to 0.99. That gap suggests overfitting, meaning past epoch 8 the model is just memorizing its training patients which isn’t helping on the unseen patients in validation. So I picked up the epoch 8 checkpoint and tested it: the test score rose to 0.53 up from 0.39.

Line chart: validation score by fine-tuning epoch climbs from 0.25 to 0.58 at epoch 8 then flattens, while the score on its own training patients rises to 0.99. Epoch 3, where RL started, is marked.

But what was it memorizing? I analyzed all of the documented diagnoses in the test set to answer one question: how often did this code show in the training data?

How many training patients had the codeUntrainedSFT, 3 epochsSFT, 8 epochs
5 or more35%73%84%
2–48%37%68%
18%28%35%
0 (never seen)2%5%1%

The pattern is hard to miss, and obvious in hindsight. The model was more likely to learn to diagnose codes that are present in the training patients.

Most of its wrong codes that were submitted were codes that it saw in the training patients. The problem was: 14% of the test set diagnoses aren’t present in the training set.

The model wasn’t learning how to code. It was learning to memorize the codes for the diagnoses it trained on. The thing is, there are roughly 74,000 billable ICD-10 codes, so our 680 patients training set was never going to cover them all, and besides: aren’t there better ways to solve this problem?

I could keep training, and risk making the model worse at other things along the way, but 74,000 codes is a lot to cram into a small add-on, and this is just like the RL sidequest: SFT is not the best way to handle this.

An agent is more than just the model and its weights, its also the harness in which it operates!

Step 4: let it look codes up

While with experience I am sure a lot of common codes can be committed to memory, I don’t think human’s memorize the 74,000 codes either! They would probably look them up in the official ICD-10-CM index, (which is published for free online). You don’t train a model’s weights to know every single code, in the agent-first world, you can give it good reference material “on the job.”

So I built a new tool for the agent, icd10_search(query)with a plain BM25 keyword search over the official 2025 code list. My first thought was in-context learning like I did for Quench AI, but the list is big for a small model’s context window. And anyway, diagnosis names usually share exact words with the official titles, so a quick test showed that keyword search puts the right code in the top eight results 85% of the time.

I altered it slightly so that the diagnosis names matched the wording observed in the training patients, then froze it so I couldn’t overfit it to the test. The misses are mostly names that are worded very differently from the official title, and 85% turned out to be plenty: the model can still pick a right code it remembers when search misses.

No embeddings, no training GPU, just plain old information retrieval with a 16 MB index that answers in about 5 milliseconds.

→ icd10_search("interstitial lung disease")
1. J84.9 Interstitial pulmonary disease, unspecified (category J84)
2. J84 Other interstitial pulmonary diseases
3. J84.178 Other interstitial pulmonary diseases with fibrosis in diseases classified elsewhere (category J84)
...

Turns out, the tool alone helped untrained models a lot: the untrained 4B went from 0.17 to 0.49, and the bigger Qwen3-30B from 0.46 to 0.64. But untrained models use tools poorly. One Qwen3-30B episode made 44 lookups, 42 of them for “cytomegalovirus esophagitis”. 😅

Step 5: tools have to be taught

So I handed the same tool to my 8-epoch fine-tuned model and… it never used it. Zero lookups, same score: 0.53.

But this made sense…it had been trained to always perform a specific three-step routine, and this new tool just wasn’t part of it.

So I rewrote the training examples. Tool-SFT is the same 680 examples with one new step: after reading the latest note, the model searches for each diagnosis, sees the real search results, and then submits the same answer as before.

Here’s what that looks like (trimmed):

→ icd10_search("Peripartum cardiomyopathy")
  1. O90.3 Peripartum cardiomyopathy (category O90)
  2. I42 Cardiomyopathy
  3. A18.84 Tuberculosis of heart ...
→ icd10_search("Impaired glucose tolerance")
  1. R73.02 Impaired glucose tolerance (oral) (category R73)
  2. R73.01 Impaired fasting glucose ...
→ icd10_search("Exclusive breastfeeding")
  No ICD-10-CM 2025 entries match "Exclusive breastfeeding". Try other words or a broader term.
→ submit_diagnosis([O90.3 Peripartum cardiomyopathy, R73.02 Impaired glucose tolerance, Z39.2 Exclusive breastfeeding, ...])

That’s it! Nothing about the answers changed. The model just sees, 2,449 times, that the right code is usually sitting right there in the search results, and learns to pick it. And sometimes it isn’t (like “exclusive breastfeeding” above), so it also learns the search isn’t a silver bullet.

So I started from the base Qwen3-4B-Instruct model again, and trained it over 8 epochs again. And the result? 0.74, up from 0.53!

Line chart: share of documented diagnoses coded right, by how many training patients carried the code. Fine-tuning alone rises from 1-5% for never-seen codes to 73-84% for codes seen five or more times. Fine-tuning with the search tool runs from 57% to 96%

Look at the left side of that chart. Without the tool, codes the model never saw in training are right 1–5% of the time. With it: 57%. Tool-SFT’s line is also much flatter: it does almost as well on codes it never saw as on codes it saw many times, because the model no longer has to remember a code to get it right. It just has to recognize it in the search results. And when the right code was in the results, it picked the wrong one only 5% of the time (versus 24% for the untrained model with the same tool). What it learned was how to choose.

So can the 4B keep up with frontier models?

Okay, so now the real question: does all of this training and tool building bring a small model closer to being able to keep up with way bigger models? I tested against 3:

Every model got the official benchmark setup with the same 200 test patients.

ModelSizeSearch toolFine-tuned hereF1Documented diagnosesFinal visitNever-seen codesCost / 1k patients
Qwen3-4B, untrained4B––0.1718%8%2%–
Qwen3-4B, SFT 8 epochs4B–✓0.5360%32%1%$0.16 GPU
Qwen3-4B, tool-SFT4B✓✓0.7485%41%57%$0.26 GPU
Kimi K2.5~1T––0.7370%58%60%$51 API
GPT-6 Lunaundisclosed––0.6970%65%50%$2 API
Gemma 3 27B*27B––0.2627%18%9%$11 API

*Gemma’s inference provider failed often, so its score is a lower bound.

Qwen3-4B with SFT and the search tool beat GPT-6 Luna (+0.07) and was neck and neck with Kimi K2.5, at a tiny fraction of the cost.

Without the tool, SFT alone greatly improved its performance, but with the tool it was able to compete with Kimi K2.5, a MoE model with over 1 trillion total parameters.

For context, my model could run on roughly $1,000 of consumer hardware. Running Kimi K2.5 yourself would take something like an 8x NVIDIA H200 server, a few hundred thousand dollars of hardware.

Scatter plot of F1 against cost per 1,000 patients on a log scale: Qwen3-4B with the tool 0.74 at about $0.26, GPT-6 Luna 0.69 at about $2, Luna with two sentences 0.81 at about $2, Kimi K2.5 0.73 at $51, Gemma 3 27B 0.26 at $11.

But the two models excel at different things. On diagnoses the chart already documents, my 4B is right 85% of the time, the best model tested, because it was specifically trained to trust and reliably use the past charts.

On the final visit, where a new diagnosis has to be reasoned out, the big models still win (58–65% vs 41%), but training moved our 4B a long way there too: from 8% untrained to 41%.

The tool did nothing for GPT-6 Luna

Of course, it’s not exactly apples-to-apples, but I wanted to see how helpful my tool could be for an off-the-shelf frontier model. I predicted that GPT-6 Luna’s score on documented diagnoses (the ones already written in the chart, the “Documented diagnoses” column above) would jump to about 85%, like my 4B’s.

To my surprise: it didn’t move! −0.01, and not because Luna ignored it! In its traces, I saw about ten searches per patient and 99% of the submitted codes came from the results. The thing is…it already knew ICD-10 codes. In fact, there was a pattern: the Qwen3-4B gained +0.31, Qwen3-30B gained +0.19, but GPT-6 Luna gained nothing. It seemed that the less a model knows, the more the tool is worth .

Dumbbell chart of F1 without and with the search tool: Qwen3-4B untrained 0.17 to 0.49 (+0.31), Qwen3-30B untrained 0.46 to 0.64 (+0.19), GPT-6 Luna 0.69 to 0.68 (-0.01).

This makes sense: OpenAI’s training data almost certainly includes the ICD-10 code list, one way or another.

But then why did my model still beat Luna with the tool?

That picky answer key strikes again!

Remember how picky the answer key was? Every answer key has conventions beyond the code list: which conditions belong on the list, and which code to use when a few would do. My 4B had learned this sense from the training, and Luna hadn’t.

The big thing was what to list. The key has about one diagnosis per visit, 3.5 per patient, and my 4B submits 3.5. Luna submits 5.8: symptoms, “history of” entries, and background chronic conditions. All reasonable things a doctor might write down! Just not what this key counts.

Bar chart of diagnoses submitted per patient: answer key 3.5, my 4B 3.5, GPT-6 Luna 5.8, Luna with two sentences 5.0.

Remember, we are using F1 which combines both precision and recall. Recall was nearly identical, meaning Luna found the right diagnoses just as often. The gap was precision: it often listed a bunch of diagnoses that the scorer consider extra. If Luna had been as precise as the 4B, its F1 would be 0.73 vs the 4B’s 0.74. Basically a tie.

It’s like hiring a chef but never telling them your preferences or allergies. No matter how much they may know about cooking, they can only make a best effort guess at what exactly you want to eat.

Upon closer examination, the answer key also has some… interesting code preferences. Four test patients with serotonin syndrome are coded as “Primary cough headache”, “Stiff-man syndrome”, and two other things, never as G90.81, whose official title is literally “Serotonin syndrome”.

Which raised a fair worry: was my 4B just learning the key’s mistakes? So I audited the labels, with two Claude Sonnet 5.5 judges and a Claude Opus 5.5 tiebreaker reviewing them without seeing any model’s answers. They flagged about 6% of the codes as wrong. Fixing those changes no model’s score by more than 0.01, and on the mislabeled diagnoses, both tool-using models usually submitted the correct code, not the key’s mistake. So no, the edge wasn’t copied errors. It was knowing what to list.

Can you just tell it?

If the edge is just knowing what to list… why not just tell Luna what to list?

Since Luna listed extra diagnoses could I help it understand the implicit rule that the key was following? Examining the training data, it seems every visit introduces at most one new diagnosis. So I added two sentences to Luna’s prompt:

The problem list should contain the diagnosis established at each encounter,
about one per visit. Don't list symptoms, findings or 'history of' entries,
and don't repeat chronic conditions unless a visit diagnosed them.

And the result? 0.69 → 0.81. The best score of the entire project, 0.06 ahead of my 4B. (I’d predicted 0.74–0.77. Too low again.) The search tool still added nothing on top.

Telling did not work on the 4B, though. Luna learned from two sentences what my 4B needed 680 worked examples to learn.

One honest caveat: I came up with this prompt after looking at the test results, so this experiment was more of a hack, not as scientific as the other experiments. It was just something I wanted to try.

So what was each model missing?

TaskQwen3-4B (small, open)GPT-6 Luna (small, frontier)
Operating the chart toolsNeeded fine-tuningAlready had it
Knowing the ICD-10 codesNeeded a search toolAlready had it
The answer key’s conventions (what to list)Needed fine-tuningTwo sentences in prompt
Reasoning about a brand-new diagnosisWeaker (41%)Much stronger (65%)

RL supplied none of these.

Bigger models bought two things with them: reasoning, and the ability to easily learn from instructions.

The 4B’s case is really about size and practicality. It gets 92% of the best score, and it’s small enough to run on a laptop!

Where else does this apply?

One of my biggest takeaways from this is that a small, cheap model can be surprisingly good at a narrow job. This matters a lot in regulated places like hospitals, banks, and government, where sending data to an API might be a non-starter. But if you want this to work, you need to:

  1. Have an eval or some reliable way to score the performance.
  2. Understand what drives its success because before you run to “distill a bigger model” or “throw compute at it” it’s clearly important to break down what things the agent specifically lacks? In this case, it was largely the ICD-10 codes and the submission instructions!

This pattern of “a reference to look things up” and “a specific format for output” shows up all over the place across diverse use cases. Lots of business work is really just map messy text to an official code or rule, which is exactly the kind of thing humans do with a reference book open.

For example:

Each of these has an official “code book,” and each is a candidate for the same recipe: small model + lookup tool + a few hundred worked examples + an eval to check it.

What I’d do differently

If I were to do this again, I would change a few things:

  1. Analyze the distribution of ICD-10 codes in the training data vs. the total universe of codes, as well as the validation/test sets. This would have quickly taught me that the training set was an incomplete training aid, and would have maybe helped me decide on the lookup tool faster.
  2. Finished the SFT runs before trying RL. I did 3 epochs, stopped, and then tried RL, but only after the RL run did I realize that I had barely scratched the surface of what SFT had to offer.

All-in-all, these are pretty minor. I wasted a few dollars, a few hours, and learned a lot in the process!

Appendix

Notes for the benchmark authors

If you’ve read the paper, you might wonder why its agents score 0.81–0.89 while my best is 0.81. The repo’s README already flags the reason in its “Known issues” section.

The agent harness used for those runs provides two tools that hand the agent the patient’s answer-key diagnoses, including the codes. The clean /env endpoints I used don’t. I measured how much that’s worth with Qwen3-30B on my validation patients: 0.73 with the leak, 0.40 with it masked, and 0.42 in my harness. So the leak is worth about +0.33, and with it masked the two harnesses more or less agree.

A few smaller things I noticed along the way, shared in the spirit of making a great benchmark even better:

(Also: don’t compare any of these scores with the paper’s physicians. That average comes from just 13 patients.)

None of this takes away from the benchmark. It’s genuinely great work, and its clean environment is what made this whole project possible. Thank you to the authors!

Side quests

Things that ate hours and might save you some:

Reproduce it