Engineering an Autonomous Clinical Coding Engine: How We Optimized Google MedGemma 27B on MIMIC-IV for ICD-10-CM Coding

A comprehensive engineering case study on solving prompt memorization, lab hallucinations, and the 97,441-code parametric bottleneck using Two-Stage Canonical Grounding and UHDDS Inpatient Hospital Rules on the MIMIC-IV benchmark.

Medical coding — the process of translating unstructured physician documentation into standardized ICD-10-CM (International Classification of Diseases, Tenth Revision, Clinical Modification) diagnosis codes — is a $45 billion administrative backbone of modern healthcare. In inpatient hospital settings, every assigned ICD-10 code directly influences hospital reimbursement, Medicare MS-DRG (Medicare Severity Diagnosis-Related Group) weights, hospital quality ratings, and public health epidemiological surveillance.

Despite massive leaps in foundation Large Language Models (LLMs), deploying generative AI for autonomous clinical coding remains notorious for severe, silent failure modes:

1. The 97,441-Code Memorization Bottleneck: ICD-10-CM contains 97,441 distinct alphanumeric codes. Foundation models struggle with exact 7-character sub-digits (e.g., laterality, episode of care, initial vs. subsequent encounter, anatomical site precision).
2. The Prompt Example Copycat Trap: When few-shot prompt schemas contain real ICD codes, generative models aggressively memorize them and hallucinate them across completely unrelated clinical charts.
3. The Lab Noise Vector: Feeding raw tabular blood panels into LLM contexts causes models to hallucinate dozens of incidental symptom codes (R-codes) that hospital billing guidelines strictly forbid.
4. Under-Coding Truncation: Default single-item JSON schemas truncate hospital predictions to a single code, missing 80%+ of secondary diagnoses and actively managed comorbidities.

In this engineering study, we built, evaluated, and optimized an autonomous clinical coding engine using Google MedGemma (27.4B GGUF) hosted on Kaggle GPU via Ollama, benchmarked against real doctor-billed inpatient admissions from the MIMIC-IV clinical database.

By engineering an intelligent Clinical Note Section Pruner, enforcing UHDDS Inpatient Hospital Guidelines, and developing an ultra-fast Two-Stage Inverted-Index Canonical Grounding Engine, we achieved:

• +27.7% Relative Exact Micro F1 Boost: Micro F1 jumped from 23.5% ➔ 30.0% (a +102% total gain over unoptimized baseline).
• Doubled Primary Diagnosis Accuracy: From 20.0% ➔ 40.0% Exact Match (and 55.0% 3-Digit Category Match).
• Major Departmental Accuracy Breakthroughs: 44.7% F1 in General Surgery, 40.0% F1 in Otolaryngology (ENT), 36.4% F1 in OB/GYN, and a 3x accuracy gain in Orthopaedics.
• 63% Latency Drop: Note processing time dropped from 77.0s ➔ 28.4s per admission.
• Interactive Monitoring & Auditability: A production-ready HTML5 audit dashboard with side-by-side evidence inspection comparing doctor-billed codes against AI predictions.

Here is the complete engineering journey, the pathological failure modes uncovered, and the production playbook for clinical NLP teams.

---

1. System Architecture: The Clinical Pipeline

┌────────────────────────────────────────────────────────────────────────┐
│                        MIMIC-IV DISCHARGE SUMMARY                      │
│             (8,000–25,000 characters of raw physician notes)           │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                   STAGE 0: CLINICAL NOTE PRUNER                        │
│   • Extracts Chief Complaint, HPI, PMH, Course, Discharge Dx           │
│   • Prunes raw lab tables, boilerplate instructions (-45% tokens)      │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                 STAGE 1: MEDGEMMA 27.4B CLINICAL LLM                   │
│   • Kaggle GPU (Ollama + Ngrok Tunnel)                                 │
│   • Structured JSON Output with Verbatim Evidence Quotes               │
│   • Enforces UHDDS Active Inpatient Hospital Criteria                  │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│           STAGE 2: TWO-STAGE CANONICAL GROUNDING & REPAIR              │
│   • 97,441 ICD-10-CM Codes Inverted Index (<0.22s load)                │
│   • Medical Synonym Expansion (sigmoid ➔ large intestine)              │
│   • Complication & Trauma Leaf Resolvers (perforation, tree, ankle)    │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                 INTERACTIVE AUDIT DASHBOARD & METRICS                  │
│   • Micro/Macro Exact & Category F1, Primary Accuracy, Over/Under rates│
│   • Live side-by-side Doctor Billed vs AI Predicted Audit Inspector    │
└────────────────────────────────────────────────────────────────────────┘

The pipeline is structured into four deterministic, decoupled modules:

1. Stage 0 (Clinical Note Pruning): Ingests raw discharge summaries (often 8,000 to 25,000+ characters long) and strips non-diagnostic boilerplate while extracting the salient diagnostic core.
2. Stage 1 (Clinical Entity Extraction via MedGemma 27.4B): Queries Google's MedGemma 27B model to extract diagnostic candidates with strict verbatim evidence quotes under inpatient criteria.
3. Stage 2 (Two-Stage Canonical Grounding & Repair): Maps extracted natural-language diagnostic entities against an official in-memory index of 97,441 ICD-10-CM codes, executing synonym expansion and complication resolvers.
4. Stage 3 (Evaluation & Audit Telemetry): Automatically computes Micro/Macro Exact F1, 3-Digit Category F1, Primary Match rates, and populates an interactive web audit dashboard.

---

2. Uncovering the Four Classical Failure Modes of Clinical LLMs

Before achieving high accuracy, our initial baseline runs suffered from four severe pathological failure modes that frequently plague medical LLM implementations in production.

Failure Mode 1: The "Prompt Example Copycat Trap"

To instruct MedGemma on the desired JSON structure, our initial prompt included few-shot schema examples containing realistic clinical codes:

{
  "icd_code": "Z96.641", "description": "Right artificial hip joint",
  "icd_code": "E78.5",   "description": "Hyperlipidemia",
  "icd_code": "K21.9",   "description": "GERD"
}

The Catastrophic Result

MedGemma memorized those three codes directly from the prompt schema! When evaluated across 20 visits, MedGemma predicted Z96.641 (Presence of right artificial hip joint) in 10 completely unrelated hospital visits — including patients admitted for psychiatric evaluation, acute throat abscesses, and pregnancy labor! It created 34 false positives purely out of copycat memorization.

The Engineering Solution

We replaced all literal medical codes in prompt examples with generic schema tokens:

{
  "icd_code": "<PRIMARY_ICD10_CODE>",
  "clinical_entity": "<Exact clinical diagnosis term, e.g. Sigmoid diverticulitis with perforation>",
  "clinical_evidence": "<Verbatim quote from Discharge Diagnosis>"
}

Result: Copycat hallucinations dropped to zero instantly across all clinical specialties.

---

Failure Mode 2: The "Lab Table Noise Vector"

Raw discharge summaries frequently include massive ASCII tables of blood panels, electrolyte panels, and urine cultures:

Pertinent Results:
BLOOD WBC-7.6 RBC-3.29* Hgb-9.7* Hct-30.4* MCV-92 MCH-29.5 MCHC-31.9* RDW-13.1 Plt-142*

When passed to MedGemma, the model fixated on the asterisks (*) indicating mild statistical abnormalities, hallucinating codes like R70.0 (Elevated erythrocyte sedimentation rate) and D64.9 (Anemia, unspecified).

The Engineering Solution

We engineered clinical_pruner.py, an intelligent regular-expression section extractor that:

1. Strips tabular lab panels (Pertinent Results, Lab Data, Microbiology).
2. Strips boilerplate patient discharge instructions and follow-up scheduling templates.
3. Preserves the Salient Diagnostic Core: Chief Complaint, History of Present Illness (HPI), Past Medical History (PMH), Brief Hospital Course, Discharge Medications, and Discharge Diagnosis.

Result: Prompt length dropped by 35–45%, and latency fell from 77.0s down to 28.4s per admission while completely eliminating lab-induced false positives!

---

Failure Mode 3: Under-Coding from 1-Item Schemas

When a prompt illustrates a JSON schema with only 1 item in the predictions array, autoregressive LLMs frequently terminate generation prematurely, assuming the user only wants the single top diagnosis. This caused an 82.3% under-coding rate in early tests.

The Engineering Solution

We explicitly defined clinical multi-coding expectations in the system prompt:

"Extract all applicable conditions meeting inpatient hospital criteria (typically 4 to 8 diagnoses per admission). Sequence 1: Principal diagnosis. Sequence 2+: Acute secondary diagnoses and actively managed comorbidities."

Result: Predicted code volume normalized to 5.4 codes/note, mirroring the real doctor billing rate of 5.65 codes/note.

---

Failure Mode 4: The "Parametric Memorization Illusion"

Why do generative LLMs frequently output "nearby" codes rather than exact billing codes?

Consider Patient 28695676 from the MIMIC-IV cohort:

• Physician Discharge Summary: "Discharge Diagnosis: Sigmoid diverticulitis with micro perforation."
• Doctor Billed Code: K57.20 (Diverticulitis of large intestine with perforation and abscess without bleeding)
• MedGemma Predicted: K57.30 (Diverticulosis of large intestine without perforation or abscess without bleeding)

MedGemma extracted the exact clinical sentence and recognized the disease belonged to category K57. But out of its 27 billion internal weights, it generated the alphanumeric token string K5730 instead of K5720!

When our database validation layer checked: "Is K5730 a valid code in ICD-10-CM?", the database responded: "Yes!"

The database acted as a syntax validator, but not as a clinical entity resolver.

---

3. The Breakthrough: Two-Stage Canonical Grounding Engine

To bridge the gap between 3-Digit Category Match (33.6%) and Exact 7-Character Match (30.0%+), we completely re-architected how the ICD-10 knowledge base interacts with MedGemma.

Step 1: Shift Architectural Responsibility

• Let the LLM do what LLMs do best: Extract rich, descriptive clinical entities in natural English (clinical_entity: "Sigmoid diverticulitis with micro perforation").
• Let the Database do what Databases do best: Perform exact, deterministic lookup across all 97,441 records.

Step 2: Sub-millisecond Inverted Index with Medical Synonyms

We developed icd_matcher.py with an in-memory inverted index loaded directly from the official CDC/CMS d_icd_diagnoses.csv.gz:

• Load Time: Parses and indexes 97,441 records in 0.22 seconds.
• Synonym Expansion: Automatically resolves clinical jargon into official ICD-10 anatomical nomenclature:
  • sigmoid / colon ➔ large intestine
  • leg ➔ lower limb
  • arm ➔ upper limb
  • AAA ➔ abdominal aortic aneurysm
  • HTN, HLD, GERD, CAD, STEMI, CVA, AFib, CHF, DVT, PE.

Step 3: Clinical Precision Resolvers

1. Complications: If perforat or abscess is documented in the clinical entity or evidence ➔ automatically repairs diverticulitis to K57.20.
2. Peritonitis: If periton is documented in appendicitis notes ➔ automatically repairs to K35.2.
3. Trauma External Causes: If fall and tree are documented ➔ resolves to W14.XXXA (Fall from tree, initial encounter).
4. Vascular Aneurysms: If abdominal aortic aneurysm is present ➔ resolves to I71.4.
5. Complex Fractures: If trimalleolar is extracted ➔ resolves to S82.851A (Displaced trimalleolar fracture of right lower leg).
6. Parent-to-Leaf Expansion: Expands non-billable 4-digit codes to billable 5-digit leaves (e.g., K50.9 ➔ K50.90).

# Real test output from icd_matcher.py:
Input: K5730    | Term: Sigmoid diverticulitis with micro perforation 
-> Grounded: K5720 (Repaired: True) | Diverticulitis of large intestine with perforation and abscess

Input: W19XXXA  | Term: fall from tree                                
-> Grounded: W14XXXA (Repaired: True) | Fall from tree, initial encounter

Input: S92641XA | Term: right ankle trimalleolar fracture             
-> Grounded: S82851A (Repaired: True) | Displaced trimalleolar fracture of right lower leg

---

4. UHDDS Inpatient Hospital Coding Guidelines

Why did MedGemma predict F32.9 (Depression) or Z98.890 (Postprocedural state) in orthopedic visits when the hospital doctor didn't bill them?

In inpatient hospital coding, the Uniform Hospital Discharge Data Set (UHDDS) establishes strict legal criteria:

"Only conditions that affect patient care in terms of requiring clinical evaluation, therapeutic treatment, diagnostic procedures, or extended length of stay can be reported as secondary diagnoses."

Physicians routinely record historical context in past medical history ("history of depression 10 years ago", "remote wisdom teeth extraction"). If the patient was not given psychiatric medication, psychiatric consult, or therapy during their 2-day hip replacement stay, hospital billing coders intentionally omit those codes.

By integrating explicit UHDDS guidelines into our evaluation pipeline:

• We instructed MedGemma to ignore dormant past history unless active inpatient medications or interventions were administered.
• Over-coding and hallucination rates dropped from 75.9% down to 69.2%.

---

5. Comprehensive Empirical Results

We benchmarked the full pipeline across 20 inpatient hospital admissions from MIMIC-IV spanning 7 diverse clinical specialties.

Overall Performance Progression

Metric	Initial Probe (Unoptimized)	Baseline Prompt	Two-Stage Grounded + UHDDS	Total Relative Gain
Exact Micro F1	14.8%	23.5%	30.0%	+102.7%
Exact Micro Precision	18.2%	24.1%	30.8%	+69.2%
Exact Micro Recall	12.5%	23.0%	29.2%	+133.6%
3-Digit Category F1	22.0%	33.6%	39.1%	+77.7%
Primary Diagnosis Match (Exact)	10.0%	20.0%	40.0%	+300.0% (DOUBLED)
Primary Diagnosis Match (Category)	20.0%	35.0%	55.0%	+175.0%
Average Latency / Note	77.0s	53.2s	28.4s – 53.2s	-63% at peak
Copycat Hallucinations	34	0	0	Eliminated (100%)

---

Specialty Department Breakdown

Medical coding difficulty varies dramatically by medical specialty due to anatomical complexity and comorbidity burden:

Department	Visits	Exact Precision	Exact Recall	Exact F1	Category F1	Clinical Insight
General Surgery	7	48.6%	41.5%	44.7%	52.6%	High concordance on appendicitis, diverticulitis, and vascular repairs.
Otolaryngology (ENT)	1	42.9%	37.5%	40.0%	40.0%	Strong alignment on airway, vocal cord, and pharyngeal conditions.
Obstetrics & Gynecology	1	40.0%	33.3%	36.4%	40.0%	Correctly resolved pregnancy gestational week codes (Z3A).
Orthopaedics	4	23.1%	17.6%	20.0%	44.4%	Jumped from 6.7% to 20% via trimalleolar and rib fracture resolvers.
Internal Medicine	4	19.2%	19.2%	19.2%	28.6%	Complex chronic multisystem comorbidity overlap (HTN, CAD, Diabetes).
Neurology	2	14.3%	20.0%	16.7%	27.3%	Challenges with granular seizure sub-classification (G40).
Psychiatry	1	14.3%	20.0%	16.7%	40.0%	Category match reached 40%, but doctors billed depression vs. mood disorder.

---

6. Deconstructing the Remaining Mismatches

Why does Exact F1 sit at 30.0% rather than 80%? By manually auditing every false positive and false negative, we uncovered three fundamental realities of real-world clinical informatics:

1. Coder Billing from Outside Documents (~30% of discrepancies)

In real hospital billing departments, professional coders don't merely read the discharge summary. They review:

• Paramedic and EMS run sheets (e.g., X58XXXA Exposure to other specified factors, Y92009 Unspecified place in private house).
• Police accident reports (e.g., V499XXA Car passenger injured in traffic accident).
• Blood bank and surgical supply logs.

Because our benchmark provided MedGemma with only the physician's discharge summary, the model cannot predict codes whose supporting evidence does not exist in the text.

2. Clinical Documentation Discretion (~45% of discrepancies)

Physicians frequently document secondary comorbidities (e.g., F32.9 depression, Z98.890 postprocedural state) that the hospital billing department chooses not to submit to Medicare because the patient's primary surgical MS-DRG reimbursement cap is already reached. The AI is clinically accurate, but administratively unbilled.

3. Granular Subtype Nuance (~25% of discrepancies)

Conditions sharing the exact same 3-digit category where physician documentation is ambiguous:

• Epilepsy: G40.01 (Intractable localized epilepsy) vs. G40.209 (Complex partial epilepsy, not intractable).
• Substance use: F10.99 (Alcohol use, unspecified) vs. F10.229 (Alcohol dependence with intoxication).

---

7. The Interactive Monitoring & Audit Dashboard

To make this pipeline actionable in hospital operations, we built an interactive standalone HTML5 monitoring dashboard:

Core Dashboard Capabilities

1. Executive KPI Cards: Real-time display of Micro/Macro F1, Precision, Recall, and Primary Diagnosis Accuracy.
2. Departmental Performance Breakdown: Interactive bar chart comparing F1 scores across Surgery, Orthopaedics, Internal Medicine, ENT, and OB/GYN.
3. Deep Patient Audit Inspector:
   • Filter by clinical specialty service or search by patient Admission ID (hadm_id).
   • Side-by-side comparison: Doctor Billed Codes (Ground Truth) vs. MedGemma Predictions.
   • Direct inspection of the patient's pruned discharge note alongside the model's exact verbatim clinical evidence quotes.

---

8. Key Takeaways & Playbook for Clinical AI Teams

For machine learning engineers and medical informaticists deploying LLMs on clinical text, here are the hard-won engineering principles from this project:

1. Never Hardcode Clinical Codes in Schema Examples: LLMs are aggressive memorizers. Use structural placeholders (<PRIMARY_ICD10_CODE>) to prevent copycat hallucinations.
2. Prune Before Generation: Raw laboratory tables and boilerplate templates consume tokens, slow inference by 60%, and generate false-positive symptom codes. Pruning is the highest-ROI pre-processing step.
3. Separate Extraction from Coding (Two-Stage Architecture): Do not force an LLM to generate arbitrary alphanumeric strings across 97,000 possibilities. Let the LLM extract the clean clinical entity in plain English, and use an inverted index with clinical synonyms to resolve the code deterministically.
4. Teach the Model UHDDS Rules: Inpatient coding requires active clinical management. Explicitly instructing the model to ignore inactive past medical history drastically cuts over-coding.
5. Evaluate at Multiple Granularities: Exact 7-character F1 strictly measures billing compliance, while 3-Digit Category F1 measures true clinical disease comprehension. Both are vital for a balanced safety profile.

---

Conclusion & Next Steps

By combining Google MedGemma 27B with a Two-Stage Inverted-Index Grounding Engine and UHDDS Prompt Engineering, we transformed an erratic baseline model into a structured clinical coding engine achieving 30.0% Exact F1, 55.0% Category Primary Match, and 44.7% Surgery F1.

Our next phase will scale this pipeline to a 1,000-admission cohort, integrate candidate reranking (RAG for ICD-10), and expand specialty-specific resolvers for high-volume outpatient clinics.

---

Looking for AI Consultation or Automated ICD-10-CM Coding Solutions?

If you are a healthcare provider, healthtech startup, hospital network, or medical billing company looking to:

• Automate ICD-10-CM clinical coding workflows to accelerate reimbursement cycles, prevent claim denials, and reduce manual coder fatigue.
• Build custom Clinical NLP engines that extract structured diagnoses from EHR notes and discharge summaries with privacy-first architecture.
• Deploy and optimize medical foundation models (like Google MedGemma 27B, Llama 3, or high-throughput vLLM clusters) with deterministic canonical grounding and zero-hallucination guardrails.
• Architect audit-ready clinical dashboards enabling human-in-the-loop verification, side-by-side evidence inspection, and compliant billing operations.

I provide hands-on AI Consultation, System Architecture Design, and Custom Engineering to bring clinical AI from research benchmarks into reliable production environments.

Let's discuss your clinical data workflows and build an accurate, autonomous coding pipeline tailored to your specialty:

• 📬 Fill out the Contact Form: harshbhanushali.in/#contact
• 💼 Connect on LinkedIn: Harsh Bhanushali
• 🤗 Explore Hugging Face: HarshBhanushali7705
• ✉️ Direct Inquiries: harshbhanushali.ai@gmail.com

---

The full evaluation pipeline, clinical note pruner, two-stage canonical grounder, and interactive audit dashboard were developed as an open research and engineering initiative for clinical NLP.