U.S. physicians lose an estimated $125 billion each year due to inaccurate coding, claim errors, and administrative inefficiencies. That figure translates to roughly $5 million per provider annually, money that disappears into denied claims and compliance audits before anyone realizes the root cause was a misapplied modifier or an under-specified diagnosis code. The technology you choose to address this problem matters more than most vendor pitches suggest. For decades, rule-based systems dominated computer-assisted coding (CAC), mapping clinical documentation to ICD-10 and CPT codes through hand-built logic trees. Large language models (LLMs) are entering the space with a fundamentally different approach. This post breaks down how each approach works at a technical level, where the current research stands on accuracy, and how performance varies across emergency medicine, cardiology, and primary care.

How Rule-Based Coding Engines Work Under the Hood

Rule-based medical coding systems operate on deterministic logic. Engineers and clinical informaticists collaborate to translate coding guidelines into structured if-then decision trees. When a clinical note enters the system, the engine scans for predefined keywords, phrases, and documentation patterns. If a note contains "chest pain," "troponin elevated," and "ST-segment elevation," the system follows a branching path to suggest a code for acute myocardial infarction.

Every coding suggestion is fully explainable. Auditors can trace exactly why a system suggested a specific ICD-10-CM code, which matters enormously for compliance. Rule-based systems also don't require training data — a significant benefit for rare diagnoses where machine learning models lack sufficient examples to learn from. A 2019 study published in AMIA Annual Symposium Proceedings found that rule-based NLP approaches using SNOMED CT-to-ICD-10 mappings performed particularly well for less-prevalent diagnostic codes, precisely because they didn't depend on high-volume training examples.

Where Static Rules Break Down

The fundamental limitation is adaptability. Coding guidelines change constantly. CMS publishes annual ICD-10-CM updates, the AMA revises CPT codes, and payer-specific rules shift quarterly. Each update requires manual reprogramming of the rule engine, a labor-intensive process that creates lag between guideline publication and system compliance.

Physicians don't document in standardized language. One cardiologist writes "reduced EF with global hypokinesis," another writes "systolic dysfunction, EF 30%," and a third dictates "pump failure with significantly decreased cardiac output." All three describe the same clinical reality, but a rule-based system must have explicit rules for each variation. CAC systems can't interpret free-flowing subjective text, which severely limits their ability to handle the narrative-driven documentation that dominates real clinical practice. A system managing 50 rules works fine. A system managing 50,000 rules becomes a maintenance nightmare where rule conflicts and edge cases multiply faster than engineering teams can resolve them.

What LLM-Based Medical Coding Actually Does Differently

Large language models take a fundamentally different approach. Instead of following predefined decision trees, LLMs process clinical text through neural networks trained on vast corpora of medical literature, coding manuals, and clinical documentation. They build contextual representations of what a clinical note means.

When an LLM reads a discharge summary describing "a 67-year-old male presenting with acute onset dyspnea, bilateral rales on auscultation, BNP of 1,800 pg/mL, and echocardiogram showing EF of 25%," it doesn't just identify individual terms. It understands the clinical narrative and maps it to the appropriate ICD-10-CM code (I50.21 for acute systolic heart failure) based on learned relationships between clinical presentations and coding conventions.

Fine-Tuning and Retrieval-Augmented Generation

Specialized approaches dramatically close this gap. A 2025 study in npj Health Systems (Nature) demonstrated that domain-specific fine-tuning increased exact matching from less than 1% to 97% for structured coding tasks. When tested against real-world clinical notes with their inherent complexity, fine-tuned models achieved 69.20% exact match and 87.16% category match, a significant improvement over unmodified LLMs.

Retrieval-augmented generation (RAG) pushes accuracy even further. By connecting an LLM to a verified database of current coding guidelines, the model can reference authoritative coding rules in real time rather than relying solely on its training data. Research from a 2024 study in the Journal of Medical Internet Research showed that a Retrieve-Rank system achieved near-perfect accuracy for ICD-10-CM code prediction by pairing LLM reasoning with structured code lookups.

The Black Box Trade-Off

LLMs' greatest strength, contextual understanding, is also their auditability challenge. Unlike rule-based systems, where you can point to the exact rule that generated a code suggestion, LLMs arrive at answers through complex statistical patterns across billions of parameters. For revenue cycle teams accustomed to deterministic audit trails, this opacity is a legitimate concern. It's a key reason why platforms like Sully.ai build their AI coding agents with a human-in-the-loop workflow, where every AI-generated code remains a draft until a human coder reviews and approves it, preserving the speed advantages of LLM-based coding while maintaining the compliance rigor that healthcare demands.

Accuracy Benchmarks: What the Research Actually Shows

ICD-10-CM Coding Performance

For diagnosis coding, fine-tuned LLMs now outperform rule-based systems in most head-to-head comparisons, particularly for complex, multi-condition encounters. Rule-based systems maintain consistent performance on low-prevalence codes because their logic doesn't depend on having seen many examples. LLMs can struggle with code they encountered infrequently during training.

CPT Coding Performance

For procedure coding, the picture is more nuanced. Traditional NLP approaches achieved an average AUROC of 0.96 and accuracy of 0.97 for predicting CPT codes from operative notes, actually outperforming more resource-intensive transformer-based models like BERT in certain settings. This suggests that for well-structured procedure documentation, the additional complexity of LLMs may not deliver proportional accuracy gains. Where LLMs pull ahead in CPT coding is with evaluation and management (E/M) codes, where the level of service depends on nuanced clinical decision-making documented in free-text notes rather than structured procedure descriptions.

Emergency Department Coding: Where LLMs Gain the Clearest Edge

Emergency medicine is a stress test for any coding system. ED encounters are fast, documentation is often fragmented across triage notes, physician assessments, nursing observations, and procedure logs, and the range of possible diagnoses spans nearly every organ system. Rule-based systems struggle here because ED documentation is highly variable. A patient presenting with abdominal pain might be documented in radically different ways depending on the physician, the urgency of the case, and whether the note was dictated in real time or completed after a shift. The keyword-matching approach that works for straightforward primary care encounters falls apart when documentation is non-linear and contextually dense.

LLM-based systems show strong results in this environment. A 2024 study published in the Annals of Emergency Medicine found that AI models predicting billing code levels for ED encounters achieved AUC values of 0.94 and 0.95 for E/M levels 4 and 5, respectively. The ensemble models reached an accuracy of 0.86 and F1 scores above 0.83 across these high-complexity levels.

What makes this particularly relevant is that the most important predictive features identified through Shapley Additive Explanations (SHAP) values included critical care documentation and discharge disposition. These are contextual factors that require understanding the entire encounter narrative, not just isolated keywords. This is exactly the type of holistic reasoning where LLMs outperform rigid rule sets. If your denial rate on level 4 and 5 E/M codes is above industry average, an LLM-based approach is likely to deliver measurably better results than a traditional CAC system.

Cardiology Coding: Specificity Demands That Challenge Both Approaches

Where Rule-Based Systems Hold Their Ground

For structured procedural coding in cardiology, rule-based systems can perform well because these documents follow relatively standardized templates. The documentation fields map predictably to specific CPT codes, and the rules can be written with high precision for known procedure types.

Where LLMs Provide the Advantage

The challenge involves office visit coding, complex care coordination, and scenarios in which a cardiologist manages multiple chronic conditions simultaneously. When a single encounter involves adjusting heart failure medications, evaluating new-onset atrial fibrillation, and ordering follow-up imaging for a previously identified valve abnormality, the E/M coding level depends on synthesizing the cumulative complexity across all three problems. LLMs are better equipped for this synthesis.

AI accuracy can drop to 70-75% for complex multi-system cardiac cases, which means neither approach has solved the cardiology coding problem entirely. The practical recommendation for cardiology practices is a hybrid strategy. Leverage rule-based logic for procedural coding where template-driven documentation aligns well with deterministic rules, and deploy LLM-based tools for the E/M coding and chronic disease management encounters where contextual reasoning determines the correct code level.

Primary Care and High-Volume Specialties: The Throughput Question

LLM-based systems excel at identifying conditions documented in the clinical narrative that should be coded but weren't, a common gap in primary care where physicians focus on the visit's chief complaint and under-document ongoing chronic conditions. For high-volume practices evaluating which approach to adopt, consider these key factors:

1. Assess your current denial rate by code category. If denials cluster around E/M leveling or HCC-relevant diagnoses, an LLM-based approach likely addresses the root cause more effectively.

2. Evaluate your documentation variability. Practices with highly templated notes may see adequate performance from rule-based systems, while those with narrative-heavy documentation need contextual AI.

3. Calculate the revenue impact of under-coding. If your risk adjustment factor (RAF) scores lag behind clinical complexity, the ROI case for LLM-based coding strengthens considerably.

4. Factor in the implementation timeline. Rule-based systems typically deploy faster with less upfront configuration, while LLM systems may require a learning period but improve continuously.

5. Consider your specialty mix. Multi-specialty groups with primary care, cardiology, and surgical subspecialties may benefit most from LLM approaches that generalize across documentation styles.

Making the Decision: A Framework Beyond the Hype

The rule-based vs. LLM debate isn't binary, and the smartest organizations are choosing based on specific workflow needs. Here's what should drive your evaluation:

• Documentation structure matters more than specialty label. If your clinicians document in highly templated formats with structured data fields, rule-based systems will perform well regardless of specialty. If your documentation is narrative-heavy, dictated, or variable across providers, LLMs will outperform rule-based approaches.

• Compliance requirements shape the architecture. Organizations under active audit scrutiny may prefer rule-based systems for their explainability, even if LLMs offer higher accuracy. The ability to point to a specific rule that generated a code suggestion is a powerful audit defense.

• Denial patterns reveal the right investment. Pull your denial data by reason code. If denials center on medical-necessity or specificity issues, those are linguistic-understanding problems that favor LLM approaches. If denials cluster around simple coding errors or misapplication of modifiers, a well-configured rule-based system may be sufficient.

• Integration depth determines real-world value. Neither approach works in isolation. The most effective implementations, such as Sully.ai's AI coding agent, combine LLM-based clinical understanding with structured coding databases and human-review workflows, blending contextual reasoning with deterministic verification.

• Scalability trajectories differ. Rule-based systems require linear engineering effort as coding guidelines change. LLM-based systems can be retrained on updated guidelines more efficiently, but require ongoing monitoring for drift and hallucination.

With payer audits and denial amounts rising 12-14% year over year, the cost of choosing poorly is increasing. Organizations that match their coding technology to their actual documentation patterns and specialty-specific complexity will capture measurably more revenue while reducing compliance risk.

The technology is moving fast. LLM-based coding is improving at a pace that rule-based systems structurally cannot match, because language models learn from data while rule engines require manual programming. But that doesn't mean every practice should switch tomorrow. Audit your current performance and identify where your revenue leakage actually originates.

Sources:

• Large Language Models Are Poor Medical Coders — NEJM AI

• Enhancing Medical Coding Efficiency Through Domain-Specific Fine-Tuned LLMs — npj Health Systems (Nature)

• Evaluating an NLP-Driven AI-Assisted ICD-10-CM Coding System — Journal of Medical Internet Research

• Artificial Intelligence to Predict Billing Code Levels of ED Encounters — Annals of Emergency Medicine

• Systematic Evaluation of Common NLP Techniques to Codify Clinical Notes — BMC Medical Informatics (PMC)

• Limitations of CAC and How to Strengthen Documentation — American Institute of Healthcare Compliance

• Computer-Assisted Diagnostic Coding Using SNOMED CT to ICD-10 Mappings — AMIA (PMC)

• Procedure Complexity and Diagnosis Severity: Coding Challenges in Cardiology — Adonis Health

• Medical Coding Error Rates: Industry-Wide Report — AMBCI

• Payer Audits and Denial Amounts Rise Again in 2025 — Fierce Healthcare