Healthcare NLP: Why Clinical Text Is Different

Part 6 of the Info2K Data Engineering for AI Series

Healthcare contains enormous amounts of valuable information in clinical notes, referral letters, discharge summaries, radiology reports, pathology reports, medication instructions, and correspondence between healthcare professionals.

Much of this information is written as natural language rather than stored in neatly structured fields.

Natural Language Processing, or NLP, can help transform this narrative information into usable knowledge. It can support activities such as:

• finding relevant clinical information;

• identifying medical concepts;

• classifying documents;

• summarising patient histories;

• supporting clinical coding;

• detecting patterns across records;

• retrieving evidence for healthcare professionals;

• preparing information for analytics and research.

However, clinical text cannot be treated as ordinary business text.

A healthcare document may contain the correct clinical term while expressing that the condition is absent, historical, uncertain, hypothetical, experienced by a family member, or no longer current.

For clinical NLP, recognising words is only the beginning.

The system must understand enough context to determine what those words mean for the patient, at that time, in that document, and for the intended clinical use.

Connecting Data Quality with Clinical Language

In Part 5: Data Quality for AI, we explored why clean-looking data is not necessarily trustworthy data.

Clinical text demonstrates this challenge particularly clearly.

A clinical NLP pipeline can successfully:

• open the document;

• extract all its text;

• identify medical terminology;

• produce a valid structured output;

• complete without errors.

Yet its output can still be clinically wrong.

For example, consider these statements:

• “The patient has diabetes.”

• “No evidence of diabetes.”

• “Family history of diabetes.”

• “At risk of developing diabetes.”

• “Diabetes was suspected but later excluded.”

• “Previous gestational diabetes, now resolved.”

A keyword system may identify diabetes in every sentence.

A reliable clinical NLP system must distinguish the very different meanings surrounding that concept.

This is why data quality for clinical text includes more than completeness and formatting. It also includes contextual accuracy, clinical meaning, provenance, temporal interpretation, and fitness for the intended use.

Clinical Text Is Written for Care, Not Primarily for Machines

Clinical documentation is created in demanding operational environments.

Healthcare professionals write notes to communicate with colleagues, document care, satisfy legal and organisational requirements, and support continuity of treatment.

The resulting language may be:

• concise;

• highly specialised;

• locally specific;

• incomplete outside its original context;

• copied or carried forward;

• written under time pressure;

• distributed across multiple document types;

• structured differently by different authors and systems.

A note may contain sentence fragments, shorthand, lists, tables, headings, abbreviations, typographical errors, or information assumed to be obvious to another clinician.

This does not necessarily make the documentation poor. It means the text was written for a particular human and clinical context.

An NLP system does not automatically share that context.

Negation Changes the Meaning Completely

Negation is one of the best-known clinical NLP challenges.

Consider:

“No evidence of pneumonia.”

A system that extracts only the concept pneumonia may incorrectly report that the patient has the condition.

Negation can appear in many forms:

• no evidence of;

• denies;

• negative for;

• ruled out;

• absent;

• without;

• not demonstrated;

• unlikely;

• free from.

Its effect may also depend on sentence structure and document section.

For example:

“The patient denies chest pain but reports shortness of breath.”

The negation applies to chest pain, not shortness of breath.

Clinical NLP therefore needs to identify both the clinical concept and the contextual expression that modifies it.

The ConText algorithm extended earlier negation work by representing properties including negation, hypothetical status, historical status and experiencer. Its authors emphasised that simply identifying a clinical condition is insufficient for understanding the patient’s state. ConText clinical NLP study

Uncertainty Is Not the Same as Absence

Clinical reasoning often involves uncertainty.

Healthcare professionals may document:

• possible infection;

• suspected fracture;

• probable diagnosis;

• cannot exclude malignancy;

• symptoms consistent with;

• low likelihood of;

• differential diagnosis includes.

These expressions do not mean that the condition is confirmed. They also do not necessarily mean that it is absent.

An NLP system that converts every mentioned condition into a positive diagnosis can create misleading structured data.

Useful representations may need to distinguish among:

• confirmed;

• suspected;

• possible;

• unlikely;

• ruled out;

• conditional;

• pending investigation.

The appropriate categories depend on the clinical task and information model. What matters is that uncertainty should not disappear during extraction or summarisation.

Time Changes Clinical Meaning

Clinical records are longitudinal.

They describe events occurring across hours, days, months, or years.

A condition may be:

• active now;

• resolved;

• historical;

• recurrent;

• newly diagnosed;

• planned for future assessment;

• present only during a previous admission.

Consider:

“The patient had asthma as a child.”

This should not automatically be represented as an active current condition.

Similarly:

“Warfarin was discontinued before surgery.”

Extracting only warfarin without the discontinuation event and its time could create an unsafe interpretation.

Clinical NLP may need to connect concepts with:

• document date;

• event date;

• admission and discharge periods;

• medication start and stop dates;

• relative expressions such as “yesterday”;

• previous and current clinical episodes;

• changes in status over time.

The meaning of clinical information is often inseparable from its temporal context.

Who Is Experiencing the Condition?

Not every condition mentioned in a patient record belongs to the patient.

Consider:

“The patient’s mother had breast cancer.”

A system may correctly identify breast cancer while incorrectly assigning it to the patient.

The experiencer may be:

• the patient;

• a family member;

• another person;

• a population described in educational material;

• a hypothetical example.

SNOMED CT explicitly recognises context as the circumstances needed to interpret a concept appropriately. Its documentation uses asthma to demonstrate the difference between a current patient condition, past history and family history, noting that clear representation of context is essential for safe use and accurate analysis of electronic health records. SNOMED CT context guidance

Concept extraction must therefore preserve who the statement is about.

Abbreviations Are Efficient, but Frequently Ambiguous

Healthcare professionals use abbreviations to communicate quickly.

The same abbreviation can have different meanings depending on specialty, organisation, document type and surrounding text.

Examples may include:

• MS: multiple sclerosis, mitral stenosis or mental status;

• PT: patient, physiotherapy or prothrombin time;

• RA: rheumatoid arthritis, right atrium or room air;

• OD: right eye, once daily or overdose, depending on context and convention.

A study of more than 2,300 discharge summaries reported that nearly one-third of the distinct abbreviation usages assessed were ambiguous. Ambiguous medical-abbreviation study

A separate comparison of clinical NLP systems found that accurately identifying and interpreting abbreviations remained challenging. Clinical NLP abbreviation study

A terminology dictionary can provide possible expansions, but surrounding clinical context is usually required to select the appropriate meaning.

This is another reason general-purpose language processing cannot simply be applied without domain-specific evaluation.

Clinical Terminology Is Rich and Variable

The same clinical idea can be expressed in many ways.

For example:

• heart attack;

• myocardial infarction;

• MI;

• acute infarction of the myocardium;

• previous myocardial infarction.

Different expressions may refer to the same underlying concept, while similar expressions may represent clinically important differences.

Clinical NLP systems often use terminology resources such as:

• SNOMED CT;

• ICD classifications;

• LOINC;

• RxNorm or local medication terminology;

• the Unified Medical Language System;

• organisation-specific dictionaries and code sets.

Terminology mapping can help normalise text into reusable concepts. However, assigning a code is not enough by itself.

The system must still preserve:

• negation;

• uncertainty;

• temporality;

• experiencer;

• severity;

• anatomical site;

• laterality;

• associated values;

• provenance;

• local meaning.

Clinical narrative has been found to use rich and varied terminology, with abbreviations and concept-recognition errors presenting significant normalisation challenges. Clinical disorder-normalisation study

Terminology gives concepts a common representation. Context determines how those concepts should be interpreted.

Structured Data and Narrative Must Remain Connected

Healthcare systems often contain both structured fields and narrative text.

A diagnosis may appear as a coded condition, while the clinical note explains why it was suspected, how certain it is, and whether it remains current.

A medication field may contain a product and dose, while the narrative explains that it was withheld, declined, changed or entered incorrectly.

Structured data supports exchange, computation and analytics. Narrative preserves nuance and clinical reasoning that may not fit neatly into predefined fields.

HL7 FHIR allows resources to include a human-readable narrative containing essential clinical and business information. Its specification also supports links between structured data and the original text from which it was derived. HL7 FHIR narrative specification

Reliable clinical NLP should preserve this relationship where practical:

Original clinical statement
          ↓
Extracted mention
          ↓
Normalised clinical concept
          ↓
Context and attributes
          ↓
Source document and evidence

A structured output should not become detached from the text that supports it.

Clinical Documents Have Structure

A clinical document is not simply a sequence of words.

Meaning may depend on its section:

• Presenting complaint

• Medical history

• Family history

• Examination

• Investigation results

• Assessment

• Plan

• Discharge medication

• Allergies

The term “stroke” in a family-history section has a different implication from the same term in the current assessment.

Tables, headings, lists and document templates can also carry meaning. A parser that extracts all words but loses structure may reduce the quality of the resulting NLP output.

Clinical-document standards such as HL7 CDA explicitly preserve narrative blocks containing paragraphs, lists, tables, captions, links and other human-readable structure. HL7 CDA narrative-block specification

Document parsing should therefore be evaluated for preservation of clinical meaning, not only text-extraction coverage.

Copying and Repetition Create Additional Risk

Clinical records may contain copied-forward information, standard templates and repeated historical content.

Repetition can be useful for continuity, but it can also create problems for NLP:

• outdated conditions may appear current;

• repeated text may be counted as multiple events;

• contradictory versions may coexist;

• copied errors may propagate;

• RAG systems may retrieve several nearly identical passages.

Deduplication cannot always be based on exact text because small changes may be clinically significant.

A reliable pipeline should preserve document dates, authorship, source, version and relationships among documents. It should also distinguish between repeated evidence and independent evidence.

Privacy Is Part of the Engineering Design

Clinical free text can contain identifiable and sensitive information in unpredictable places.

Names, addresses, dates, contact details, medical-record numbers, family relationships, workplaces and rare conditions may appear inside narrative text.

De-identification is therefore not equivalent to removing a few structured columns.

For example, US HHS guidance describes two HIPAA de-identification approaches, Safe Harbor and Expert Determination, and notes that even properly de-identified information can retain some re-identification risk. HHS de-identification guidance

Organisations must apply the privacy, health-information and research requirements relevant to their own jurisdiction and use case.

Clinical NLP architecture should consider:

• whether identifiable text is necessary;

• where processing occurs;

• who can access source and derived data;

• how outputs are logged;

• whether temporary data is retained;

• how models and vendors handle information;

• whether generated outputs can reveal sensitive details;

• how re-identification risk is evaluated;

• how consent and permitted use are enforced.

Privacy cannot be added after the NLP system has already copied sensitive information into uncontrolled environments.

Evaluation Must Reflect the Clinical Task

A single overall accuracy score is rarely sufficient.

Evaluation should reflect what the system is expected to do and the consequences of its errors.

Depending on the use case, evaluation may include:

• concept-identification precision and recall;

• terminology-normalisation accuracy;

• negation accuracy;

• uncertainty classification;

• temporal-relation accuracy;

• experiencer identification;

• document-classification performance;

• evidence retrieval;

• summary faithfulness;

• subgroup performance;

• privacy leakage;

• clinically significant error analysis.

The costs of false positives and false negatives may differ.

Missing a non-critical research concept is not the same as incorrectly identifying an active medication allergy.

Evaluation should therefore include representative documents, realistic workflows, domain experts and explicit review of clinically significant errors.

Generative AI Does Not Remove These Requirements

Large language models can interpret language more flexibly than many traditional rule-based systems.

They may help with:

• information extraction;

• summarisation;

• classification;

• terminology explanation;

• question answering;

• retrieval;

• document drafting.

However, fluent language does not guarantee correct clinical interpretation.

A generative system may:

• convert uncertainty into certainty;

• omit qualifying information;

• merge events from different dates;

• attribute family history to the patient;

• generate unsupported details;

• cite evidence that does not support its conclusion.

Generative AI therefore needs the same foundations discussed throughout this series:

• authoritative data;

• preserved context;

• governance;

• privacy;

• quality controls;

• traceable evidence;

• evaluation;

• monitoring;

• human oversight.

The model is a component of the clinical information system, not a replacement for its engineering and safety responsibilities.

AI Should Assist Healthcare Professionals

Healthcare NLP can reduce the effort required to find, organise and interpret large volumes of information.

It can help people:

• locate relevant passages;

• organise longitudinal histories;

• identify potential documentation gaps;

• prepare information for review;

• support coding and research workflows;

• retrieve relevant guidance.

But final responsibility must remain clear.

Qualified professionals provide:

• clinical judgement;

• understanding of the patient;

• interpretation of uncertainty;

• ethical responsibility;

• verification;

• accountability.

The most valuable goal is not unrestricted automation.

It is well-designed assistance that gives people better access to relevant, contextual and verifiable information.

A Practical Healthcare NLP Checklist

Before using clinical text in an NLP or generative AI system, ask:

1. What clinical or operational task will the system support?

2. Who created the source text, and for what purpose?

3. Can the system identify negation?

4. Can it distinguish confirmed, suspected and ruled-out conditions?

5. Can it determine whether information is current or historical?

6. Can it identify who experienced the condition?

7. How are abbreviations and local terminology handled?

8. Are extracted concepts mapped to appropriate terminologies?

9. Is document structure preserved?

10. Can every derived output be traced to its source?

11. Are conflicting or copied records handled appropriately?

12. Are privacy and access controls enforced throughout the pipeline?

13. Does the evaluation dataset represent the intended environment?

14. Are clinically significant errors reviewed separately?

15. Is qualified human oversight built into the workflow?

16. Can the system express uncertainty or decline when evidence is insufficient?

These questions help turn clinical NLP from a text-processing demonstration into a governed healthcare capability.

The Central Takeaway

Clinical text is different because its meaning depends on more than the words it contains.

Meaning depends on:

• context;

• negation;

• uncertainty;

• time;

• experiencer;

• terminology;

• document structure;

• provenance;

• privacy;

• intended clinical use.

A system that identifies a medical term has not necessarily understood the clinical statement.

Reliable healthcare NLP must preserve the path from the original narrative to the extracted concept, its contextual meaning, and the evidence that supports it.

Because in healthcare, transforming information into knowledge is not only a technical objective.

It is also a matter of trust, privacy, safety and human responsibility.

Continue the Info2K Data Engineering for AI Series

This article is Part 6 of the Info2K Data Engineering for AI series.

Previously:

• Part 1: The Hidden Data Engineering Behind Successful AI Projects

• Part 2: What Does AI-Ready Data Actually Mean?

• Part 3: ETL vs ELT for Modern AI Systems

• Part 4: Building Reliable RAG Pipelines

• Part 5: Data Quality for AI: Why Clean Data Is Not Enough

Coming next:

• Part 7: Responsible AI Starts with Responsible Data

• Part 8: Designing Cloud Data Platforms for AI

• Part 9: Modern Data Engineering as the Foundation of Enterprise AI

Follow Info2K as we continue exploring how information becomes reliable, governed, contextual and usable knowledge.

About Info2K

Info2K : Information to Knowledge

Info2K shares practical insights across Data Engineering, AI Engineering, Healthcare NLP, Cloud Data Platforms, Digital Health and responsible technology.

Our focus is on connecting engineering, domain knowledge, governance, reliability and real-world outcomes.

Website: https://info2k.co.nz LinkedIn: https://www.linkedin.com/company/info2k