Skip to content
DEXPRADEXPRA home

Article · Data platforms

What a healthcare data platform is made of

The same two words describe an integration engine and a cloud lakehouse. Naming the parts, six components and two shared services, is what makes two very different platforms comparable.

DEXPRA · · 14 min read

Ask two people at the same hospital what their data platform is and you can get two different systems. One means the integration engine: forty interfaces, HL7 v2 feeds from the laboratory and the admissions system, FHIR APIs for the patient portal, a message queue that never stops. The other means the cloud lakehouse the analytics team has been building for two years, where a decade of encounters sits in tables a model can be trained on.

Both are right, and the two systems have almost nothing in common except the name. A procurement conversation in which the two sides mean different ones is not an unusual failure. It is the normal case, and it is why so many platform projects deliver something that works and is not what anyone needed.

This piece names the parts. It starts from the five-stage picture on our home page, opens the data platform stage into the six components that picture already lists, and adds the two services every component depends on and none of them owns. It ends on where the standards stop, which is the reason two platforms built on identical standards can still produce different numbers.

Two systems that share a name

The cleanest way to tell them apart is the question each one answers. An integration engine and the operational stores behind it answer questions about one patient, now: what is this patient’s latest potassium, is there an allergy on file, which bed are they in. An analytics platform answers questions about populations over time: how did potassium results relate to readmission across ten years of patients, which clinics have the longest wait from referral to first visit.

Everything else follows from that. The first system is built for freshness, measured in seconds, and keeps only as much history as it needs to route and serve. The second is built for history, measured in years, and is fresh only to within a day, by design. One fixes its schema on write, interface by interface. The other lands data raw and models it in tiers above, the arrangement lakehouse vendors call bronze, silver and gold.

Two systems, one name

Both get called the data platform. Bring either column forward to read it on its own; nothing leaves the table.

Showing every column at equal weight.

DimensionIntegration engine and operational storeMoves and servesAnalytics platformAccumulates and answers
The questionIntegration engine and operational storeWhat is this patient’s latest potassium, right now?Analytics platformHow did potassium results relate to readmission, across ten years of patients?
Unit of workIntegration engine and operational storeOne message, one patient, one transactionAnalytics platformA population, a cohort, a time series
How freshIntegration engine and operational storeSeconds. An answer that arrives late is a wrong answerAnalytics platformHours to a day, by design
How much historyIntegration engine and operational storeEnough to route a message and serve the recordAnalytics platformAll of it, versioned, so last year’s number can be computed again
Shape of the dataIntegration engine and operational storeThe source’s own, translated in flight: HL7 v2, FHIR, DICOMAnalytics platformA model built for reading: dimensional, OMOP, or flattened FHIR
Schema fixedIntegration engine and operational storeOn write, interface by interfaceAnalytics platformOn read at the raw tier, then modelled in the tiers above it
Typical technologyIntegration engine and operational storeInterface engines, FHIR servers, message queuesAnalytics platformWarehouses and lakehouses, and the EHR vendor’s own analytic model
Who uses itIntegration engine and operational storeApplications, and clinicians through themAnalytics platformAnalysts, researchers, quality teams, model builders
Governed byIntegration engine and operational storeInterface specifications, and the standards on the wireAnalytics platformMostly the organisation’s own decisions. The last section of this piece is about why
Bought as the otherIntegration engine and operational storeAsked for ten years of potassium, it searches message logs nobody indexedAnalytics platformAsked to stop a dose, it answers after the dose has been given

The procurement mistake

Most organisations need both, joined by the interoperability stage between them. The expensive mistake is not lacking one of them. It is buying one and calling it the other, and finding out which you bought the first time somebody asks it the wrong question.

Bring either column forward; nothing is removed from the table. The last row is the one to read before signing anything.

Most organisations need both, joined by the interoperability stage between them. The rest of this piece is about the second.

Where the platform sits in the stack

Our home page draws healthcare data as five stages: sources, interoperability, data platform, applied AI and machine learning, and outcomes. The stage before the platform has one job, which is getting data out of source systems in a standard shape. That is what HL7 v2, DICOM, FHIR and the IHE profiles are for, and it is the subject of the interoperability series on this site.

The data platform stage adds three things the interoperability stage does not have and is not meant to have: history, a single model that everything downstream reads, and the evidence of how every number was made.

The data platform stage, opened into its partsFive stages run left to right: sources, fragmented by default; interoperability, standards made usable; data platform, foundations you can build on; applied AI and machine learning, built to reach production; and outcomes, where it has to land. The data platform stage is opened below into six components: the canonical model, which answers what shape everything downstream reads; data contracts, what each source has promised to send; lineage and audit trail, how a number was made and who has seen it; access control, who may see a record and for what purpose; quality monitoring, whether the data is fit for the use in front of it; and event and bulk delivery, whether data leaves one change at a time or all at once. Under all six, in a dashed band that belongs to none of them, are the two shared services: patient identity and terminology.SOURCESFragmentedby defaultINTEROPERABILITYStandards,made usableDATA PLATFORMFoundations youcan build onAPPLIED AI / MLBuilt to reachproductionOUTCOMESWhere it hasto landINSIDE THE DATA PLATFORM STAGE · SIX COMPONENTS01CANONICAL MODELWhat shape does everythingdownstream read?OMOP · SQL ON FHIR02DATA CONTRACTSWhat has each sourcepromised to send?ODCS · GENERAL-PURPOSE03LINEAGE & AUDIT TRAILHow was this number made,and who has seen it?PROVENANCE · AUDITEVENT04ACCESS CONTROLWho may see this record,and for what purpose?SECURITY LABELS · CONSENT05QUALITY MONITORINGIs this data fit for theuse in front of it?KAHN · OHDSI DQD06EVENT & BULK DELIVERYOne change at a time,or everything at once?SUBSCRIPTIONS · BULK DATASHARED BY ALL SIXInside none of themPATIENT IDENTITYTERMINOLOGY
The five stages from the home page, with the data platform stage opened. The component names are the home page’s own.

The gap between the two stages shows up in national data. ONC’s survey of US hospitals tracks four steps of exchange: sending a record, receiving one, finding one by query, and integrating a received record into the EHR without anyone retyping it. In 2023, 84% of hospitals routinely sent records. Only 51% routinely integrated the ones they received (ONC Data Brief No. 71).

US hospitals that send, receive, find and integrate records, 2023A horizontal bar chart of the share of US non-federal acute care hospitals engaging in each domain of interoperable exchange in 2023, split into routinely and sometimes. Send: 84 percent routinely, 8 more sometimes. Receive: 73 percent routinely, 13 more sometimes. Find: 76 percent routinely, 7 more sometimes. Integrate, meaning the EHR takes the record in without manual entry: 51 percent routinely, 27 more sometimes. All four domains together: 43 percent routinely, 27 more sometimes. The gap between routinely sending, at 84 percent, and routinely integrating, at 51 percent, is 33 points. Source: ONC Data Brief No. 71, May 2024, from the 2023 AHA Annual Survey IT Supplement.SHARE OF US HOSPITALS, 2023ROUTINELYSOMETIMES0%25%50%75%100%SENDa summary of care record84%+8RECEIVEone from outside73%+13FINDquery outside sources76%+7INTEGRATEinto the EHR, no retyping51%+2733 POINTS FROM SENT TO USEDALL FOURsend, receive, find and integrate43%+27Source: ONC Data Brief No. 71, May 2024. 2023 AHA Annual Survey IT Supplement, US non-federal acute care hospitals,n = 2,547. Integrate means the EHR takes a received summary of care record in without manual entry.
Integration in the survey means into the EHR, one document type at a time. A platform does the same step for every source it has.

That last step is the hard one, and it is the step a data platform repeats at scale. Receiving a record is a transport problem the standards largely solve. Using it means knowing whose record it is, what its codes mean, and whether it can be trusted, and those three questions are where most of the work in a platform goes.

It is easiest to see by following one fact through all five stages. The panel below takes a single potassium result from the laboratory system to a population query, with the payload as it actually looks at each stage.

One result, five shapes

A single potassium result, followed through the five stages. Choose a stage to see the payload as it looks there, what that stage added, and what it still cannot tell you.

HL7 v2.5.1 · ORU^R01

The laboratory system sends a message

A serum potassium of 5.9 mmol/L, flagged high, leaves the laboratory system as an HL7 v2 result. It is in the lab’s own vocabulary: the test is called K, a local code that means something inside this lab and nowhere else.

HL7 v2 · four segments

MSH|^~\&|LIS|GENHOSP|ENGINE|GENHOSP|20261014081500||ORU^R01^ORU_R01|MSG-000913|P|2.5.1
PID|1||004417^^^GENHOSP^MR||TEST^PATIENT||19580302|F
OBR|1||LAB-88214|K^Potassium^L|||20261014074000
OBX|1|NM|K^Potassium^L||5.9|mmol/L|3.5-5.1|H|||F|||20261014081200

What this stage added

  • The fact itself: a value, a unit, a range, a flag and a time

What it still cannot tell you

  • What K means to anyone outside this laboratory
  • Whether record 004417 is the same person as a record somewhere else
  • Anything about this patient before this morning
Codes checked against LOINC, OHDSI’s own configuration and HL7 v2.5.1. The patient, the hospital and the record numbers are invented.

Read the right-hand list at each stage and the two shared services appear by themselves. Whether record 004417 here is the same person as A-77310 elsewhere is a question of identity. Whether the laboratory’s K means LOINC 2823-3 is a question of terminology. Neither belongs to any one stage, and the platform cannot be right without both.

The six components of a data platform

The home page diagram lists six components inside the platform stage. Each answers one question, and the useful way to read them is by what goes wrong when one is missing. None of those failures raises an error. The pipeline keeps running and the number is simply wrong.

Six components, and what each one is for

Read the middle line of each card. That is the failure a missing component produces, and none of them raises an error.

01

Canonical model

What shape does everything downstream read?

Without it · Every consumer builds its own flattening of the source, and the same question gets three answers.

  • OMOP CDM
  • SQL on FHIR
02

Data contracts

What has each source promised to send?

Without it · A source upgrade renames a field and the pipeline keeps running, filling a column with nulls nobody notices.

  • ODCS
03

Lineage and audit trail

How was this number made, and who has seen it?

Without it · A regulator, an auditor or a clinician asks where a figure came from and the honest answer is a reconstruction.

  • FHIR Provenance
  • AuditEvent
04

Access control

Who may see this record, and for what purpose?

Without it · The platform becomes the one place the source system’s permissions and the patient’s directives stop applying.

  • Security labels
  • Consent
05

Quality monitoring

Is this data fit for the use in front of it?

Without it · Every record validates and the dataset is still wrong: half the encounters missing a value, a unit changed in March.

  • Kahn framework
  • OHDSI DQD
06

Event and bulk delivery

Does data leave one change at a time, or all at once?

Without it · Population extracts run as millions of single calls, or alerts wait for tonight’s batch.

  • Bulk Data
  • Subscriptions
Patient identityTerminologyShared by all six · inside none of them

The two services every component depends on and no component owns. Which is why they are so often added last, and why adding them last means rebuilding the canonical model around them.

Canonical model

The canonical model is the one shape everything downstream reads. Data may arrive as HL7 v2 segments, FHIR resources and flat files, and it still has to land in a form an analyst, a quality measure and a model can all query. In practice there are three candidate shapes: the EHR vendor’s own dimensional model, the OMOP Common Data Model maintained by the OHDSI community, and FHIR itself, flattened into tables.

Each is good at something different. The vendor’s model tracks its source closely and is maintained for you, and it stops at the vendor’s boundary. OMOP is built for observational research and has the largest analytic tool ecosystem in the field, at the price of mapping every local code into its standard vocabulary. Its current version, 5.5, was released in August 2026, and most of the tools have not caught up with its new fields yet.

Three candidate shapes

What each canonical model is for, what it costs to keep, and where it stops. Bring one forward to read it alone.

Showing every column at equal weight.

DimensionVendor dimensionalThe EHR vendor’s modelOMOP CDMOHDSI · v5.5FHIR, flattenedSQL on FHIR
Defined byVendor dimensionalThe vendorOMOP CDMOHDSI, an open research communityFHIR, flattenedAn HL7 implementation guide, over FHIR profiles you choose
Built forVendor dimensionalThe vendor’s own reporting tools, against its own sourceOMOP CDMObservational research: cohorts, comparative studies, studies across institutionsFHIR, flattenedAnalytic views over FHIR data that run on more than one engine
ShapeVendor dimensionalFacts and dimensionsOMOP CDMPerson-centred event tables beside a standardised vocabularyFHIR, flattenedOne flat table per view definition
CodesVendor dimensionalAs the source system holds themOMOP CDMMapped to standard concepts drawn from LOINC, SNOMED CT, RxNorm and othersFHIR, flattenedWhatever the FHIR resource carries
Version nowVendor dimensionalThe vendor’s, on the vendor’s scheduleOMOP CDMv5.5, released August 2026. Most tools do not support its new fields yetFHIR, flattened2.0.0 published October 2024. 3.0.0 in ballot, targeting FHIR R6
Does wellVendor dimensionalTracks its source closely, and is maintained for youOMOP CDMThe largest analytic tool ecosystem in the field, and studies that run unchanged across sitesFHIR, flattenedStays close to the API. A view is a standard document, not code in one engine
Stops atVendor dimensionalThe vendor’s boundary. Joining other sources is your problemOMOP CDMMapping effort, and a design for research rather than operationsFHIR, flattenedDeliberately limited expressivity, and only as good as the FHIR you are sent
You maintainVendor dimensionalThe extensions you add to itOMOP CDMThe extract, transform and load, and every mapping in itFHIR, flattenedThe view definitions, and the profiles they assume
Usual readerVendor dimensionalOperational and financial reportingOMOP CDMResearchers and epidemiologistsFHIR, flattenedAnalysts, and engineers building marts

The cost of carrying two

Most platforms end up carrying two of these, usually a reporting shape and a research one. The cost of the second is reconciliation: the same cohort counted in both has to come out the same, and nothing makes it come out the same except a test somebody writes and keeps running.

Versions as at October 2026. OMOP and SQL on FHIR are both moving.

The third shape has a standard of its own. SQL on FHIR defines a ViewDefinition: a small JSON document that names a FHIR resource type, selects elements from it with FHIRPath, and produces a flat table. Version 2.0.0 was published in October 2024. Version 3.0.0 is in ballot, and it makes the ViewDefinition a FHIR resource in its own right, targeting FHIR R6. The point of a view is portability. The same definition runs on more than one engine, so the logic that flattens a record is a standard document rather than code inside one product.

serum-potassium.ViewDefinition.json
{
  "resourceType": "ViewDefinition",
  "name": "serum_potassium",
  "status": "draft",
  "resource": "Observation",
  "constant": [{ "name": "potassium", "valueCode": "2823-3" }],
  "select": [{
    "column": [
      { "name": "id",         "path": "getResourceKey()" },
      { "name": "patient_id", "path": "subject.getReferenceKey(Patient)" },
      { "name": "effective",  "path": "effective.ofType(dateTime)" },
      { "name": "value",      "path": "value.ofType(Quantity).value" },
      { "name": "unit",       "path": "value.ofType(Quantity).code" }
    ]
  }],
  "where": [{
    "path": "code.coding.exists(system = 'http://loinc.org' and code = %potassium)"
  }]
}

Most platforms end up carrying two of these, typically a reporting shape and a research one. The cost of the second model is not storage. It is reconciliation: the same cohort counted in both has to agree, and nothing makes it agree except a test somebody writes and keeps running.

Data contracts

A data contract is what a source has promised to send. It covers the schema, the meaning of each field, the units, how often data arrives and who to tell when any of that changes. Without one, the platform depends on the source staying the same, and sources do not stay the same. Every upgrade, every local configuration change and every new picklist value is a chance for a field to be renamed or repurposed.

The failure is rarely a pipeline that stops. It is a pipeline that keeps running: a renamed column fills with nulls, a unit changes in March, a code list gains a value nothing downstream expects. A contract turns that into an event somebody sees, because data that breaks the promise is held back instead of loaded.

What a data contract covers
PartWhat it promisesWhat breaks without it
SchemaThe fields, their types, and which are requiredA renamed field loads as nulls
SemanticsWhat each field means, and which code system it usesA local code is reused for something else
UnitsThe unit of every measurementA value changes scale and no rule notices
TimelinessHow often data arrives, and how late it may beA feed stops and yesterday is reported as today
OwnershipWho changes the source, and who must be toldAn upgrade lands on a Friday with no notice

There is no healthcare standard for data contracts. There is a general one: the Open Data Contract Standard, a Linux Foundation project whose version 3.1.0 describes a dataset’s schema, quality rules, service levels and owners in YAML. It knows nothing about clinical data, which is the part a healthcare team has to supply.

Lineage and audit trail

Lineage answers how a number was made. Audit answers who has seen the data. They are different records about different parts of the pipeline: lineage describes what each transformation did, and provenance describes where each dataset came from and under what permission. In FHIR terms, Provenance records how a resource came to hold its value and AuditEvent records system events such as who read what. A platform usually needs both.

Lineage attaches to the edges of a pipeline, provenance attaches to its nodesA pipeline runs left to right: source, transform, feature store, training set. A lineage band above connects down to the three transformations between those datasets. A provenance band below connects up to the four datasets themselves. Lineage records what was done to the data; provenance records where each dataset came from and under what permission.LINEAGE · WHAT HAPPENED TO THE DATAAttaches to the edges: every transformation, join, filter and snapshotPROVENANCE · WHERE IT CAME FROM, AND ON WHAT PERMISSIONAttaches to the nodes: origin, author, consent, agreement, permitted purposeSOURCETRANSFORMFEATURE STORETRAINING SET
From our earlier piece on the data platform as an evidence package, which covers what regulators ask of training data.

Lineage has to work at the level of the attribute, not the table, because the questions that matter are about attributes. Which encounters fed this readmission rate. Which version of the potassium mapping was in force when this cohort was drawn. Who was in the training data for this model. Our piece on the evidence package takes the last of those in detail, starting from the 95.5% of FDA-cleared AI devices whose summaries never said who was in the training data.

In Ontario the audit side has statutory weight coming. PHIPA was amended in 2020 to require custodians that handle personal health information electronically to keep an electronic audit log of every access, recording the type of information, the date and time, the person and the patient. That section has not yet been proclaimed in force. The Information and Privacy Commissioner has expected custodians to be able to audit electronic access since its HO-013 order in 2014, so in practice the duty is already real.

Access control

A platform is a new place the data lives. The permissions the source system enforced do not travel with the data by themselves, and neither do the patient’s own directives. Access control in a platform has to be asserted again, and in healthcare it is a decision in six layers rather than one.

The six layers of an access decisionSix questions, asked in order. One, who is asking: a clinician, an analyst or a service account, carried by SMART scopes and directory roles. Two, for what purpose, carried by HL7 purpose-of-use codes such as TREAT, HOPERAT for operations, HRESCH for research, HPAYMT for payment and PUBHLTH for public health. Three, how sensitive the data is, carried by security labels: a confidentiality level of normal, restricted or very restricted, then sensitivity codes such as SUD for substance use disorder, MH for mental health, HIV, and SEX for sexual and reproductive health. Four, whether the patient has said no: in Ontario the PHIPA lockbox, represented in FHIR as a Consent. Five, whether it is an emergency: break-the-glass, coded ETREAT or BTG, which grants access with a stated reason and guarantees a review. Six, whether it can be shown afterwards: an audit trail, carried by FHIR AuditEvent and IHE ATNA; PHIPA section 10.1 on audit logs is enacted and not yet proclaimed.THE QUESTIONWHAT CARRIES THE ANSWER01WHOWho is asking?A clinician, an analyst, or a serviceaccount acting for an application.SMART SCOPESDIRECTORY ROLE02WHYFor what purpose?The same person treats today andresearches tomorrow. Purpose is per request.TREATHOPERATHRESCHHPAYMTPUBHLTH03WHATHow sensitive is this data?Labels travel on the record: a confidentialitylevel, then the kind of sensitivity.N · R · VSUDMHHIVSEX04WHOSE WISHESHas the patient said no?In Ontario, a lockbox withholds or withdrawsconsent for health care purposes.PHIPA LOCKBOXFHIR CONSENT05OVERRIDEIs this an emergency?Break-the-glass grants access with a statedreason, and guarantees a review of it.ETREATBTG06AUDITCan it be shown afterwards?Who saw what, when, and why. PHIPA s. 10.1is enacted and not yet proclaimed.AUDITEVENTIHE ATNACodes from HL7 v3 ActReason, Confidentiality and ActCode, checked against HL7’s terminology server. A structure, not legal advice.
Who, why, what, whose wishes, override, audit: six questions, each carried by something different.

The layers are worth separating because each is carried by something different. Who is asking is identity and role. Why they are asking is purpose of use, which HL7 codes as TREAT for treatment, HOPERAT for operations and HRESCH for research, and which matters because the same analyst may do quality work in the morning and research in the afternoon. What they are asking for is carried by security labels on the record itself: a confidentiality level, then a sensitivity code such as SUD for substance use disorder or MH for mental health.

The fourth layer is the patient’s own say. In Ontario that is what the Information and Privacy Commissioner calls the lockbox: a patient may withhold or withdraw consent to the use and disclosure of their information for health care purposes, under sections 37(1)(a), 38(1)(a) and 50(1)(e) of PHIPA. A platform that copies data out of a source without the directive attached has quietly removed it. In the United States the nearest analogue is narrower: records from substance use disorder programs under 42 CFR Part 2, where since the 2024 rule a single consent can cover future treatment, payment and operations.

Then the override, and the record of it. Break-the-glass access, coded ETREAT or BTG, exists because a rule that blocks an emergency is worse than no rule. It grants access with a stated reason and guarantees that somebody reviews it afterwards, which brings the audit trail back in.

Quality monitoring

Clinical data quality is its own discipline. General data observability tools watch row counts, schema drift and freshness, and they do not know that an adult does not weigh 15 kg or that a discharge cannot come before an admission. The vocabulary most healthcare work uses comes from Kahn and colleagues in 2016: conformance asks whether values follow the rules, completeness asks whether they are present, and plausibility asks whether they are believable. Each can be checked against the data’s own expectations, which Kahn calls verification, or against something outside it, which is validation.

The distinction that matters for a platform is between what one record can show and what only the whole dataset can. A FHIR validator sees one resource at a time. It can reject a code outside its value set or a missing required element. It cannot see that blood pressure has gone missing from a third of this month’s visits, that one person has two records, or that a site’s HbA1c values have drifted away from a reference laboratory’s.

Data quality, in Kahn’s terms

Eighteen example checks for clinical data. Filter by category, or by what a check has to see. Filtering narrows what is shown; every check stays in the page.

Category

What the check has to see

Showing 18 of 18 · 9 need the whole dataset

  • Conformance · ValueVerificationOne record

    A coded value outside its value set

    Sex sent as a local code where an administrative gender code is required

    Caught by · A resource validator, against the binding

  • Conformance · ValueVerificationOne record

    A code that does not exist

    A LOINC code absent from the release the platform has loaded

    Caught by · The terminology service, validating the code

  • Conformance · RelationalVerificationOne record

    A required element missing

    An Observation with no status

    Caught by · A resource validator. Kahn’s own table also files a missing required value under completeness

  • Conformance · RelationalVerificationAcross records

    A reference to nothing

    An Observation whose subject is a Patient the platform never received

    Caught by · A referential check across the dataset

  • Conformance · ComputationalVerificationOne recordKahn’s example

    A derived value that does not recompute

    A stored BMI that disagrees with the height and weight recorded beside it

    Caught by · A recomputation rule

  • Conformance · ComputationalValidationOne record

    A published algorithm, recomputed

    An eGFR that the published equation, given the same inputs, does not reproduce

    Caught by · Recomputation against the published equation

  • CompletenessVerificationAcross recordsKahn’s example

    A value that stopped arriving

    Discharge time missing for three days running

    Caught by · A query over time

  • CompletenessVerificationAcross records

    An optional field that is usually there

    Blood pressure absent from far more outpatient visits this month than last

    Caught by · A query comparing periods

  • CompletenessValidationAcross recordsKahn’s example

    A gap an outside event explains

    Diagnosis codes drop when a classification changes

    Caught by · Comparison with a known external change

  • Plausibility · UniquenessVerificationAcross recordsKahn’s example

    One person, two records

    The same patient registered twice under two record numbers

    Caught by · Identity resolution

  • Plausibility · AtemporalVerificationOne record

    An impossible value

    An adult weight of 15 kg

    Caught by · A range rule, per concept and per unit

  • Plausibility · AtemporalVerificationOne record

    A unit that does not fit the concept

    Serum potassium recorded in a unit outside the approved list

    Caught by · An approved-unit list per concept. OHDSI’s tool keeps one: 8753 or 9557 for this test

  • Plausibility · AtemporalValidationOne record

    A valid code that means something else

    SNOMED CT 365761000 mapped as serum potassium. It is Sodium level – finding

    Caught by · A lookup of what the code means, checked against what was meant

  • Plausibility · AtemporalVerificationAcross recordsKahn’s example

    One fact contradicting another

    A prostate cancer diagnosis on a patient recorded as female

    Caught by · A rule across two resources

  • Plausibility · AtemporalValidationAcross recordsKahn’s example

    A distribution that disagrees with a reference

    HbA1c values here differ from a reference laboratory’s for the same population

    Caught by · An external benchmark

  • Plausibility · TemporalVerificationOne recordKahn’s example

    Time running backwards

    A discharge recorded before the admission

    Caught by · A rule on the encounter’s period

  • Plausibility · TemporalVerificationAcross records

    An event before birth

    A result dated before the patient’s date of birth

    Caught by · A rule across two resources

  • Plausibility · TemporalVerificationAcross recordsKahn’s example

    A season that did not happen

    No winter rise in respiratory visits

    Caught by · A time-series check

What one widely used tool checks

OHDSI’s Data Quality Dashboard defines 27 check types and files each under Kahn. Counted from its own check list, version 2.9.0.

Conformance

Do values follow the rules?

9 types · 1 validation

Completeness

Are values present?

6 types · 2 validation

Plausibility

Are values believable?

12 types · 2 validation

Twenty-two of the 27 are verification: checks against the data’s own rules. Five compare against something outside it. The external benchmarks are the part a team has to supply.

Kahn et al., eGEMs, 2016. Check types counted from the OHDSI Data Quality Dashboard’s own list, version 2.9.0.

Thresholds are configuration, and configuration is a clinical decision. OHDSI’s Data Quality Dashboard ships plausibility bounds for a handful of measurements and approved units for many more, and every one of them is a judgement somebody made. A platform that runs the checks without anyone owning the thresholds has automated a guess.

One example in the panel is worth singling out. SNOMED CT 365761000 is a valid code. It means Sodium level, a finding. Mapped as serum potassium, it passes every format check there is, and only a lookup of what the code means, compared with what was meant, catches it. That is a quality check, and it is also the reason terminology is a service rather than a spreadsheet.

Event and bulk delivery

Data leaves a platform in two shapes. Events go one change at a time, as they happen: a result is finalised, a patient is transferred, a subscriber is told. Bulk goes all at once: a population extract, a nightly load, a research dataset. They are separate components because they fail differently and are governed by different specifications.

For events, FHIR R5 introduced topic-based subscriptions, and the Subscriptions R5 Backport guide, version 1.1.0, brings them to R4 servers, which is what most production systems in Canada and the United States still run. For bulk, the FHIR Bulk Data Access guide defines an asynchronous export. It is now at version 3.0.0, and every certified EHR in the United States has to support it for groups of patients.

A FHIR Bulk Data export, step by stepA sequence between three parties: the platform, the FHIR server and the file server. The platform kicks off an export with a GET to the Group export operation and the header Prefer: respond-async. The server answers 202 Accepted with a Content-Location header giving a status URL. The platform polls the status URL; while the export runs the server answers 202 with optional X-Progress and Retry-After headers, and 429 if polled too often. When the export is complete the server answers 200 OK with a manifest listing transactionTime, the output files and any error files. The platform fetches each output file, which is newline-delimited FHIR JSON, one resource per line. Finally the platform sends DELETE to the status URL, answered 202, after which the files may be removed. The platform keeps transactionTime as the _since value for its next export.THE PLATFORMBulk Data clientFHIR SERVERThe EHR, usuallyFILE SERVERMay be elsewhereREPEATGET [base]/Group/[id]/$exportPrefer: respond-async202 AcceptedContent-Location: the status URLGET [status URL]202 · X-Progress · Retry-AfterPoll too often and the answer is 429GET [status URL]200 OK + manifesttransactionTime, output[ ], error[ ]GET [file URL] for each outputapplication/fhir+ndjsonOne resource per lineDELETE [status URL]202 AcceptedFiles may now be removedKEEP transactionTimeIt is the _since of the next export, which is how a nightly load stays incremental.
From the FHIR Bulk Data Access guide, version 3.0.0. Every certified US EHR has to answer the group export.

Claims that streaming makes bulk export obsolete come mostly from the people selling the streaming. Streaming a FHIR store into an analytic engine is real and useful. It does not replace the one population interface every certified EHR is obliged to offer.

The two shared services: patient identity and terminology

Identity and terminology appear in every component and belong to none. The canonical model needs one person to file records under. Contracts need code systems to say what a field means. Lineage needs to record which mapping version was used. Access control needs to know whose directive applies. Quality checks need both to find duplicates and wrong codes. Because they belong to no component they are usually added last, and adding them last means rebuilding the canonical model around them.

Patient identity

An enterprise master patient index decides when two records describe one person. Inside one EHR the record number does most of that work. Across systems, and especially across organisations, records have to be matched on what they share: a health number, names, a date of birth, a postal code.

The standard method is the one Fellegi and Sunter published in 1969. For each field it compares two probabilities: m, how often the field agrees when the records really are the same person, and u, how often it agrees by coincidence when they are not. Agreement adds log₂(m/u) to a running score, disagreement subtracts, and the total is compared with two thresholds. Above the upper one the records are linked. Below the lower one they are not. Between the two, a person decides. Open-source implementations such as Splink, from the UK Ministry of Justice, estimate m and u from the data itself.

Are these two records one person?

A Fellegi–Sunter match, field by field. Set what each field does between the two records, or start from a case. The m and u values are illustrative; a real index estimates them from its own data.

  • Health number

    Provincial. Rarely shared by chance, sometimes mistyped

    m 0.95 · u 0.0001 · agree +13.21 · disagree −4.32

    +13.21
  • Last name

    Changes on marriage; common names coincide

    m 0.92 · u 0.01 · agree +6.52 · disagree −3.63

    +6.52
  • First name

    Nicknames, spellings, initials

    m 0.88 · u 0.02 · agree +5.46 · disagree −3.03

    −3.03
  • Date of birth

    Reliable, and shared by twins

    m 0.97 · u 0.0005 · agree +10.92 · disagree −5.06

    +10.92
  • Sex

    Agrees half the time by chance alone

    m 0.98 · u 0.5 · agree +0.97 · disagree −4.64

    +0.97
  • Postal code

    People move

    m 0.8 · u 0.005 · agree +7.32 · disagree −2.31

    +7.32

The weights, added up

Each bar starts where the one above it ended. Agreement pushes right, disagreement pushes left, a missing field moves nothing. The dashed lines are the two thresholds below.

Total weight
+22.63
prior −13.29 plus six fields
Match probability
>99.99%
2^W ÷ (1 + 2^W)
Decision
Link
Above the upper threshold

Total weight +22.63, match probability >99.99%, decision Link.

Set by how candidate pairs were chosen

+4.50

probability 95.8%

−4.50

probability 4.2%

Why somebody owns the thresholds

Try the twins. Same surname, same birth date, same address, and the model links them, because it assumes each field disagrees independently and twins are exactly the case where they do not. Raise the upper threshold far enough to stop it and look at what else falls into the review band, which somebody has to staff. Neither number comes with the software. Both are decisions, and both change every count downstream.

Illustrative m and u values, not any real index’s. Try the twins.

Two things in that arithmetic are not technical. The thresholds are decisions, and every count downstream depends on them. And a match model’s accuracy depends on the population it runs on. Canada has provincial health numbers, which change the problem considerably against a United States baseline with no national patient identifier, so accuracy figures from a US deployment say little about a Canadian one.

Terminology

A terminology service answers what a code means, whether it is valid in the release in use, what it maps to, and whether one concept falls under another. Underneath it are code systems that each release on their own clock.

A year of scheduled terminology releasesA twelve-month calendar with one row per code system. SNOMED CT International is released monthly, on the first of each month: twelve releases. The SNOMED CT Canadian Edition, maintained by Canada Health Infoway, is released quarterly, in February, May, August and November: four. LOINC is released in February and August, and its steward asks users to adopt each version within 90 days: two, each followed by a three-month adoption window. RxNorm has a full release on the first Monday of each month with weekly updates between: twelve plus the weeklies. ICD-10-CM is released on 1 October with an update on 1 April: two. ICD-10-CA has no releases, because CIHI has frozen it at its 2022 version while preparing for ICD-11. Altogether, thirty-two scheduled releases a year before RxNorm’s weekly updates.CODE SYSTEMA YEARJFMAMJJASONDSNOMED CT INTERNATIONALMonthly, on the 1st12SNOMED CT CANADIAN EDITIONFeb, May, Aug and Nov · Infoway4LOINCFeb and Aug · adopt within 90 days2RXNORMFirst Monday monthly · weekly updates12+ICD-10-CM1 October, with a 1 April update2ICD-10-CAFrozen at its 2022 version · CIHINO RELEASES · PREPARING FOR ICD-110Scheduled releaseWeekly updateAdoption window the steward asks for32 SCHEDULED RELEASES A YEAR, BEFORE RXNORM’S WEEKLY UPDATES
Cadences from SNOMED International, Canada Health Infoway, Regenstrief, the US National Library of Medicine, CDC and CIHI.

A platform that maps to five common code systems absorbs around thirty scheduled releases a year, before RxNorm’s weekly updates, and any of them can retire a code a mapping depends on. That is why a mapping needs a version and an owner, and why a published number should be able to say which terminology release it was computed under. Canada adds two particulars: its own SNOMED CT edition from Infoway, released in March and September, and a diagnosis classification that is standing still, because CIHI has frozen ICD-10-CA at its 2022 version while it prepares for ICD-11.

Four questions that decide the shape of a platform

No reference architecture survives contact with a particular organisation unchanged. Four questions decide which parts of a platform carry the weight: how many sources there are and who controls them, whether anything needs an answer in under a second, who reads what comes out, and where the data is allowed to live.

Four questions that decide the shape

Answer for a platform you know. The eight parts below reweight as you go, each with the reason.

01How many sources, and who controls them?
02Does anything need an answer in under a second?
03Who reads what comes out?
04Where is the data allowed to live?

Heaviest for these answers · Lineage and audit trail · Quality monitoring

  • Canonical model

    Standard

    Models need features computed as of a moment, so the model has to keep history that can be replayed.

  • Data contracts

    Standard

    Several internal systems, each upgrading on its own calendar. Write down what each one promised.

  • Lineage and audit trail

    Heavy

    Which rows trained which version of the model is the first question after anything goes wrong.

  • Access control

    Standard

    Re-assert the source system’s rules here. The platform is a new place the data lives.

  • Quality monitoring

    Heavy

    A model learns whatever the data gets wrong, and then repeats it at scale.

  • Event and bulk delivery

    Standard

    Events for the feeds that matter within minutes, bulk for everything else.

  • Patient identity

    Standard

    Your own systems disagree about who is who more often than anyone expects. One index across them.

  • Terminology

    Standard

    Your own local codes mapped once, then kept current as the code systems release.

A Canadian region is residency, not sovereignty

Under 18 U.S.C. §2713 a US-based provider can be compelled to produce data it holds anywhere, including in its Canadian regions. PHIPA itself has no in-Canada storage rule; it holds the custodian to reasonable safeguards, and storage outside Canada becomes a privacy impact assessment question. British Columbia dropped its blanket in-Canada rule for public bodies in 2021 in favour of an assessment. Check backups, logs, support access and AI services region by region. A pointer to the questions, not legal advice.

A weighting of the parts, not a product recommendation.

The residency question deserves one plain statement, because it is so often answered wrong. Putting data in a Canadian region of a US-based cloud provider is residency, not sovereignty: the US CLOUD Act lets US authorities compel a provider to produce data it holds anywhere. PHIPA does not itself require data to stay in Canada. It holds the custodian responsible for reasonable safeguards, which makes storage outside Canada a matter for a privacy impact assessment rather than a prohibition. Our piece on on-premises and hybrid clinical AI covers the case where the data cannot leave the building at all.

Where the standards stop

The interoperability series on this site is about the standards that govern how data arrives: HL7 v2, DICOM, FHIR, the IHE profiles. At the edges of the platform stage that coverage is good. Bulk export and subscriptions have specifications, and US certification requires the first.

Inside the stage it is uneven. Measure logic is standardised: CMS writes its electronic clinical quality measures in CQL, a normative HL7 standard. Analytic views have SQL on FHIR. The research model has OMOP, a community standard with real reach. Data contracts have a general-purpose specification from outside healthcare.

What governs each part of the data platform stageThree groups. At the edges, how data arrives and leaves: messages and documents in are governed by normative standards, HL7 version 2, DICOM, FHIR R4 core resources and IHE profiles; bulk extract by FHIR Bulk Data version 3.0.0, trial use, which certified US EHRs must support; events out by the Subscriptions R5 Backport guide, trial use. Inside, the model and the logic: measure logic by CQL 1.5.3, normative, which is how CMS writes its eCQMs; analytic views by SQL on FHIR 2.0.0, a published implementation guide, with 3.0.0 in ballot targeting R6; the research model by the OMOP CDM version 5.5, a community standard maintained by OHDSI; data contracts by the Open Data Contract Standard, a general-purpose specification that is not healthcare-specific. Governed by nothing but the organisation’s own decisions: identity thresholds, when two records become one person; quality thresholds, how complete is complete enough; mapping versions, which terminology release a number was computed under; and lineage granularity, which FHIR Provenance can carry but no standard sets.PART OF THE STAGEWHAT GOVERNS ITAT THE EDGES · HOW DATA ARRIVES AND LEAVESMessages and documents inHL7 v2, DICOM, FHIR R4 core resources, IHE profilesNORMATIVEBulk extractFHIR Bulk Data v3.0.0. Certified US EHRs must support itTRIAL USEEvents outSubscriptions R5 Backport v1.1.0, for R4 serversTRIAL USEINSIDE · THE MODEL AND THE LOGICMeasure logicCQL 1.5.3. How CMS eCQMs are writtenNORMATIVEAnalytic viewsSQL on FHIR 2.0.0. Version 3.0.0 in ballot, targeting R6PUBLISHED IGResearch modelOMOP CDM v5.5, maintained by OHDSICOMMUNITYData contractsOpen Data Contract Standard v3.1.0. Not healthcare-specificGENERAL-PURPOSEGOVERNED BY YOUR OWN DECISIONSIdentity thresholdsWhen two records become one personNONEQuality thresholdsHow complete is complete enough for this useNONEMapping versionsWhich terminology release a number was computed underNONELineage granularityFHIR Provenance can carry it. Nothing says how fineNONE
Statuses as at October 2026.

Then there are four things nothing governs: when two records become one person, how complete is complete enough, which terminology release a number was computed under, and how fine-grained lineage has to be. These are the decisions that move a number most, and each organisation makes them on its own. That is why two platforms built on identical standards and fed identical FHIR can report different readmission rates, and why any honest comparison of platforms has to start from the parts rather than from the product names.

If you are choosing, building or rebuilding one of these, tell us what it is and we will tell you what we think.

Working on one of these problems?

Tell us what you’re working on in interoperability, data platforms, or clinical AI, and we’ll tell you what we think.

Get in touch