On-Premises & Hybrid Clinical AI Framework

The inference stays. Everything else still travels

More clinical AI now runs where the data is made: inside the imaging device, on a server beside it, or on a platform the hospital operates itself. The reasons are good ones and most of them are physical. A decision taken during acquisition or inside a procedure cannot wait on a network round trip. A whole-slide image or a tomosynthesis study costs more to move than a model does. A tool that triages intracranial haemorrhage has to keep working when the uplink is down. And a great many hospital policies forbid sending pixel data anywhere at all, whatever the law permits.

What none of that removes is the data movement. Model updates still have to arrive. Performance evidence, audit records and the cases needed to monitor and retrain the model still have to leave. So the design question in a hybrid deployment is not whether data crosses the site boundary. It is what crosses, in which direction, in what form, and on whose authority.

That question is answerable, and answering it is most of the work. This piece walks the five places the inference can run, inventories everything that crosses the boundary in both directions, maps the standards onto each path, and sets out the obligations that follow the model onto the hospital network. It assumes the surrounding ground covered in the six handoffs between a scanner and a chart and what makes a function a regulated device.

Why the inference moves on site, and what that costs

Four arguments get made for local inference, and they are not equally strong. Latency is the one most often cited and the hardest to pin down: there is no generic figure for the round trip a given model needs, because it depends on the study size, the link, the model and the hardware. The argument holds where a device acts on the output inside the acquisition or the procedure, and it is weaker than it sounds for a radiologist reading a worklist an hour later.

Availability is the more defensible driver, and it is usually undersold. A hospital either has a downtime procedure that assumes the tool is there, or it does not. Once a department routes triage through a model, a severed fibre is a clinical event rather than an IT ticket. Data volume is similarly concrete: moving multi-gigabyte studies for every case has a recurring cost that a one-time appliance does not, and the arithmetic is specific to a site rather than general.

Residency and policy are the fourth, and in practice the decisive one. Hospital security policy routinely exceeds what the law requires, and a policy forbidding unredacted images from leaving the building ends the architecture discussion regardless of what the privacy analysis would have permitted.

The other side deserves stating plainly, because a paper that only argues one way is not much use. Central inference keeps one version in service, which means one thing to validate, one thing to monitor and one thing to roll back. It needs no GPU in anybody’s rack, no cooling, and no local change advisory board. A fleet of on-premises appliances trades those away for latency, availability and residency, and the trade is worth making often but not always.

Five places the inference can run

The five below are not a maturity ladder, and the last is on the list because the contrast is useful rather than because it is wrong. They differ most in one thing: where the regulated device boundary falls, and therefore who answers for the result.

The hospital AI platform row is where the boundary gets genuinely contested. The vendor certifies the model; the hospital owns the orchestration, the resource allocation and the version selection, and those decisions change what the model sees. IEC 80001-1 is at Edition 2.0 and allocates risk management across the manufacturer, the healthcare delivery organisation and the IT provider, rather than assigning it to the hospital as the 2010 first edition did.

What crosses the boundary

This is the inventory that should exist as a design document and usually does not. Eleven paths, both directions, each with an identifiability, a frequency, an authority and a standard that carries it.

Two rows in that list deserve more than a cell. The first is the legal basis for sending performance evidence out. Evaluating a model against reality means comparing its output to the signed report, the pathology or the follow-up, and that ground truth lives in the record rather than in the modality. In the United States this is health care operations: quality assessment and improvement including outcomes evaluation, done by a business associate under a contract, with the minimum necessary standard applying because it applies to operations even though it does not apply to treatment. Patient authorisation is not the mechanism, and asking for it would imply the custodian needed it.

The second is the proviso attached to that same definition, which is the sharpest thing in this subject. Quality assessment is an operation provided obtaining generalizable knowledge is not the primary purpose of any study that results. Consider what that means for a hybrid deployment. One outbound packet of disagreement cases can be monitoring whether this hospital’s model still works, which is squarely operations. The same packet can be improving the vendor’s model for every hospital, which is arguably the pursuit of generalizable knowledge. Nothing in the packet distinguishes the two. The difference is in what happens downstream, and it is almost never written into the contract that permits the transfer.

The third thing worth naming is the row most inventories omit: complaint, incident and adverse-event information. It is the one outbound path that is obligatory rather than contractual, which makes it the one path a hospital cannot switch off and a vendor cannot decline to receive.

The standards on each path, and the asymmetry between them

Read the inventory’s last column downwards and something shows up that is easy to miss when the paths are discussed one at a time. The evidence coming out is carried by standards. The updates going in are carried by whatever the vendor chose.

On the way in there is no healthcare standard for distributing a model. Weights, configuration and site calibration arrive as container images, signed archives and vendor-specific formats. This is worth stating plainly rather than implying a standard exists, because a hospital that assumes model distribution is standardised will not ask the questions that make it auditable.

On the way out, DICOM carries the results. Studies reach the model by C-STORE from older modalities or by DICOMweb for anything recent. Findings come back as Structured Reporting, and the thing to avoid is well established: text burned into the pixels makes the finding opaque to everything downstream and forces the next system into optical character recognition. IHE AI Results leans on the TID 1500 family rather than TID 1500 alone, with TID 1501 appearing more often than 1500 and TID 1410 and 1411 for planar and volumetric measurements, alongside Segmentation, Parametric Map and Presentation State objects.

AIR is at Revision 1.3 and introduces two transactions of its own: Display Analysis Result [RAD-136] and Query Analysis Results [RAD-137], between an Evidence Creator, an Image Manager and an Image Display. It is worth knowing that it is Trial Implementation, as is every imaging AI profile. Its companion AI Workflow for Imaging has been at Revision 1.1, Trial Implementation, since August 2020.

The profile for the path this paper is about

The most relevant profile is also the newest and the least discussed. AI Result Assessment for Imaging, Revision 1.1, Trial Implementation, 12 June 2025, covers the storage and communication of assessments of AI results. Its own stated purposes include post-market surveillance and the detection of data drift, with metrics such as rejected findings over time and positive predictive value, and it covers machine assessment as well as human. That is precisely the outbound evidence path.

And then it stops, in a sentence worth reading carefully: the assessment process itself is not defined by this profile. The supplement goes on to note the consequence, which is that without additional established process the use of assessment results for retraining models is limited.

That is the honest shape of this subject. The transport for the disagreement-case path is standardised. The authority to act on what it carries is not, and no profile is going to supply it. Deciding who reviews a flagged case, on what criteria, and what happens when the model turns out to be right more often than the reader is governance that belongs to the deployment.

The broader clinical context still moves on HL7. Version 2 ORU and order messages carry reports and orders between the hospital information system and the imaging systems, and FHIR is what queries the record for ground truth: DiagnosticReport, Observation, ImagingStudy, Device and DeviceMetric, with Bulk Data export for anything at dataset scale. For devices at the bedside rather than in imaging, IEEE 11073 Service-oriented Device Connectivity and the IHE device profiles carry the high-frequency data instead.

Accountability runs on IHE ATNA and on FHIR AuditEvent. One caution: AuditEvent was substantially restructured between R4B and R5, so a requirement naming its elements has to name its version too. And AuditEvent is not Provenance. Provenance records the lineage of a resource, which is who asserted this value and from what. AuditEvent records system events, including the failed access attempt and the policy change. A deployment generally needs both, for different questions.

{
  "ConceptNameCodeSequence": {
    "CodeValue": "126000",
    "CodingSchemeDesignator": "DCM",
    "CodeMeaning": "Imaging Measurement Report"
  },
  "Finding": {
    "CodeValue": "429891001",
    "CodingSchemeDesignator": "SCT",
    "CodeMeaning": "Lesion"
  },
  "Measurement": {
    "Value": 24.5,
    "CodeValue": "mm",
    "CodingSchemeDesignator": "UCUM"
  }
}

The point of that shape is the last three lines of each block. A number with a coding scheme behind it can be parsed into a warehouse and compared across sites; the same number burned into a pixel cannot. An audit record for the same inference names the model version, which is the question a review asks first.

{
  "resourceType": "AuditEvent",
  "category": [{ "coding": [{ "code": "110100", "display": "Application Activity" }] }],
  "action": "E",
  "recorded": "2026-10-10T14:22:05Z",
  "outcome": { "code": "0" },
  "agent": [{
    "who": { "display": "Inference engine v2.4.1" },
    "requestor": false
  }],
  "entity": [{ "what": { "reference": "ImagingStudy/CT-HEAD-89421" } }]
}

De-identification at the boundary

When cases leave for central analysis they have to be stripped, and the mechanism is the DICOM PS3.15 Annex E confidentiality profile. Six action codes apply to every attribute, and the nested-sequence behaviour that trips up most implementations is not a separate trap: it is written into the definitions. X removes a sequence along with all its items. K keeps a value unchanged for a non-sequence attribute and still cleans sequences. C reapplies the rules recursively inside each item. A flat loop over top-level tags does not merely miss something subtle; it fails to do what the codes say.

Two options matter here more than their profile placement suggests. Clean Structured Content is the one for an AI result, because an AI result is structured content, and a pipeline that de-identifies the image headers and ships the structured report untouched has exported the findings with whatever identifiers the report carried. And the option for recognisable faces reconstructable from head and neck volumes does exist, contrary to a common claim that header de-identification cannot reach the problem. PS3.15 defines a Clean Recognizable Visual Features Option; it sits outside the attribute table because it operates on pixel data. What the standard withholds is the method, not the requirement, which means the defacing implementation is yours to choose and yours to evidence.

Utility survives this if the options are chosen deliberately. Retain Longitudinal Modified Dates shifts dates consistently so disease progression stays analysable without exposing service dates, and Retain UIDs keeps a patient’s studies linkable across time. De-identification and pseudonymisation are not the same thing: where a crosswalk is kept, the data stays subject to privacy law for whoever holds the key, which is a fact about the key-holder rather than about the packet.

Federated learning, at the strength the evidence supports

Training across sites without moving cases is the obvious answer to all of this, and it is a real one. Local nodes train on local data and only the weight updates go to an aggregator. What it is not is a proof of privacy. Gradient inversion attacks can recover training data from updates, and the honest summary of the literature is narrower than the headline: fidelity falls as batch size and image resolution rise, and the strongest published reconstructions assume small batches, lower resolution, prior knowledge, or a server that modifies the model.

That last condition is the one that decides a hospital’s exposure and the one most often left out. An honest-but-curious aggregator is a different threat from an aggregator that can change the architecture to make a single sample recoverable. Which of those the agreement permits, and what would detect the second, is the question to ask. Differential privacy, secure multi-party computation and homomorphic encryption all mitigate, and all cost something in accuracy, latency or both.

The regulation follows the model onto the network

Deployment location changes how a manufacturer meets its obligations. It does not change whether they apply. A model on a hospital appliance drags a lifecycle of cybersecurity and change-control duties onto that network behind it.

In the United States, section 524B of the Food, Drug, and Cosmetic Act defines a cyber device as one containing software, connecting to a network and susceptible to cybersecurity threats, which describes every deployment in this piece. It requires cybersecurity information in the premarket submission, including a software bill of materials. FDA’s current cybersecurity guidance is the February 2026 final, which supersedes the June 2025 version. For an appliance in the field the hard word in the SBOM obligation is "current": what is installed at a site drifts from what shipped.

Change control is the mechanism that makes a hybrid deployment legal rather than merely possible. Without it, retraining a model on local data is a modification requiring a new submission. With an authorised predetermined change control plan, modifications inside its bounds proceed without one. The authority is section 515C, added by FDORA in 2022, and the final guidance was issued in December 2024 and reissued with updates in August 2025, widening its scope from machine-learning-enabled to all AI-enabled device software functions. A plan describes three things: the modifications, the methodology for developing and validating and implementing them, and an impact assessment.

One nearby document is worth naming precisely because of its status. FDA’s guidance on lifecycle management and marketing submissions for AI-enabled device software functions is still draft, dated 7 January 2025, and sits on the agency’s lower-priority list. Anyone planning a submission needs to know which documents they cannot yet rely on.

A regulator has written the per-site case down

The most useful sentence available on this subject is not American. Health Canada’s pre-market guidance for machine learning-enabled medical devices, finalised on 1 April 2026, allows a change control plan to specify changes that apply non-uniformly based on the unique characteristics of a clinical site or patient, and expects labelling to cover local validation and calibration.

That is a regulator contemplating, in writing, a model that behaves differently at one hospital than another by design rather than by accident. It is the clearest official acknowledgement that per-site behaviour is a thing to be specified and evidenced instead of a deviation to be eliminated. The same guidance names the risks it expects to see analysed, and the list reads like a description of on-premises practice: automation bias, alarm fatigue, algorithmic bias across identity characteristics, and performance degradation caused by changes in clinical practice or in input format and quality.

For the European Union the dates are what matter here, and they moved. Regulation 2026/1744 amended the AI Act’s application dates: the high-risk duties reach standalone Annex III systems on 2 December 2027 and product-embedded Annex I systems, which is where a medical device meeting Article 6(1) sits, on 2 August 2028. The transparency duties in Article 50 were not postponed.

The standards each side owns

On the hospital side, IEC 80001-1 Edition 2.0 governs risk management where medical devices sit on an IT network, and allocates it across three parties rather than one. On the manufacturer side, IEC 81001-5-1 sets security activities across the health software lifecycle, ANSI/AAMI SW96:2023 is the recognised method for premarket security risk management, and AAMI TIR97:2019 covers the postmarket side, which is where on-premises patching actually lives.

One artefact is worth singling out because hospitals routinely do not get it. The Manufacturer Disclosure Statement for Medical Device Security, ANSI/NEMA HN 1-2019, documents twenty-three groups of security controls. It is voluntary, which in practice means it exists for a given device only if the buyer asks for it. A hospital that never asks never finds out it could have.

Who the vendor is to the hospital

The contractual question underneath all of this is what role the vendor occupies, because the role decides what the outbound data is allowed to be.

In Ontario the structure is worth getting right because it is commonly described wrongly, as three roles of descending weight. It is not. An electronic service provider is defined under PHIPA’s regulation so as to exclude agents, so the two are mutually exclusive rather than adjacent. A health information network provider is defined by serving two or more custodians primarily to let them disclose to one another, whether or not it is also an agent, which means a single-site appliance vendor almost never is one. The practically useful consequence: a custodian using a non-agent electronic service provider is not thereby disclosing personal health information to it at all, and that is the provision that makes the on-premises appliance model clean rather than merely defensible.

Running a fleet of them

A model at one site is a deployment. The same model at thirty sites, at four versions, against eleven scanner configurations, is an operation, and it is where most of the real difficulty lives.

Updates go out through version pinning and staged rollout, and the mechanism that makes this safe is shadow mode: the new model processes local studies and logs its inferences without showing them to anyone, so the site can confirm it behaves on this population and this equipment before it reaches a reader. It is also the only step in the loop a hospital controls unilaterally, which makes it the right thing to insist on.

The drift that matters most is caused by the hospital rather than by its patients. Replacing a scanner, changing a contrast protocol or updating a reconstruction algorithm alters the statistical distribution of what the model sees, and the failure mode is silent: accuracy degrades without anything erroring. Health Canada names changes in input format and quality as a cause of performance degradation, which is a citation for what every deployment team already suspects. The practical consequence is a standing procedure: when the estate changes, the vendor is told.

Two network facts shape the rest. Connections are outbound-only in any hospital worth deploying in, because an inbound firewall rule letting a vendor dial in gets refused by security review, and discovering that late costs a deployment window. And when the link drops, the local application has to keep delivering inferences while buffering audit records and telemetry, then catch up. What happens when the buffer fills is the part that is usually undefined.

Contracts and data rights

The clauses that decide how a hybrid deployment actually behaves are not the technical ones, and they are the ones most often left vague.

• Who owns the telemetry, and what "derived data" includes. If derived data covers models improved using this hospital’s cases, that clause is where the generalizable-knowledge boundary gets crossed, silently, years before anyone looks.
• What outbound data may be used for. Monitoring this site’s model and improving the vendor’s product are different purposes with different legal footing. A contract that permits "performance analysis" has settled nothing.
• Audit rights. Can the hospital see what left the building? Most agreements do not say, and a privacy impact assessment that cannot answer it is incomplete.
• Incident notification, in both directions. The hospital needs to hear about a vulnerability in the appliance; the vendor needs to hear about a scanner replacement.
• Exit and data return. What happens to the weights, the site calibration and the accumulated audit log when the contract ends. The appliance is on the hospital’s floor, which makes this more concrete and more negotiable than a cloud exit.

The loop is the point

Put the pieces together and the architecture is not a pipeline with a model at the end of it. It is a circuit that crosses the site boundary exactly twice.

Studies are acquired and inference runs locally, which is the whole reason for the architecture. The result reaches the report and a reader agrees with it or does not. That assessment, with drift statistics and audit records beside it, crosses the boundary outbound. Somewhere else it becomes monitoring, then retraining, then a signed model package, which crosses back inbound within the bounds of an authorised change control plan, waits in shadow mode, and goes into service to be measured again.

Every crossing in that circuit has an identifiability, an authority and a standard, and the standards are uneven: normative on the results side, trial implementation for the AI profiles, and absent for model distribution. The work of deploying clinical AI on hospital infrastructure is mostly the work of being explicit about those crossings, because nothing in the technology forces anyone to be.

Which is the reason the thesis holds in the form it started: the inference stays on site, and the evidence, the updates and the accountability still travel. Moving the computation closer to the patient does not reduce the governance. It relocates it, from a vendor’s cloud to a boundary that runs through the middle of somebody’s hospital.

The lists above are the questions worth settling before go-live rather than after. If you are working through them on a real deployment and want another read on the answers, write to us.