EHR and interoperability testing checks that clinical data leaves one system and arrives in another complete, correctly coded and attached to the right patient. For HL7 v2 that means validating pipe-delimited messages, acknowledgements and field mappings across each interface. For FHIR it means checking resources against profiles, search behaviour and SMART on FHIR authorization, usually with conformance tools such as Inferno. Above both sits workflow testing inside the EHR itself, such as an order placed in Epic that must return a result to the right chart. Negative cases matter as much as the happy path.
HL7 vs FHIR: What Changes for the Tester
Both standards come from HL7 International, but a test plan for one does not transfer to the other. HL7 describes Version 2 as the most widely implemented healthcare messaging standard (HL7 product brief). The first V2 release dates from October 1987. FHIR is a resource-based standard with a web API and JSON or XML formats (FHIR overview). The current FHIR release is R5. US certification still names FHIR Release 4.0.1 for its standardized API criterion (ASTP/ONC test method for 170.315(g)(10)).
| Aspect | HL7 v2 | FHIR |
|---|---|---|
| Unit of exchange | Message made of segments (MSH, PID, OBR, OBX) | Resource (Patient, Observation, Encounter) |
| Encoding | Pipe and caret delimited text | JSON or XML |
| Transport | Usually MLLP over TCP through an interface engine | RESTful HTTP API, plus messaging and documents |
| Typical trigger | Event, for example ADT^A01 admit or ORU^R01 result | Client request (read, search, create) or subscription |
| Rules live in | Site interface specifications and local Z-segments | Base specification, implementation guides and profiles |
| Security model | Network controls around the interface engine | OAuth 2.0 with SMART App Launch scopes |
| Main tooling | Interface engine test harnesses, message validators | FHIR validator, Inferno test kits, API clients |
HL7 v2 testing is mostly about the agreement between two specific systems, because each site bends the standard. FHIR testing starts from a shared, machine-readable definition, so much of it can be automated against published profiles. Most hospitals run both side by side.
Healthcare Interoperability Testing in Five Levels
A plan that covers only one of these levels tends to pass validation and then fail in the clinic.
| Level | Question it answers | Typical evidence |
|---|---|---|
| 1. Message and resource validation | Is each message or resource well formed and does it follow the specification? | Validator reports, schema and profile checks |
| 2. Conformance | Does the system meet a named implementation guide or certification criterion? | Inferno or other test-kit results, conformance statements |
| 3. Integration | Do two real systems exchange data correctly, with the agreed mappings and code sets? | Interface test logs, mapping sign-off, ACK and error handling |
| 4. End-to-end workflow | Does a clinical task complete across systems, as a clinician performs it? | Scripted workflow runs in a test EHR, screenshots, audit trail |
| 5. Negative and edge cases | What happens with bad, late, duplicate or partial data? | Failure-path test results, reconciliation reports |
Levels 1 and 2 are cheap to automate in the build pipeline. Levels 3 and 4 need the partner system or a sandbox and usually set the schedule. Level 5 holds most patient-safety risk and deserves its own test design.
HL7 Interface Testing: Messages, Acknowledgements and Mappings
An HL7 v2 interface test starts from the interface specification signed by both sides, not from the standard alone. For each message type in scope, testers prepare sample messages that cover every event trigger, then check five things:
- Structure: required segments present, segment order correct, delimiters declared in MSH used consistently.
- Field mapping: each field lands in the right place in the receiving system, including repeating fields such as multiple identifiers in PID-3.
- Code sets: local codes translate to the receiver's values; LOINC, SNOMED CT or local lab codes map without silent loss.
- Acknowledgements: the receiver returns an ACK with the expected code (AA, AE or AR), and the sender reacts correctly to an error or reject.
- Sequencing: an admit, transfer and discharge processed out of order does not leave the patient in the wrong location or the wrong encounter.
ADT feeds deserve the most attention because every other interface depends on patient identity. Merge and link events (for example ADT^A40) are a common source of misfiled results. For results interfaces (ORU), testers also check units, reference ranges, abnormal flags and how corrected or cancelled results replace the earlier version on the chart.
FHIR Testing: Profiles, Searches and SMART Authorization
FHIR validation, as the specification defines it, covers structure, cardinality, value domains, terminology bindings, invariants, profile rules and business rules (FHIR validation). A FHIR test plan usually adds three layers on top:
- Profile conformance. Each resource the server returns is checked against the profiles it claims, for example US Core in the United States, including Must Support elements.
- Search and paging. Required search parameters, combinations,
_include, sorting and paging links behave as the CapabilityStatement says they do. - Authorization. SMART App Launch flows (standalone and EHR launch), scope enforcement, token expiry and refresh, and that a patient-scoped token cannot read another patient's data.
The US standardized API criterion names FHIR R4.0.1, the US Core implementation guide, SMART App Launch and Bulk Data Access as required standards (170.315(g)(10) test method). Apps that consume those APIs should test against the same versions, including bulk-export performance.
Inferno FHIR Testing and Other Conformance Tools
Inferno is an open-source tool for creating, running and sharing automated conformance tests for FHIR, hosted by ASTP/ONC on HealthIT.gov (Inferno). Its test kits can run on a local machine or behind a firewall, which matters when test data must stay inside the network. The ONC Certification (g)(10) Standardized API Test Kit is an approved test method for the 170.315(g)(10) criterion (test kit page). That kit covers SMART App Launch, US Core and Bulk Data.
| Tool type | What it does | Where it fits |
|---|---|---|
| Inferno test kits | Automated conformance runs against named FHIR implementation guides | Level 2, before certification or partner onboarding |
| FHIR validator (HL7) | Validates individual resources against base spec and profiles | Level 1, inside CI pipelines |
| Interface engine harness | Replays HL7 v2 messages, captures ACKs and transformations | Levels 1 and 3 for v2 feeds |
| EHR vendor sandboxes | Test endpoints and synthetic patients from the EHR vendor | Levels 3 and 4 |
Passing a conformance kit shows that a server follows the guide, not that a specific hospital's data, mappings and workflows work. It is an entry ticket to integration testing, not a replacement.
EHR Integration Testing and Epic EMR Workflows
EHR integration testing combines interface checks with the screens and work queues clinicians use. For Epic, the public developer site open.epic.com publishes API and interface specifications and a developer guide. Project-specific build, test environments and interface details come from the customer organisation's Epic team, so a test plan should name who owns each environment and refresh.
Workflow scripts follow a clinical task from start to finish:
- Registration to chart: a patient registered in Epic appears in the connected app with the same identifiers, and a later demographic update follows.
- Order to result: an order placed in the EHR reaches the lab or imaging system, and the result files to the right encounter with the right ordering provider.
- Scheduling: appointments booked, moved or cancelled through an integration show the same status in both systems.
- Embedded apps: a SMART on FHIR app launched from inside the EHR receives the right patient and user context.
Regression matters most here. EHR upgrades and local build changes can break an interface that passed last quarter, so a smoke suite per interface should run after every upgrade window. Access control and audit-log checks for these releases are covered in the HIPAA and FDA compliance testing guide.
Negative and Edge-Case Tests for Clinical Interfaces
Most interface incidents come from data the happy path never sends. A negative set should include:
| Case | Example | Expected behaviour |
|---|---|---|
| Missing required data | ORU without patient identifier | Rejected with an error ACK, routed to an error queue, nobody's chart updated |
| Patient mismatch | Two patients with the same name and date of birth | No automatic match; manual reconciliation work item |
| Duplicates | Same message resent after a timeout | Processed once; no duplicate result or charge |
| Out-of-order events | Discharge arrives before admit | Held or reconciled, not dropped |
| Unknown codes | Local lab code with no mapping | Visible failure, not a blank field |
| Authorization | Expired token, wrong scope, revoked consent | 401 or 403, no data returned, event logged |
| Downtime | Receiver offline for hours | Queue holds messages; backlog replays in order |
Run every case with synthetic patients; real PHI brings HIPAA obligations into the test lab.
Which Ranked Vendors Publish HL7 and FHIR Testing Evidence
Among the 12 companies in the ranking, public interoperability evidence ranges from named message types to a single mention of HL7 on a service page. The comparison table shows the wider scores, and the scenario picks name best and alternative choices for HL7/FHIR and Epic work.
- ScienceSoft describes HL7 CCD and ADT interface validation with Postman and custom tools in a care-management case (case). Another of its cases covers integration testing of HIE software with multiple EHRs (case).
- Citrusbug Technolabs lists ADT, ORU and ORM message testing for HL7 v2.x, FHIR R4 and SMART on FHIR flows, C-CDA document checks and HIPAA 5010 EDI transactions (service page).
- a1qa publishes an EHR case with functional, compatibility, cybersecurity and integration testing toward HIPAA and ONC certification (case).
- ImpactQA describes EPIC software testing of patient data, provider workflows, appointment scheduling and billing (service page).
- DeviQA lists HL7 v2 and FHIR R4 among the standards it tests (healthcare page).
- BetterQA names HL7/FHIR integration in its CardiaSync project (projects).
QualityLogic and XBOSoft mention interoperability testing without naming HL7 or FHIR. Mindfire Solutions' Athena and Cerner integration cases describe development work, not testing.
What to Ask an EHR Testing Services Company
These questions ask for artefacts rather than claims:
- Which HL7 v2 message types and FHIR implementation guides have you tested, and can you show a redacted test case for each?
- How do you build synthetic patients and messages, and how do you keep PHI out of interface test environments?
- Which conformance tools do you run (Inferno, the HL7 FHIR validator, interface engine harnesses), and at which pipeline stage?
- Have you tested inside an Epic, Oracle Health or athenahealth environment, and who provided access?
- What does your negative-case catalogue for ADT and results interfaces contain?
Score the answers against the same evidence rules used in the ranking methodology. For a full selection process, RFP outline and pilot design, see the guide on how to choose a healthcare software testing company.
HL7, FHIR and EHR Testing FAQ
What is the difference between HL7 and FHIR?
HL7 Version 2 is an event-driven messaging standard: systems send pipe-delimited messages such as an admit or a lab result, usually through an interface engine. FHIR, also from HL7 International, exposes clinical data as resources through a web API in JSON or XML, secured with OAuth 2.0 and SMART scopes. For testers, v2 work centres on site-specific mappings and acknowledgements, while FHIR work centres on profile conformance, searches and authorization.
What does HL7 interface testing check?
HL7 interface testing checks that each message type in the interface specification is built correctly and processed correctly by the receiving system. Testers verify segment structure, field mappings, code translations, acknowledgement codes and the handling of events that arrive out of order or twice. ADT feeds get the most attention, because a wrong patient match or a failed merge can attach results and orders to the wrong chart.
How do you test a FHIR API?
FHIR API testing starts with validating returned resources against the base specification and the profiles the server claims, such as US Core. It then checks search parameters, paging and the CapabilityStatement, followed by SMART App Launch flows, scope enforcement and token handling. Teams add performance checks for bulk export and broad searches, plus negative tests that confirm a token for one patient cannot read another patient's records.
What is Inferno used for in FHIR testing?
Inferno is an open-source FHIR conformance testing tool hosted by ASTP/ONC on HealthIT.gov. Teams use its test kits to run automated checks against named implementation guides, and its ONC Certification (g)(10) kit is an approved test method for the US standardized API criterion. Kits can run locally or behind a firewall. A passing run shows conformance to the guide; integration and workflow testing with real partner systems are still needed.
What is EHR integration testing?
EHR integration testing confirms that an external system and an EHR exchange data correctly and that clinical tasks complete across both. It covers interface messages or API calls, identifier and code mappings, error queues, and end-to-end workflows such as registration, orders, results and scheduling. It is repeated after EHR upgrades and local build changes, because an interface that passed earlier can break when either side changes.
How is Epic EMR testing different from general EHR testing?
Epic EMR testing follows the same interface and workflow principles, but environments, build and interface details come from the customer organisation's Epic team, so access and refresh schedules shape the plan. Epic publishes API and interface specifications on open.epic.com. Test scripts usually follow registration, orders and results, scheduling and SMART on FHIR apps launched inside Epic, with a regression smoke suite after every upgrade window.
What are the levels of healthcare interoperability testing?
A practical model uses five levels. Message and resource validation checks structure against the standard. Conformance testing checks a named implementation guide or certification criterion. Integration testing checks exchange between two real systems with agreed mappings. End-to-end workflow testing follows a clinical task across systems. Negative and edge-case testing covers bad, late, duplicate and partial data, where most patient-safety risk sits.
Which testing companies publish HL7 and FHIR integration testing evidence?
Among the twelve ranked vendors, Citrusbug Technolabs lists HL7 v2 ADT, ORU and ORM message testing, FHIR R4 and SMART on FHIR flows, and C-CDA checks on its testing page. a1qa publishes an EHR case with integration testing toward HIPAA and ONC certification. DeviQA lists HL7 v2 and FHIR R4 among tested standards. ImpactQA describes EPIC testing of provider workflows, scheduling and billing. Ask each one for redacted test cases.