User Group Monthly Meeting January 10, 2018 2:00 PM ET Agenda - - PowerPoint PPT Presentation

▶
user group
SMART_READER_LITE
LIVE PREVIEW

User Group Monthly Meeting January 10, 2018 2:00 PM ET Agenda - - PowerPoint PPT Presentation

Apr 05, 2024 153 likes •678 views

HL7 Immunization User Group Monthly Meeting January 10, 2018 2:00 PM ET Agenda Welcome Poll: Which perspective do you primarily identify yourself with? Functional Guide vol. 2: CDC Endorsed Data Elements, community review in

slide-1
SLIDE 1

HL7 Immunization User Group

Monthly Meeting January 10, 2018 2:00 PM ET

slide-2
SLIDE 2

Agenda

▪ Welcome

▪ Poll: Which perspective do you primarily identify yourself with? ▪ Functional Guide vol. 2: CDC Endorsed Data Elements, community review in January

▪Updates

▪ Poll results ▪ SISC

▪Todays Topic: Data Quality and ACK Messages

slide-3
SLIDE 3

HL7 User Group Poll Topics

Nathan Bunker, AIRA

slide-4
SLIDE 4

User Group Topics for 2019

Area Topic Yes Maybe No Score

VXU/ACK Messages Proper acknowledgement of message by IIS 21 7 49 HL7 Processing Support Interaction and support for lot inventory tracking 20 5 45 VXU/ACK Messages Appropriate triggers for when message should be sent 18 8 1 42 VXU/ACK Messages Proper processing of message by IIS 15 12 42 HL7 Processing Support Integration of vaccination matching with VXU and QBP 19 6 1 42 VXU/ACK Messages Proper construction of message by EHR 16 10 2 38 HL7 Processing Support Integration of patient matching with VXU and QBP 19 4 2 38 HL7 Processing Support Supporting data quality activities and reporting 17 4 38 Special Focus Areas Vaccine NDC 18 1 37 HL7 Processing Support Interaction and support for Vaccines for Children program 16 7 2 35 Special Focus Areas Updating and deleting vaccinations 17 1 1 33 Special Focus Areas Vaccine eligibility and vaccine funding source 16 3 1 33

slide-5
SLIDE 5

SISC Update

Craig Newman, Altarum

slide-6
SLIDE 6

Acknowledgements to Identify Data Quality Issues

Nathan Bunker, AIRA

slide-7
SLIDE 7

Resources on AIRA Repository:

  • Guidance for HL7 ACK Messages to Support Interoperability

Keyword: ACK messages

  • National Set of Error Codes

Keyword: error codes

slide-8
SLIDE 8

Resources, cont.

  • Data Validation Guide for the IIS

Onboarding Process

  • IIS Data Quality Practices: Monitoring

and Evaluating Data Submissions

slide-9
SLIDE 9

ACK Guidance

  • Created after HL7 v2.5.1 r1.5 completed
  • AIRA Measurement and Testing
  • Unable to understand the meaning of many ACKs received
  • Was the patient and vaccination history accepted by the IIS or not?
  • Source of the problem
  • Many ACKs were not properly following r1.5
  • Even if they were, the meaning was ambiguous
slide-10
SLIDE 10

ACK Guidance

  • ACK Guidance was created to clarify implementation of ACK
  • Just 6 pages including the examples
  • Adds additional constraints on r1.5, but still compatible
  • Defines valid combinations of:
  • MSA-1 (Acknowledgement Code)
  • ERR-4 (Severity)
  • Better defines the concept of “error”
slide-11
SLIDE 11

ACK Guidance

  • When the HL7 v2.5.1 r1.5 discusses an error, it could refer to:
  • General concept of an error as in something unexpected,

unwanted, or wrong

  • Error (ERR) segment
  • Severity code of Error (E)
  • Acknowledgement Code of Application Error (AE)
  • ACK Guidance
  • Something unexpected, unwanted, or wrong
slide-12
SLIDE 12

ACK Guidance

  • MSA-1 can be derived from ERR-4 fields
  • ERR-4 documents how serious the error is
  • Decision of severity is left to the determination of each IIS
  • However the distinction between E, W, and I must be preserved
  • The expectation on the sender must be consistent across IIS
slide-13
SLIDE 13

ACK Guidance

  • Transaction successful / Transaction was not successful
  • Misleading language in original description
  • A warning (W) doesn’t indicate success if there also an error (E)
slide-14
SLIDE 14

ACK Guidance

  • After a lot of discussion with the community this simple table

was created:

slide-15
SLIDE 15

ACK Guidance

  • What about MSA-1 (Acknowledgement Code)?
  • Must be consistent with the values in all the ERR-4 fields
slide-16
SLIDE 16

ACK Guidance

slide-17
SLIDE 17

ACK Guidance

  • What about correction and resubmission?
  • IIS might not process part of a message
  • Not stated how much of a message must be resubmitted
  • ACK is not well suited to indicate which parts were accepted
  • Senders should fix issue and resubmit all data
  • IIS will deduplicate information, if needed
slide-18
SLIDE 18

ACK Guidance – Specific Scenarios

Scenario (possible values an IIS could send) ERR-4 MSA-1 NDC for administered vaccination is not valid, vaccination can’t be read by IIS, sender needs to fix and resend E AE Expiration date for administered vaccination is missing Does the sender need to fix this? W I AE AA Patient phone number is missing How important is this to fix? W I no ERR AE AA AA Observation is sent, it’s valid in the guide, but the IIS doesn’t need the information or can’t process it yet I no ERR AA AA Refusal is sent correctly, but the IIS doesn’t accept refusals I no ERR AA AA

slide-19
SLIDE 19

ACK Guidance – Specific Scenarios

Scenario (possible values an IIS could send) ERR-4 MSA-1 A required HL7 field is empty, but the IIS doesn’t need this information and can still process the message without problems E W I AE AE AA

slide-20
SLIDE 20

ACK Guidance

  • Take home messages for IIS
  • Upgrade ACK messages to meet guidance
  • Use E, W, and I correctly, must match your IIS business rules
  • Communicate early changes to ACK messages with submitters
  • Aligning with this standard will
  • Reduce communication issues
  • Allow submitters to fix problems without asking more questions
  • Allow EHR vendors to better automate and support IIS submissions
slide-21
SLIDE 21

Data Quality in AART Discovery Reports

  • AART Discovery Reports
  • Exploratory testing of IIS functionality
  • All tests have to have a “right” answer to check results against
  • Section was created to test ability of IIS to identify data quality

problems and report them back in ACK

  • Problems we had to solve:
  • Which data quality issues should an IIS be sensitive to?
  • Which really should be reported back and which should be silent?
  • What error severity level should be expected?
slide-22
SLIDE 22

Data Quality in AART Discovery Reports

  • Original solution
  • List of issues from an older project to test a data quality tool
  • Sorted into three broad priority errors: High, Medium, Low
  • Submit message with no data quality issues
  • Submit message with the data quality issue
  • Expect at least one additional ERR segment for data quality issue
  • Not looking at any values in ERR segment, just presence of one more ERR
slide-23
SLIDE 23

Data Quality in AART Discovery Reports

  • Current solution
  • Continue with same set of issues in the same categories
  • Send message with one data quality issue
  • Expect IIS to return an ERR segment indicating:
  • There is an issue in the field where we put the data quality issue
  • Do not look at the value in ERR-4
  • No other expectations for any of the other fields
  • Current problems:
  • Still don’t know which data quality issues an IIS must be able to message back
slide-24
SLIDE 24

Data Quality in AART Discovery Reports

  • Potential future solutions:
  • Continue in Discovery testing
  • Could look for specific errors codes to be returned
  • Could test for support of proposed National Error Codes
  • Would need to get community decisions on some type of priority

and categorization of which error checks need to be supported

  • Would probably never look at what value is actually being placed in

ERR-4

  • Other functional testing handles this area, for example if an IIS says they

accepted a message with MSA-1 of AA and no error segments then the functional test might expect to query and retrieve the submitted information

slide-25
SLIDE 25

National Error Codes

  • Next step after ACK guidance, what codes should be sent in
  • ERR-3 (HL7 Error Code)
  • ERR-5 (Application Error Code)
  • Recommendations are still within the r1.5 standard
  • NIST testing software should allow these to be used
  • Not required for IIS to support, maybe a requirement for

future version

  • For now, EHRs are probably not looking at these fields much
slide-26
SLIDE 26

National Error Codes

  • ERR-3 (HL7 Error Codes)
  • We can’t change these in our guide, they are fixed by HL7
  • 1 – Illogical Date Error
  • 2 – Invalid Date
  • 3 – Illogical Value Error
  • 4 – Invalid Value
  • 5 – Table Value Not Found
  • 6 – Required Observation Missing
  • 7 – Required Data Missing
  • Not sufficient to convey the variety of errors IIS might send back
slide-27
SLIDE 27

National Error Codes

  • A set of general classes of errors was defined:
  • Existing (error codes documented in Release 1.5)
  • Conflicting Data (data within a single message is internally inconsistent)
  • Inappropriate Date
  • Invalid Data
  • Lookup (an expected record cannot be found based on data in the

message)

  • Message Construction (structural issues based on local business rules)
  • Missing Data
  • Processing Error
  • Data Sharing
slide-28
SLIDE 28

National Error Codes

slide-29
SLIDE 29

National Error Codes

  • IIS should not create or use their own local codes
  • IIS are not expected to support every code defined
  • Implementation of these codes recommended when:
  • Upgrade their ACK/RSP messaging capabilities
  • Implement new data quality checks and validations (i.e. recognize

new error conditions)

  • Implement expanded error reporting
  • Contact AIRA if you can’t find a national code that meets a

specific scenario

slide-30
SLIDE 30

National Error Codes

  • Expectations on receiving systems:
  • Must gracefully handle values in ERR-5 that they don’t recognize
  • Must be able to handle ERR-5 being empty
  • The long term vision is that ERR-5 will be useful to EHR’s as

they improve their handling of errors

  • IIS must not redefine national codes, so that ERH’s can have

confidence to take specific action for specific codes

slide-31
SLIDE 31

Message Quality Evaluator (MQE)

  • Open-source application
  • https://github.com/immregistries/mqe/wiki
  • Measures data quality in HL7 messages
  • Makes “Detections” for each message in a batch of messages
  • Do not indicate severity (E, W, or I)
  • Only that an issue exists
  • Summarizes these detections into reports
slide-32
SLIDE 32

Message Quality Evaluator (MQE)

  • The core Validation engine that makes the Detections is or

will be used in different places:

  • MQE application
  • Data-at-Rest software (being developed by NIST)
  • HL7 message testing (on NIST website)
  • AART testing
slide-33
SLIDE 33

HL7 Data Quality

Kevin Snow, Envision Technology Partners

slide-34
SLIDE 34

HL7 Data Quality

Harmonizing HL7 Validation with the AIRA Data Validation Guide and National Error Code Set Guidance

Jan 2019

slide-35
SLIDE 35

Completeness Consistency Accuracy Timeliness Validity

slide-36
SLIDE 36

The 5Ws (and H)

Who

  • The IIS

What

  • Implement or harmonize with existing data accuracy checks and error codes

Where • HL7 Response Messages When • In real time Why

  • Quick to find, quick to fix. Easier when in harmony with existing resources.

How

  • …and request for feedback!
slide-37
SLIDE 37

Resources Used

https://immregistries.org/resource/

slide-38
SLIDE 38

High Priority accuracy validation

BR-101: Vaccination Encounter Date must not be before Patient Date of Birth

HL7 Data Quality Statement AIRA-DV-BR-101: RXA-3 (DateTimeStartOfAdministration) must not be before PID-7 (DateTimeOfBirth) when RXA-21 (ActionCode) is not valued "D“. When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 2204^Vaccination Date Too Long Ago^HL70533

slide-39
SLIDE 39

High Priority accuracy validation

BR-103: Vaccination Encounter Date must be less than or equal to (before or the same as) the Submission Date

HL7 Data Quality Statement AIRA-DV-BR-103: RXA-3 (DateTimeStartOfAdministration) must be less than or equal to (before or the same as) MSH-7 (DateTimeOfMessage) when RXA-21 (ActionCode) is not valued "D". When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 1^Illogical Date error^HL70533

slide-40
SLIDE 40

High Priority accuracy validation

BR-107: Every administered vaccine should be recorded as a single Vaccination Event (e.g., combo vaccine should be recorded as 1 event rather than separate events for each component)

Not Implemented Was unsure how to implement due to:

  • Immunizations can be split into multiple messages
  • Even if it’s in a single message the components could be out of order
slide-41
SLIDE 41

High Priority accuracy validation

BR-114: Vaccination Encounter Date should not be the same as the Patient Date of Birth unless it is on the list of vaccines recommended for administration on the date of birth, e.g., HepB

HL7 Data Quality Statement AIRA-DV-BR-114: RXA-3 (DateTimeStartOfAdministration) should not be the same as PID-7 (DateTimeOfBirth) unless it is on the list of vaccines recommended for administration on the date of birth, e.g., HepB when RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D“. When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 1^Illogical Date error^HL70533

slide-42
SLIDE 42

High Priority accuracy validation

BR-116: Manufacturer and CVX Code should not contradict one another

Not Implemented Was unsure how to implement due to:

  • How to address case of manufacturer buys out another manufacturer. Does

the code change? Do we need history of this to allow multiple values? Is there a challenge with keeping this up to date?

Functional Guide Vol 2-Review Draft 1.7.19.docx

slide-43
SLIDE 43

High Priority accuracy validation

BR-118: Vaccination Encounter Date should not be after the lot number expiration date

HL7 Data Quality Statement AIRA-DV-BR-118: RXA-3 (DateTimeStartOfAdministration) should not be after RXA-16 (SubstanceExpirationDate) when RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D". When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 2001^Conflicting Administration Date and Expiration Date^HL70533

slide-44
SLIDE 44

Medium Priority accuracy validation

BR-119: Route and Site should not contradict each

  • ther for a given Vaccine Type and Patient’s age

HL7 Data Quality Statement AIRA-DV-BR-119: RXR-1 (Route) and RXR-2 (AdministrationSite) contradict each other for the given Vaccine Type in RXA-5 (AdministeredCode) and Patient’s age on RXA-3 (DateTimeStartOfAdministration) when RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D". When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 3^Illogical Value error^HL70533

slide-45
SLIDE 45

High Priority accuracy validation

BR-121: Administered vaccinations coded with an “unspecified” CVX code (should have specific Vaccine Types, e.g., Hib PRP-OMP; unspecified vaccine types, e.g., Hib, unspecified formulation)

HL7 Data Quality Statement AIRA-DV-BR-121: RXA-5 (AdministeredCode) should not be valued with an “unspecified” vaccine when the first occurrence of RXA-9.1 is valued "00" and RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D". When the above data quality statement is violated then return ERR-3 (HL7ErrorCode) 3^Illogical Value error^HL70533

slide-46
SLIDE 46

High Priority accuracy validation

BR-130: Doses should not be recorded as given before the minimum patient age or after the maximum patient age for that particular vaccine

HL7 Data Quality Statement AIRA-DV-BR-130: Patient's age on RXA-3 (DateTimeStartOfAdministration) should not be before the minimum patient age or after the maximum patient age for the given Vaccine Type in RXA-5 (AdministeredCode) when RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D". When the above data quality statement is violated then return Next slide…

slide-47
SLIDE 47

BR-130 cont.

ERR-3 (HL7ErrorCode) 3^Illogical Value error^HL70533 ERR-7 (DiagnosticInfor mation) NumericPath: RXA[1].3[1].1, NamePath: ORDER[0]/RXA/DateTimeStartOfAdministration/Time, RuleId: , ApplicationErrorCode: AIRA-DV-BR-130, AIRA Data Validation Guide Rule: BR 130, Maximum Date: 20180101, Administration Date: 20181217, Days Difference: 350 ERR-8 (UserMessage) RXA-3 (DateTimeStartOfAdministration): Patient's age on this date should not be before the minimum patient age or after the maximum patient age for the given Vaccine Type in RXA-5 (AdministeredCode) when RXA-20 (CompletionStatus) is valued "CP" or "PA" and RXA-21 (ActionCode) is not valued "D". Please see BR-130 in the AIRA Data Validation Guide. When the above data quality statement is violated then return

slide-48
SLIDE 48

Test and Document Like Crazy

slide-49
SLIDE 49
slide-50
SLIDE 50

Next Meeting

February 14, 2019 2:00 pm ET / 11:00 am PT

slide-51
SLIDE 51

More Information

  • Web Links
  • Subscribe to immunization group

http://www.hl7.org/participate/UserGroups.cfm?UserGroup=Immunization

  • Public User Group Wiki

http://www.hl7.org/special/committees/iug/index.cfm

  • Private User Group Wiki

http://iugwiki.hl7.org/

  • HL7 Press Release

http://www.hl7.org/documentcenter/public_temp_F760602A-1C23-BA17- 0C0D326E635471F9/pressreleases/HL7_PRESS_20140402.pdf

  • AIRA Press Release

http://www.immregistries.org/events/2014/04/10/hl7-immunization-user-group

slide-52
SLIDE 52

Contact Information

If you have any questions or comments: ▪ Kim Salisbury-Keith Kim.SalisburyKeith@health.ri.gov ▪ Nathan Bunker nbunker@immregistries.org ▪ Kevin Snow ksnow@envisiontechnology.com ▪ Danny Wise Danny.Wise@allscripts.com Thank you!