Demaq: A Foundation for Declarative XML Message Processing - - PowerPoint PPT Presentation

▶
demaq a foundation for declarative xml message processing
SMART_READER_LITE
LIVE PREVIEW

Demaq: A Foundation for Declarative XML Message Processing - - PowerPoint PPT Presentation

Nov 29, 2022 396 likes •626 views

Demaq: A Foundation for Declarative XML Message Processing Alexander Bhm Carl-Christian Kanne Guido Moerkotte University of Mannheim Carl-Christian Kanne, January 8, 2007 Demaq - p. 1/20 XML Messaging XML Introduction XML Messaging

slide-1
SLIDE 1

Carl-Christian Kanne, January 8, 2007 Demaq - p. 1/20

Demaq: A Foundation for Declarative XML Message Processing

Alexander Böhm Carl-Christian Kanne Guido Moerkotte

University of Mannheim

slide-2
SLIDE 2

Introduction

  • XML Messaging
  • Networks of XML Queues
  • Messaging Rules
  • ∞-tier Architectures
  • State of the Art

Demaq Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 2/20

XML Messaging

XML XML XML XML XML XML XML XML XML XML XML

Manufacturer Supplier C Supplier B Supplier A Customer Customer

SOA, Web Services, AJAX, RSS/Atom...

slide-3
SLIDE 3

Introduction

  • XML Messaging
  • Networks of XML Queues
  • Messaging Rules
  • ∞-tier Architectures
  • State of the Art

Demaq Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 3/20

Networks of XML Queues

XML XML XML XML XML XML XML XML XML XML XML

slide-4
SLIDE 4

Introduction

  • XML Messaging
  • Networks of XML Queues
  • Messaging Rules
  • ∞-tier Architectures
  • State of the Art

Demaq Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 4/20

Messaging Rules

■ "If a request for an offer comes in, forward it to the

legal, finance, and planning departments"

■ "If the delivery of all items has been confirmed,

send a completion message to the customer"

slide-5
SLIDE 5

Introduction

  • XML Messaging
  • Networks of XML Queues
  • Messaging Rules
  • ∞-tier Architectures
  • State of the Art

Demaq Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 5/20

∞-tier Architectures

XML

XML

Java Java C# Java C# Network

Messaging System DBMS App Code App Server TP Monitor

slide-6
SLIDE 6

Introduction

  • XML Messaging
  • Networks of XML Queues
  • Messaging Rules
  • ∞-tier Architectures
  • State of the Art

Demaq Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 6/20

State of the Art

t o p ic = ( ( QIjmsSession ) t_sess ) . getTopic ( " strmadmin " , " oe_queue " ) ; t_pub = t_sess . createPublisher ( t o p ic ) ; db_conn = ( ( QIjmsSession ) t_sess ) . getDBConnection ( ) ; agent = new QIjmsAgent ( " e x p lic it _ e n q " , n u l l ) ; adt_msg = ( ( QIjmsSession ) t_sess ) . createAdtMessage ( ) ; lcr_data = new St r in g Bu f f e r ( ) ; lcr_data . append (" <ROW_LCR " ) ; lcr_data . append ( " xmlns= ’ h t t p : / / xmlns . tentacle .com/ streams / schemas / lc r ’ " ) ; lcr_data . append ( " xmlns : xsi = ’ h t t p : / / www.w3. org /2001/XMLSchema−instance ’ " ) ; lcr_data . append ( " xsi : schemaLocation = ’ h t t p : / / xmlns . tentacle .com/ streams / schemas / l c r " ) ; lcr_data . append ( " h t t p : / / xmlns . tentacle .com/ streams / schemas / l c r / streamslcr . xsd ’ > " ) ; lcr_data . append (" < source_database_name>source_dbname</ source_database_name > " ) ; . . . MORE DOCUMENT CONSTRUCTION HERE . . . xml_lcr = tentacle . xdb .XMLType . createXML ( db_conn , lcr_data . t o St r in g ( ) ) ; adt_msg . setAdtPayload ( xml_lcr ) ; ( ( QIjmsMessage) adt_msg ) . setSenderID ( agent ) ; System . out . p r i n t l n ( " Publish message 3 − XMLType containing LCR R O W" ) ; r e c i p L i s t = new QIjmsAgent [ 1 ] ; r e c i p L i s t [ 0 ] = new QIjmsAgent ( " e x p lic it _ d q " , n u l l ) ; ( ( QIjmsTopicPublisher ) t_pub ) . publish ( topic , adt_msg , r e c i p L i s t ) ; t_sess . commit ( ) ;

slide-7
SLIDE 7

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 7/20

Demaq Application

Network

Queue Queue Queue Queue Queue Gateway Gateway

Demaq Application

■ Complete ■ Declarative ■ Executable

slide-8
SLIDE 8

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 8/20

Demaq Language(s)

Network

Gateway Gateway Queue Queue Queue Queue Queue Queue Queue Queue Queue Queue Gateway Gateway

Demaq Application Application Rules (QML) Message Queues (QDL)

Demaq Language

slide-9
SLIDE 9

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20

Demaq QML Rules

■ "If the delivery of all items has been confirmed,

send a completion message to the customer"

slide-10
SLIDE 10

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20

Demaq QML Rules

■ "If the delivery of all items has been confirmed,

send a completion message to the customer" XML messages

− →

new XML messages

slide-11
SLIDE 11

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20

Demaq QML Rules

■ "If the delivery of all items has been confirmed,

send a completion message to the customer" XQuery Update Facility + Queuing Primitives XML messages

− →

new XML messages

slide-12
SLIDE 12

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 10/20

Demaq Sample Rule

create rule sendComplete for orderMsgs i f ( / / deliverymsg / @type = " confirm " ) then l e t $ordered := fn : count (qs : slice ( ) / / ordermsg / / item ) l e t $delivered := fn : count (qs : slice ( ) / / deliverymsg / item ) where $ordered eq $delivered return do enqueue <done> { / / orderID } </done> into customerReply

slide-13
SLIDE 13

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 11/20

Demaq Server

Network

Gateway Gateway Queue Queue Queue Queue Queue Queue Queue Queue Queue Queue Gateway Gateway

Demaq Application Application Rules (QML) Message Queues (QDL)

Transport Engine

Demaq Server

Rule

Transactional Store

Demaq Language

XML

slide-14
SLIDE 14

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 12/20

Messages all the way

■ Everything is an XML message ◆ Rule Input ◆ Rule Output ◆ Errors ◆ Timeouts ■ Messages are processed once, but kept "forever" ■ Message History ◆ captures process state ◆ organized into slices (virtual queues) ◆ declarative expiration

slide-15
SLIDE 15

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 13/20

Demaq Project

Network

Gateway Gateway Queue Queue Queue Queue Queue Queue Queue Queue Queue Queue Gateway Gateway

Demaq Application Application Rules (QML) Message Queues (QDL) QEP QEP QEP

Transport Engine

Demaq Language Demaq Compiler Demaq Server

Rule

Transactional Store

XML

slide-16
SLIDE 16

Introduction Demaq

  • Demaq Application
  • Demaq Language(s)
  • Demaq QML Rules
  • Demaq Sample Rule
  • Demaq Server
  • Messages all the way
  • Demaq Project
  • Thank you

Appendix Carl-Christian Kanne, January 8, 2007 Demaq - p. 14/20

Thank you http://demaq.net

http://db.informatik.uni-mannheim.de

slide-17
SLIDE 17

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 15/20

Slices

Orders Queue

... ... Queues Slices

Slice for customer 23

...

Slice for customer 42

... ... ...

Delivery Notifications Queue Requests Queue 23 47 7 42 9 42 23 23 42 42 23 23 23 23 23 47 15

create property customerID fixed queue requests ,

  • rders ,

d e l i v e r y N o t i f i c a t i o n s value / / customerID create slicing customers on customerID

slide-18
SLIDE 18

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 16/20

Slice usage

■ Merge parallel control flow

Rating Check Setup Costs Check Plant Capacity Check Export Restrictions Check Credit Send/Refuse Offer

create property c o r r e l a t i o n I D fixed queue creditCheck , exportCheck , plantCheck , setupCheck value / / c o r r e l a t i o n I D create slicing checkResults on c o r r e l a t i o n I D create rule merge for checkResults i f ( count (qs : slice ( ) ) eq 4) then . . .

slide-19
SLIDE 19

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 17/20

Error Handling

■ Plenty of error sources in distributed applications ◆ Application-related (dynamic errors) ◆ Message-related (invalid XML, wrong schema) ◆ Network-related (disconnects, routing, . . . ) ◆ . . . ■ Message-based error handling ■ Error queues, e.g. for rules

create rule errorSource for foo errorqueue errors

slide-20
SLIDE 20

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 18/20

A Demaq Rule

create rule newOfferRequest for customerMsgs i f ( / / offerRequest ) then l e t $customerInfo := <requestCustomerInfo reqID = " { / / requestID }" > <customer > { / / customerID } </ customer> </ requestCustomerInfo > l e t $expor tRestr i c ti on In fo := . . . l e t $plantCapacityInfo := . . . return do enqueue $customerInfo into finance , do enqueue $expor tRestr ictionsInfo into legal , do enqueue $plantCapacityInfo into supplier

slide-21
SLIDE 21

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 19/20

Work in progress

■ Optimization across rules ■ Optimization/verification across sites ■ Template Folding [XIMEP06] ■ Rules driven by XML Schema validation

slide-22
SLIDE 22

Introduction Demaq Appendix

  • Slices
  • Slice usage
  • Error Handling
  • A Demaq Rule
  • Work in progress
  • Demaq Goals

Carl-Christian Kanne, January 8, 2007 Demaq - p. 20/20

Demaq Goals

■ Declarative XML message processing language ◆ Move work from programmer to system ◆ Data independence ◆ Optimizable ■ Execution Engine ◆ Reliability ◆ Scalability ◆ Reuse DB system knowledge