RISMA: A Rule-based Interval State Machine Algorithm for Performance - - PowerPoint PPT Presentation

▶

Nov 08, 2023 431 likes •730 views

RISMA: A Rule-based Interval State Machine Algorithm for Performance Analysis, Alerts Generation, and Monitoring Real-Time Data Processing S. Laban 1 and A.I. El-Desouky 2 Science and Technology 2013 Conference Vienna, Austria. 1 Processing

SLIDE 1

RISMA: A Rule-based Interval State Machine Algorithm for Performance Analysis, Alerts Generation, and Monitoring Real-Time Data Processing

S. Laban1 and A.I. El-Desouky2

Science and Technology 2013 Conference

Vienna, Austria.

1 Processing Systems Officer – Preparatory Commission for the Comprehensive Nuclear-

Test-Ban Treaty (CTBT) Organization, Vienna International Centre, P.O. Box 1200, A-1400, Vienna, Austria.

E-Mail: shaban.laban@ctbto.org

2 Professor - Computer and Systems Department, Faculty of Engineering, Mansoura

University, Egypt.

SLIDE 2

Page 2

Motivation Background RISMA: A Rule-based Interval State Machine Algorithm for Performance Analysis, Alerts Generation, and Monitoring Real-Time Data Processing Interval state machine (ISM) State transition diagram and relationships Algorithm rules and priorities Implementation Results and analysis Algorithm derived information

Alerts
Operation performance indicator
Timeliness
Data availability

SHI examples RN examples Conclusion and future work

Outline

SLIDE 3

Motivation

The monitoring of real-time systems is a challenging and complicated process. The monitored data and their properties are rapidly, not necessarily linear, and continuously changing with time. Need to detect anomalous behaviors

monitored systems and determine/understand the sequence in which they occurred. Produce necessary and timely alerts during the workflow monitoring of such systems. Need to identify and use Key Performance Indicators (KPI). Interval-based theorems can be used for monitoring interval-based data

processing. However, increasing the number of non-linear intervals makes

the implementation of such theorems a complex and time consuming process. knowledge-based systems are fast and efficient and used for monitoring different systems. However, implementation with real-time data is not straight forward . Can we use knowledge-based systems with interval-based theorems in monitoring interval-based data t?

IDC Page 3

SLIDE 4

Background

IDC Page 4

The interval-based or period-based theorems have been discussed, analyzed, and used by many researches in Artificial Intelligence (AI), philosophy, and linguistics. As explained by Allen, there are 13 relations between any two intervals. However, processing a very large number of similar non-ordered intervals with classical interval-based theorems is a complicated and time consuming process. This is due to the need to infer an unlimited number of relationships between these intervals. The most popular inference engines used in artificial intelligence community is based on the C Language Integrated Production System (CLIPS). CLIPS shell is a forward-chaining rule-based tool which was originally developed at NASA's Johnson Space Centre and is based

n the Rete algorithm.

The inference engine of CLIPS is characterized by its explanation facility that provides valuable information to the user about how the inference engine arrived at its conclusions.

SLIDE 5

IDC Page 5

Interval State Machine (ISM)

The proposed ISM approach is used to model any interval-based data. The algorithm identifies four main different states for any similar non-ordered intervals as follows: NEW, LATEST, OLD, and DELETED. The Latest Moving Interval Approach

As the received intervals are not necessarily ordered. The algorithm is designed to maintain and make sure that the LATEST state is always having the maximum ending time using its internal necessary rules. This can be achieved by applying the following constraints and assumptions:

1. The processed intervals are not necessarily linearly ordered intervals.
2. There will be one and only one LATEST interval.
3. The LATEST interval should have the maximum end time for all the existing

permanent intervals.

4. There could be as many as of OLD intervals or none.
5. The primary state of any received interval is NEW and the algorithm will sequentially

process/infer them.

6. There will be one temporary NEW interval.
7. After inferring the NEW interval, it should not exist anymore and will be converted to

either OLD, LATEST, or DELETED.

8. The DELETED intervals are removed from memory.

SLIDE 6

IDC Page 6

State transition diagram

SLIDE 7

IDC Page 7

18 direct relationships

(Stations & Channels)

NEW-LATEST relationships
11 relationships
OLD-LATEST relationships
1 relationship
OLD-OLD relationships
6 relationships

States relationships

Alert-Alert relationships

(Stations)

The meets relationship is used to

concatenate similar alerts.

The equals relationships is mostly

used to end any onGoing alerts or complement the missing information

f the onGoing alerts.

SLIDE 8

Algorithm rules and priorities

IDC Page 8

Priority Rules 1 generate-alert-incomplete-information-for-NEW-interval 2 alert-incomplete-information-ended 3 alert-meets-alert 4 alert-complements-alert 5 new-after-latest new-during-latest new-equals-latest new-overlaps-latest new-meets-latest new-ends-latest new-starts-latest 6 new-before-latest new-contains-latest new-overlapped_by-latest newt-met_by-latest 7

ld-meets-latest

ld-during-old
ld-equals-old
ld-overlaps-old
ld-starts-old
ld-ends-old

ld-meets-old

10 generate-alert-missing-data

Begin RISMA For Each NEW Interval /* handling alerts */ ……. /* manipulating and sorting the NEW intervals */ IF not exits LATEST then Create LATEST with NEW Information. IF NEW after LATEST then LATEST information = NEW information and Delete NEW. IF NEW equals or during LATEST then Delete NEW. IF NEW overlaps or meets LTEST then Begin time of LATEST = begin time of NEW and Delete NEW. IF NEW starts LATEST then End time of LATEST = maximum end time of both and Delete NEW. IF NEW finishes LATEST then Begin time of LATEST = minimum begin time of both and Delete NEW. IF LATEST after NEW then NEW becomes OLD. IF LATEST during NEW then Begin and end times of LATEST equal begin and end times of NEW respectively and Delete OLD. IF LATEST overlaps or meets NEW then End time of LATEST = end time of NEW and Delete NEW. IF OLD meets LATEST then Begin time of LATEST = begin time of OLD and Delete NEW. /* manipulating and sorting the OLD intervals */ IF OLDi equals or during OLDj then Delete OLDi. IF OLDi overlaps or meets OLDj then Begin time of OLDj = begin time of OLDi and Delete OLDi. IF OLDi starts OLDj then End time of OLDj = maximum end time of both and Delete OLDi. IF OLDi finishes OLDj then Begin time of OLDj = minimum begin time of both and Delete OLDi. Finally: /* generate missing data alerts */ IF OLDmax precedes LATEST then Generate missing data alert with begin time=end time of OLDmax and end time = begin time of LATEST. IF OLDi precedes OLDj then Generate missing data alert with begin time=end time of OLDi and end time = begin time of OLDj. End RISMA

SLIDE 9

Implementation (1/2)

IDC Page 9

Field Description Id

Unique Identification number for the interval

Class

Category of the interval

Name

Subcategory or property of the interval or Station Name

Begin Time

The begin time of the interval

End Time

The end time of the interval

State

The state of the interval

;; definition of a generic Knowledge-Based Class (defclass KBO (is-a USER ) (slot hasType (type SYMBOL) ) (slot hasId (type SYMBOL) ) (slot hasSource (type SYMBOL) ) (slot hasCategory (type SYMBOL) ) (slot hasSubCategory (type SYMBOL) ) (slot hasClass (type SYMBOL) ) (slot hasName (type SYMBOL) ) (slot hasState (type SYMBOL)) (slot hasCount (type NUMBER) (default 1)) (slot hasCreationTime (type NUMBER) (default-dynamic (time))) (slot hasModificationTime (type NUMBER) (default-dynamic (time))) ) ;; definition of Real-Time Knowledge-Based Class ;; represents a generic Interval Object (defclass RTKBO (is-a KBO ) (slot hasBeginTime (type NUMBER) (default -1)) (slot hasEndTime (type NUMBER) (default -1)) ) ;; definition of CLF Class of type Interval (defclass CLF (is-a RTKBO) (slot hasCategory (type SYMBOL) (default CDData) ) (slot hasClass (type SYMBOL) (default CLF) ) (slot hasResentFrames (type NUMBER) (default 0) ) (slot hasChannels (type STRING)(default "")) (slot hasId (type NUMBER) ) (slot hasMissingChan (type SYMBOL) ) (slot hasReceivedTime (type NUMBER) (default -1)) ) ;; definition of Alert Class – another type Interval (defclass alert (is-a RTKBO) (slot hasStartTime (type NUMBER) (default -1)) (slot hasStopTime (type NUMBER) (default -1)) (slot hasChecktime (type NUMBER) (default -1)) (slot hasClass (type SYMBOL) (default alert) ) (slot hasSeverity (type SYMBOL) (default routine)) (slot hasInfo (type STRING)) ) ;; Rule for generating Alert for incomplete-information (defrule alert-incomplete-information (declare (salience 10000)) ?n <-(object (is-a CLF) (hasName ?name) (hasId ?id) (hasState NEW) (hasEndTime ?et) (hasBeginTime ?bt) (hasChannels ?ch) (hasMissingInfo YES)) => (bind ?d (str-cat ?ch)) (bind ?current_time (time)) (bind ?inst (symbol-to-instance-name (sym-cat alert_chan_missing_ ?name _ ?id) ) ) (make-instance (gensym*) of alert_clf (hasChecktime (send ?n get-hasReceivedTime) ) (hasName (sym-cat ?name)) (hasId ?id)(hasChannels ?d)(hasBeginTime ?bt) (hasEndTime ?et) (hasType CDDatat) (hasState

nGoing)

(hasClass alert_clf) (hasStartTime (send ?n get-hasCreationTime) ) ) ) ;; Similar Alerts will be ended once missing info completed (defrule alert-incomplete-information-ended (declare (salience 9999)) ?a1 <-(object (is-a alert_clf) (hasChecktime ?ct1) (hasName ?name)(hasEndTime ?et)(hasBeginTime ?bt) (hasChannels ?d1) ) ?a2 <-(object (is-a alert_clf) (hasChecktime ?ct2) (hasName ?name)(hasEndTime ?et)(hasBeginTime ?bt) (hasChannels ?d2)) (test (is-complete ?d1 ?d2)) (test (> ?ct2 ?ct1)) => (modify-instance ?a1 (hasStopTime (max (send ?a1 get-hasStartTime) (send ?a2 get- hasStartTime))) (hasState Ended) (hasCount (+ (send ?a2 get-hasCount) (send ?a1 get-hasCount))) (hasOtherChannels (str-cat (send ?a1 get-hasOtherChannels) " " ?d2) ) (hasMergeCount (+ (send ?a2 get-hasMergeCount) (send ?a1 get-hasMergeCount))) (hasOtherTimes (str-cat (send ?a1 get-hasOtherTimes) " " (send ?a2 get-hasStartTime)) ) ) (send ?a2 delete) ) ;; Similar Alerts can be concatenated or joined (defrule alert-meets-alert (declare (salience 9998)) ?a1 <-(object (is-a alert_clf) (hasChecktime ?ct1) (hasName ?name)(hasEndTime ?et1)(hasBeginTime ?bt1) (hasChannels ?d)) ?a2 <-(object (is-a alert_clf) (hasChecktime ?ct2) (hasName ?name)(hasEndTime ?et2)(hasBeginTime ?et1) (hasChannels ?d)) (test (> ?ct2 ?ct1)) => (modify-instance ?a1 (hasEndTime ?et2) (hasCount (+ 1 (send ?a1 get-hasCount))) (hasMeetsCount (+ (send ?a2 get-hasMeetsCount) (send ?a1 get-hasMeetsCount))) (hasChecktime ?ct2) ) (send ?a2 delete) )

SLIDE 10

Implementation (2/2)

IDC Page 10 ;; Normal operation of stations (defrule clf-new-met_by-latest (declare (salience 9800)) ?o <-(object (is-a CLF) (hasName ?name) (hasId ?idl)(hasState LATEST) (hasBeginTime ?btl) (hasEndTime ?btn) ) ?n <-(object (is-a CLF) (hasName ?name) (hasId ?idn) (hasState NEW) (hasBeginTime ?btn) (hasEndTime ?etn) ) (test (<> ?idl ?idn ) ) => (send ?n delete) (modify-instance ?o (hasEndTime ?etn) (hasCount (+ (send ?o get-hasCount) 1) ) ) ) ;; In case of data gaps (defrule clf-new-after-latest (declare (salience 9900)) ?o <-(object (is-a CLF) (hasName ?name) (hasId ?idl)(hasState LATEST) (hasBeginTime ?btl) (hasEndTime ?etl) ) ?n <-(object (is-a CLF) (hasName ?name) (hasId ?idn) (hasState NEW) (hasBeginTime ?btn) (hasEndTime ?etn) ) (test (<> ?idl ?idn) ) (test (> ?btn ?etl) ) => (make-instance (symbol-to-instance-name (sym-cat ?btl)) of CLF (hasState OLD) (hasId ?btl) (hasName ?name) (hasSource (send ?o get-hasSource)) (hasBeginTime ?btl) (hasEndTime ?etl)(hasId ?btl) (hasCount (send ?o get-hasCount)) ) (modify-instance ?o (hasBeginTime ?btn)(hasEndTime ?etn) (hasId ?idn) (hasCount (send ?n get-hasCount)) ) (send ?n delete) )

PyClips: It is an extension module for the Python language that embeds full CLIPS functionality in Python applications.

SLIDE 11

Algorithm derived information (1/2)

Alert equations

Total Alerts = Total Final Intervals -1 Total Alerts = Total Alerts - Total Alerts Total Ended Alerts = Total Fired Rules of “alert-incomplete-information- ended” Total onGoing Alerts = Total Alerts - Total Ended Alerts

Operation performance indicator

Operation Performance Indicator (OPI) =

!" !# $!" %&'(!) * %&

Where: Algorithm Management Rules 0 OldMeetsLatest 3 AlertMeetsAlert 3 AlertEnded

IDC Page 11

SLIDE 12

Timeliness

Timeliness =

5 !" !& * 6 7,9 5 !" !# ∗ 100

Where: M 6 =,>: NEW met by LATEST

Data availability

Interval data availability=

∑ CDEFGHIJ IK HJGLEMFN OGFGLO PQRSTUV WXYWVU QZ WXTUZUYURTUV [WT\ T\U WXTUZ]SP !# "&!

∗ 100 Total data availability =

∑ "&! ^ 5 "&!

∗ 100

IDC Page 12

Algorithm derived information (2/2)

SLIDE 13

SHI examples (1/10)

IDC Page 13

SLIDE 14

SHI examples (2/10)

IDC Page 14

SLIDE 15

SHI examples (3/10)

IDC Page 15

SLIDE 16

SHI examples (4/10)

IDC Page 16

SLIDE 17

SHI examples (5/10)

IDC Page 17

SLIDE 18

SHI examples (6/10)

IDC Page 18

SLIDE 19

SHI examples (7/10)

IDC Page 19

SLIDE 20

SHI examples (8/10)

IDC Page 20

SLIDE 21

SHI examples (9/10)

IDC Page 21

SLIDE 22

SHI examples (10/10)

IDC Page 22

SLIDE 23

RN examples (1/2)

IDC Page 23

SLIDE 24

RN examples (2/2)

IDC Page 24

SLIDE 25

Conclusion

The proposed intelligent algorithm uses the Interval State Machine (ISM) approach to model any number of interval-based data into well-defined states as well as inferring them. An interval-based state transition model and methodology are presented to identify the relationships between the different states

f the proposed algorithm.

By using such model, the unlimited number of relationships between similar large numbers of intervals can be reduced to only 18 direct relationships using the proposed well-defined states. For any period of time and any number of intervals, it was possible to analyze the monitored system behavior, generate alerts, and deduce derive important systems performance indicators.

IDC Page 25

SLIDE 26

Future work

Evaluate and revise the current derived information.
Derive new key performance indicators.
Study the possibility of extending the proposed

algorithm with association rules.

Extend the algorithm to investigate the possible

Alert-Data relationships and benefits.

Finally, study the possibility of extending the

proposed algorithm by applying rules for instants and instants-intervals relationships.

Implement other inference engines.

IDC Page 26

SLIDE 27

References

Allen, J. F., Maintaining knowledge about temporal intervals, Communications of the ACM 26 (1983), pp. 832–

843.

J. F. Allen., Towards a General Theory of Action and Time, Artificial Intelligence, 23(2): 123–154, 1984.
Halpern, J. and Y. Shoham, A propositional modal logic of time intervals, Journal of the ACM 38 (1991), pp. 935–

962.

Bresolin, D., D. Della Monica, V. Goranko, A. Montanari and G. Sciavicco, Metric propositional neighborhood

logic: Expressiveness, decidability, and undecidability, in: Proc. of the 19th European Conference on Artificial Intelligence (ECAI 2010), 2010, pp. 695–700.

Bresolin, D., V. Goranko, A. Montanari and G. Sciavicco, On Decidability and Expressiveness of Propositional

Interval Neighborhood Logics, in: Proc. of the International Symposium on Logical Foundations of Computer Science, LNCS 4514 (2007), pp. 84–99.

Bresolin, D., V. Goranko, A. Montanari and G. Sciavicco, Propositional interval neighborhood logics:

Expressiveness, decidability, and undecidable extensions, Annals of Pure and Applied Logic 161 (2009), pp. 289 – 304.

Bresolin, D., V. Goranko, A. Montanari and G. Sciavicco, Right propositional neighborhood logic over natural

numbers with integer constraints for interval lengths, in: Proc. of the 7th IEEE Conf. on Software Engineering and Formal Methods, 2009, pp. 240–249.

Bresolin, D., A. Montanari and P. Sala, An optimal tableau-based decision algorithm for Propositional

Neighborhood Logic, in: Proc. of STACS 2007, LNCS 4393 (2007), pp. 549–560.

Bokyoung Kang, Dongsoo Kim, Suk-Ho Kang, Real-time business process monitoring method for prediction of

abnormal termination using KNNI-based LOF prediction, Expert Systems with Applications, Volume 39, Issue 5, April 2012, pp. 6061-6068.

Rete: a fast algorithm for the many pattern/many object pattern match problem. Artificial Intelligence, Volume

19, 1982,pp. 17-37.

Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO),

http:/www.ctbto.org/.

IDC Page 27

SLIDE 28

International Data Centre Page 28