Next Generation LearnLib (NGLL) Bernhard Steffen, Falk Howar, Maik - - PowerPoint PPT Presentation

next generation learnlib ngll
SMART_READER_LITE
LIVE PREVIEW

Next Generation LearnLib (NGLL) Bernhard Steffen, Falk Howar, Maik - - PowerPoint PPT Presentation

Next Generation LearnLib (NGLL) Bernhard Steffen, Falk Howar, Maik Merten , Oliver Bauer TU Dortmund SFM2011 Overview Practical aspects in active automata learning LearnLib: mature library for active atomata learning NGLL: framework,


slide-1
SLIDE 1

Next Generation LearnLib (NGLL)

SFM2011

Bernhard Steffen, Falk Howar, Maik Merten, Oliver Bauer TU Dortmund

slide-2
SLIDE 2

Overview

  • Practical aspects in active automata learning
  • LearnLib: mature library for active atomata

learning

  • NGLL: framework, providing infrastructure for

practical application

  • Tool-demo: Modeling learning setups with

LearnLib & LearnLib Studio

slide-3
SLIDE 3

Active automata learning

Learner MQ-Oracle EQ-Oracle Σ={a,b} a ∈ L? no ? no, bb ∈ L!

a a a a b b b b a a,b b

slide-4
SLIDE 4

Practical challenges

Behavioral models

Test-driver

<presence type=… /> <iq type= “result“ /> Available OK

reset interfacing real systems:

  • alphabet generation
  • abstraction
  • data

equivalence queries membership queries

Interface description etc.

slide-5
SLIDE 5

NGLL: Structure & components

Algorithms (LearnLib)

  • Learning algorithms
  • Counterexample analysis
  • Equivalence query realizations

Core/Utils (LearnLib)

  • (Small) automata library
  • Statistics
  • Logging
  • Import/export (dot, ppt, xml etc.)
  • Simulation environments

Case studies (NGLL)

  • Test-driver tools
  • Data mappers
  • Abstraction

LearnLib Studio (NGLL)

  • Visual composition of learning

experiments

slide-6
SLIDE 6

LearnLib: Algorithms

  • Learning algorithms
  • Angluin’s L*
  • Rivest and Schapire’s “Reduced Observation Table”
  • Non-uniform observation table
  • DHC
  • Handling Counterexamples
  • Maler and Pnueli
  • Shahabaz and Groz
  • Rivest and Schapire
  • Equivalence queries
  • Chow’s “W-Method”
  • Random walks
  • Hopcroft and Karp
  • Evolving hypothesis

All algorithms work for DFA + Mealy machines

slide-7
SLIDE 7

LearnLib: Enhancements

  • POTS with 3 parties (CADP)
  • 39.979 states
  • > 2 million transitions
  • > 270 equivalence queries

Old c++ LearnLib (2009)

  • 3 hours
  • 7GB memory
  • > 690 million MQ

Java NGLL (today)

  • 20 minutes (-90%)
  • 2,3GB memory (-60%)
  • ca. 593 million MQ (-15%)
slide-8
SLIDE 8

NGLL: Test-drivers (example)

search( “Scotland” ) authenticate(falk,**** )

Falk’s … … Scotland2010

browse( ) uploadPhoto( , )

Scotland2010

OK ERROR register(… createAlbum(… getAlbum(…

A photo service

slide-9
SLIDE 9

Beyond plain DFA / Mealy machines

session ssid-123 albums [‚Fa…', ... ... s e a r c h ( ‚ S c

  • t

l a n d ' ) [‚Falk‘s …', ‚Rebus‘s Scot…'

albums = search(si, query)

session ssid-123 albums [‚Fa…', ...

search(si): if (si == session) return matchingAlbums(query);

PhotoService Client PhotoService

slide-10
SLIDE 10

Beyond plain DFA / Mealy machines

session ssid-123 albums [‚Fa…', ... ... s e a r c h ( ‚ S c

  • t

l a n d ' ) [‚Falk‘s …', ‚Rebus‘s Scot…' session ssid-123 albums [‚Fa…', ...

albmus←search(si, query) when (si==session): session ssid-123 albums [‚Falk‘s …', session ssid-123 albums

  • k

PhotoService Client PhotoService

Symbolic Mealy machines with registers:

slide-11
SLIDE 11

Building test-drivers

Interface description Datatypes Instance pools Semantics (effects) Reset strategy Generate instrumentation code Generate abstract / concrete mapping Build test-driver SUT instrumentation Abstract interface alphabet Test-driver

slide-12
SLIDE 12

Test execution

p=lower(i) lift(List<Packet> …)

slide-13
SLIDE 13

Generating test-drivers from WSDLs

reg.t=auth(“falk”,… / error

Proxy Test-driver

Mapping

Register / Instance pool

Abstraction/Data driver

concrete queries

<WSDL>

standard tools

System Learner

invocations abstract queries

Setup defs.

<message … / try { reg.t=proxy.auth( “falk”,… return “ok”; } catch (Exception… return “error”; t←auth(u1,… / error <message …

slide-14
SLIDE 14

Summary

  • LearnLib: mature library for active automata

learning

  • NGLL: Infrastructure to learn real systems
  • LearnLib Studio: visual composition /

debugging of learning experiments

  • www.learnlib.de
slide-15
SLIDE 15

Hands on …

Tool demo

slide-16
SLIDE 16

Example: SendAGift24.com

Post-mortem analysis of a legacy service

slide-17
SLIDE 17

17

Scenario

/** * This is the interface to SendAGift24.com * * SendAGift24 is a multi-billion .com that is specialized in providing * a service where people can shop gifts for friends. * * * Sadly all system backups disappeared into the cloud, so the service * source code was retrieved by decompiling obfuscated binaries. I wonder if * there is a technology to find out how this service works. * * The CEO of the company told me that "user" and "password" are fitting * credentials, so at least security is up to modern standards. */ public interface Service { public long a(String a, String b) throws Exception; public String[] b(long l) throws Exception; public Object[] c(long l) throws Exception; public boolean d(long l, String s, Object o) throws Exception; }

slide-18
SLIDE 18

18

Test-driver design

  • Handle data-dependencies
  • instance-pool for, e.g., credentials
  • store invocation results of previous invocations for inter-method

dependencies

  • fill in parameters from data store
  • Translate learning symbols into concrete invocations
  • Lift concrete system output, lower abstract parameters
  • Ensure every query is executed on initial state
slide-19
SLIDE 19

19

Implementation

  • sendagift.learner.ServiceAlphabet
  • alphabet of abstract symbols
  • can be used with AddAlphabetByClassName SIB
  • sendagift.learner.ServiceDriver
  • application-specific test-driver
  • sendagift.learner.ServiceOracle
  • glue between learner and test-driver
  • can be used with QueryOracleByClassName SIB
slide-20
SLIDE 20

20

Learned result

  • a == openSession
  • b == getFriends
  • c == getGifts
  • d == sendGift
  • gift and friend have to be

known before sendGift can be invoked

  • friend and gift bound to session