Situation coverage testing for autonomous robots Patrizio - - PowerPoint PPT Presentation

Patrizio Pelliccione Associate Professor, University of L’Aquila, Italy Associate Professor, Chalmers|GU

www.patriziopelliccione.com

Situation coverage testing for autonomous robots

H2020-ICT-2016

In the near future…

In the near future??

Situation coverage

• Take care about the range and diversity of the

situations in which we simulate and/or test autonomous robots

• Make sure that the situation coverage of our

testing is adequate

Situation coverage for autonomous robots

Mission, set of tasks, that the robot should accomplish Context in which the mission will be executed

Situation coverage for autonomous robots

Mission, set of tasks, that the robot should accomplish Context in which the mission will be executed

What are the typical missions robots are asked to perform?

Specification Patterns

• What kind of missions are specified in practice?
• Identification of missions already specified in practice (i.e. papers, documents of

robotic companies)

• Definition of a catalogue of mission specification patterns
• Tool support for assisting users in the specification of missions via the use and

instantiation of patterns

Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019 http://roboticpatterns.com

Specification Patterns

• Methodology
• Collection of mission requirements
• We collected mission requirements from papers within the robotic scientific

literature and from software engineering venues (mostly absent)

• Classification of mission concerns
• We linguistically decomposed each mission requirement into terms describing

concerns inherent in it

• Utilizing the concerns obtained from the mission requirements in the form of

terms, we created a graph structure that represents relations between mission concerns

• If a node represents some temporal behavior that concerns movement, it is a

candidate pattern

• Pattern identification and formulation

Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019 http://roboticpatterns.com

How to formulate the patterns?

“A robot r shall visit the two locations l1 and l2 in this order”

How to formulate the patterns?

“A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2

How to formulate the patterns?

“A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2 Is it possible to visit l2 before l1 and then to visit l2? Ambiguity

How to formulate the patterns?

“A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2 Is it possible to visit l2 before l1 and then to visit l2?

Φ1=<>((r in l1) && <>(r in l2)) vs. φ2 = φ1 && ((!r in l2)U(r in l1))

Intuitive but rigorous

Patterns identification

Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019 http://roboticpatterns.com

Specification Patterns

Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019 http://roboticpatterns.com

Specification Patterns

Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019 http://roboticpatterns.com

An example of pattern

Automating tool: PsAlM

http://roboticpatterns.com/psalm/

Claudio Menghi, Christos Tsigkanos, Thorsten Berger, Patrizio Pelliccione, “PsALM: Specification of Dependable Robotic Missions,” in 41st International Conference on Software Engineering (ICSE), Demonstrations Track, 2019.

Main components

• f the tool

http://roboticpatterns.com/psalm/

Claudio Menghi, Christos Tsigkanos, Thorsten Berger, Patrizio Pelliccione, “PsALM: Specification of Dependable Robotic Missions,” in 41st International Conference on Software Engineering (ICSE), Demonstrations Track, 2019.

Validation

• RQ1: To what extent are real-world, natural-language mission

requirements expressible using our pattern catalog?

• RQ2: To what extent are real-world mission specifications

expressible using our pattern catalog?

• RQ3: Does the pattern catalogue support the formulation of

mission requirements and specifications in a set of real-world scenarios defined in collaboration with our industrial partners?

Validation

• RQ1: How effective is the pattern catalog in capturing mission

requirements and producing mission specifications?

• We investigated
• How the pattern catalog supports the specification of mission

requirements

• How the pattern catalog reduces ambiguities in mission requirements
• We collected mission requirements in natural language from available

requirements in 11 different robotic applications produced from Spectra (428 mission requirements) and LTLMoP (16 mission requirements)

• The results show that most of the mission requirements (370 over 436 )

were expressible using the pattern catalog, which is a reasonable coverage for pattern catalog usage. The 66 mission requirements that are not covered suggested the introduction of new patterns

How to specify complex missions?

• Specification patterns enable the specification of “simple” missions and
• f one single robot
• PsAlM supports the composition of patterns to specify more complex

missions, but it supports only AND and OR operators

• We developed PROMISE to overcome these limitations
• Patterns are basic building blocks
• We added operators to compose patterns for specifying complex missions

for multi-robots

Domain Specific Language to specify missions

Sergio Garcia, Patrizio Pelliccione, Claudio Menghi, Thorsten Berger, Tomas Bures, “Higher-Level Mission Specification for Multiple Robots,” in 12th ACM SIGPLAN International Conference on Software Language Engineering (SLE), 2019.

Operators

• f the DSL

Operators

• f the DSL

Promise tool chain

Abstraction Layers DSL: Domain Specific Language Intermediate Language Patterns NuSMV Spectra Simbad Co4robots Planner

SERA (Self- adaptive dEcentralized Robotic Architecture)

l

• c

a l m i s s i

• n

[ r

• b
• t

1 ]

Global mission manager Global mission decomposer

global mission

Mission Management Layer Local mission manager

local mission high level specification

High level specification manager Change Management Layer Component Control Layer

local mission[robot1]

Adaptation manager Plan executor

plan

Motion planning & control

call [gestures robot state environment] method call

Legend

component unit

Central station Robot

robot status local mission shared map sharing l

• c

a l m i s s i

• n

[ r

• b
• t

4 ]

Gestures recognition Detection executor SLAM Robot interface Sensors interface Hardware & drivers

[robot state, mission status environment] Communication & collaboration manager

Information manager

[call status] [plan status] notify

• S. García, C. Menghi, P. Pelliccione, T. Berger and R. Wohlrab, "An Architecture for Decentralized, Collaborative, and Autonomous Robots,"

2018 IEEE International Conference on Software Architecture (ICSA), Seattle, WA, 2018, pp. 75-7509.

Validation

• RQ1: does PROMISE effectively support the specification of

complex missions?

We specified three of the missions proposed for the 2018’s edition of RoboCup@Home (Stage II) + the mission in the example

Validation

• RQ2: how simple is it to specify missions with PROMISE?
• We conducted two user studies:
• The first to have feedback to improve

the language

• The second, the final validation
• All the participants of the second study were able

to completely solve the three proposed blocks and validate them using the generated natural language file within the expected time frame (30 minutes for each block).

• They were also able to validate at least the first

two blocks through simulation within the time frame.

Patrizio Pelliccione Associate Professor, University of L’Aquila, Italy Associate Professor, Chalmers|GU

www.patriziopelliccione.com H2020-ICT-2016

Situation coverage testing for autonomous robots