Keeping Evolving Requirements and Acceptance Tests Aligned with - - PowerPoint PPT Presentation

keeping evolving requirements
SMART_READER_LITE
LIVE PREVIEW

Keeping Evolving Requirements and Acceptance Tests Aligned with - - PowerPoint PPT Presentation

Keeping Evolving Requirements and Acceptance Tests Aligned with Automatically Generated Guidance Sofija Hotomski Eya Ben Charrada Martin Glinz Agenda 1. Context, background and motivation 2. Our approach and tool 3. Evaluation 4. Summary


slide-1
SLIDE 1

Keeping Evolving Requirements and Acceptance Tests Aligned with Automatically Generated Guidance

Eya Ben Charrada Sofija Hotomski Martin Glinz

slide-2
SLIDE 2

Agenda

2

  • 1. Context, background and

motivation

  • 2. Our approach and tool
  • 3. Evaluation
  • 4. Summary
slide-3
SLIDE 3

Requirements and acceptance tests – an example

Requirement id: REQ1 Title: Group chat management – adding users to a group Text: A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin can modify personal data and the status of a user. Test id: AT1 Title: Test for REQ1 (Group chat management – adding users to a group) Prerequisites: Internet connection is needed. Steps: Expected results:

  • 1. Log-in as user

Login is successful

  • 2. Open the page

The page is opened and the list with users’ groups

  • f groups is visible
  • 3. Click on the button The group members are listed

"Manage group” next and there is a button “Add users” to the Group 1. in the upper, right corner. … …

3

slide-4
SLIDE 4

Motivation

4

slide-5
SLIDE 5

Motivation

4

slide-6
SLIDE 6

Motivation

4

TESTERS ARE NOT AWARE

slide-7
SLIDE 7

Motivation

4

TESTERS ARE NOT AWARE

slide-8
SLIDE 8

Motivation

4

TESTERS ARE NOT AWARE BUGS FOR FEATURES

slide-9
SLIDE 9

Motivation

5

slide-10
SLIDE 10

Motivation

Non-aligned documents

5

slide-11
SLIDE 11

Motivation

Poor communication Non-aligned documents

5

slide-12
SLIDE 12

Motivation

Poor communication Non-aligned documents

5

False expectations

slide-13
SLIDE 13

Motivation

Poor communication Non-aligned documents

=>

5

False expectations

slide-14
SLIDE 14

Motivation

Poor communication Non-aligned documents

=>

5

False expectations

slide-15
SLIDE 15

Motivation

Poor communication Non-aligned documents

=>

5

False expectations

&

slide-16
SLIDE 16

Motivation

Poor communication Non-aligned documents

=>

5

False expectations

& &

slide-17
SLIDE 17

State of the art

Keep software documentation aligned using Information Retrieval and Natural Language Processing methods for identifying impacted documents.

6

slide-18
SLIDE 18

State of the art

Gaps in the current methods:

  • Less applied between requirements and acceptance

tests

  • Only identify which documents are impacted by a

change, but not suggest how to manage the change

7

slide-19
SLIDE 19

The goals of our approach (GuideGen)

  • 1. Support test engineers in making decisions on

how to modify impacted acceptance tests,

  • 2. Ensure on-time and smooth communication of

changes between requirements and test engineers.

When requirements evolve:

8

slide-20
SLIDE 20

The goals of our approach (GuideGen)

  • 1. Support test engineers in making decisions on

how to modify impacted acceptance tests,

  • 2. Ensure on-time and smooth communication of

changes between requirements and test engineers.

When requirements evolve:

8

Generate guidance

slide-21
SLIDE 21

The goals of our approach (GuideGen)

  • 1. Support test engineers in making decisions on

how to modify impacted acceptance tests,

  • 2. Ensure on-time and smooth communication of

changes between requirements and test engineers.

When requirements evolve:

8

Generate guidance Notification system

slide-22
SLIDE 22

GuideGen overview

1. 2.

3.

9

slide-23
SLIDE 23

An example of requirement changes and generated suggestions

A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

slide-24
SLIDE 24

An example of requirement changes and generated suggestions

  • 1. Delete steps or their parts which verify that the

addition of a new user must be first approved by the admin. A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

slide-25
SLIDE 25

An example of requirement changes and generated suggestions

  • 1. Delete steps or their parts which verify that the

addition of a new user must be first approved by the admin. A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

  • 2. Make sure that now the user can modify personal

data of that user. Add the steps or modify the expected results which verify this activity.

slide-26
SLIDE 26

An example of requirement changes and generated suggestions

  • 1. Delete steps or their parts which verify that the

addition of a new user must be first approved by the admin. A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

  • 3. Delete steps or their parts which verify that the

admin can modify the status of a user.

  • 2. Make sure that now the user can modify personal

data of that user. Add the steps or modify the expected results which verify this activity.

slide-27
SLIDE 27

An example of requirement changes and generated suggestions

  • 1. Delete steps or their parts which verify that the

addition of a new user must be first approved by the admin. A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

  • 3. Delete steps or their parts which verify that the

admin can modify the status of a user.

  • 4. Add new steps or modify existing steps to verify that
  • nly user can modify its status.
  • 2. Make sure that now the user can modify personal

data of that user. Add the steps or modify the expected results which verify this activity.

slide-28
SLIDE 28

An example of requirement changes and generated suggestions

  • 1. Delete steps or their parts which verify that the

addition of a new user must be first approved by the admin. A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a

  • user. Only user can modify

its status. The admin must be logged-in in order to modify personal data of a user.

10

  • 3. Delete steps or their parts which verify that the

admin can modify the status of a user.

  • 4. Add new steps or modify existing steps to verify that
  • nly user can modify its status.
  • 5. Add new steps or modify existing steps to verify that

the admin must be logged-in in order to modify personal data of a user.

  • 2. Make sure that now the user can modify personal

data of that user. Add the steps or modify the expected results which verify this activity.

slide-29
SLIDE 29
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq:

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-30
SLIDE 30
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-31
SLIDE 31
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-32
SLIDE 32
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-33
SLIDE 33
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-34
SLIDE 34
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-35
SLIDE 35
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-36
SLIDE 36
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-37
SLIDE 37
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-38
SLIDE 38
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user.

  • 1. Identify relevant change patterns – sentence level
slide-39
SLIDE 39
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-40
SLIDE 40
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-41
SLIDE 41
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-42
SLIDE 42
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-43
SLIDE 43
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-44
SLIDE 44
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-45
SLIDE 45
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-46
SLIDE 46
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-47
SLIDE 47
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-48
SLIDE 48
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-49
SLIDE 49
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-50
SLIDE 50
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-51
SLIDE 51
  • 1. A user can add new users

to the group.

  • 1. A user can add new users

to the group. The sentences in oldReq: The sentences in newReq: Similarity Score (S):

  • 2. The addition of a new

user must be first approved by the admin.

  • 3. The admin can modify

personal data and the status of a user.

  • 2. The admin and the user

can modify personal data of that user.

  • 3. Only user can modify

its status.

S = 1 S = 0.36 S = 0.47 Unchanged sentences

  • 4. The admin must be

logged-in in order to modify personal data of a user. Similarity threshold = 0.6

  • 1. Identify relevant change patterns – sentence level
slide-52
SLIDE 52
  • 1. Identify relevant change patterns – word level

1

  • The
  • DET
  • 2
  • det
  • 2
  • admin
  • NOUN
  • 4
  • nsubj

3

  • can
  • VERB
  • 4
  • aux

4

  • modify
  • VERB
  • ROOT

5

  • personal
  • ADJ
  • 6
  • amod

6

  • data
  • NOUN
  • 4
  • dobj

7

  • and
  • CONJ
  • 6
  • cc

8

  • the
  • DET
  • 9
  • det

9

  • status
  • NOUN
  • 6
  • conj

10

  • f
  • ADP
  • 9
  • prep

11

  • a
  • DET
  • 12
  • det

12

  • user
  • NOUN
  • 10
  • pobj

13

  • .
  • PUNCT
  • punct
  • ID
  • TEXT
  • WORD

CLASS

  • DEPENDENCY
  • GRAMMATICAL
  • NUMBER
  • FUNCTION

12

slide-53
SLIDE 53
  • 1. Identify relevant change patterns

1.3 Identify change types using an adapted TextDiff algorithm The admin <add>and</add> <add>the</add> <add>user</add> can modify personal data <del>and</del> <del>the</del> <del>status</del>

  • f <add>that</add>

<del>a</del> user.

13

slide-54
SLIDE 54
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

slide-55
SLIDE 55
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

slide-56
SLIDE 56
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

An object is deleted => Delete steps or their parts which verify that

slide-57
SLIDE 57
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

An object is deleted => Delete steps or their parts which verify that

1

  • The
  • DET
  • 2
  • det
  • 2
  • admin
  • NOUN
  • 4
  • nsubj

3

  • can
  • VERB
  • 4
  • aux

4

  • modify
  • VERB
  • ROOT

5

  • personal
  • ADJ
  • 6
  • amod

6

  • data
  • NOUN
  • 4
  • dobj

7

  • and
  • CONJ
  • 6
  • cc

8

  • the
  • DET
  • 9
  • det

9

  • status
  • NOUN
  • 6
  • conj

10

  • f
  • ADP
  • 9
  • prep

11

  • a
  • DET
  • 12
  • det

12

  • user
  • NOUN
  • 10
  • pobj

13

  • .
  • PUNCT
  • punct
  • ID
  • TEXT
  • WORD

CLASS

  • DEPENDENCY
  • GRAMMATICAL
  • NUMBER
  • FUNCTION
slide-58
SLIDE 58
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

An object is deleted => Delete steps or their parts which verify that

1

  • The
  • DET
  • 2
  • det
  • 2
  • admin
  • NOUN
  • 4
  • nsubj

3

  • can
  • VERB
  • 4
  • aux

4

  • modify
  • VERB
  • ROOT

5

  • personal
  • ADJ
  • 6
  • amod

6

  • data
  • NOUN
  • 4
  • dobj

7

  • and
  • CONJ
  • 6
  • cc

8

  • the
  • DET
  • 9
  • det

9

  • status
  • NOUN
  • 6
  • conj

10

  • f
  • ADP
  • 9
  • prep

11

  • a
  • DET
  • 12
  • det

12

  • user
  • NOUN
  • 10
  • pobj

13

  • .
  • PUNCT
  • punct
  • ID
  • TEXT
  • WORD

CLASS

  • DEPENDENCY
  • GRAMMATICAL
  • NUMBER
  • FUNCTION

=> the admin can modify the status of a user.

slide-59
SLIDE 59
  • 2. Generate guidance

2.1 Identify static and dynamic parts of a suggestion The admin and the user can modify personal data and the status of that a user.

14

An object is deleted => Delete steps or their parts which verify that

1

  • The
  • DET
  • 2
  • det
  • 2
  • admin
  • NOUN
  • 4
  • nsubj

3

  • can
  • VERB
  • 4
  • aux

4

  • modify
  • VERB
  • ROOT

5

  • personal
  • ADJ
  • 6
  • amod

6

  • data
  • NOUN
  • 4
  • dobj

7

  • and
  • CONJ
  • 6
  • cc

8

  • the
  • DET
  • 9
  • det

9

  • status
  • NOUN
  • 6
  • conj

10

  • f
  • ADP
  • 9
  • prep

11

  • a
  • DET
  • 12
  • det

12

  • user
  • NOUN
  • 10
  • pobj

13

  • .
  • PUNCT
  • punct
  • ID
  • TEXT
  • WORD

CLASS

  • DEPENDENCY
  • GRAMMATICAL
  • NUMBER
  • FUNCTION

=> the admin can modify the status of a user.

slide-60
SLIDE 60
  • 3. Notify subscribed parties

1. E-mail:

  • Affected test id and description
  • Relevant guidance
  • Summarized changes

2. Warnings

15

slide-61
SLIDE 61

Tool prototype

16

slide-62
SLIDE 62

Tool prototype

17

slide-63
SLIDE 63

Tool prototype

18

slide-64
SLIDE 64

Tool prototype

19

slide-65
SLIDE 65

Tool prototype

20

slide-66
SLIDE 66

Tool prototype

21

slide-67
SLIDE 67

Tool prototype

22

slide-68
SLIDE 68

Tool prototype

23

slide-69
SLIDE 69

Tool prototype

23

slide-70
SLIDE 70

Tool prototype

24

slide-71
SLIDE 71

Evaluation – study design

25

Dataset/Company Type of requirements Number of generated suggestions Number of experts

DS1/C1 User story 28 2 DS2/C2 User story 37 2 DS3/C3 Unstructured text 197 3

slide-72
SLIDE 72

Evaluation - results

26

slide-73
SLIDE 73

Evaluation - results

26

Correct actions

slide-74
SLIDE 74

Evaluation - results

26

Grammatically correct Correct actions

slide-75
SLIDE 75

Evaluation - results

26

Grammatically correct Complete Correct actions

slide-76
SLIDE 76

Evaluation - results

26

Grammatically correct Complete Understandable Correct actions

slide-77
SLIDE 77

Evaluation - results

26

Grammatically correct Complete Understandable Self-explanatory Correct actions

slide-78
SLIDE 78

Evaluation - results

26

Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Correct actions

slide-79
SLIDE 79

Evaluation - results

26

Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions

slide-80
SLIDE 80

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions

slide-81
SLIDE 81

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2)

slide-82
SLIDE 82

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2) Almost all (> 95%)

slide-83
SLIDE 83

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2) Almost all (> 95%) Almost all (> 95%)

slide-84
SLIDE 84

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2) Almost all (> 95%) Almost all (> 95%) ± 73%

slide-85
SLIDE 85

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2) Almost all (> 95%) Almost all (> 95%) ± 73% < 10% (DS1, DS2), ± 30%(DS3)

slide-86
SLIDE 86

Evaluation - results

26

> 80% (user stories), ±67% (unstructured) Grammatically correct Complete Understandable Self-explanatory Redundant/Unnecessary Missing Correct actions > 80% (DS1, DS3), 67.5% (DS2) Almost all (> 95%) Almost all (> 95%) ± 73% < 10% (DS1, DS2), ± 30%(DS3) < 6%

slide-87
SLIDE 87

Summary

27

slide-88
SLIDE 88

Summary

27

slide-89
SLIDE 89

Summary

27

slide-90
SLIDE 90

Summary

27

slide-91
SLIDE 91

Summary

27

slide-92
SLIDE 92

Summary

27

www.guidegen.org

slide-93
SLIDE 93

Summary

27

www.guidegen.org hotomski@ifi.uzh.ch