Automated Repair of Layout Cross Browser Issues Using Search-Based - - PowerPoint PPT Presentation

▶
automated repair of layout cross browser issues using
SMART_READER_LITE
LIVE PREVIEW

Automated Repair of Layout Cross Browser Issues Using Search-Based - - PowerPoint PPT Presentation

Nov 08, 2022 359 likes •989 views

Automated Repair of Layout Cross Browser Issues Using Search-Based Techniques Sonal Mahajan * , Abdulmajeed Alameer * , Phil McMinn + , William G. J. Halfond * * University of Southern California, USA + University of Sheffield, UK Work supported

slide-1
SLIDE 1

Automated Repair of Layout Cross Browser Issues Using Search-Based Techniques

Sonal Mahajan*, Abdulmajeed Alameer*, Phil McMinn+, William G. J. Halfond*

*University of Southern California, USA +University of Sheffield, UK

Work supported by NSF Grant CCF-1528163.

slide-2
SLIDE 2

2

Background Information

slide-3
SLIDE 3

2

Background Information

Cross Browser Issues (XBIs) — Inconsistencies in appearance or behavior across browsers.

Developers maintain consistent cross-browser user experience

C h a l l e n g i n g

End-users get a consistent cross-browser user experience

C

  • n

v e n i e n t

slide-4
SLIDE 4
  • 23,000 posts on
  • Appearance XBIs are prevalent — 90% [Choudhary et. al. ’13]
  • Reason for Occurrence

– Difference in browser implementations of HTML and CSS

Cross Browser Issues (XBIs)

3

Layout XBIs appear in 57% of the websites [Choudhary et. al. ‘13]

Layout XBIs — inconsistent layout of HTML elements in a web page across different browsers

slide-5
SLIDE 5

4

slide-6
SLIDE 6

4

slide-7
SLIDE 7

4

slide-8
SLIDE 8
  • 1. Detection is expensive

– Large number of browsers and platforms

  • 2. Localization is difficult

– Complex layouts and styles

Challenges in Repairing XBIs

5

  • 3. No standardized ways to repair

– Resolve XBIs on a case by case basis

  • 4. Repair must not introduce new XBIs

– Precisely modify problematic UI elements

slide-9
SLIDE 9
  • 1. Detection is expensive

– Large number of browsers and platforms

  • 2. Localization is difficult

– Complex layouts and styles

Challenges in Repairing XBIs

5

X-PERT [ICSE’13]

Choudhary et. al.

✔

  • 3. No standardized ways to repair

– Resolve XBIs on a case by case basis

  • 4. Repair must not introduce new XBIs

– Precisely modify problematic UI elements

slide-10
SLIDE 10
  • 1. Detection is expensive

– Large number of browsers and platforms

  • 2. Localization is difficult

– Complex layouts and styles

Challenges in Repairing XBIs

5

X-PERT [ICSE’13]

Choudhary et. al.

✔

  • 3. No standardized ways to repair

– Resolve XBIs on a case by case basis

  • 4. Repair must not introduce new XBIs

– Precisely modify problematic UI elements

Manual

slide-11
SLIDE 11
  • 1. Detection is expensive

– Large number of browsers and platforms

  • 2. Localization is difficult

– Complex layouts and styles

Challenges in Repairing XBIs

5

X-PERT [ICSE’13]

Choudhary et. al.

✔

  • 3. No standardized ways to repair

– Resolve XBIs on a case by case basis

  • 4. Repair must not introduce new XBIs

– Precisely modify problematic UI elements

Manual

Goal — Automatically find fixes that can repair the layout XBIs detected in a web page.

slide-12
SLIDE 12

Two Key Insights

6

  • 1. Guided search-based techniques can be used to

generate repairs.

</> HTML { } CSS

px values

Large repair search space Competing constraints

slide-13
SLIDE 13

Two Key Insights

7

  • 2. Impact of layout XBIs can be quantified by a fitness

function leveraging their visual manifestation.

Fitness function computes numeric closeness

  • f candidate

solutions to the required solution.

Number of XBIs detected (e.g., using X-PERT) Layout similarity (size and position of the bounding boxes of HTML elements)

XBIs

slide-14
SLIDE 14

Our Approach

8

XBIs

✗

✔

Page Under Test (PUT)

  • Ref. Browser

Test Browser Repaired Page (PUT’)

  • S1. Initial XBI

Detection

  • S2. Extract Root

Causes

  • S3. Search for

Candidate Fixes

  • S4. Search for the Best

Combination of Candidate Fixes

  • S5. Check

Termination Criteria

Y N

slide-15
SLIDE 15

<label, <e1, e2>>

Obtains set of XBIs when PUT is rendered in reference and test browsers

  • S1. Initial XBI Detection

9

XBIs X-PERT

[Choudhary et. al. ICSE’13]

XPaths of elements showing layout XBI Descriptor of correct layout position

<top-alignment, </html/body/div[3]/div/div, /html/body/div[3]/div>>

  • Ref. Browser

Test Browser

e1 e2

slide-16
SLIDE 16

Extracts the root causes relevant to each XBI

  • S2. Extract Root Causes

10 <e1, p1, v> <e2, p1, v> . . . < >

lement,

e

roperty,

p

alue

v

label1 p1, …, pn

. . .

labeln p1, …, pn

<label, <e1, e2>>

slide-17
SLIDE 17

</html/body/div[3]/div/div, margin-top, -20px> </html/body/div[3]/div/div, top, 0px> </html/body/div[3]/div, margin-top, 0px> </html/body/div[3]/div, top, 0px>

Extracts the root causes relevant to each XBI

  • S2. Extract Root Causes

10

top-alignment margin-top, top

. . .

<top-alignment, </html/body/div[3]/div/div, /html/body/div[3]/div>>

< >

lement,

e

roperty,

p

alue

v

slide-18
SLIDE 18

minimize layout deviation of e in reference and test browsers

Produces individual candidate fixes for each root cause

  • S3. Search for Candidate

Fixes

11 < >

lement,

e

roperty,

p

alue

v’

alue,

v

slide-19
SLIDE 19

Produces individual candidate fixes for each root cause

  • S3. Search for Candidate

Fixes

11

AVM

< >

lement,

e

roperty,

p

alue

v

< >

lement,

e

roperty,

p

alue

v’

alue,

v

Root cause Candidate fix Alternating Variable Method (AVM) Search

slide-20
SLIDE 20

Alternating Variable Method

[Korel 1990], [Kempka et. al. 2015] 12

AVM

✗

✔

start point

(v)

  • ptimal value

(v’)

Exploratory Moves

– Probe neighboring values to establish direction of fitness score improvement – Add small delta values [-1, 1] and observe impact on fitness

Pattern Moves

– Accelerate fitness score improvements in the established direction – Add delta values in exponentially increasing step sizes

✔

slide-21
SLIDE 21

Fitness Function

13

Quantify relative layout deviation of PUT rendered in reference and test browsers for every candidate v'

lement

e

eighbor

n

eighbor

n

eighbor

n

eighbor

n

  • 1. Δpos: Difference

in location

  • 2. Δsize: Difference in size
  • 3. Δnpos: Difference in

location of neighbors

Fitness score = (w1 * Δpos) + (w2 * Δsize) + (w3 * Δnpos)

slide-22
SLIDE 22
  • 1. Δpos: Difference in Location

14

lement

e

lement

e

Reference browser Test browser

slide-23
SLIDE 23
  • 1. Δpos: Difference in Location

14

lement

e

lement

e

Reference browser Test browser

( ) ( ) ( ) ( )

slide-24
SLIDE 24
  • 1. Δpos: Difference in Location

14 Δpos = DTL + DBR

slide-25
SLIDE 25
  • 1. Δpos: Difference in Location

14 Δpos = DTL + DBR Δpos decreases as the boxes move closer

slide-26
SLIDE 26
  • 2. Δsize: Difference in Size

15

lement

e

lement

e

Reference browser Test browser

slide-27
SLIDE 27
  • 2. Δsize: Difference in Size

15 Δsize = |WR - WT| + |HR - HT|

WR WT HR HT

slide-28
SLIDE 28
  • 2. Δsize: Difference in Size

15 Δsize = |WR - WT| + |HR - HT| Δsize decreases as the boxes become similar in size

WR WT HR HT

slide-29
SLIDE 29

16

  • 3. Δnpos: Difference in Location
  • f Neighbors

lement

e

Neighbors — elements that are at a distance of N hops from e in the HTML DOM tree

  • f the PUT
slide-30
SLIDE 30

17 1 1 1

  • 3. Δnpos: Difference in Location
  • f Neighbors

lement

e

eighbor

n

Neighbors — elements that are at a distance of N hops from e in the HTML DOM tree

  • f the PUT

N = 1

slide-31
SLIDE 31

18 1 1 1 1 1 1

  • 3. Δnpos: Difference in Location
  • f Neighbors

1

lement

e

eighbor

n

Neighbors — elements that are at a distance of N hops from e in the HTML DOM tree

  • f the PUT

N = 2

slide-32
SLIDE 32
  • 3. Δnpos: Difference in Location
  • f Neighbors

19

lement

e

lement

e

Reference browser Test browser

eighbor

n

eighbor

n

slide-33
SLIDE 33
  • 3. Δnpos: Difference in Location
  • f Neighbors

19 Δnpos = DTL + DBR

lement

e

eighbor

n

slide-34
SLIDE 34
  • 3. Δnpos: Difference in Location
  • f Neighbors

19 Δnpos = DTL + DBR

lement

e

eighbor

n

Δnpos decreases as e’s boxes move closer, causing n’s boxes to also move closer

slide-35
SLIDE 35

Fitness score = (w1 * Δpos) + (w2 * Δsize) + (w3 * Δnpos) = (1 * 40) + (2 * 0) + (0.5 * 40) = 60

20

</html/body/div[3]/div/div, margin-top, -20px>

AVM

Root causes

Running Example — S3

e

n

(10, 0) (10, -20) (10, 100) (10, 80)

DTL = DBR = 20

Reference browser Test browser

margin-top = -20px

slide-36
SLIDE 36

Fitness score = (w1 * Δpos) + (w2 * Δsize) + (w3 * Δnpos) = (1 * 40 10) + (2 * 0) + (0.5 * 40 10) = 60 15

20

</html/body/div[3]/div/div, margin-top, -20px>

AVM

</html/body/div[3]/div/div, margin-top, -20px, 20px>

Root causes Candidate fixes

Running Example — S3

e

n

(10, 0) (10, 100) Reference browser Test browser

margin-top = -20px 15px

(10, -5) (10, 95)

slide-37
SLIDE 37

Fitness score = (w1 * Δpos) + (w2 * Δsize) + (w3 * Δnpos) = (1 * 40 10) + (2 * 0) + (0.5 * 40 10) = 60 15

20

</html/body/div[3]/div/div, margin-top, -20px> </html/body/div[3]/div/div, top, 0px> </html/body/div[3]/div, margin-top, 0px> </html/body/div[3]/div, top, 0px>

AVM

</html/body/div[3]/div/div, margin-top, -20px, 20px> </html/body/div[3]/div/div, top, 0px, 20px>

Root causes Candidate fixes

Running Example — S3

e

n

(10, 0) (10, 100) Reference browser Test browser

margin-top = -20px 15px

(10, -5) (10, 95)

slide-38
SLIDE 38

Finds subset of candidate fixes representing the best

  • verall repair

– Combine candidate fixes to fully resolve an XBI – Interaction of fixes may have duplicating, multiplying, or competing effects

  • S4. Search for the Best

Combination of Candidate Fixes

21

</html/body/div[3]/div/div, margin-top, -20px, 20px> </html/body/div[3]/div/div, top, 0px, 20px>

Candidate fixes

OR

✔

slide-39
SLIDE 39

Finds subset of candidate fixes representing the best

  • verall repair

– Combine candidate fixes to fully resolve an XBI – Interaction of fixes may have duplicating, multiplying, or competing effects

  • S4. Search for the Best

Combination of Candidate Fixes

21

</html/body/div[3]/div/div, margin-top, -20px, 20px> </html/body/div[3]/div/div, top, 0px, 20px>

Candidate fixes

AND

✗

slide-40
SLIDE 40

Finds subset of candidate fixes representing the best

  • verall repair
  • S4. Search for the Best

Combination of Candidate Fixes

22

Biased Random Search

< >

lement,

e

roperty,

p

alue

v’

alue,

v

< >

lement,

e

roperty,

p

alue

v’

alue,

v

< >

lement,

e

roperty,

p

alue

v’

alue,

v

. . .

< >

lement,

e

roperty,

p

alue

v’

alue,

v

< >

lement,

e

roperty,

p

alue

v’

alue,

v

. . .

Set of candidate fixes Subset of candidate fixes giving the best repair

slide-41
SLIDE 41

Biased Random Search

23

  • Selection

– A candidate fix is selected with a probability

improvement in fitness in S3 maximum improvement in S3 across all fixes

slide-42
SLIDE 42

Biased Random Search

23

  • Selection

– A candidate fix is selected with a probability

  • Fitness Function

– Minimizing goal — number of XBIs

  • Termination Conditions
  • 1. All XBIs are fixed
  • 2. Maximum threshold of repairs to be tried has been reached
  • 3. A sequence of repairs with no fitness improvement

X-PERT

[Choudhary et. al. ICSE’13]

slide-43
SLIDE 43

Determines whether to terminate or proceed to S2 for another iteration. Approach terminates if:

  • 1. All XBIs resolved
  • 2. New XBIs same as previous iteration XBIs
  • 3. New number of XBIs more than previous iteration
  • S5. Check Termination Criteria

24

slide-44
SLIDE 44

Produce Repaired Page

25

✔

  • 1. @-moz-document url-prefix() {
  • 2. html > body > div:nth-of-type(3) > div:nth-of-type(1) > div:nth-of-type(1) {

3. margin-top: 1.7% !important; /* 20px */ 4. }

  • 5. }

Generate test browser specific CSS repair patch

CSS repair patch

</html/body/div[3]/div/div, margin-top, -20px, 20px>

Repair

Browser specific CSS qualifier Directs layout engine to use this code if browser is Firefox

slide-45
SLIDE 45

Produce Repaired Page

25

✔

  • 1. @-moz-document url-prefix() {
  • 2. html > body > div:nth-of-type(3) > div:nth-of-type(1) > div:nth-of-type(1) {

3. margin-top: 1.7% !important; /* 20px */ 4. }

  • 5. }

Generate test browser specific CSS repair patch

CSS repair patch

</html/body/div[3]/div/div, margin-top, -20px, 20px>

Repair

XPath is converted to CSS selector

slide-46
SLIDE 46

Produce Repaired Page

25

✔

  • 1. @-moz-document url-prefix() {
  • 2. html > body > div:nth-of-type(3) > div:nth-of-type(1) > div:nth-of-type(1) {

3. margin-top: 1.7% !important; /* 20px */ 4. }

  • 5. }

Generate test browser specific CSS repair patch

CSS repair patch

</html/body/div[3]/div/div, margin-top, -20px, 20px>

Repair

Convert to relative fix value

slide-47
SLIDE 47
  • RQ1: How effective is our approach at reducing layout

XBIs?

  • RQ2: What is the impact on the cross-browser

consistency of the page when the suggested repairs are applied?

  • RQ3: How long does our approach take to find repairs?

Empirical Evaluation

26

slide-48
SLIDE 48
  • Tool

– Approach implemented in

  • Methodology

– For each subject

  • Select reference and test browsers
  • Run XFix 30 times to mitigate non-determinism in search

– Human study for impact on cross-browser consistency

Experimental Protocol

27 RQ1 and RQ3 RQ2 v51.0 v11.0.33 v46.0.1

slide-49
SLIDE 49

Subjects

28 Name #HTML #CSS Ref Test

BenjaminLees 317 1,525 CH FF Bitcoin 207 1,957 FF IE Eboss 439 789 IE FF EquilibriumFans 340 868 CH FF GrantaBooks 325 6,545 FF IE HenryCountyOhio 300 983 IE FF HotwireHotel 1,457 10,618 FF IE IncredibleIndia 251 2,172 IE FF Leris 195 1,262 FF CH Minix3 118 821 IE CH Newark 598 17,426 FF IE Ofa 578 5,381 IE CH PMA 497 13,743 FF IE StephenHunt 497 13,743 FF IE WIT 300 3,249 FF IE

slide-50
SLIDE 50
  • RQ1. Reduction of XBIs

29

Reduction (%) in X-PERT reported XBIs before and after repair

slide-51
SLIDE 51
  • RQ1. Reduction of XBIs

29

ReducYon (%)

25 50 75 100

BenjaminLees Bitcoin Eboss EquilibriumFans GrantaBooks HenryCountyOhio HotwireHotel IncredibleIndia Leris Minix3 Newark Ofa PMA StephenHunt WIT

Mean = 86% Median = 93%

slide-52
SLIDE 52
  • RQ1. Reduction of XBIs

29

ReducYon (%)

25 50 75 100

BenjaminLees Bitcoin Eboss EquilibriumFans GrantaBooks HenryCountyOhio HotwireHotel IncredibleIndia Leris Minix3 Newark Ofa PMA StephenHunt WIT

Mean = 86% Median = 93% Reduction in X-PERT reported XBIs

Remaining XBIs? Hypothesis: XBIs reported due to pixel-level differences,

would not be human perceptible.

Answer: Confirmed. Mean 99% (median 100%) reduction in

human-observable XBIs.

slide-53
SLIDE 53
  • RQ2. Impact on Cross-Browser

Consistency

30

Similarity (consistency) ratings given by participants of the human study

. . .

slide-54
SLIDE 54
  • RQ2. Impact on Cross-Browser

Consistency

30

Users

1 2 3 4 5 6 7 8 9 10 11

BenjaminLees Bitcoin Eboss EquilibriumFans GrantaBooks HenryCountyOhio HotwireHotel IncredibleIndia Leris Minix3 Newark Ofa PMA StephenHunt WIT

improved same decreased Improved = 78% Same = 14% Decreased = 8%

slide-55
SLIDE 55
  • RQ2. Impact on Cross-Browser

Consistency

30

Users

1 2 3 4 5 6 7 8 9 10 11

BenjaminLees Bitcoin Eboss EquilibriumFans GrantaBooks HenryCountyOhio HotwireHotel IncredibleIndia Leris Minix3 Newark Ofa PMA StephenHunt WIT

improved same decreased Improved = 78% Same = 14% Decreased = 8%

High discordance? Inherent browser-level differences Text intensive and contain specific fonts rendering differently in browsers

Consistency judgement likely influenced by such differences.

slide-56
SLIDE 56

Total running time of XFix

  • RQ3. Time Needed to Run XFix

31

slide-57
SLIDE 57
  • RQ3. Time Needed to Run XFix

31

43 sec 2 hours

Median = 14 min S4: Search for the Best Combination of Candidate Fixes S3: Search for Candidate Fixes Other: S1, S2, and S5

slide-58
SLIDE 58

Artifact

Open source tool release Evaluation data

  • Subjects
  • Human study documents

32

https://github.com/sonalmahajan/xfix

slide-59
SLIDE 59

Summary

33

slide-60
SLIDE 60

Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, William G. J. Halfond spmahaja@usc.edu alameer@usc.edu p.mcminn@sheffield.ac.uk halfond@usc.edu

Work supported by NSF Grant CCF-1528163

Thank you

Automated Repair of Layout Cross Browser Issues Using Search-Based Techniques

slide-61
SLIDE 61
  • RQ4. Similarity of Repair Patches to

Real-world Websites’ Code

35

Analyzed 480 Alexa websites

slide-62
SLIDE 62
  • RQ4. Similarity of Repair Patches to

Real-world Websites’ Code

35

Browser specific code present in 80% websites

P r e v a l e n t

Size of browser specific code 3 7 10 13 FF IE CH

Avg = 9

XFix generates realistic repair patches

C

  • m

p a r a b l e

@-moz-document url-prefix() { html > body > div:nth-of-type(3) { height: 90% !important; margin-top: 1% !important; } }

size = 2