Fighting Layout Bugs Techniques to automatically verify the work of - - PowerPoint PPT Presentation

▶
fighting layout bugs
SMART_READER_LITE
LIVE PREVIEW

Fighting Layout Bugs Techniques to automatically verify the work of - - PowerPoint PPT Presentation

Mar 15, 2023 155 likes •1.13k views

Fighting Layout Bugs Techniques to automatically verify the work of HTML and CSS programmers QCon London 2010 Who am I? Michael Tamm System Architect Selenium committer Conference Speaker author of numerous JUnit tests ;)

slide-1
SLIDE 1

Fighting Layout Bugs

QCon London 2010 Techniques to automatically verify the work of HTML and CSS programmers

slide-2
SLIDE 2

2 / 96 Fighting Layout Bugs – QCon London 2010

Who am I?

Michael Tamm

 System Architect  Selenium committer  Conference Speaker  author of numerous

JUnit tests ;)

 big fan of IntelliJ IDEA,

Hamcrest, Mockito, FindBugs, ...

slide-3
SLIDE 3

3 / 96 Fighting Layout Bugs – QCon London 2010

You all know this ...

Presentation Layer Application Layer Business Logic Layer Persistence Layer

slide-4
SLIDE 4

4 / 96 Fighting Layout Bugs – QCon London 2010

Agenda

Technique #1 Continuous HTML validation Technique #2 Continuous CSS validation Technique #3 Detect typical layout bugs using JavaScript and image processing Technique #4 Automatically detect violations of the design rules of your website

slide-5
SLIDE 5

5 / 96 Fighting Layout Bugs – QCon London 2010

Agenda

Technique #1 Continuous HTML validation Technique #2 Continuous CSS validation Technique #3 Detect typical layout bugs using JavaScript and image processing Technique #4 Automatically detect violations of the design rules of your website

slide-6
SLIDE 6

6 / 96 Fighting Layout Bugs – QCon London 2010

Idea

Prevent layout bugs by always delivering valid HTML pages.

slide-7
SLIDE 7

7 / 96 Fighting Layout Bugs – QCon London 2010

How?

A Browser Extension?

 Only works in one browser.  Does not validate pages automatically.  Can not make the build fail

slide-8
SLIDE 8

8 / 96 Fighting Layout Bugs – QCon London 2010

How?

Scan files during the build process?

 Doesn't work for templates (*.jsp, *.vm, ...)

slide-9
SLIDE 9

9 / 96 Fighting Layout Bugs – QCon London 2010

How?

Validate HTML during your frontend tests!

 Option 1: A separate test, which validates

a well defined set of pages

 Option 2: Inside every frontend test,

each time you get a new page

 Should use the W3C Markup Validation Service

slide-10
SLIDE 10

10 / 96 Fighting Layout Bugs – QCon London 2010

W3C Markup Validation Service

slide-11
SLIDE 11

11 / 96 Fighting Layout Bugs – QCon London 2010

W3C Markup Validation Service

slide-12
SLIDE 12

12 / 96 Fighting Layout Bugs – QCon London 2010

W3C Markup Validation Service

slide-13
SLIDE 13

13 / 96 Fighting Layout Bugs – QCon London 2010

W3C Markup Validation Service

slide-14
SLIDE 14

14 / 96 Fighting Layout Bugs – QCon London 2010

How to fail faster than waiting for CI?

Use a javax.servlet.Filter!

 Works in all browsers.  Can validate HTML automatically.  Should not hamper daily work.  Should use the W3C Markup Validation Service too.

slide-15
SLIDE 15

15 / 96 Fighting Layout Bugs – QCon London 2010

The W3CMarkupValidationFilter class

How it works:

 Inside the doFilter method:

responseWrapper = new TeeHttpServletResponse(response); filterChain.doFilter(request, responseWrapper);

 The TeeHttpServletReponse writes the reponse

a) immediatly back to the browser and b) into an internal buffer.

 After doFilter returns, the HTML is fetched from, the

buffer and send to the W3C Markup Validation Service.

 A small JavaScript snippet is appended to the response

(after the closing </html> tag), which displays either a green or a red box, depending on the validation result.

slide-16
SLIDE 16

16 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

Demo

slide-17
SLIDE 17

17 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

slide-18
SLIDE 18

18 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

slide-19
SLIDE 19

19 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

slide-20
SLIDE 20

20 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

slide-21
SLIDE 21

21 / 96 Fighting Layout Bugs – QCon London 2010

W3CMarkupValidationFilter in action

slide-22
SLIDE 22

22 / 96 Fighting Layout Bugs – QCon London 2010

Some final thoughts

Try it out

http://w3c-markup-validation-filter.googlecode.com/

How to deal with ads?

 Do you need to integrate invalid HTML into

your web page provided from a third party?

 You can add a switch to your web application

to disable all ads.

 You can extend the W3CMarkupValidationFilter

and strip out those snippets, which should not be validated.

slide-23
SLIDE 23

23 / 96 Fighting Layout Bugs – QCon London 2010

Agenda

Technique #1 Continuous HTML validation Technique #2 Continuous CSS validation Technique #3 Detect typical layout bugs using JavaScript and image processing Technique #4 Automatically detect violations of the design rules of your website

slide-24
SLIDE 24

24 / 96 Fighting Layout Bugs – QCon London 2010

Idea

Prevent layout bugs by always delivering valid CSS.

slide-25
SLIDE 25

25 / 96 Fighting Layout Bugs – QCon London 2010

How?

A Browser Extension?

 Only works in one browser.  Does not validate pages automatically.  Can not make the build fail

slide-26
SLIDE 26

26 / 96 Fighting Layout Bugs – QCon London 2010

How?

Scan files during the build process?

 CSS is not only in *.css files, but also

in HTML (templates) inside <style> elements and/or style attributes

slide-27
SLIDE 27

27 / 96 Fighting Layout Bugs – QCon London 2010

How?

Scan files during the build process!

 CSS is not only in *.css files, but also

in HTML (templates) inside <style> elements and/or style attributes

 Nevertheless, why not validate the *.css files?

slide-28
SLIDE 28

28 / 96 Fighting Layout Bugs – QCon London 2010

CSS Validation Service

slide-29
SLIDE 29

29 / 96 Fighting Layout Bugs – QCon London 2010

How?

Should you use the CSS Validation Service?

 Complains about some CSS hacks, e.g.

... { *display: ... }

 Doesn't complain about:

* html ... { ... }

 Complains about unknown CSS properties / values, e.g.

... { -moz-border-radius: ... } ... { filter: progid:DXImageTransform.... } ... { ...: expression(...) }

slide-30
SLIDE 30

30 / 96 Fighting Layout Bugs – QCon London 2010

Example

slide-31
SLIDE 31

31 / 96 Fighting Layout Bugs – QCon London 2010

Example

slide-32
SLIDE 32

32 / 96 Fighting Layout Bugs – QCon London 2010

How?

Validate CSS during your frontend tests!

 This is another reasonable option.

A javax.servlet.Filter!

 Haven't done that yet,

but sounds like a nice idea to me.

slide-33
SLIDE 33

33 / 96 Fighting Layout Bugs – QCon London 2010

But!

Valid CSS is not good enough ...

slide-34
SLIDE 34

34 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: invalid URLs in the CSS

slide-35
SLIDE 35

35 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: HTML and CSS do not match

slide-36
SLIDE 36

36 / 96 Fighting Layout Bugs – QCon London 2010

Agenda

Technique #1 Continuous HTML validation Technique #2 Continuous CSS validation Technique #3 Detect typical layout bugs using JavaScript and image processing Technique #4 Automatically detect violations of the design rules of your website

slide-37
SLIDE 37

37 / 96 Fighting Layout Bugs – QCon London 2010

Idea

Detect layout bugs by simulating the human eye.

slide-38
SLIDE 38

38 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text truncated at vertical edge

slide-39
SLIDE 39

39 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text truncated at horizontal edge

slide-40
SLIDE 40

40 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text overlaps vertical edge

slide-41
SLIDE 41

41 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text overlaps vertical edge

slide-42
SLIDE 42

42 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: not enough space between text and edge

slide-43
SLIDE 43

43 / 96 Fighting Layout Bugs – QCon London 2010

How to automatically detect those layout bugs?

 We need to know, which pixels

belong to text.

 We need to know, which pixels

belong to a vertical or horizontal edge.

 If text pixels and edge pixels overlap,

we have found a layout bug.

 We need to somehow report

a found layout bug.

slide-44
SLIDE 44

44 / 96 Fighting Layout Bugs – QCon London 2010

1.) Detecting text

How the SimpleTextDetector works:

 Inject jQuery into currently displayed page

 jQuery('*').css('color', '#000000');

 Take screenshot

 jQuery('*').css('color', '#fffff');

 Take another screenshot  Compare the two screenshots:

All different pixels are probably text.

slide-45
SLIDE 45

45 / 96 Fighting Layout Bugs – QCon London 2010

SimpleTextDetector in action

slide-46
SLIDE 46

46 / 96 Fighting Layout Bugs – QCon London 2010

SimpleTextDetector in action

slide-47
SLIDE 47

47 / 96 Fighting Layout Bugs – QCon London 2010

SimpleTextDetector in action

slide-48
SLIDE 48

48 / 96 Fighting Layout Bugs – QCon London 2010

SimpleTextDetector in action

slide-49
SLIDE 49

49 / 96 Fighting Layout Bugs – QCon London 2010

SimpleTextDetector in action

slide-50
SLIDE 50

50 / 96 Fighting Layout Bugs – QCon London 2010

2.) Detecting vertical edges

How the SimpleEdgeDetector works:

 Inject jQuery into currently displayed page

 jQuery('*').css('color', 'transparent');

 Take screenshot  Find vertical pixel sequences of a certain minimal

length, where each pixel has the same color (or a very similar one)

 Only select those, which have a high contrast

to the left or right

 Only select those, which have a certain

minimal length

slide-51
SLIDE 51

51 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-52
SLIDE 52

52 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-53
SLIDE 53

53 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-54
SLIDE 54

54 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-55
SLIDE 55

55 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-56
SLIDE 56

56 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-57
SLIDE 57

57 / 96 Fighting Layout Bugs – QCon London 2010

SimpleEdgeDetector in action

slide-58
SLIDE 58

58 / 96 Fighting Layout Bugs – QCon London 2010

So, we have the text pixels ...

slide-59
SLIDE 59

59 / 96 Fighting Layout Bugs – QCon London 2010

… and the edge pixels

slide-60
SLIDE 60

60 / 96 Fighting Layout Bugs – QCon London 2010

3.) buggy pixels = text pixels && edge pixels

slide-61
SLIDE 61

61 / 96 Fighting Layout Bugs – QCon London 2010

3.) buggy pixels = text pixels && edge pixels

slide-62
SLIDE 62

62 / 96 Fighting Layout Bugs – QCon London 2010

4.) Report the found layout bug

How LayoutBug.markBuggyPixels works:

 Replace each buggy pixel by a circle

having a radius of 5 pixels – this leads to one or more buggy areas

 Find the outlines of the buggy areas  Blend those outlines as thick red lines

into the screenshot of the analyzed page

slide-63
SLIDE 63

63 / 96 Fighting Layout Bugs – QCon London 2010

LayoutBug.markBuggyPixels() in action

slide-64
SLIDE 64

64 / 96 Fighting Layout Bugs – QCon London 2010

LayoutBug.markBuggyPixels() in action

slide-65
SLIDE 65

65 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text is not readable because of low contrast

slide-66
SLIDE 66

66 / 96 Fighting Layout Bugs – QCon London 2010

How to automatically detect those layout bugs?

 We need to know, which pixels

belong to text. (Already solved ;)

 We need to know the background color around text.  We need to know the actual text color.

slide-67
SLIDE 67

67 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

What is it?

 A library for automatic detection

  • f layout bugs in web pages

 Open Source:

http://fighting-layout-bugs.googlecode.com/

 Implemented in Java  Uses a Firefox browser

remotely controlled by Selenium 2

slide-68
SLIDE 68

68 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

 What does it offer?

 DetectNeedsHorizontalScrolling

slide-69
SLIDE 69

69 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: unintended horizontal scroll bar

slide-70
SLIDE 70

70 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

 What does it offer?

 DetectNeedsHorizontalScrolling  DetectInvalidImageUrls

slide-71
SLIDE 71

71 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: Invalid references to image resources

 Where can those be located?

 In the src attribute of <img> tags  In the style attributes of all tags  In the text content of <style> elements  In linked CSS files  "/favicon.ico" or href attribute of

<link rel="icon" ...> or <link rel="shortcut icon" ...>

 What does invalid mean?

 <img> tags with no or empty src attribute  Malformed URL  HTTP GET returns 4xx or 5xx repsonse  Content-Type does not start with "image/"

slide-72
SLIDE 72

72 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

 What does it offer?

 DetectNeedsHorizontalScrolling  DetectInvalidImageUrls  DetectTextNearOrOverlappingHorizontalEdge  DetectTextNearOrOverlappingVerticalEdge  DetectTextWithTooLowContrast

slide-73
SLIDE 73

73 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

 How to get started?

slide-74
SLIDE 74

74 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library in action

Demo

slide-75
SLIDE 75

75 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library in action

slide-76
SLIDE 76

76 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library in action

slide-77
SLIDE 77

77 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library in action

slide-78
SLIDE 78

78 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library in action

slide-79
SLIDE 79

79 / 96 Fighting Layout Bugs – QCon London 2010

Outlook

More typical layout bugs ...

slide-80
SLIDE 80

80 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: different layout in different browsers

slide-81
SLIDE 81

81 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: different layout in different browsers

slide-82
SLIDE 82

82 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: unintended scrollbars for <div> or <iframe>

slide-83
SLIDE 83

83 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: broken borders

slide-84
SLIDE 84

84 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: text partly hidden

slide-85
SLIDE 85

85 / 96 Fighting Layout Bugs – QCon London 2010

Typical layout bug: Incorrect indentation

slide-86
SLIDE 86

86 / 96 Fighting Layout Bugs – QCon London 2010

Fighting Layout Bugs library

 Currently not much more than a proof of concept  Shall become something like FindBugs  FindBugs:

Finds typical programming bugs in Java classes

 Fighting Layout Bugs:

Finds typical layout bugs in web pages

 Contributions are welcome

http://fighting-layout-bugs.googlecode.com/ http://groups.google.com/group/fighting-layout-bugs

slide-87
SLIDE 87

87 / 96 Fighting Layout Bugs – QCon London 2010

Some final thoughts

When should the detectors be executed?

 Option 1: In a separate frontend test, which visits a

well defined set of pages or crawls your website

 Option 2: At certain points in your frontend test, e.g.

assertThat(currentPage, hasNoLayoutBugs());

 Option 3: Every time you get a new page

  • r click on something in your frontend tests

What about false alarms?

 Option 1: Configuration file  Option 2: Special HTML comments or attributes  Option 3: Tweak Fighting Layout Bugs

slide-88
SLIDE 88

88 / 96 Fighting Layout Bugs – QCon London 2010

Tweaking Fighting Layout Bugs

 Supress false alarms because of animation

by using AdvancedTextDetector

slide-89
SLIDE 89

89 / 96 Fighting Layout Bugs – QCon London 2010

Tweaking Fighting Layout Bugs

 Supress false alarms by modifying

default thresholds

slide-90
SLIDE 90

90 / 96 Fighting Layout Bugs – QCon London 2010

Agenda

Technique #1 Continuous HTML validation Technique #2 Continuous CSS validation Technique #3 Detect typical layout bugs using JavaScript and image processing Technique #4 Automatically detect violations of the design rules of your website

slide-91
SLIDE 91

91 / 96 Fighting Layout Bugs – QCon London 2010

Idea

Prevent layout bugs by treating your design as a nonfunctional requirement, for which you write tests.

slide-92
SLIDE 92

92 / 96 Fighting Layout Bugs – QCon London 2010

How to test a certain design?

 Transform your design in testable rules

 You have a grid?

Verify it's edges.

 You have reoccuring design elements?

Verify their presence and their location.

 Think TDBF (Test Driven Bug Fixing ;)

slide-93
SLIDE 93

93 / 96 Fighting Layout Bugs – QCon London 2010

Example: www.motor-talk.de

slide-94
SLIDE 94

94 / 96 Fighting Layout Bugs – QCon London 2010

Example: www.motor-talk.de

(1) Find top left corner of „Forum“ box

slide-95
SLIDE 95

95 / 96 Fighting Layout Bugs – QCon London 2010

Example: www.motor-talk.de

(1) Find top left corner of „Forum“ box (2) Find the other corners (3) Verify that all 4 corners have common x,y-coordinates

slide-96
SLIDE 96

96 / 96 Fighting Layout Bugs – QCon London 2010

The last slide

Thank You!

http://w3c-markup-validation-filter.googlecode.com/ http://fighting-layout-bugs.googlecode.com/ http://groups.google.com/group/fighting-layout-bugs