Because you cant fix what you dont know is broken About me - - PowerPoint PPT Presentation

▶
because you can t fix what you don t know is broken about
SMART_READER_LITE
LIVE PREVIEW

Because you cant fix what you dont know is broken About me - - PowerPoint PPT Presentation

Sep 29, 2023 150 likes •814 views

Because you cant fix what you dont know is broken About me Initiator of the Automated Error Reporting Initiative Project Lead of Eclipse Code Recommenders Eclipse Committer since 2010 Plug-in Developer for 10+ years

slide-1
SLIDE 1

Because you can’t fix what you don’t know is broken…

slide-2
SLIDE 2

About me

2

  • Initiator of the Automated Error Reporting Initiative
  • Project Lead of Eclipse Code Recommenders
  • Eclipse Committer since 2010
  • Plug-in Developer for 10+ years (Eclipse 2.1)
  • Member of the Eclipse Architecture Council
  • Co-Lead of Java User Group Darmstadt
  • CEO of Codetrails
  • Passion to improve developers day-to-day work with

intelligent and practical tools - mostly using Data Mining on Big (Software Engineering) Data.

@marcelbruch marcelbruch

slide-3
SLIDE 3

Adding a new character to the plot

3

Software
 User

{ — — – }

Application
 Developer Product Manager

slide-4
SLIDE 4

Imagine the following situation…

4

Software
 User

{ — — – }

Application
 Developer Product Manager

slide-5
SLIDE 5

But when the user starts the app…

5

slide-6
SLIDE 6

6

Can you feel their pain…?

This image is not available in your country

slide-7
SLIDE 7

In the logs you’d find…

7

slide-8
SLIDE 8

✘ ✘ ✘ The tragedy of manual error reports

8

✘ ✘

!

maintenance

✘ ✘ ✘

Release SR 2 production SR 1

slide-9
SLIDE 9

✘

!

The tragedy of manual error reports

9

maintenance Release SR 2 production SR 1

✘ ✘ ✘

slide-10
SLIDE 10

10

Not too happy but feeling okay…

This image is not available in your country

slide-11
SLIDE 11

What can we do?

11

Application Developer

?

{ — — – }

slide-12
SLIDE 12

The Automated Error Reporting Initiative

12

slide-13
SLIDE 13

Live Demo

13

slide-14
SLIDE 14

About server responses,
 Bugzilla, and immediate feedback

„Keep users informed about your progress…“

14

slide-15
SLIDE 15

Systems Overview

15

AERI Web UI Eclipse Bugzilla Eclipse IDEs „Direct Reporter Feedback“

slide-16
SLIDE 16

Not yet tracked problems…

16

slide-17
SLIDE 17

Committers needinfo…

17

Status: – – – Resolution: – – –

Bug 442500 – ….

slide-18
SLIDE 18

Fixed…

18

Status: – – – Resolution: – – –

Bug 442500 – ….

slide-19
SLIDE 19

Won’t Fix…

19

Status: – – – Resolution: – – –

Bug 442500 – ….

slide-20
SLIDE 20

Invalid (aka log messages)

20

Status: – – – Resolution: – – –

Bug 442500 – ….

slide-21
SLIDE 21

About caching, filtering,
 anonymization, and other shortcuts…

„What goes to eclipse.org?“

21

slide-22
SLIDE 22

Do we send all errors to Eclipse…?

22

really

slide-23
SLIDE 23

The error reporting funnel

23

✘

structural filter

slide-24
SLIDE 24

#1: Structural Filtering

24

java.net.UnknownHostException: marketplace.eclipse.org at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) at java.net.Socket.connect(Socket.java:538) at com.example.HttpClient.openServer(HttpClient.java:527) at com.example.http.HttpClient.<init>(HttpClient.java:211) at com.example.HttpURLConnection.connect(HttpURLConnection.java:932) at org.eclipse.recommenders.internal.news.rcp.PollFeedJob.run(PollFeedJob.java:69) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) Error connecting to ‘http://marketplace.eclipse.org/rss.xml’.

slide-25
SLIDE 25

#2: Did I already sent this?

25

✘

structural filter

personal filter

slide-26
SLIDE 26

#3: Did anyone else sent this?

26

✘

structural filter

personal filter global hist.

slide-27
SLIDE 27

structural filter

personal filter

If you hit a known issue…

27

✘

global hist.

slide-28
SLIDE 28

#4: Review

28

review

✘

structural filter

personal filter global hist.

slide-29
SLIDE 29

#4: Review

29

slide-30
SLIDE 30

The server side

Walkthrough through the committer front-end

30

slide-31
SLIDE 31

The core concepts

31

slide-32
SLIDE 32

The Core Concepts

32

✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘

Incidents Problems Projects

slide-33
SLIDE 33

The committer front-end

33

slide-34
SLIDE 34

Creating bug reports

34

slide-35
SLIDE 35

Creating bug reports

35

slide-36
SLIDE 36

Notifying committers

36

slide-37
SLIDE 37

Project Alerts

37

slide-38
SLIDE 38

Weekly or daily email digests

38

slide-39
SLIDE 39

Some fun facts

e = mc2 —> errors = (more code)2

39

slide-40
SLIDE 40

Disclaimer

Don’t draw preliminary conclusions! Please consult your personal statistican…

slide-41
SLIDE 41

How many milestones testers?

450 simrel committers 1200 eclipse committers Testers? ~7000 error reporters

slide-42
SLIDE 42

Milestones adoption

42

RC3 RC2 RC1 M7 M6 M5 M4 M3 M2 M1 1000 220 160 650 450 400 560 600

Do new milestones introduce new problems?

70 10 # reporters one week high SR2 SR1 R Luna Mars

slide-43
SLIDE 43

350 700 1050 1400

4.4.0 4.4.1 4.4.2 M1 M2 M3 M4 M5 M6 M7 RC1 RC2 RC3

Milestones adoption

43

Christmas Addition to all EPP packages RCs Addition to first EPP packages

slide-44
SLIDE 44

How hard do they suffer?

44

300 - 3000 reports 25 testers 50-300 reports 330 testers 3-50 reports 3600 testers 1-2 reports 3000 testers

slide-45
SLIDE 45

What do they report?

45

750 1500 2250 3000

‚Normal‘ Errors Out of Memory Freezes Stack Overflows 3rd Party Involved Top 100 of 7000

Me

slide-46
SLIDE 46

What kind of errors are send?

46

3rd Party Involved 16 % Stack Overflows 0 % Freezes 9 % Out Of Memory 1 % 'Normal' Errors 74 %

slide-47
SLIDE 47

What are the most popular exceptions?

47

32 % 12 % 25 % 3 % 8 % 2 % 18 %

slide-48
SLIDE 48

What are the most popular exceptions?

48

Other 32 % No {Class | Method | Field} DefFoundError 12 % NullPointerException 25 % IndexOutOfBoundsException 3 % IllegalArgumentException 8 % ClassCastException 2 % {File} IOException 18 %

slide-49
SLIDE 49

NullPointerExceptions by Projects

49

1000 2000 3000 NullPointerException

Code Recommenders 65 Statisticians, note: Projects automatically marked as „being responsible“ fixing these NullPointerExceptions but does not mean the NPE happened in their code.

slide-50
SLIDE 50

Tip: Enable null analysis

50

@Override protected void run(@Nullable List<Incident> slice) { for (Incident incident : slice) {

slide-51
SLIDE 51

JDT offers a lot…

51

slide-52
SLIDE 52

Tip: Never return null

52

@Override public Optional<String> getReason() { return Optional.ofNullable(cause); }

Use java.util.Optional or com.google.common.base.Optional instead

slide-53
SLIDE 53

Oh Class, Were Art Thou?

53

Use proper import ranges and define uses on your exports to get rid of 
 {Class | Method | Field} DefNotFoundErrors, LinkageErrors, and some

  • ClassCastExceptions. OSGI needs that information to work properly…

Export-Package: org.eclipse.recommenders.utils; uses:="org.osgi.framework,

  • rg.eclipse.core.runtime,

com.google.common.base, com.google.common.collect" Import-Package: com.google.common.annotations;version="[15.0.0,16.0.0)",

slide-54
SLIDE 54

Index out of bounds?

54

a b c d e

  • 1 and array.length do not work. No matter how often you try…

There is no tool support for this. Just unit tests and code that tries to fail gracefully.

slide-55
SLIDE 55

Tip: If error messages are not helpful…

55

try { // … } catch (Exception e) { log(„An error occurred“, e); }

21% of all logged errors do not contain stack traces.

try { // … } catch (Exception e) { log(„Invalid argument {}“, value, e); }

slide-56
SLIDE 56

Are your users ready for Java 8?

56

both
 2,5%

Java 1.7 33%

Java 1,8
 64%

Java 1.9 0,3%

Which VM do users run the Eclipse IDE with?

slide-57
SLIDE 57

Do people anonymize their messages?

57

Sometimes 3 % No 81 % Yes 16 %

Are you concerned that your exception message may reveal sensitive data?

slide-58
SLIDE 58

Bugfixing Rates

Speak plain: Does it work?

58

slide-59
SLIDE 59

Bugzilla Summary

59

1170 bugs 275 fixed (24%) 100 invalid 70 not eclipse 467 in progress (40%) 172 duplicates (15%) 69 works 4 me 28 wont fix

Bugzilla

slide-60
SLIDE 60

Fixed bugs by projects

60

More (35) 84

  • 11

EGit 15 Papyrus 19 Recommenders 41 Xtext 42 Oomph 58

slide-61
SLIDE 61

In progress & fixed bugs

61 30 60 90 120 Oomph TMF Recommenders Papyrus EGit

  • 97

19 14 17 29 20 11 15 19 41 42 58 Fixed In Progress

slide-62
SLIDE 62

Future Work?

Feature requests are tracked in bugzilla. Make sure your request is in there and vote for it.

Client:

  • Separate view to „review, analyze and send later“
  • Making client reusable in other (Eclipse- or SWT-based) applications
  • Additional client-side automated analysis

Server:

  • Ensure scalability of the system
  • Improve duplicate detections
  • Go Neon if feedback is good.

Hosting an error reporting service for Java software?

  • Need one? TTML.

62

slide-63
SLIDE 63

How to reach EPP Logging

63

  • Code:

Check out the EPP Logging Oomph profile

  • Wiki

: https://wiki.eclipse.org/EPP/Logging

  • Manual:

https://dev.eclipse.org/recommenders/community/confess/#/about

  • Forum:

https://www.eclipse.org/forums/index.php/f/69/

  • Bugzilla: https://bugs.eclipse.org/bugs/buglist.cgi?component=logging&product=EPP
  • Mail:

error-reports-inbox@eclipse.org

slide-64
SLIDE 64

Credits

  • Daniel Haftstein, Codetrails – Eclipse client conception and implementation
  • Marcel Bruch, Codetrails – Server-side conception and implementation
  • Webmasters, Eclipse Foundation – Eclipse server infrastructure & integration
  • Thanks to many early adopters and committers providing feedback

64

slide-65
SLIDE 65

Feedback

65

slide-66
SLIDE 66

Contact Us

66

Codetrails GmbH Robert-Bosch-Str. 7 64293 Darmstadt, Germany www: codetrails.com phone: 06151 / 2767 092 email: info@codetrails.com

  • Dr. Marcel Bruch

CEO mobile: 0179 / 1317 721 email: marcel.bruch@codetrails.com