Hardware and JVM Design Trends? Those Dont Affect Me! Or Do They? - - PowerPoint PPT Presentation

▶
hardware and jvm design trends those don t affect me or
SMART_READER_LITE
LIVE PREVIEW

Hardware and JVM Design Trends? Those Dont Affect Me! Or Do They? - - PowerPoint PPT Presentation

Jun 30, 2023 190 likes •714 views

Hardware and JVM Design Trends? Those Dont Affect Me! Or Do They? Dan Heidinga Eclipse OpenJ9 Project Lead Interpreter Lead, IBM Runtimes @danheidinga DanHeidinga Important disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS

slide-1
SLIDE 1

Hardware and JVM Design Trends? Those Don’t Affect Me! Or Do They?

Dan Heidinga Eclipse OpenJ9 Project Lead Interpreter Lead, IBM Runtimes @danheidinga DanHeidinga

slide-2
SLIDE 2

Important disclaimers

§ THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. § WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION

CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

§ ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED

  • ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR

INFRASTRUCTURE DIFFERENCES.

§ ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. § IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT

PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.

§ IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT

OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

§ NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: – CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS

OR THEIR SUPPLIERS AND/OR LICENSORS

2

slide-3
SLIDE 3

Who am I?

§ One of the Project Leads for the Eclipse OpenJ9 JVM. § I've been involved with virtual machine development at IBM

since 2007 and lead IBM’s Interpreter team working on OpenJ9.

§ I've represented IBM on both the JSR 292 ('invokedynamic')

and JSR 335 ('lambda') expert groups and lead J9's implementation of both JSRs. And am now involved in Project Valhalla and Panama.

§ I’ve also maintain the bytecode verifier and deal with various

  • ther parts of the runtime.

3

slide-4
SLIDE 4

§ Moore’s law and singled threaded performance § The multicore explosion § Multi-tenant: bare metal, virtualization & containers § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

4

Trends

slide-5
SLIDE 5

§ Moore’s law and singled threaded performance § The multicore explosion § Multi-tenant: bare metal, virtualization & containers § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

5

Trends

slide-6
SLIDE 6

Moore’s Law

6

https://commons.wikimedia.org/wiki/File:Moore%27s_Law_Transistor_Count_1971-2016.png

§ The number of transistors doubles

every two years

§ Amazing single threaded

performance improvements

slide-7
SLIDE 7

Benchmark wars: Peak performance is #1

7

§ Cooperative suspend (1999) § Adaptive compilation (1999) § Escape analysis and stack allocating

  • bjects (2001)

§ Real Time Specification for Java (2005 -

2011)

§ Dynamic Ahead of Time compiled code

in production (2006)

§ Hot code replacement (2007) § Compressed references (2007) § Metronome soft real-time GC (2008)

Resource sharing? Not so much!

slide-8
SLIDE 8

Long lead times affect JVM design

Time between ordering and racking a server? Often 6+ months Buy the biggest server you can afford

9

slide-9
SLIDE 9

Long lead times affect JVM design

Time between ordering and racking a server? Often 6+ months Buy the biggest server you can afford

10

The JVM better use all of it!

slide-10
SLIDE 10

§ Moore’s law and singled threaded performance § The multicore explosion § Multi-tenant: bare metal, virtualization & containers § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

11

Trends

slide-11
SLIDE 11

Welcome to the multicore revolution

12

§ ~2006 RIP Moore’s Law? § Singled threaded performance no

longer growing by leaps and bounds

§ Welcome to the multicore

revolution

https://commons.wikimedia.org/wiki/File:Moore%27s_Law_Transistor_Count_1971-2016.png

slide-12
SLIDE 12

java.util.concurrent brings ForkJoin to the masses

13

§ ForkJoinPool brought the concept of work stealing to user code in Java 7 – The same technique that Java’s GC’s used to parallelize garbage

collection

§ Java 8 continued the trend towards parallelism with the introduction of

Lambda, ParallelStreams and Spliterators

§ The focus still on peak performance – but now it’s up to you (and the platform) to write good multithread code

slide-13
SLIDE 13

Common Characteristics

Deployments: infrequent Startup: Small fraction of up time Focus on peak performance

14

slide-14
SLIDE 14

Aside: Development and Debugging

15

https://xkcd.com/303/

slide-15
SLIDE 15

Trade peak performance for faster startup

  • Xquickstart (OpenJ9)
  • client (Hotspot)

16

slide-16
SLIDE 16

Adaptive JIT Compilation

17

§ Methods start out running bytecode form directly § After many invocations (or via sampling) code get compiled at ‘cold’ or ‘warm’ level § Low overhead sampling thread is used to identify hot methods § Methods may get recompiled at ‘hot’ or ‘scorching’ levels (for more

  • ptimizations)

§ Transition to ‘scorching’ goes through a temporary profiling step

cold hot scorching profiling interpreter warm

slide-17
SLIDE 17

§ Moore’s law and singled threaded performance § The multicore explosion § Multi-tenant: bare metal, virtualization & containers § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

18

Trends

slide-18
SLIDE 18

19

http://www.fanpop.com/clubs/home-improvement-tv-show/images/33144922/title/wilson-wallpaper

slide-19
SLIDE 19

Footprint and Startup

SharedClasses cache

20

slide-20
SLIDE 20

Footprint and Startup

21

Classfile ROMClass J9RAMClass

slide-21
SLIDE 21

ShareClasses: ROM pays off

22

JVM 1 JVM 2 JVM 3

slide-22
SLIDE 22

ShareClasses: ROM pays off

23

JVM 1 JVM 2 JVM 3

slide-23
SLIDE 23

ShareClasses: ROM pays off

24

JVM 1 JVM 2 JVM 3

Shared Classes Cache

Faster startup, Smaller footprint

slide-24
SLIDE 24

Balloon drivers

slide-25
SLIDE 25

–XX:+IdleTuningGcOnIdle -XX:+IdleTuningCompactOnIdle

https://developer.ibm.com/javasdk/2017/09/25/still-paying-unused-memory-java-app-idle/

“…around 30 percent of data center servers have not delivered information or computing services in the last six months.”

slide-26
SLIDE 26

Container Support

§ Container detection mechanism (-XX:+UseContainerSupport) – This allows JVM to tune itself based on limits imposed by the container. § Increase default heap size when running in container – JVM is typically the only application in the container – Reserve a larger chunk of container memory for heap § Better utilization of memory available to the container § -XX:MaxRAMPercentage and -XX:InitialRAMPercentage – Existing options for adjusting heap -Xmx/-Xms accept absolute values. – Cumbersome to adjust –Xmx/-Xms when container memory limit can vary – Adjust the heap based on percentage of the physical memory rather than an

absolute value.

§ Adjust Runtime.availableProcessors() based on cgroup limits § Add container limits to the Javacore file

slide-27
SLIDE 27

OpenJ9 JIT memory auto-scaling based on container limits

  • Docker container without swap space configured -> Exceed limit? Kaboom!
  • Docker container with swap space configured -> Exceed limit? Swap! Performance cost
  • JIT must be aware of container memory limits to avoid causing performance or functional

instability!

  • JIT compiler scratch memory usage is auto-scaled based on container memory limit
  • Reduce number of compilation threads as container limit is approached
  • Lower optimization level of compiled methods as container limit is approached
  • Different sized containers running same application on OpenJ9 could behave differently
  • Ramp up to peak throughput could be slower in smaller containers
  • Peak memory usage could be lower in smaller containers
slide-28
SLIDE 28

29

Trends

§ Moore’s law and singled threaded performance § The multicore explosion § Multi-tenant: bare metal, virtualization & containers § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

slide-29
SLIDE 29

The new world

30

slide-30
SLIDE 30

Continuous integration

31

slide-31
SLIDE 31

Continuous deployment

32

Checkout Compile Test Deploy

slide-32
SLIDE 32

Key idea of cloud

Making it faster and cheaper to get ideas into production with less risk

33

slide-33
SLIDE 33

Cloud: horizontal scaling

34

slide-34
SLIDE 34

Microservices

35

slide-35
SLIDE 35

Serverless / Function as a Service

https://github.com/apache/incubator-openwhisk/blob/master/docs/images/OpenWhisk.png

36

slide-36
SLIDE 36

New World Characteristics

Deployments: frequent, multiple times a day Startup: Larger fraction of uptime Lots of “first” runs: fewer opportunities to reuse work from previous runs

37

slide-37
SLIDE 37

The JVM needs to change

38

slide-38
SLIDE 38

“Dynamic” AOT through ShareClasses

39

Shared Classes Cache AOT ROM Classes

$ java –Xshareclasses ...

slide-39
SLIDE 39

Shared cache enablement

  • Enabled with –Xshareclasses
  • Population of the cache happens naturally & transparently at runtime
  • Applicable to boot, extension, & application loaders

& all URLClassloader-subclasses

  • Helper APIs to enable non-URLClassloaders (manual)
  • Disclaim the JIT and meta data after startup
  • Reduce footprint further
  • Debug data is only paged in as needed
slide-40
SLIDE 40

New AOT ramp up and throughput on Liberty Daytrader7

OpenJ9 default JIT vs –Xtune:virtualized with old AOT vs –Xtune:virtualized with new AOT

  • New AOT ramps up faster than the old AOT and stabilizes at higher throughput
  • New AOT cuts AOT vs JIT compiled code throughput delta roughly in half
slide-41
SLIDE 41

Jprofiling: Ramp up faster

§Improved mechanism for profiling compiled code. –Careful placement of counters –Conditionally patch code to enable / disable §Reduces the dependency on interpreter profiling. àGets into compiled code faster àDon’t sacrifice profiling quality

42

S I A C D B E F G H X

https://github.com/eclipse/openj9/blob/master/doc/compiler/JProfiling.md

slide-42
SLIDE 42

§ Moore’s law and singled threaded performance § The multicore explosion § Virtualization enters the arena § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

43

Trends

slide-43
SLIDE 43

Spectre & related security issues

44

https://upload.wikimedia.org/wikipedia/commons/thumb/2/25/Spectre_with_text.svg/702px-Spectre_with_text.svg.png

slide-44
SLIDE 44

§ Moore’s law and singled threaded performance § The multicore explosion § Virtualization enters the arena § The deployment strategy shift § Spectre: Caught passing bad checks § Cloud as an opportunity

45

Trends

slide-45
SLIDE 45

The JVM needs to keep changing

46

slide-46
SLIDE 46

JIT as a Service

47

App JIT App JIT App JIT App JIT

slide-47
SLIDE 47

JIT as a Service

48

App App App App JIT

slide-48
SLIDE 48

Take aways

§ The application life cycle has changed – no longer dominated by uptime. – Frequent restarts are the new reality. – Startup is now critical to your application! § The JVM is becoming a good neighbor – Intelligent resource usage matters… including giving them back § The free lunch is over – Expect to spend more and more time looking at resource usage, performance

and footprint.

49

slide-49
SLIDE 49

https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=openj9 https://adoptopenjdk.net/releases.html?variant=openjdk8&jvmVariant=openj9 https://github.com/eclipse/openj9 https://openj9.slack.com/

50

slide-50
SLIDE 50

Questions?

51

Dan Heidinga Eclipse OpenJ9 Project Lead Interpreter Lead, IBM Runtimes @danheidinga DanHeidinga

slide-51
SLIDE 51

52

Legal Notice

IBM and the IBM logo are trademarks or registered trademarks of IBM Corporation, in the United States, other countries or both. Java and all Java-based marks, among others, are trademarks or registered trademarks of Oracle in the United States, other countries or both. Other company, product and service names may be trademarks or service marks of others. THE INFORMATION DISCUSSED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, AND IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, SUCH INFORMATION. ANY INFORMATION CONCERNING IBM'S PRODUCT PLANS OR STRATEGY IS SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.