Metascala A tiny DIY JVM https://github.com/lihaoyi/Metascala Li - - PowerPoint PPT Presentation

▶
metascala
SMART_READER_LITE
LIVE PREVIEW

Metascala A tiny DIY JVM https://github.com/lihaoyi/Metascala Li - - PowerPoint PPT Presentation

Nov 04, 2022 251 likes •596 views

Metascala A tiny DIY JVM https://github.com/lihaoyi/Metascala Li Haoyi haoyi@dropbox.com Scala Exchange 2nd Dec 2013 Who am I? Li Haoyi Write Python during the day Write Scala at night What is Metascala? A JVM in 3000 lines of

slide-1
SLIDE 1

Metascala

A tiny DIY JVM

https://github.com/lihaoyi/Metascala

Li Haoyi haoyi@dropbox.com Scala Exchange 2nd Dec 2013

slide-2
SLIDE 2

Who am I?

Li Haoyi Write Python during the day Write Scala at night

slide-3
SLIDE 3

What is Metascala?

  • A JVM
  • in 3000 lines of Scala
  • Which can load & interpret java programs
  • And can interpret itself!
slide-4
SLIDE 4

Size comparison

  • Metascala:

~3,000 lines

  • Avian JVM: ~80,000 lines
  • OpenJDK:

~1,000,000 lines

slide-5
SLIDE 5

Basic Usage

Closure’s class file is given to VM to load/parse/execute Result is extracted from VM into host environment Captured variables are serialized into VM’s environment Create a new metascala VM Plain Old Java Object Any other classes necessary to evaluate the closure are loaded from the current Classpath No global state

slide-6
SLIDE 6

It’s Metacircular!

Need to give the outer VM more than the 1mb default heap Simpler program avoids initializing the scala/java std libraries, which takes forever under double-interpretation. Takes a while (~10s) to produce result VM inside a VM!

slide-7
SLIDE 7

Limitations

  • Single-threaded
  • Limited IO
  • Slowww
slide-8
SLIDE 8

Performance Comparison

  • OpenJDK:

1.0x

  • Metascala:

~100x

  • Meta-Metascala: ~10000x
slide-9
SLIDE 9

Why Metascala?

  • Fun to explore the innards of the JVM
  • An almost-fully secure Java runtime!
  • Small size makes fiddling fun
slide-10
SLIDE 10

Why Metascala?

  • Fun to explore the innards of the JVM
  • An almost-fully secure Java runtime!
  • Small size makes fiddling fun
slide-11
SLIDE 11

Quick Tour

Immutable Defs: ~380 loc Native Bindings: ~650 loc Bytecode SSA transform: ~650 loc Runtime data structures: 820 loc Binary heap & Copying GC: 132 loc DIY scala-pickling: 132 loc “This is a VM”: 243 loc

slide-12
SLIDE 12

Quick Tour: Tests

Tests for basic Java features GC Fuzz-tests Test Metacircularity! Scala std lib usage

slide-13
SLIDE 13

What’s a Heap?

Fig 1. A Heap

slide-14
SLIDE 14

What’s a Garbage Collector?

Blit (copy) all roots to new heap Scan the already copied things for more things and copy them too Stop when you’ve scanned everything Not pseudocode

slide-15
SLIDE 15

Why Metascala?

  • Fun to explore the innards of the JVM
  • An almost-fully secure Java runtime!
  • Small size makes fiddling fun
slide-16
SLIDE 16

Limited Instruction Count

slide-17
SLIDE 17

And Limited Memory!

Not an OOM Error! We throw this ourselves

slide-18
SLIDE 18

Explicitly defined capabilities

Every external call has to be explicitly defined and enabled

slide-19
SLIDE 19

Security Characteristics

  • Finite instruction count
  • Finite memory
  • Well-defined interface to outside world
  • Doesn’t rely on Java Security Model at all!
  • Still some holes…
slide-20
SLIDE 20

Security Holes

  • Classloader can read from anywhere
  • Time spent classloading not accounted
  • Memory spent on classes not accounted
  • GC time not accounted
  • “native” methods’ time/memory not

accounted

slide-21
SLIDE 21

Basic Problem

User code resource consumption is bounded VM’s runtime resource usage can be made to grow arbitrarily large

User Code Classes Runtime Data Structures Native method calls Garbage Collector Outside World

slide-22
SLIDE 22

Possible Solution

Put a VM Inside a VM! Works, ... but 10000x slowdown

Outside World

User Code Classes Runtime Data Structures Native method calls Garbage Collector Outside World Fixed Unaccounted Costs

slide-23
SLIDE 23

Another Possible Solution

Move more components into virtual runtime Difficult to bootstrap correctly WIP

Outside World User Code Classes Runtime Data Structures Native method calls Garbage Collector

slide-24
SLIDE 24

Why Metascala?

  • Fun to explore the innards of the JVM
  • An almost-fully secure Java runtime!
  • Small size makes fiddling fun
slide-25
SLIDE 25

Live Demo

slide-26
SLIDE 26

Ugliness

  • Long compile times
  • Nasty JVM Interface
  • Impossible Debugging
slide-27
SLIDE 27

Long compile times

  • 100 lines/s
  • Twice as slow (50 lines/s) on my older

machine!

slide-28
SLIDE 28

Real World

Nasty JVM Interface

Ideal World

User Code Std Library VM Initialized Std Library User Code VM Initialized Nasty Language VM Interface Lazy- Initialization means repeated dives back into lib/native code Clean Interfaces Linear Initialization

slide-29
SLIDE 29

Java’s dirty little secret

WTF! I’d never use these things! The Verbosity of Java with the Safety of C

slide-30
SLIDE 30

You probably do

Almost every Java program ever uses these things. What happens if you don’t have them

slide-31
SLIDE 31

Next Steps

  • Maximize correctness

○ Implement Threads & IO ○ Fix bugs (GC, native calls, etc.)

  • Solidify security characteristics

○ Still plenty of unaccounted-for memory/processing ○ Some can be hosted “within” VM itself

  • Simplify Std-Lib/VM interface

○ Try using Android Std Lib?

slide-32
SLIDE 32

Possible Experiments

  • Native codegen instead of an interpreter?

○ Generate/exec native code through JNI ○ Heap is already a binary blob that can be easily passed to native code

  • Bytecode transforms and optimizations?

○ Already in SSA form

  • Continuations, Isolates, Value Classes?
  • Port the whole thing to Scala.Js?
slide-33
SLIDE 33

Metascala: a tiny DIY JVM

Ask me about:

  • Single Static Assignment form
  • Copying Garbage Collection
  • sun.misc.Unsafe
  • Warts of the .class file format
  • Capabilities-based security
  • Abandoned approaches