Project Application Development Building an IJVM emulator Sebastian - - PowerPoint PPT Presentation

Project Application Development

Building an IJVM emulator Sebastian ¨ Osterlund

s.osterlund@vu.nl

Department of Computer Science Vrije Universiteit Amsterdam

June, 2020

Outline

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

Outline

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

Project Application Development

VUSec VUSec

Topics: Memory safety, Fuzz Testing, Microarchitectural Attacks (RIDL/ MDS).

Goal of this course

The main aim of this course is to give the student a more hands-on practical experience with programming. By implementing a larger project with a basis in material covered during the first year of the Computer Science bachelor, you will have the opportunity to gain more experience developing software.

The PAD team

Sebastian ¨ Osterlund (me) Atze van der Ploeg (lecturer) Andrea di Dio (Q&A sessions) Floris Gorter Kristian Laursen Ruby Bar-Ner Aron Heijstek Eduard Haivas Daniel K¨

• ves

Anthony Wilkes Dorukhan Arslan Frank de Jong Eduardo Lira Peter Kementzey

Support

https://discuss.ridl.eu Read the rules of the discussion board (do not post long code snippets, etc.)

Links

Material available at: http://vu-programming.gitlab.io/ pad/website/

Setup of the course

Fewer than four weeks (!!) for one deliverable. The deliverable is split into five smaller sub-modules, each building

• n the previous one.

Four meetings to discuss your progress with your Teaching Assistant. The deliverable is to be implemented individually. Finally, one last meeting to evaluate your submitted program (Monday June 29th). This evaluation can be scheduled at an earlier date on an individual basis. Deadline: Friday June 26th 23:59 CEST.

The assignment

tl;dr

Implement a subset of the Java Virtual Machine in C

(I)JVM?

An Instruction Set Architecture (ISA) consisting of a subset of the JVM ISA, created for educational purposes. Only operations on integers are considered Your task is to create a program that can execute IJVM binaries. I.e., implement a subset of the Java Virtual Machine.

The five modules

The course consists of 6 modules (5 mandatory), spread over 4 meetings

Module 4: Local variables and constant pool.

Module 6: (Additional features).

Grading

How is my submission graded?

40% based on basic tests (must pass all) 30% based on advanced tests 10% based on style and general impression (this includes the impression during the final evaluation meeting). A higher grade can be achieved by implementing additional functionality (see chapter 6 of the course manual). Pick-and-choose as you like. Capped at 30%.

!

WARNING: very high workload! (at least in the first week)

Additional info

Be well prepared for your first meeting! You should already be well on your way implementing the first task. Try to get the first three modules done as quickly as possible. The toughest part is getting everything to work correctly. Enrollment for the meetings opens today at 18:00 CEST. See Canvas. The manual is available right now! Start reading it today.

Additional info

There will be four Q&A sessions:

Today 2.06 at 13:30 Tuesday 9.06 at 13:30) Tuesday 16.06 at 13:30) Tuesday 23.06 at 13:30)

Support

https://discuss.ridl.eu Read the rules of the discussion board (do not post long code snippets, etc.)

Outline

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

Virtual machines

Virtual machines

Run an emulated/ virtual version of a computer system. ISA: Instruction Set Architecture

x86, x86-64, RISC-V, JVM, AARCH64

Virtual machines

Virtual machines

IJVM

IJVM: Integer Java Virtual Machine Subset of the JVM. Has 24 instructions. Created for educational purposes by Andy Tanenbaum Stack machine

Stack

LIFO datastructure Operations:

pop() top()/ peek() push() size()

Implemented using an array and a counter (stack pointer)

Stack machine

A (simple) machine operating on a stack RPN (Reverse Polish Notation) calculator 12+, 111++, 112-1++ ISA: +, -, *, 0-9

RPN example

RPN example

RPN example

RPN example

RPN example

RPN example

RPN example

RPN example

RPN example complete

RPN example

#define STACK_SIZE 1024

int stack[STACK_SIZE];

int sp;

int pop()

{

assert(sp >= 0);

return stack[sp--];

}

int top()

{

assert(sp >= 0);

return stack[sp];

}

void push(int n)

{

assert(sp < STACK_SIZE - 1);

stack[++sp] = n;

}

stack array sp stack pointer

RPN example

IJVM

Back to the IJVM

IJVM

Stack operations: BIPUSH, POP, IADD, ISUB . . . Branching: GOTO, IFEQ, IFICMPEQ, IFLT Memory: ILOAD, ISTORE, IINC, LDC W I/O: IN, OUT Method invocation: INVOKEVIRTUAL, IRETURN HALT, ERR In total 24 instructions, complete list in the manual

Decoding

BIPUSH 0x1)

0x0010, IINC 0x2 0x1)

Decoding

BIPUSH 0x1)

0x0010, IINC 0x2 0x1) 0x10 0x1?

Decoding

BIPUSH 0x1)

0x0010, IINC 0x2 0x1) 0x10 0x1? BIPUSH 0x1 0x84 0x1 0x5?

Decoding

BIPUSH 0x1)

0x0010, IINC 0x2 0x1) 0x10 0x1? BIPUSH 0x1 0x84 0x1 0x5? IINC 0x1 0x5

Decoding

Note: some arguments are signed while others are not

Signed vs. unsigned

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM memory model

pc program counter

incremented after each instruction modified by branching instructions

sp stack pointer memory consists of local frames

used for methods keeps a link to the previous frame

IJVM programs

IJVM executes bytecode Not really human-readable. . . Java ASsembly (JAS) We have written our custom assembler for this course (goJASM)

Assembler

Translates text into machine code Translates variable names into indices labels into offsets

JAS program

.main L1: BIPUSH 0xA // push 10 to stack L2: BIPUSH 0x1 ISUB // Subtract 1 DUP IFEQ END // Jump to end if zero BIPUSH 0x31 OUT // Print 1 GOTO L2 // Repeat loop END: HALT .end-main

IJVM program

./gojasm -o count.ijvm count.jas xxd count.ijvm

00000000: 1dea dfad 0001 0000 0000 0000 0000 0000 ................ 00000010: 0000 0010 100a 1001 6459 9900 0910 31fd ........dY....1. 00000020: a7ff f6ff ....

IJVM program

00000000: 1dea dfad 0001 0000 0000 0000 0000 0000 ................ 00000010: 0000 0010 100a 1001 6459 9900 0910 31fd ........dY....1. 00000020: a7ff f6ff ....

IJVM program

L1:

BIPUSH 0xA // push 10 to stack

L2:

BIPUSH 0x1

ISUB // Subtract 1

DUP

IFEQ END // Jump to end if zero

BIPUSH 0x31

OUT // Print 1

GOTO L2 // Repeat loop

END:

HALT 00000000: 1dea dfad 0001 0000 0000 0000 0000 0000 ................ 00000010: 0000 0010 100a 1001 6459 9900 0910 31fd ........dY....1. 00000020: a7ff f6ff ....

Recap

So far we have seen:

stack (machines) instruction decoding assembly branching

Methods

Last part: methods

Methods

re-usable subroutines for example add(a, b) also supports recursion last thing you have to implement (but can be

• challenging. . . )

.main BIPUSH 0x0 // OBJREF BIPUSH 0x2 BIPUSH 0x3 INVOKEVIRTUAL add BIPUSH 0x30 IADD OUT HALT .end-main .method add(a, b) ILOAD a ILOAD b IADD IRETURN .end-method

Invocation

Return

IJVM

IJVM: done!

Outline

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

Where are we?

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

History

Developed by Dennis Ritchie at Bell Labs 1960-1973 Used for developing UNIX No 2 on TIOBE programming language popularity index Still used for most systems programming nowadays!

Famous programs written in C

Operating systems kernels: Windows, OSX, Linux , Unix Wolfenstein 3D, DOOM, Quake (1/2/3) Git Most linux tools (ls,awk, etc) Embedded systems (your microwave?) Much much more

Hello World

C vs. C++

No classes No generics/ templates No references (only pointers) Much smaller standard library No operator overloading No namespaces No streams (cin/ cout) No exception handling No RAII

C vs. C++

Upside: more control and is (arguably) easier to master

C vs. C++

Note: C++ is a superset of C

This means that most C code compiles as C++

C is notorious for memory errors

Source of most (serious) errors and vulnerabilities!

(In fact Sebastian’s research is on memory safety)

Basics

#include <stdio.h> // Include standard I/O header // main is called on start, with the number of command // line arguments stored in argc, and the pointers to the // arguments in argv. argv is an array of char pointers. int main(int argc, char **argv) { char *binary_name = argv[0]; if (argc < 2) { printf("expected 2 command line arguments, got %d\n", argc - 1); return 1; // return status code 1 for error } char *name = argv[1]; char *code = argv[2]; printf("Hello %s, you entered the code %s\n", name, code) return 0; // return status code 0 for normal execution }

What we will cover

Warning

I will be very brief. Feel free to ask questions at any time!

Constants in C

Define constant as #define MINUTES_PER_HOUR 60 This is essentially a text replace by the preprocessor Cannot use C++ construct (const int MINUTES_PER_HOUR = 60;)

Type alias

define a type using: typedef existingtype newtype typedef int32_t word_t; Note: (u)intX t defined in stdint.h

Pointers

A memory item that references an address Reference using &. char *myptr; char mychar = 0x42; myptr = &mychar; myptr points to the address of mychar char *myptr = &mychar;. Can be confusing! View char * as the type. Dereference using *: *myptr == 0x42;

Arrays

A contiguous list of elements of a certain type Many ways to define:

int myarr[1024]; int myarr[] = {1,2,3}; int *myarr = malloc(1024);

Dereference an element:

First element: myarr[0]; n:th element: myarr[n];

Is essentially a pointer: myarr[42] is same as 42[myarr] is same as *(myarr + 42) No bounds checking!

Types of Memory management

Static memory mangement (global variables) Automatic memory management (stack) Manual memory management, manual allocation/ deallocation on the heap

Heap

Allocate using void *malloc(size_t n); (stdlib.h) Allocates n bytes of contiguous memory Returns a pointer Valid until void free(void *ptr) is called on that pointer

Pointer/ array

int *myArray = malloc(sizeof(int) * 5); free(myArray);

Strings

Unlike C++, C has no string type! A string is an array of characters, terminated by a NULL (’\0’) character. char mystr[] = {’h’, ’e’, ’l’, ’l’, ’o’, ’\0’}; char *mystr = "hello"; You can use it just like an array: mystr[2] == ’l’ Or point to a substring: char *mystr2 = &mystr[1]; // "ello" Built-in functions in C standard library (string.h): strdup(), strcat(), etc. Do not use ’==’ to compare strings! Use strcmp().

Structs

A composite data type (record) Physically groups a list of variables under one name struct tag_name { type member1; type member2; /* declare as many members as desired, * but the entire structure size must * be known to the compiler. */ }; struct tag_name mystruct; Access fields: mystruct.member1 If you have a pointer: mystructptr->member1

I/O

Writing:

printf("Hello\n "); fprintf(stderr, "OH NO!\n "); Read printf documentation! (man printf)

Reading:

char c = getc(stdin); fread(buf, sizeof(char), 1, stdin); scanf(...);

I/O

Raw access using fopen(),fread(),fwrite() #include <stdio.h> int main(void) { FILE *f = fopen("myfile.txt", "r"); char buf[10]; fread(buf, sizeof(char), 10, f); }

Using variables from other file

Access variable from other file (translation unit) using the extern keyword extern int mycounter; static variables cannot be accessed from other file (private)

Switch statement

switch (c) {

case ’+’:

a = pop();

b = pop();

push(a + b);

break;

case ’-’:

a = pop();

b = pop();

push(b - a);

break;

case ’*’:

a = pop();

b = pop();

push(a * b);

break;

case EOF:

// Terminate on CTRL-D

printf("= %d\n", top());

exit(0);

break;

case ’\n’:

printf("= %d\n", top());

break;

default:

push(c - ’0’); // Push ASCII character as integer

}

Code style

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” – Martin Fowler

Code style

Code should explain itself to reader Keep functions short (≈≤ 20 lines) Use meaningful names Do not repeat yourself (DRY) Avoid global variables if possible

What is wrong here?

void bad(int* a, int len_a, int* b, int len_b){

int sum_a = 0;

for(int i = 0 ; i < len_a; i++){

sum_a+= a[i];

}

int average_a = sum_a / len_a;

int sum_b = 0;

for(int i = 0 ; i < len_b; i++){

sum_b+= a[i];

}

int average_b = sum_b / len_b;

...

}

Better

double average(int* a, int len) {

double sum = 0;

for(int i = 0 ; i < len; i++){

sum+= a[i];

}

return sum / len;

}

void better(int* a, int len_a, int* b, int len_b){

double average_a = average(a,len_a);

double average_b = average(b,len_b);

...

}

If you are copy and pasting code, you are doing something wrong. Do not repeat yourself (DRY) Easier to read

What is wrong here?

void iadd(int* pc, word_t* stack, int* sp, bool* halted) {

...

}

void isub(int* pc, word_t* stack, int* sp, bool* halted) {

...

}

void bipush(byte a, int* pc, word_t* stack, int* sp, bool* halted) {

...

}

Better

typedef struct {

int pc;

word_t* stack;

int sp;

bool halted

} jvm_state;

void iadd(jvm_state* state) {

...

}

void isub(jvm_state* state) {

...

}

void bipush(byte a, jvm_state* state) {

...

}

If you are copy and pasting code, you are doing something wrong. Do not repeat yourself (DRY) Easier to read

What does this do?

#include <stdio.h>

void main(){

int f, c;

int l = 0;

int u = 300;

int s = 20;

for(f = l ; f <= u ; f+=s ) {

c = 5 * (f

• 32) / 9;

printf("%d\t%d\n",f,c);

}

}

Better

#include <stdio.h>

int fahr_to_celcius(int fahr) {

return 5 * (fahr - 32) / 9;

}

void print_fahrenheit_to_celcius_table() {

int fahr, celcius;

int lower_fahr = 0;

int upper_fahr = 300;

int step_fahr = 20;

for(fahr = lower_fahr; fahr <= upper_fahr; fahr+= step_fahr) {

printf("%d\t%d\n",fahr,fahr_to_celcius(fahr));

}

}

void main(){

print_fahrenheit_to_celcius_table();

}

A word about style

Use more restrictive compiler flags -Wall -Werror -Wpedantic

• Wextra. The compiler can prevent many common bugs by using

pedantic flags (see Makefile for complete list) Avoid memory leaks (test with valgrind and LLVM sanitizers)

Outline

1

Introduction

2

IJVM

3

The C Programming Language

4

Assignment

Overview

Skeleton

How to checkout skeleton

https://github.com/VU-Programming/pad-skeleton-c.git

How to build

make ./ijvm mybinary.ijvm

How to test

make test1 ./test1 make testbasic make testall

Skeleton

include: header files src: source files (work here) tests: automatic tests tools: other tools (goJASM)

Grading

How is my submission graded?

40% based on basic tests (must pass all) 30% based on advanced tests 10% based on style and general impression (this includes the impression during the final evaluation meeting). A higher grade can be achieved by implementing additional functionality (see chapter 6 of the course manual). Pick-and-choose as you like. Capped at 30%.

Tests

tests/testX.c: 5 basic tests (need to pass all) tests/testadvanced(X|stack).c: 20 advanced test cases tests/testbonusheap.c: test for heap additional functionality Example: pass all basic tests and all cases in testadvanced1,2,3,4,5 ⇒ 40% + 10

20 × 30% = 55%. If you then get 0% for style, you pass the

course with a 6

Useful tools

gdb: debugger xxd/ hexdump: print binaries in hex git: version control Editor/ IDE: vim, emacs, Atom, Visual Studio, CLion

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

Assignment recap

Networking, Heap, Debugger, Snapshoting, Hardening

!

Get started today!

If you have trouble getting started, there is a Q&A session later today

Plan of action

How do I start?

text size(), get text(), get program counter(). You can add these to src/machine.c

code on how to read files

better understanding of the binary format

I do not pass the test? What now?

I do not pass the test? What now?

You can debug the test using gdb ./test1 (Use lldb or see special instructions for macOS)

I do not pass the test? What now?

I do not pass the test? What now?

!!

Be prepared for your first meeting

We expect you to be mostly done with module 1

!!!

Let your TA know if you will be absent

If you don’t cancel, the TA is free to move your timeslot around

!!!!

But most of all have fun!

enjoy debugging memory errors. . .

Support

24/7 @

https://discuss.ridl.eu

VUSec VUSec

For info on (bachelor) projects, visit https://vusec.net