VXA : A Virtual Architecture for Durable Compressed Archives Bryan - - PowerPoint PPT Presentation
VXA : A Virtual Architecture for Durable Compressed Archives Bryan Ford Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology http://pdos.csail.mit.edu/~baford/vxa/ The Ubiquity of Data Compression
– Archive/Backup/Distribution: ZIP, tar.gz, ... – Multimedia streams: mp3, ogg, wmv, ... – Office documents: XML-in-ZIP – Digital cameras: JPEG, proprietary RAW, ... – Video camcorders: DV, MPEG-2, ...
— S Q — L U — A R C — Z O O — L H a r c — Z I P — b z i p 2 — g z i p — c
p r e s s — 7 z Lossless Compression — R A R 1980 1985 1990 1995 2000 2005
— S Q — L U — A R C — Z O O — L H a r c — Z I P — b z i p 2 — g z i p — c
p r e s s — 7 z Lossless Compression — R A R — M P E G
— D V — W M V 7 Video Encoding Audio Encoding — M P E G
— M P E G
— M P 3 — R e a l A u d i
A A C — F L A C — V
b i s — S
e n s
— Q u i c k T i m e — J P E G — G I F — P N G — J P E G 2 Image Encoding — T I F F — W M A 7 — W M A 9 — W M V 9 — W M V 8 — F L I C — W A V — A I F F — 8 S V X — A N I M — A I F F
— I L B M — P C X — T G A — B M P 1980 1985 1990 1995 2000 2005
– Inconvenient:
– Impedes data portability:
– Threatens long-term data usability:
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector) — 68000 — MIPS — ARM — PA-RISC — PowerPC — DEC Alpha — Itanium Other Architectures — SPARC
— 68000 — MIPS — ARM — PA-RISC — PowerPC — DEC Alpha — Itanium Other Architectures — SPARC
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
— 68000 — MIPS — ARM — PA-RISC — PowerPC — DEC Alpha — Itanium Other Architectures — SPARC
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
— 68000 — MIPS — ARM — PA-RISC — PowerPC — DEC Alpha — Itanium Other Architectures — SPARC
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
— 68000 — MIPS — ARM — PA-RISC — PowerPC — DEC Alpha — Itanium Other Architectures — SPARC
1980 1985 1990 1995 2000 2005 x86 Architecture — 8086 — 80386 (32-bit) — x86-64 (64-bit) — SSE (FP vector) — MMX (int vector)
Itanic
Make archive self-extracting (data + executable decoder) To extract data, archive reader runs embedded decoder
D D
Archive
Encoder Decoder
D D
Archive
Encoder Decoder
D
Emulator
D
Archive
Encoder Decoder
D
x86 Emulator
D
Archive
Encoder Decoder
D
Fast x86 Emulator
D
Archive
Encoder Decoder
D1
General Compressor Decoder1
D1
General Compressor Decoder1
D1 D2 D3
General Compressor Decoder1 Image Compressor Decoder2 Audio Compressor Decoder3
D1 D2 D3 General Compressor Image Compressor Audio Compressor Decoder1 Decoder2 Decoder3
D4 D5
Image Format Recognizer Decoder4 Audio Format Recognizer Decoder5 D1 D2 D3 General Compressor Image Compressor Audio Compressor Decoder1 Decoder2 Decoder3
x86 Emulator
D4 D5 D1 D2 D3
x86 Emulator Decoder1
D4 D5 D1 D2 D3
x86 Emulator Decoder1 Decoder2 Decoder3
D4 D5 D1 D2 D3
x86 Emulator
D4 D5 D1 D2 D3
Decoder1 Decoder2 Decoder3
x86 Emulator Decoder4 Decoder5
D4 D5 D1 D2 D3
Decoder1 Decoder2 Decoder3
Central Directory Audio file Audio file Image file
Central Directory Audio file FLAC Decoder Audio file Image file JP2 Decoder
Central Directory Audio file (FLAC-encoded) FLAC Decoder Audio file (FLAC-encoded) Image file (JP2-encoded) JP2 Decoder
Central Directory Audio file (FLAC-encoded) FLAC Decoder (deflated) Audio file (FLAC-encoded) Image file (JP2-encoded) JP2 Decoder (deflated)
– Can be written in any language, safe or unsafe – Compiled using ordinary tools (GCC)
– read stdin, write stdout, malloc, next file, exit
– open files, windows, devices, network connections, ... – get system info: user name, current time, OS type, ...
(up to 1GB) vxUnZIP Process Address Space
Decoder Address Space VXA System Calls vx32 Emulator library
– Secure fault isolation
– Data sandboxing via
– Code sandboxing via
– No privileges or
Code Rewriting
(up to 1GB) Flat-Model Code/Data Segment
Decoder Data Segment (LDT)
Controlled Procedure Calls
vx32 Emulator library Transformed code cache
– Portable but slow “fallback” instruction interpreter
– Fast x86-to-PowerPC binary translator
– Hopefully more in the future
– Can be used for other sandboxing applications
– not important for long-term archival storage, but... – very important for common short-term uses of archives:
– Athlon 64 3000+ PC running SuSE Linux 9.3 – Measure user-mode CPU time (not wall-clock time)
– Emulated vs native execution – Running on x86-32 vs x86-64 host environment
zlib bzip2 jpeg jp2 flac vorbis
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
native x86-32 vx32 on x86-32 native x86-64 vx32 on x86-64
Normalized User-mode Execution Time
zlib bzip2 jpeg jp2 flac vorbis
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
native x86-32 vx32 on x86-32 native x86-64 vx32 on x86-64
Normalized User-mode Execution Time
– Storage cost amortized over all files of same type – Relative overhead depends on size of archive
zlib bzip2 jpeg jp2 flac vorbis 10 20 30 40 50 60 70 80 90 100 110 120 130
Decoder C library
Compressed code size (KBytes)