FileSender Terabyte Challenge Ren e Klomp Edwin Schaap System and - - PowerPoint PPT Presentation

▶

Jan 13, 2023 196 likes •447 views

FileSender Terabyte Challenge Ren e Klomp Edwin Schaap System and Network Engineering University of Amsterdam February 6, 2013 Ren e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 1 / 24 What is FileSender?

SLIDE 1

FileSender Terabyte Challenge

Ren´ e Klomp Edwin Schaap

System and Network Engineering University of Amsterdam

February 6, 2013

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 1 / 24

SLIDE 2

What is FileSender?

Meant to be easy for user and sysadmin No extra software requirements Default setup reaches only 150Mb/s

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 2 / 24

SLIDE 3

Research Question

Can we identify bottlenecks in the FileSender application and how can we improve the transfer speeds by reducing or removing these bottlenecks.

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 3 / 24

SLIDE 4

Environment

Tested and excluded the hardware as the bottleneck (except HDD)

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 4 / 24

SLIDE 5

The bottleneck

time (s) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 No delay time (s) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 50ms delay time (s) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 100ms delay time (s) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 150ms delay time (s) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 200ms delay

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 5 / 24

SLIDE 6

Client side implementation

Remove Gaps JavaScript webworkers Parallelisation Concurrent TCP streams (like GridFTP)

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 6 / 24

SLIDE 7

Server side implementation

Handle out-of-order chunks X-Start-Byte HTTP header File based queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 7 / 24

SLIDE 8

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 8 / 24

SLIDE 9

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 9 / 24

SLIDE 10

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 10 / 24

SLIDE 11

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 11 / 24

SLIDE 12

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 12 / 24

SLIDE 13

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 13 / 24

SLIDE 14

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 14 / 24

SLIDE 15

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 15 / 24

SLIDE 16

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 16 / 24

SLIDE 17

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 17 / 24

SLIDE 18

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 18 / 24

SLIDE 19

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 19 / 24

SLIDE 20

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 20 / 24

SLIDE 21

Server side implementation (Example)

Destination File Queue

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 21 / 24

SLIDE 22

SSL performance

Depends on cipher AES-NI

Mb/s CAMELLIA128 CAMELLIA256 AES128 AES256 RC4 3DES 1000 2000 3000 4000 5000 6000

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 22 / 24

SLIDE 23

Terabyte Challenge

Transferred in 3 hours and 19 minutes (@702Mb/s) Harddisk as bottleneck

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 23 / 24

SLIDE 24

Upload completed...

Questions?

made possible by

Ren´ e Klomp, Edwin Schaap (UvA) FileSender Terabyte Challenge February 6, 2013 24 / 24