Performance measurement and tuning of remote acquisition Lukasz - - PowerPoint PPT Presentation

▶
performance measurement and tuning of remote acquisition
SMART_READER_LITE
LIVE PREVIEW

Performance measurement and tuning of remote acquisition Lukasz - - PowerPoint PPT Presentation

Jan 22, 2023 249 likes •1.1k views

Performance measurement and tuning of remote acquisition Lukasz Makowski February 2, 2016 Location Netherlands Forensic Institute Supervisor : Ruud Schramp Agenda 1 Remote acquisition - research motivation introduction 2 Research scope and

slide-1
SLIDE 1

Performance measurement and tuning of remote acquisition

Lukasz Makowski February 2, 2016

slide-2
SLIDE 2

Location

Netherlands Forensic Institute Supervisor : Ruud Schramp

slide-3
SLIDE 3

Agenda

1 Remote acquisition - research motivation introduction 2 Research scope and questions posed 3 Approach & methods taken 4 Results 5 Future work

slide-4
SLIDE 4

Forensic acquisition

”Old-school” approach:

slide-5
SLIDE 5

Forensic acquisition

”Old-school” approach:

slide-6
SLIDE 6

Forensic acquisition

slide-7
SLIDE 7

Forensic acquisition

The bottlenecks in the current process:

slide-8
SLIDE 8

Forensic acquisition

The bottlenecks in the current process: quantity : regular disk size increases

slide-9
SLIDE 9

Forensic acquisition

Data source : http://www.mkomo.com/cost-per-gigabyte

slide-10
SLIDE 10

Forensic acquisition

The bottlenecks in the current process: quantity : regular disk size increases

slide-11
SLIDE 11

Forensic acquisition

The bottlenecks in the current process: quantity : regular disk size increases staffing : forensic experts cannot be easily multiplied :(

slide-12
SLIDE 12

Forensic acquisition

The bottlenecks in the current process: quantity : regular disk size increases staffing : forensic experts cannot be easily multiplied :( legal : court approval takes time

slide-13
SLIDE 13

Forensic acquisition

The bottlenecks in the current process: quantity : regular disk size increases staffing : forensic experts cannot be easily multiplied :( legal : court approval takes time But there is a possible solution! (at least to the first two points . . . )

slide-14
SLIDE 14

Forensic triage - the cure for pain?

Triage is the process of determining the priority

  • f patients’ treatments

based on the severity of their condition. This rations patient treatment efficiently when resources are insufficient for all to be treated immediately.

Source : https://en.wikipedia.org/wiki/Triage Source : https://cartadvocate.files.wordpress.com/2015/03/img 3788.jpg

slide-15
SLIDE 15

Forensic triage - the cure for pain?

slide-16
SLIDE 16

Forensic triage - the cure for pain?

slide-17
SLIDE 17

Remote triage

Remote triage - problem:

slide-18
SLIDE 18

Remote triage

Remote triage - approach:

slide-19
SLIDE 19

Remote triage

Remote triage’ issues:

slide-20
SLIDE 20

Remote triage

Remote triage’ issues: WAN links introduce whole subset of problems (delay, bandwidth, packet loss, . . . )

slide-21
SLIDE 21

Remote triage

Remote triage’ issues: WAN links introduce whole subset of problems (delay, bandwidth, packet loss, . . . ) iSCSI uses TCP in transport layer (TCP limitations inherited)

slide-22
SLIDE 22

Remote triage

Remote triage’ issues: WAN links introduce whole subset of problems (delay, bandwidth, packet loss, . . . ) iSCSI uses TCP in transport layer (TCP limitations inherited) iSCSI is not well suited to WAN links

slide-23
SLIDE 23

Remote triage - issues

Essentially the problem can be synthesized to simple question :

slide-24
SLIDE 24

Remote triage - issues

Essentially the problem can be synthesized to simple question : How to make the remote triage as efficient as possible?

slide-25
SLIDE 25

Remote triage - issues

Areas where the speed-up can be potentially achieved:

slide-26
SLIDE 26

Remote triage - issues

Areas where the speed-up can be potentially achieved: TCP protocol tuning

slide-27
SLIDE 27

Remote triage - issues

Areas where the speed-up can be potentially achieved: TCP protocol tuning iSCSI stack tuning

slide-28
SLIDE 28

Remote triage - issues

Areas where the speed-up can be potentially achieved: TCP protocol tuning iSCSI stack tuning Acquisition I/O optimisation

slide-29
SLIDE 29

Remote triage - issues

Areas where the speed-up can be potentially achieved: TCP protocol tuning iSCSI stack tuning Acquisition I/O optimisation

  • Yes. . . TCP and iSCSI options left in the defaults
slide-30
SLIDE 30

Research scope

Acquisition I/O optimisation :

slide-31
SLIDE 31

Research scope

Acquisition I/O optimisation : Is it feasible to enhance a transfer rate for acquisition performed on the iSCSI block device?

slide-32
SLIDE 32

Research scope

Acquisition I/O optimisation : Is it feasible to enhance a transfer rate for acquisition performed on the iSCSI block device? Which techniques an application can use to improve on the transmission rate?

slide-33
SLIDE 33

Research scope

Acquisition I/O optimisation : Is it feasible to enhance a transfer rate for acquisition performed on the iSCSI block device? Which techniques an application can use to improve on the transmission rate? How a link delay influences the experiment?

slide-34
SLIDE 34

Research scope

Researching on potential I/O optimisation methods:

slide-35
SLIDE 35

Research scope

Researching on potential I/O optimisation methods: prefetching (implies the usage of cache)

slide-36
SLIDE 36

Research scope

Researching on potential I/O optimisation methods: prefetching (implies the usage of cache)

read-ahead

slide-37
SLIDE 37

Research scope

Researching on potential I/O optimisation methods: prefetching (implies the usage of cache)

read-ahead read-behind

slide-38
SLIDE 38

Research scope - prefetching

Read-ahead : read block-size → cache MISS → read block-size+read-ahead

slide-39
SLIDE 39

Research scope - prefetching

slide-40
SLIDE 40

Research scope - prefetching

Read-ahead : read block-size → cache HIT

slide-41
SLIDE 41

Research scope

Researching on potential I/O optimisation methods: prefetching (implies the usage of cache)

read-ahead read-behind

slide-42
SLIDE 42

Research scope

Researching on potential I/O optimisation methods: prefetching (implies the usage of cache)

read-ahead read-behind

parallelism

slide-43
SLIDE 43

Research scope - parallelism

Single process, waiting for the reply

slide-44
SLIDE 44

Research scope - parallelism

More processes, an attempt to utilise the wait time

slide-45
SLIDE 45

Research scope - parallelism

Source : http://www.potaroo.net/ispcol/2005-06/fig4.jpg

slide-46
SLIDE 46

Methods - creating triage.py

Goals:

slide-47
SLIDE 47

Methods - creating triage.py

Goals: Repeatable triage process (tests)

slide-48
SLIDE 48

Methods - creating triage.py

Goals: Repeatable triage process (tests) Two modes : sequential & parallel

slide-49
SLIDE 49

Methods - creating triage.py

Goals: Repeatable triage process (tests) Two modes : sequential & parallel Adjustable parallel workers number

slide-50
SLIDE 50

Methods - creating triage.py

Solution:

slide-51
SLIDE 51

Methods - parallelism

  • Multiprocessing. Making The SleuthKit (TSK) parallel.
slide-52
SLIDE 52

Methods - prefetching

Cache implementation : Fusecoraw1

1https://homepages.staff.os3.nl/˜delaat/rp/2013-2014/p71/report.pdf

slide-53
SLIDE 53

Methods - prefetching

Expanding fusecoraw with read-ahead, read-behind functionality. Simplified approach.

slide-54
SLIDE 54

Methods - prefetching

Reads issued to the FUSE filesystem are being extended by the additional read().

slide-55
SLIDE 55

Methods - prefetching

slide-56
SLIDE 56

Methods - Lab setup

slide-57
SLIDE 57

Methods - Lab setup

Constant delay applied : 0, 10, 20 [ms]

slide-58
SLIDE 58

Experiments performed

relative delay (ms) test performed prefetching parallelism repetitions X X 3 10 X X 3 20 X X 3

Table : Test sets summary

slide-59
SLIDE 59

Experiments performed

Chosen metrics: Average throughput (tcpdump + tcptrace) Elapsed time (GNU time)

slide-60
SLIDE 60

Experiments performed

Prefetching

read ahead read behind 8192 65536 X X X 8192 X X

  • 65536

X

  • X

Table : Chosen read-ahead and read-behind values

slide-61
SLIDE 61

Results

Prefetching (Read-ahead & read-behind)

slide-62
SLIDE 62

Results

Prefetching (Read-ahead & read-behind)

slide-63
SLIDE 63

Results

Prefetching tests observations

slide-64
SLIDE 64

Results

Prefetching tests observations Average throughput may indicate the triage process speed-up, but . . .

slide-65
SLIDE 65

Results

Prefetching tests observations Average throughput may indicate the triage process speed-up, but . . . It’s better to look at the execution time

slide-66
SLIDE 66

Results

Prefetching tests observations Average throughput may indicate the triage process speed-up, but . . . It’s better to look at the execution time When no delay was introduced; read-ahead of 8KiB, had the smallest mean execution time

slide-67
SLIDE 67

Results

Prefetching tests observations Average throughput may indicate the triage process speed-up, but . . . It’s better to look at the execution time When no delay was introduced; read-ahead of 8KiB, had the smallest mean execution time With the delay; I/O without prefetching had the smallest time metric

slide-68
SLIDE 68

Experiments performed

Parallelism

directory scanner file fetcher 1 2 4 1 X

  • 2
  • X
  • 4
  • X

Table : triage.py workers setup

slide-69
SLIDE 69

Results

Parallelism

slide-70
SLIDE 70

Results

Parallelism

slide-71
SLIDE 71

Results

Parallelism test observations

slide-72
SLIDE 72

Results

Parallelism test observations Elapsed time barchart suggests that 8 workers perform surprisingly well for the delayed link

slide-73
SLIDE 73

Results

Parallelism test observations Elapsed time barchart suggests that 8 workers perform surprisingly well for the delayed link However, the throughput chart does not record expected speed-up (the differences are small)

slide-74
SLIDE 74

Results

Parallelism test observations Elapsed time barchart suggests that 8 workers perform surprisingly well for the delayed link However, the throughput chart does not record expected speed-up (the differences are small) Probably the external factor which influenced the test

  • ccurred (caching?)
slide-75
SLIDE 75

Lessons learnt

slide-76
SLIDE 76

Lessons learnt

OS tries to be your best friend. It optimises/caches whenever it can. Not necessarily bad, but it has to be understood while designing the tests.

slide-77
SLIDE 77

Lessons learnt

OS tries to be your best friend. It optimises/caches whenever it can. Not necessarily bad, but it has to be understood while designing the tests. Trying to abstract the research from the components it will eventually need to rely on, is close to agreeing that its results may become ”abstract”.

slide-78
SLIDE 78

Future work

slide-79
SLIDE 79

Future work

Follow up on the I/O optimisation techniques (extend presented tests)

slide-80
SLIDE 80

Future work

Follow up on the I/O optimisation techniques (extend presented tests) Try to reuse tuning knowledge from the papers which investigated iSCSI sequential writes over the delayed links

slide-81
SLIDE 81

Future work

Follow up on the I/O optimisation techniques (extend presented tests) Try to reuse tuning knowledge from the papers which investigated iSCSI sequential writes over the delayed links Assess chosen iSCSI implementation against Analysis of iSCSI Short Blocks Access paper criteria

slide-82
SLIDE 82

Future work

Follow up on the I/O optimisation techniques (extend presented tests) Try to reuse tuning knowledge from the papers which investigated iSCSI sequential writes over the delayed links Assess chosen iSCSI implementation against Analysis of iSCSI Short Blocks Access paper criteria Is getting the work done without TCP possible? Exploring ATA over Ethernet (AoE) feasibility for the remote acquisition

slide-83
SLIDE 83

Q&A

Questions?