SLIDE 1
1
Natural Language Processing
Machine Translation III
Dan Klein – UC Berkeley
2
Natural Language Processing Machine Translation III Dan Klein UC - - PowerPoint PPT Presentation
Natural Language Processing Machine Translation III Dan Klein UC - - PowerPoint PPT Presentation
Natural Language Processing Machine Translation III Dan Klein UC Berkeley 1 Syntactic Models 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Syntactic Decoding 29 30 31 32 33 34 35 36
SLIDE 2
SLIDE 3
3
SLIDE 4
4
SLIDE 5
5
SLIDE 6
6
SLIDE 7
7
SLIDE 8
8
SLIDE 9
9
SLIDE 10
10
SLIDE 11
11
SLIDE 12
12
SLIDE 13
13
SLIDE 14
14
SLIDE 15
15
SLIDE 16
16
SLIDE 17
17
SLIDE 18
18
SLIDE 19
19
SLIDE 20
20
SLIDE 21
21
SLIDE 22
22
SLIDE 23
23
SLIDE 24
24
SLIDE 25
25
SLIDE 26
26
SLIDE 27
27
SLIDE 28
28
SLIDE 29
29
Syntactic Decoding
SLIDE 30
30
SLIDE 31
31
SLIDE 32
32
SLIDE 33
33
SLIDE 34
34
SLIDE 35
35
SLIDE 36
36
SLIDE 37
37
SLIDE 38
38
SLIDE 39
39
SLIDE 40
40
SLIDE 41
41
SLIDE 42
42
SLIDE 43
43
SLIDE 44
44
SLIDE 45
45
SLIDE 46
46
SLIDE 47
47
SLIDE 48
48
SLIDE 49
49
Flexible Syntax
SLIDE 50
50
Soft Syntactic MT: From Chiang 2010
SLIDE 51
51
SLIDE 52
52
Hiero Rules
From [Chiang et al, 2005]
SLIDE 53
53
SLIDE 54
54
SLIDE 55
55
SLIDE 56
56
SLIDE 57
57
SLIDE 58
58
SLIDE 59
59
SLIDE 60
60
SLIDE 61
61
SLIDE 62
62
SLIDE 63
63
SLIDE 64
64
SLIDE 65
65
Exploiting GPUs
SLIDE 66
66
Lots to Parse
≈2.6 billion words
SLIDE 67
67
Lots to Parse
≈6 months (CPU)
SLIDE 68
68
Lots to Parse
≈3.6 days (GPU)
SLIDE 69
69
CPU Parsing
- NLP algorithms achieve speed by exploiting
sparsity.
>98% sparsity
Slide credit: Slav Petrov
[Petrov & Klein, 2007]
SLIDE 70
70
CPU Parsing
Grammar S NP VP
× ×××
Skip Spans Skip Rules
SLIDE 71
71
CPU Parsing
CPU
SLIDE 72
72
CPU Parsing
CPU
SLIDE 73
73
CPU Parsing
CPU CPU
SLIDE 74
74
The Future of Hardware
SLIDE 75
75
The Future of Hardware
SLIDE 76
76
The Future of Hardware
SLIDE 77
77
The Future of Hardware
16384
SLIDE 78
78
The Future of Hardware
32 Threads
SLIDE 79
79
The Future of Hardware
Warp
add.s32 %r1, %r631, %r0; ld.global.f32 %f81, [%r1]; ld.global.f32 %f82, [%r34]; mul.ftz.f32 %f94, %f82, %f81; mov.f32 %f95, 0f3E002E23; mov.f32 %f96, 0f00000000; mad.f32 %f93, %f94, %f95, %f96; shl.b32 %r2, %r646, 8; add.s32 %r3, %r658, %r2; shl.b32 %r4, %r3, 2; add.s32 %r5, %r631, %r4; mul.lo.s32 %r6, %r646, 588; shl.b32 %r7, %r6, 1; add.s32 %r8, %r5, %r7; ld.global.f32 %f83, [%r8]; mul.ftz.f32 %f98, %f82, %f83;
SLIDE 80
80
Warps
Warp
SLIDE 81
81
Warps
Warp Divergence
SLIDE 82
82
Warps
SLIDE 83
83
Warps
SLIDE 84
84
Warps
Warp Divergence
SLIDE 85
85
Warps
Warp Divergence
SLIDE 86
86
Warps
✔ ✗
Coalescence
SLIDE 87
87
Designing GPU Algorithms
Dense, Uniform Computation
Warp Coalescence
SLIDE 88
88
Designing GPU Algorithms
Irregular, Sparse Regular, Dense
CPU
GPU
×
× ×××
SLIDE 89
89
Designing GPU Algorithms
Irregular, Sparse Regular, Dense
CPU
GPU
×
× ×××
[Canny, Hall, and Klein, 2013]
× ×××
SLIDE 90
90
Designing GPU Algorithms
CKY Algorithm
SLIDE 91
91
CKY Parsing
for each sentence: for each span (begin, end): for each split: for each rule (P ‐> L R): score[begin, end, P] += ruleScore[P ‐> L R] * score[begin, split, L] * score[split, end, R]
Grammar Application Item Queue
SLIDE 92
92
CKY Parsing
for each sentence: for each span (begin, end): for each split: applyGrammar(begin, split, end)
Item Queue Grammar Application
SLIDE 93
93
CKY Parsing
for each parse item in sentence: applyGrammar(item)
Item Queue Grammar Application
SLIDE 94
94
CKY Parsing
for each parse item in sentence: applyGrammar(item)
CPU GPU
SLIDE 95
95
GPU Parsing Pipeline
CPU GPU
Queue
(i, k, j) (1, 2, 4) (1, 3, 4)
…
Grammar S NP VP
(0, 1, 3) (0, 2, 3)
3 2
(0, 1, 3)
SLIDE 96
96
Parsing Speed
[Canny, Hall, and Klein, 2013]
GPU 190 s/sec CPU 10 s/sec 100 200 300 400 500
Sentences per second
SLIDE 97
97
Exploiting Sparsity
Grammar S NP VP
× ×××
CPU Queuing GPU Application
SLIDE 98
98
Exploiting Sparsity
Grammar S NP VP
GPU Application
Grammar S NP VP
GPU Application
SLIDE 99
99
Exploiting Sparsity
Warp
(1, 2, 4) (0, 1, 3) (0, 2, 3) (1, 3, 4)
…
(2, 3, 5) (2, 4, 5) (3, 4, 6)
3 2
SLIDE 100
100
Exploiting Sparsity
(1, 2, 4) (0, 1, 3) (0, 2, 3) (1, 3, 4)
…
(2, 3, 5) (2, 4, 5) (3, 4, 6)
S NP VP PP … S NP VP PP … S NP VP PP … S NP VP PP … S NP VP PP … S NP VP PP … S NP VP PP …
SLIDE 101
101
Exploiting Sparsity
Warp Divergence
SLIDE 102
102
Exploiting Sparsity
Grammar S NP VP
GPU Application
SLIDE 103
103
Exploiting Sparsity
NP
NP NP PP
VP
VP VP PP
S
S NP VP
PP
PP IN NP
NP
(i, k, j)
…
(0, 1, 3) (0, 2, 3)
VP
(i, k, j)
…
(0, 1, 3) (0, 2, 3)
S
(i, k, j)
…
(0, 1, 3) (0, 2, 3)
PP
(i, k, j)
…
(0, 1, 3) (0, 2, 3)
Queue
(i, k, j)
…
(0, 1, 3) (0, 2, 3)
SLIDE 104
104
Exploiting Sparsity
CPU GPU
NP Queue
(i, k, j) (1, 2, 4) (1, 3, 4)
…
(0, 1, 3) (0, 2, 3)
NP
NP NP PP
NP
NP NP PP
NP
NP NP PP
NP
NP NP PP
NP
NP NP PP
SLIDE 105
105
Exploiting Sparsity
CPU GPU
VP Queue
(i, k, j) (1, 2, 4) (1, 3, 4)
…
(0, 1, 3) (0, 2, 3)
NP
NP NP PP
NP
NP NP PP
NP
NP NP PP
VP
VP VP NP
SLIDE 106
106