[PPT] - SMB3 Multichannel with Samba/CTDB and Gluster Gnther Deschner PowerPoint Presentation

SLIDE 1

SMB3 Multichannel with Samba/CTDB and Gluster

Günther Deschner

<gd@samba.org>

Sachin Prabhu

<sprabhu@redhat.com>

SLIDE 2

SambaXP 2018, Slide 2

A g e n d a



S a mb a / C T D B C l u s t e r i n g w i t h G l u s t e r F S



S M B 3 M u l t i c h a n n e l



O p l

c

k / L e a s e b r e a k f a i l u r e s a n d M u l t i c h a n n e l

D

e mo



M u l t i c h a n n e l a n d t e s t i n g



I P f a i l

v

e r w i t h M u l t i c h a n n e l a n d C T D B



F u r t h e r r e a d i n g & Q / A

SLIDE 3

Samba/CTDB clustering with GlusterFS

SLIDE 4

SambaXP 2018, Slide 4

R e d H a t G l u s t e r S t

r

a g e ( R H G S )



“ R e d H a t G l u s t e r S t

r

a g e p r

v

i d e s a n

p

e n , s

f

t w a r e

d

e fi n e d s t

r

a g e s

l

u t i

n

a c r

s

s p h y s i c a l , v i r t u a l , a n d c l

u

d r e s

u

r c e s . ”



S M B s t

r

a g e

n

t

p
f

G l u s t e r F S u s i n g S a mb a



C T D B f

r

c l u s t e r i n g



v f s _ g l u s t e r f s mo d u l e f

r

S a mb a

(

u s e s l i b g f a p i f

r

s t

r

a g e I / O )



C u r r e n t R e l e a s e :

R

H G S 3 . 3 . 1 w i t h S a mb a 4 . 7 . 3

ff

e r i n g S M B 3 f e a t u r e s



S M B M u l t i c h a n n e l

n

l y a s “ T e c h P r e v i e w ”

SLIDE 5

What was SMB Multichannel again?

SLIDE 6

SambaXP 2018, Slide 6

S M B M u l t i c h a n n e l



S M B 3 p e r f

r

ma n c e a n d r e l i a b i l i t y f e a t u r e



A v a i l a b l e s i n c e Wi n d

w

s 2 1 2



M a x i mi z e t h r

u

g h p u t

M

u l t i p l e T C P t r a n s p

r

t c

n

n e c t i

n

s a g g r e g a t e d i n

n

e s e s s i

n
M

u l t i p l e N I C s ( N I C t e a m i n g , R D M A )

M

u l t i p l e C P U C

r

e s w i t h R S S ( R e c e i v e S i d e S c a l i n g )



I n c r e a s e f a u l t t

l

e r a n c e

M

u l t i c h a n n e l s e t u p s c

m

p e n s a t e T C P f a i l u r e s

n

c h a n n e l s



A u t

ma

t i c c

n

fi g u r a t i

n
F

e a t u r e i s a u t

m

a t i c a l l y a n d t r a n s p a r e n t l y e n a b l e d w h e n p r e r e q u i s i t e s a r e m e t :

C

l i e n t a n d S e r v e r s u p p

r

t S M B 3

A

u t

m

a t i c d e t e c t i

n
f

m a t c h i n g i n t e r f a c e s

SLIDE 7

SambaXP 2018, Slide 7

S M B M u l t i c h a n n e l



S i n g l e N I C w i t h R S S

SMB Client SMB Server

10GbE

10GbE 10GbE

Switch 10GbE

SLIDE 8

SambaXP 2018, Slide 8

S M B M u l t i c h a n n e l



M u l t i p l e N I C s

SMB Client SMB Server

10GbE

1GbE 1GbE

Switch 1GbE

10GbE

1GbE 1GbE

Switch 1GbE

SLIDE 9

SambaXP 2018, Slide 9

S M B M u l t i c h a n n e l



O v e r v i e w ( ( c ) M i c r

s
f

t )

SLIDE 10

SambaXP 2018, Slide 10

S M B M u l t i c h a n n e l i n S a mb a



F i r s t i mp l e me n t a t i

n

i n S a mb a 4 . 4 ( 2 1 6 )

“

s e r v e r m u l t i c h a n n e l s u p p

r

t = y e s ”

U

s e s f d

p

a s s i n g s

a

l l c h a n n e l s p

i

n t t

n

e s m b d

E

x p e r i m e n t a l f e a t u r e , s i n c e n

t

a l l s c e n a r i

s

a r e c

v

e r e d



C u r r e n t l i mi t a t i

n

s :

O

p l

c

k a n d l e a s e b r e a k n

t

M u l t i c h a n n e l a w a r e a n d d

e

s n

t

a t t e m p t t

r

e t r y

M

u l t i c h a n n e l n

t

t e s t a b l e i n a u t

b

u i l d

N
i

n t e r a c t i

n

w i t h c t d b ’ s f a i l

v

e r i p m a n a g e m e n t

SLIDE 11

Oplock/Lease break failures and Multichannel

SLIDE 12

SambaXP 2018, Slide 12

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



O p l

c

k / L e a s e B r e a k a r e i s s u e d b y t h e S e r v e r



“ T h e S M B 2 O p l

c

k B r e a k N

t

i fi c a t i

n

p a c k e t i s s e n t b y t h e s e r v e r w h e n t h e u n d e r l y i n g

b

j e c t s t

r

e i n d i c a t e s t h a t a n

p

p

r

t u n i s t i c l

c

k (

p

l

c

k ) i s b e i n g b r

k

e n , r e p r e s e n t i n g a c h a n g e i n t h e

p

l

c

k l e v e l . ”



“ T h e S M B 2 L e a s e B r e a k N

t

i fi c a t i

n

p a c k e t i s s e n t b y t h e s e r v e r w h e n t h e u n d e r l y i n g

b

j e c t s t

r

e i n d i c a t e s t h a t a l e a s e i s b e i n g b r

k

e n , r e p r e s e n t i n g a c h a n g e i n t h e l e a s e s t a t e . ”



M u l t i p l e c h a n n e l s c a n b e a v a i l a b l e f

r

s e n d i n g b r e a k n

t

i fi c a t i

n

s



Wh i c h

n

e i s c h

s

e n ?



Wh a t h a p p e n s

n

c h a n n e l f a i l u r e ?

SLIDE 13

SambaXP 2018, Slide 13

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



Wh a t h a p p e n s w h e n a c h a n n e l f a i l s ?

SMB Client SMB Server

10GbE

1GbE 1GbE

Switch 1GbE

10GbE

1GbE 1GbE

Switch 1GbE

SLIDE 14

SambaXP 2018, Slide 14

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



Wh a t h a p p e n s w h e n a c h a n n e l f a i l s ?

SMB Client SMB Server

10GbE

1GbE 1GbE

Switch 1GbE

10GbE

1GbE 1GbE

Switch 1GbE

SLIDE 15

SambaXP 2018, Slide 15

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



M i c r

s
f

t I n t e r

p

L a b 2 1 6 / 2 1 7 r e s e a r c h

p

l

c

k a n d l e a s e b e h a v i

r

w i t h mu l t i c h a n n e l

n

Wi n d

w

s

s

m b t

r

t u r e t e s t s u i t e



H

w

t

s

i mu l a t e c h a n n e l f a i l u r e ?

i

p t a b l e s t

d

r

p

p a c k e t s



T e s t i n g r e s u l t s f

r
p

l

c

k b r e a k :

s

m b t

r

t u r e / / s e r v e r / s h a r e

U

u s e r %p a s s w

r

d s m b 2 . m u l t i c h a n n e l .

p

l

c

k

O

p l

c

k b r e a k i s s e n t

n

t h e l a s t c r e a t e d c h a n n e l

R

e t r y i s n

t

a t t e m p t e d a t a l l



T e s t i n g r e s u l t s f

r

l e a s e b r e a k :

s

m b t

r

t u r e / / s e r v e r / s h a r e

U

u s e r %p a s s w

r

d s m b 2 . m u l t i c h a n n e l . l e a s e s

L

e a s e b r e a k i s s e n t

n

t h e fi r s t c r e a t e d c h a n n e l

R

e t r y i s a t t e m p t e d

n

t h e fi r s t c

n

n e c t e d c h a n n e l

SLIDE 16

SambaXP 2018, Slide 16

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



T C P p r

p

e r t i e s d u r i n g r e t r y :

Wi

n d

w

s 2 1 2 a n d 2 1 6 w i l l s t a r t r e t r y i n g 1 t i m e s a f t e r i n a c t i v i t y

f

1 s e c

n

d s w i t h a 1 s e c

n

d i n t e r v a l

SLIDE 17

SambaXP 2018, Slide 17

O p l

c

k / L e a s e B r e a k w i t h M u l t i c h a n n e l



S a mb a d

e

s c u r r e n t l y n

t

d e a l w i t h O p l

c

k / L e a s e B r e a k r e t r i e s w i t h mu l t i c h a n n e l



S a mb a i mp l e me n t a t i

n

p r

t
t

y p e d e mo



T O D O :

M

a k e s u r e w e c a l c u l a t e a n d v e r i f y d e l i v e r y

f

b r e a k r e s p

n

s e s ( c

m

p a r e s e n d a n d r e c e i v e q u e u e p a c k e t c

u

n t e r s )

C

l e a n u p d i s c

n

n e c t e d / f a i l e d c h a n n e l s

T

C P s e t t i n g s t

s

p e e d u p d i s c

v

e r y

f

f a i l e d c h a n n e l s ( j u s t a s

n

Wi n d

w

s )

Wh

y a r e

p

l

c

k b r e a k n

t

i fi c a t i

n

s n e v e r r e t r i e d ?

SLIDE 18

Multichannel and testing

SLIDE 19

SambaXP 2018, Slide 19

S e l f t e s t s u p p

r

t f

r

M u l t i c h a n n e l



E v e r y c

mmi

t i n S a mb a i s r u n t h r

u

g h a u t

ma

t e d t e s t i n g d u r i n g a u t

b

u i l d



F

r

e n a b l i n g S M B M u l t i c h a n n e l b y d e f a u l t i t mu s t b e t e s t e d p e r ma n e n t l y a n d a u t

ma

t i c a l l y



S a mb a a u t

ma

t e d t e s t i n g u s e s a b s t r a c t i

n

l i b r a r i e s ( c w r a p .

r

g ) :

s
c

k e t _ w r a p p e r , n s s _ w r a p p e r , u i d _ w r a p p e r , r e s

l

v _ w r a p p e r , p a m_ w r a p p e r , e t c .



F d

p

a s s i n g ?



P r e s e n t s c e n a r i

h

a r d c

d

e d h a c k ( l e t s s

c

k e t w r a p p e r w

r

k w /

f

d

p

a s s i n g )

SLIDE 20

SambaXP 2018, Slide 20

S e l f t e s t s u p p

r

t f

r

M u l t i c h a n n e l



S u p p

r

t f

r

f d

p

a s s i n g w

r

k e d

n

b y A n n

p

C S , A n d r e a s S c h n e i d e r a n d M i c h a e l A d a m



F u t u r e d e v e l

p

me n t s t

s
c

k e t _ w r a p p e r

ma

k e s

c

k e t _ w r a p p e r t h r e a d

s

a f e = > d

n

e a n d me r g e d

s

w i t c h t

mma

p

e

d fi l e f

r

s h a r e d me mo r y a mo n g p r

c

e s s e s = > d

n

e

p

r

t

e c t i

n

w i t h p t h r e a d r

b

u s t mu t e x e s = > d

n

e

i

mp l e me n t f d

p

a s s i n g = > w i p

(

p l a n ) s e n d s

c

k e t _ i n f

a

r r a y i n d e x e s a s t h e f d a r r a y i n s t e a d

f

a c t u a l f d s v i a p i p e a n d c r e a t e n e w f d s t r u c t u r e b a s e d

n

t h e c

r

r e s p

n

d i n g i n d e x e s r e c e i v e d a t t h e

t

h e r e n d .

SLIDE 21

Multichannel and CTDB

SLIDE 22

SambaXP 2018, Slide 22

M u l t i c h a n n e l a n d C T D B



T y p i c a l l y : mu l t i p l e p u b l i c i p a d d r e s s e s p e r c l u s t e r



I p a d d r e s s e s c a n b e mo v e d w i t h i n t h e c l u s t e r



I p a d d r e s s e s c a n s p a w n

v

e r mu l t i p l e n

d

e s



Wi t h mu l t i c h a n n e l a n d f d

p

a s s i n g a l l i p a d d r e s s e s

f
n

e mu l t i c h a n n e l s e s s i

n

* mu s t * r e s i d e

n

t h e s a me n

d

e



C u r r e n t s

l

u t i

n

s :

N
/

e t c / c t d b / p u b l i c _ a d d r e s s e s fi l e a n d h a r d c

d

e d i p a d d r e s s e s

I

n d i v i d u a l / e t c / c t d b / p u b l i c _ a d d r e s s e s fi l e s p e r n

d

e



R e q u i r e me n t : a u t

ma

t i c c

n

fi g u r a t i

n

a n d t r a n s p a r e n t f a i l

v

e r

SLIDE 23

SambaXP 2018, Slide 23

M u l t i c h a n n e l a n d C T D B



Q u e r y f

r

F S C T L Q U E R Y N E T WO R K I N T E R F A C E I N F O

n

I P 1

Node1 Node2 Node3

IP1 IP2 IP3

SMB Client

IP4 IP5 IP6 IP7 IP8 IP9

SLIDE 24

SambaXP 2018, Slide 24

M u l t i c h a n n e l a n d C T D B



M u l t i p l e c h a n n e l s ( I P 1 , I P 2 , I P 3 ) b

u

n d t

s

a me S M B 3 s e s s i

n

Node1 Node2 Node3

IP1 IP2 IP3

SMB Client

IP4 IP5 IP6 IP7 IP8 IP9

SLIDE 25

SambaXP 2018, Slide 25

M u l t i c h a n n e l a n d C T D B



I n t e r f a c e / C h a n n e l f a i l u r e f

r

I P 1

Node1 Node2 Node3

IP1 IP2 IP3

SMB Client

IP4 IP5 IP6 IP7 IP8 IP9

SLIDE 26

SambaXP 2018, Slide 26

M u l t i c h a n n e l a n d C T D B



C T D B w

u

l d mi g r a t e I P 1 t

a

n

t

h e r N

d

e . . .

Node1 Node2 Node3

IP1 IP2 IP3

SMB Client

IP4 IP5 IP6 IP7 IP8 IP9

SLIDE 27

SambaXP 2018, Slide 27

M u l t i c h a n n e l a n d C T D B



. . a n d f a i l

v

e r t h e c l i e n t ! M u l t i c h a n n e l c

u

l d n

t

b e r e

e

s t a b l i s h e d

Node1 Node2 Node3

IP1 IP2 IP3

SMB Client

IP4 IP5 IP6 IP7 IP8 IP9

SLIDE 28

SambaXP 2018, Slide 28

M u l t i c h a n n e l a n d C T D B



P

s

s i b l e f u t u r e s

l

u t i

n

f

r

a u t

ma

t i c c

n

fi g u r a t i

n

:

M
n

i t

r

a l l N I C s i n t h e c l u s t e r u s i n g C T D B a n d mo v e a l l c h a n n e l s

n

f a i l u r e

T

r a n s p a r e n t f a i l

v

e r Wi t n e s s i n t e r f a c e ( R P C s e r v e r d e p e n d e n c i e s )

S

M B 3 . 1 . 1 t r e e c

n

n e c t c

n

t e x t r e d i r e c t i

n
T

B D

SLIDE 29

SambaXP 2018, Slide 29

F u r t h e r r e a d i n g



M i c r

s
f

t P r

t
c
l

D

c

u me n t a t i

n

:

M

S

S

M B 2 , M S

S

WN



V a r i

u

s M i c r

s
f

t T e c h n e t a r t i c l e s

SLIDE 30

SambaXP 2018, Slide 30

Q u e s t i

n

s a n d a n s w e r s



M a i l g d @s a mb a .

r

g , s p r a b h u @r e d h a t . c

m



# s a mb a

t

e c h n i c a l

n

i r c . f r e e n

d

e . n e t

SLIDE 31

Thank you for your attention!

www.redhat.com www.samba.org <gd@samba.org>