An FPGA Platform for Hyperscalers - Slides Presentation August 2017 - PDF document

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/319346565 An FPGA Platform for Hyperscalers - Slides Presentation · August 2017 CITATIONS READS 0 472 1 author: Francois Abel IBM 36 PUBLICATIONS 481 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: cloudFPGA View project All content following this page was uploaded by Francois Abel on 01 September 2017. The user has requested enhancement of the downloaded file.

A n F P G A P l a t f o r m f o r H y p e r s c a l e r s F . A b e l , J . We e r a s i n g h e , C . H a g l e i t n e r , B . We i s s , S . P a r e d e s I B M R e s e a r c h – Z u r i c h , S w i t z e r l a n d H o t I n t e r c o n n e c t s 2 5 , S a n t a C l a r a , C A , A u g . 2 9 - 3 0 , 2 0 1 7

Wh a t i s a h y p e r s c a l e r ? ■ D e f i n i t i o n – C l o u d o p e r a t o r t h a t r u n s s e v e r a l h u n d r e d s o f t h o u s a n d s o f s e r v e r s – * O p e r a t o r w i t h mo r e t h a n U S $ [ 1 , 4 , 8 ] b i l l i o n i n a n n u a l r e v e n u e f r o m: • a s e r v i c e s u c h a s I a a S , P a a S o r S a a S ( e . g . , A WS , R a c k s p a c e , G o o g l e ) • I n t e r n e t , s e a r c h , s o c i a l n e t w o r k i n g ( e . g . , S a l e s f o r c e , A D P , G o o g l e ) • E - c o mme r c e / p a y me n t p r o c e s s i n g ( e . g . , A ma z o n , A l i b a b a , e B a y ) ■ H y p e r s c a l e r s i n 2 0 1 7 : – * * 2 4 h y p e r s c a l e c o mp a n i e s o p e r a t i n g 3 0 0 + d a t a c e n t e r s H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 2 S o u r c e s : * C i s c o G l o b a l C l o u d I n d e x a n d * * S y n e r g y R e s e a r c h G r o u p

H y p e r s c a l e s e r v e r – H o w d o e s i t l o o k ? M i c r o s o f t ' s c loud server architecture F a c e b o o k ' s Y o s e mi t e s l e d V 1 w i t h f o u r M o n o L a k e s e r v e r s S o u r c e s : H o w M i c r o s o f t D e s i g n s i t s C l o u d - S c a l e S e r v e r s , M i c r o s o f t , 2 0 1 4 a n d I n t r o d u c i n g " Y o s e mi t e " : t h e f i r s t o p e n s o u r c e mo d u l a r c h a s s i s f o r h i g h - p o w e r e d mi c r o s e r v e r s , F a c e b o o k , 2 0 1 5 . H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 3

Wh y d o w e n e e d a n e w p l a t f o r m? ■ T r a d i t i o n a l b u s a t t a c h me n t → F P G A a s a c o - p r o c e s s o r C P U C P U F P G A F P G A S e r v e r N o d e S e r v e r N o d e F P G A N o d e P C I e – – – + + + S e r v e r h o mo g e n e i t y – – I n c r e a s e d s e r v e r c o s t & p o w e r – – + P e r f o r ma n c e b o o s t – – – + N u mb e r o f a c c e l e r a t o r s p e r s e r v e r – Wo r k l o a d f l e x i b i l i t y a n d mi g r a t i o n – – H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 4

[ 1 ] D i s a g g r e g a t i o n o f t h e F P G A f r o m t h e s e r v e r ■ N e t w o r k a t t a c h me n t → F P G A a s a p e e r p r o c e s s o r C P U F P G A F P G A F P G A S e r v e r N o d e F P G A N o d e F P G A N o d e F P G A N o d e P C I e E t h e r n e t S w i t c h D C N e t w o r k + + + S e r v e r h o mo g e n e i t y + + + N u mb e r o f F P G A s p e r s e r v e r + + + S e r v e r c o s t & p o w e r + + P e r f o r ma n c e b o o s t + + M a n a g e me n t + Wo r k l o a d f l e x i b i l i t y & mi g r a t i o n + + / - - L a r g e - s c a l e d i s t r i b u t e d a p p l i c a t i o n s [ 1 ] J . We e r a s i n g h e e t a l . , “ E n a b l i n g F P G A s i n h y p e r s c a l e d a t a c e n t e r s , ” i n 2 0 1 5 I E E E I n t ' l C o n f . o n C l o u d a n d B i g D a t a C o mp u t i n g , B e i j i n g , C h i n a , 2 0 1 5 . H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 5

S t a n d a l o n e n e t w o r k - a t t a c h e d F P G A 1 ) R e p l a c e P C I e I / F w i t h i n t e g r a t e d N I C ( i N I C ) T M 3 M S P D 0 8 S e r i e s F P G A M o d u l e D R A M D D R 4 D D R 4 P WR GTY 2 ) T u r n F P G A J T A G (x8) c a r d i n t o a s e l f - c o n t a i n e d a p p l i a n c e x72 x72 I O PMBus EN PG MON I O H P I O H P I O F P G A x 3 2 H R P S o C P e r v a s i v e x 4 8 H P E mi f PCIe K i n t e x (x4) P S o C A R M I2C M o n U l t r a S c a l e i N I C C o r t e x - M 3 C f g ~ 1 5 % F l a s h B P I M P S S E J T A G J T A G G T H G T H G T H C o n f i g 10GBASE PCIe PCIe SATA JTAG USBv2 KR (x6) (x8) (x8) (x2) C o n n e c t o r B a c k p l a n e 3 ) R e p l a c e t r a n s c e i v e r s w / b a c k p l a n e c o n n e c t i v i t y H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 6

O n e c a r r i e r s l e d = 3 2 F P G A mo d u l e s ■ O u r f i r s t F P G A mo d u l e u s e s a X i l i n x K i n t e x U l t r a s c a l e K U 0 6 0 – A mi d - r a n g e F P G A w i t h h i g h p e r f o r ma n c e / p r i c e a n d l o w w a t t a g e H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 7

O n e c a r r i e r s l e d = 3 2 F P G A mo d u l e s ■ O u r f i r s t F P G A mo d u l e u s e s a X i l i n x K i n t e x U l t r a s c a l e K U 0 6 0 – A mi d - r a n g e F P G A w i t h h i g h p e r f o r ma n c e / p r i c e a n d l o w w a t t a g e × 1 × 1 6 6 4 0 G E x 8 6 4 0 G b / s E t h e r n e t s w i t c h H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 8

T w o c a r r i e r s l e d s p e r c h a s s i s = 6 4 F P G A s F i g u r a t i v e p i c t u r e L e g e n d ( p e r s l i c e ) : B a l a n c e d ( i . e . n o o v e r - s u b s c r i p t i o n ) [ = = ] x 8 4 0 G b E u p l i n k s ( 3 2 0 G b / s ) b e t w e e n t h e n o r t h a n d s o u t h l i n k s o f [ – – ] x 3 2 1 0 G b E F P G A - t o - S w i t c h l i n k s ( 3 2 0 G b / s ) t h e E t h e r n e t s w i t c h [ – – ] x 3 2 1 0 G b E r e d u n d a n t l i n k s [ – – ] x 3 2 1 0 G b E F P G A - t o - F P G A l i n k s ██ ] [ x 1 6 P C I e x 8 G e n 3 S P x 1 S e r v i c e P r o c e s s o r H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 9

S i x t e e n c h a s s i s p e r r a c k = 1 0 2 4 F P G A s 1 0 2 4 F P G A s → 2 . 8 M D S P s , 1 5 2 x 1 0 F i x e d - P o i n t M u l t i p l y - A c c u mu l a t e s / s 1 0 T b / s b i - s e c . B w – 1 6 T B D D R 4 – 4 0 k W ma x . H o t I n t e r c o n n e c t s 2 5 – A u g . 2 0 1 7 10