1 Destination site Chat and Games with Avatars Revenue over $40 - - PowerPoint PPT Presentation

▶
1 destination site chat and games with avatars revenue
SMART_READER_LITE
LIVE PREVIEW

1 Destination site Chat and Games with Avatars Revenue over $40 - - PowerPoint PPT Presentation

Mar 13, 2024 43 likes •554 views

1 Destination site Chat and Games with Avatars Revenue over $40 million annually Profitable Engineering staff > 50 people 2 NO STAGING CLUSTER NO QA REVIEW COMMIT CODE TO LIVE IN PRODUCTION 3 4 RELEASE DEVELOP PRAY FOR CUSTOMERS

slide-1
SLIDE 1

1

slide-2
SLIDE 2

2

Destination site Chat and Games with Avatars Revenue over $40 million annually Profitable Engineering staff > 50 people

slide-3
SLIDE 3

3

NO STAGING CLUSTER NO QA REVIEW

COMMIT CODE TO LIVE IN PRODUCTION

slide-4
SLIDE 4

4

slide-5
SLIDE 5

5

DEVELOP RELEASE PRAY FOR CUSTOMERS

slide-6
SLIDE 6

6

DEVELOP RELEASE LEARN ITERATE

slide-7
SLIDE 7

7

slide-8
SLIDE 8

8

WAY EASIER TO FIND REGRESSIONS IN SMALL BATCHES OF COMMITS

VS.

slide-9
SLIDE 9

9

WAY EASIER TO FIND REGRESSIONS IN SMALL BATCHES OF COMMITS

VS.

LESS THAN 30 LINES OF CODE TO INVESTIGATE

slide-10
SLIDE 10

10

FAST RESPONSE TIMES

slide-11
SLIDE 11

11

slide-12
SLIDE 12

Chapter 5 – Reducing Batch Size

Reducing batch size… reduces cycle time reduces variability in flow accelerates feedback reduces risk reduces overhead Large batches… reduce efficiency inherently lower motivation and energy cause exponential cost and schedule growth lead to even larger batches The entire batch is limited by its worst element

DON’T TAKE MY WORD FOR IT…

slide-13
SLIDE 13

All tests pass? Local tests pass, engineer commits code Lots and lots of tests run Code deployed to all servers Metrics good? Code deployed to % of servers Metrics still good? Rollback (Blocks) Revert commit (Blocks) No Yes No Yes No Yes Win!

slide-14
SLIDE 14
slide-15
SLIDE 15

All tests pass? Lots and lots of tests run Code deployed to all servers Metrics good? Code deployed to % of servers Metrics still good? Rollback (Blocks) Revert commit (Blocks) No Yes No Yes No Yes Win! Local tests pass, engineer commits code

slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

CODE HERE!

slide-19
SLIDE 19

All tests pass? Code deployed to all servers Metrics good? Code deployed to % of servers Metrics still good? Rollback (Blocks) Revert commit (Blocks) No Yes No Yes No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run

slide-20
SLIDE 20

CONTINUOUS INTEGRATION

slide-21
SLIDE 21

21

slide-22
SLIDE 22

Code deployed to all servers Metrics good? Code deployed to % of servers Metrics still good? Rollback (Blocks) Yes No Yes No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Revert commit (Blocks) No

slide-23
SLIDE 23
slide-24
SLIDE 24

No Revert commit (Blocks) Code deployed to all servers Metrics still good? Rollback (Blocks) No Yes No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Yes Metrics good? Code deployed to % of servers

slide-25
SLIDE 25

25

CLUSTER IMMUNE SYSTEM

slide-26
SLIDE 26

SYSTEM PERFORMANCE BUSINESS PERFORMANCE

IS IT IMPORTANT TO YOUR BUSINESS? MONITOR IT

slide-27
SLIDE 27

No Revert commit (Blocks) Code deployed to all servers Metrics still good? Yes No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Yes Metrics good? Code deployed to % of servers Rollback (Blocks) No

slide-28
SLIDE 28
slide-29
SLIDE 29

Rollback (Blocks) No Revert commit (Blocks) No No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Yes Metrics good? Code deployed to % of servers Code deployed to all servers Metrics still good? Yes

slide-30
SLIDE 30

No Revert commit (Blocks) No Yes Win! Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Yes Metrics good? Code deployed to % of servers Code deployed to all servers Metrics still good? Yes Rollback (Blocks) No

slide-31
SLIDE 31

Rollback (Blocks) No Revert commit (Blocks) No No Local tests pass, engineer commits code Lots and lots of tests run All tests pass? Yes Metrics good? Code deployed to % of servers Code deployed to all servers Metrics still good? Yes Yes Win!

slide-32
SLIDE 32

32

slide-33
SLIDE 33

33

START WITH SANDBOX JUST PUSH! START WITH PRODUCTION BUILD CONFIDENCE NEW PRODUCT ESTABLISHED PRODUCT IDEAL TIME FOR FAILURES!!! AUTOMATE DEPLOYS

COMMIT TO MAKING FORWARD PROGRESS

slide-34
SLIDE 34

EXPECT SOME HURDLES

slide-35
SLIDE 35

35

The ¡Trickier ¡Bits ¡

  • Catching ¡issues ¡that ¡fail ¡slowly ¡

– SELECT * FROM growing_table WHERE 1 ¡

  • Some ¡cri9cal ¡areas ¡cause ¡hard ¡lock-­‑ups ¡

– MySQL ¡ – Memcached ¡

  • Lack ¡of ¡test ¡coverage ¡of ¡older ¡code ¡

– Not ¡an ¡issue ¡if ¡you ¡start ¡with ¡test ¡coverage ¡

  • Outsourcing ¡
  • Standard ¡code ¡review ¡tools ¡
slide-36
SLIDE 36
slide-37
SLIDE 37

BUILD ISOLATION!

FASTER TEST RUNS BUY HARDWARE SORT TESTS BY SPEED DEPENDENCY INJECTION AND… THE HYPOTHESIS BUILDER!

slide-38
SLIDE 38
slide-39
SLIDE 39

RUN THEM THAT WAY!

BUILD SYSTEMS ARE CRITICAL BUSINESS FUNCTIONS

slide-40
SLIDE 40
slide-41
SLIDE 41

CODE HERE!

slide-42
SLIDE 42

42

$experiment = array( ‘name’ -> ‘GDCEurope2011’, 'initial_rollout' => '0', 'branches' => array( 'Enhanced' => 0.5, // can specify multiple weighted branches ) ); // Helper function for readability function user_should_see_test_feature( $uid = NULL ) { return get_branch_for_customer( $uid, $experiment ) ) == 'Enhanced'; // assigns customer if first time called } // The experiment branch if( user_should_see_test_feature( $uid ) ) { // behavior for experiment } else { // control behavior }

slide-43
SLIDE 43

43

CONTROL PARTICIPANTS CLOSE EXPERIMENTS

slide-44
SLIDE 44

44

UNDESIRED RESULT DESIRED RESULT

NO STATISTICAL SIGNIFICANCE

slide-45
SLIDE 45

Work Remaining

Cycle Start Planned Cycle End

THE “PERFECT” SCHEDULE

IF YOU HIT THIS CONSISTENTLY YOU ARE SAND-BAGGING!

slide-46
SLIDE 46

Work Remaining

Cycle Start Planned Cycle End

REALITY STRIKES RELEASE DRIVES FEATURES

Drop or push Outstanding Issues Incomplete Features Tech Review Refactoring When releases happen every 15 minutes, this can be arbitrary

slide-47
SLIDE 47

Work Remaining

Cycle Start Cycle End Goal

FEATURES DRIVE RELEASE

Overage Cycle End

slide-48
SLIDE 48

48

DEVELOP RELEASE LEARN ITERATE RELEASE LEARN ITERATE

slide-49
SLIDE 49

49

EMPOWERED!

slide-50
SLIDE 50

50

Thank ¡You! ¡ ¡Any ¡Ques9ons? ¡

BreH ¡G. ¡DurreH ¡

bdurreH@imvu.com ¡ TwiHer: ¡@bdurreH ¡ Tweet ¡this ¡talk! ¡ IMVU recognized as:

  • Inc. 500

http://bit.ly/dv52wK

Red Herring 100:

http://bit.ly/bbz5Ex

Best Place to Work:

http://bit.ly/aAVdp8

(and we're hiring) http://www.imvu.com/jobs

#gdceurope ¡

slide-51
SLIDE 51

51

Thanks! ¡

  • Single ¡Lane ¡Traffic ¡Photo: ¡

hHp://www.flickr.com/photos/onebigchickenman/4869442019/ ¡