How to get away with murder refactoring @qcmaude This is me (and - - PowerPoint PPT Presentation

How to get away with murder refactoring

@qcmaude

This is me (and Chewbacca). I work at & live in San Francisco.

2 /57

How to Get Away with Refactoring @qcmaude

Drummondville, QC

3 /57

How to Get Away with Refactoring @qcmaude

McGill

Rent the Runway

4 /57

How to Get Away with Refactoring @qcmaude

This presentation contains pictures of cake and other baked goods courtesy of The Great British Bake Off.

WARNING:

5 /57

How to Get Away with Refactoring @qcmaude

1. Refactoring

• what it is
• why do it
• 2. In Practice
• 3. Lessons Learned

Agenda

6 /57

How to Get Away with Refactoring @qcmaude

Refactoring

• n. the process of restructuring existing code (e.g.

the factoring) without changing its external behaviour a definition

7 /57

How to Get Away with Refactoring @qcmaude

Monique’s Millefeuille

1. Place 1/10 inch-thick puff pastry in the oven at 350˚F. 2. Whisk the eggs, sugar, vanilla and flour in a bowl. 3. Add boiled milk to the bowl. 4. Return mixture to saucepan for a few minutes until it has a cream- like texture. 5. When the puff pastry is cooked, cut into three even pieces. 6. Layer the crème pâtissière between layers of pastry and let cool.

Boil milk in medium saucepan.

(white all-purpose flour is best)

s h

• u

l d t a k e 1 m i n u t e s * c r è m e p â t i s s i è r e i s t h e c r e a m

• l

i k e m i x t u r e w e j u s t m a d e about 15 minutes on medium heat.

8 /57

How to Get Away with Refactoring @qcmaude

9 /57

How to Get Away with Refactoring @qcmaude

but, in the world of business, it’s a little more complicated than that ...

10 /57

How to Get Away with Refactoring @qcmaude

Reasons to Refactor

1. for fun

• 2. out of boredom
• 3. “happen to be passing by”
• 4. to make the code more legible
• r extendable

11 /57

How to Get Away with Refactoring @qcmaude

“If it ain’t broken, don’t fix it.”

12 /57

How to Get Away with Refactoring @qcmaude

1. shift in product requirements

• 2. adopting a new technology
• 3. improving performance

Reasons to Refactor

13 /57

How to Get Away with Refactoring @qcmaude

14 /57

How to Get Away with Refactoring @qcmaude

Refactoring can …

• cause serious regressions
• unearth dormant bugs
• easily grow in scope
• introduce unnecessary complexity

15 /57

How to Get Away with Refactoring @qcmaude

So, let’s not kid ourselves ...

16 /57

How to Get Away with Refactoring @qcmaude

Refactoring

• n. the process by which we take a pile of poo and

turn it into a shinier pile of poo the real definition

17 /57

How to Get Away with Refactoring @qcmaude

In Practice

18 /57

How to Get Away with Refactoring @qcmaude

May 2017

19 /57

How to Get Away with Refactoring @qcmaude

What’s the actual problem?

Narrow it down:

20 /57

How to Get Away with Refactoring @qcmaude

21

public and private channels DMs and group DMs unread

/57

How to Get Away with Refactoring @qcmaude

22

By the numbers

# of public & private channels on top five teams* 80 301 69 296 60 029 55 043 49 697 5 841 454 1 926 918 1 819 719 1 527 894 1 524 953 # of channels on top five teams*

* only looks at totals on a single team (non-aggregate view of Enterprise customers)

8 338 590 102 569 100 647 66 311 61 101 # of channel memberships on top five users*

/57

How to Get Away with Refactoring @qcmaude

We have two tables that store nearly identical information:

1. teams_channels stores a row for each public channel 2. groups stores a row for each private channel or group DM

23 /57

How to Get Away with Refactoring @qcmaude

Correspondingly, we have yet another two tables that store nearly identical information:

1. teams_channels_members stores a row for each user’s membership in a public channel 2. groups_members stores a row for each user’s membership in a private channel or group DM

24 /57

How to Get Away with Refactoring @qcmaude

25

teams_channels groups groups_members teams_channels_members teams_ims

channel membership

public channels private channels DMs C123456 G123456 D123456

Slack in mid-2017

/57

How to Get Away with Refactoring @qcmaude

We end up with a ton of similar queries to two tables and lots of 
 
 UNION
 UNION ALL
 LEFT (OUTER) JOIN
 
 which isn’t great for performance

26 /57

How to Get Away with Refactoring @qcmaude

27

SQL Performance 102

• UNION removes duplicate records.
• UNION ALL returns all columns (no extra

distinctness check).

• LEFT OUTER JOIN is faster than LEFT

INNER JOIN.

• Use EXPLAIN all day every day.

/57

How to Get Away with Refactoring @qcmaude

do some stuff do some other stuff

28 /57

How to Get Away with Refactoring @qcmaude

Why was it designed this way originally?

Get some context:

29 /57

How to Get Away with Refactoring @qcmaude

1. Security: keeping private channel information in a separate table isolates it

• 2. Product history: channels and

private channels seemed like vastly different concepts

• 3. Inability to travel into the future

30 /57

How to Get Away with Refactoring @qcmaude

Brainstorm a Solution & Identify the Challenges

Put on you thinking cap:

31 /57

How to Get Away with Refactoring @qcmaude

Remember this?

32

teams_channels groups groups_members teams_channels_members teams_ims

channel membership

public channels private channels DMs C123456 G123456 D123456

/57

How to Get Away with Refactoring @qcmaude

Consolidate into a single channels_members table

33 /57

How to Get Away with Refactoring @qcmaude

34

teams_channels groups teams_ims

channel membership

public channels private channels DMs C123456 G123456 D123456

channels_members

Let’s do this!

/57

How to Get Away with Refactoring @qcmaude

It might not be so simple ...

• SQL queries are scattered

throughout the code

• About 400 callsites
• ☝ embedded in old, crufty code

from 3 years ago

• Little to no unit test coverage

35 /57

How to Get Away with Refactoring @qcmaude

Convince your team!

Sell, sell, sell:

36 /57

How to Get Away with Refactoring @qcmaude

1. Our biggest customers won’t be able to handle the additional latency in a few months

• 2. Opportunity to easily modify a key table
• 3. Develop a pattern for future data

consolidations

37 /57

How to Get Away with Refactoring @qcmaude

Write a Detailed Plan of Action

Map it out:

38 /57

How to Get Away with Refactoring @qcmaude

39 /57

How to Get Away with Refactoring @qcmaude

Be careful: don’t bite off more than you can chew!

40

How to Get Away with Refactoring @qcmaude

Expect the unexpected!

41 /57

How to Get Away with Refactoring @qcmaude

A few other tips

1. Get feedback from folks on other teams

• 2. Think outside your codebase: could your

changes affect other services, other folks’ pipelines, third-party developers, etc.

• 3. Be generous in your estimates

42 /57

How to Get Away with Refactoring @qcmaude

Execute!

Go, go, go:

43 /57

How to Get Away with Refactoring @qcmaude

Feature flags are your friend.

44 /57

How to Get Away with Refactoring @qcmaude

Bugs are inevitable.

45 /57

How to Get Away with Refactoring @qcmaude

In case you’d forgotten

46

teams_channels groups teams_ims

channel membership

public channels private channels DMs C123456 G123456 D123456

channels_members

/57

How to Get Away with Refactoring @qcmaude

47

teams_ims

channel membership

public channels private channels DMs C123456 G123456 D123456

channels_members channels

One step further

/57

How to Get Away with Refactoring @qcmaude

1. Dark mode: Read from both tables, compare the results, return the value from the old table.

• 2. Light mode: Read from both tables, compare

the results, return the value from the new table.

48 /57

How to Get Away with Refactoring @qcmaude

1. Dark mode in dev environments.

• 2. Dark mode in production for a few weeks.
• 3. Light mode in dev for a two weeks.
• 4. Light mode to our team for one week.
• 5. Light mode to increasing % of teams in

production over a 2 week period.

49 /57

How to Get Away with Refactoring @qcmaude

Document, document, document.

50 /57

How to Get Away with Refactoring @qcmaude

51

“I’m afraid I don’t understand, I’m sorry!”

~8.2 million unique DMs

/57

How to Get Away with Refactoring @qcmaude

Soft Deletion


 team_id channel_id user_id … date_joined … date_deleted

52 /57

How to Get Away with Refactoring @qcmaude

Celebrate!

Do the boogie:

53 /57

How to Get Away with Refactoring @qcmaude

Clean Up

You’re not done yet:

54 /57

How to Get Away with Refactoring @qcmaude

Refactoring can be a win-win for both engineers & your business but … it has to be carefully scoped & planned out.

55

In Review

/57

How to Get Away with Refactoring @qcmaude

56 /57

How to Get Away with Refactoring @qcmaude

Merci!

Send questions, puns, & concerns to @qcmaude