Rails in the Large: Building the Biggest (Enterprise) Rails - - PowerPoint PPT Presentation

▶
rails in the large
SMART_READER_LITE
LIVE PREVIEW

Rails in the Large: Building the Biggest (Enterprise) Rails - - PowerPoint PPT Presentation

Feb 24, 2024 133 likes •1.06k views

Thought Works Thought Works Rails in the Large: Building the Biggest (Enterprise) Rails Application in the World PAUL GROSS software developer / consultant NEAL FORD software architect / meme wrangle r Thought Works Thought Works

slide-1
SLIDE 1
slide-2
SLIDE 2

ThoughtWorks ThoughtWorks

NEAL FORD software architect / meme wrangler

ThoughtWorks

nford@thoughtworks.com 3003 Summit Boulevard, Atlanta, GA 30319 www.nealford.com www.thoughtworks.com blog: memeagora.blogspot.com twitter: neal4d PAUL GROSS software developer / consultant

ThoughtWorks

pgross@thoughtworks.com 200 E. Randolph St, 25th Floor, Chicago, IL 60601-6501 pgross@thoughtworks.com www.pgrs.net www.thoughtworks.com

Rails in the Large:

Building the Biggest (Enterprise) Rails Application in the World

slide-3
SLIDE 3

what OVE.com does

slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6

the pursuit

Go for the one that’ll beat the

  • ne that’ll beat the one you

last did

slide-7
SLIDE 7

.NET Rails

slide-8
SLIDE 8
slide-9
SLIDE 9

Business Analyst Project manager Developer Tech Lead

quick start: october 2006

slide-10
SLIDE 10

Added one pair every 2 weeks Started with 2 pairs 8 or 9 pairs by July

inception: Jan 17, 2007

slide-11
SLIDE 11

6 quality assurance 11 pairs of developers 8 business analysts client principle project manager

now

iteration manager

slide-12
SLIDE 12

spikes are your friends! technology isn’t as important as responsiveness to business needs don’t try to convince too early demonstration over arguments

lessons learned

slide-13
SLIDE 13

infrastructure

Rock is for Rookies: males have a tendency to lead with Rock on their

  • pening throw
slide-14
SLIDE 14

pairing workstations XServe (Selenium Grid)

physical infrastructure

BA standalone QA integrated QA UAT (sneak peak)

slide-15
SLIDE 15

deployment stack

10 web boxes 2 image servers background server memcache 4 database servers

slide-16
SLIDE 16

technical stats

slide-17
SLIDE 17

environment

slide-18
SLIDE 18
slide-19
SLIDE 19

Mac OS X rocks scale infrastructure opportunistically... ...but don’t wait too long have fun

lessons learned

slide-20
SLIDE 20

testing

Scissors on First: play scissors as your

  • pening move against a

more experienced player

slide-21
SLIDE 21

disconnected unit tests

UnitRecord and the evolution of unit tests that don’t hit the database

http://github.com/dan-manges/unit-record

slide-22
SLIDE 22

unit tests

slide-23
SLIDE 23

the rule:

unit tests don’t hit the database mock everything

slide-24
SLIDE 24

functional tests

slide-25
SLIDE 25

no mocking allowed in functional tests

tests that hit the database are slooooow

slide-26
SLIDE 26

DeepTest

http://github.com/qxjit/deep-test

slide-27
SLIDE 27
slide-28
SLIDE 28

DistributedDeepTest

slide-29
SLIDE 29

prefer factories over fixtures

slide-30
SLIDE 30

Selenium grid

slide-31
SLIDE 31

new instances added as needed

slide-32
SLIDE 32

core trunk build (commit build) Selenium view builds trunk Selenium view builds release core-release, externals, web services, datasets trunk + search infrastructure

cc.rb instances

slide-33
SLIDE 33

deployment & testing

slide-34
SLIDE 34

write smart tests fight the battle to keep tests fast invent stuff if you have to scale development infrastructure just like production infrastructure

lessons learned

slide-35
SLIDE 35

knowledge transfer

Paper is the least obvious

  • f opening moves.
slide-36
SLIDE 36

project Mingle on the wall

slide-37
SLIDE 37

cc_board

http://github.com/qxjit/cc_board/

slide-38
SLIDE 38

play theme song upon successful checkin play a song when a build breaks

slide-39
SLIDE 39

jukebox.rb

http://subversion.hammersforge.com/ jukebox.rb/trunk/ currently in alpha

slide-40
SLIDE 40

pairing stations

adium no email

slide-41
SLIDE 41

internal Jabber server chat rooms

devs BAs QAs shared buddy list

slide-42
SLIDE 42

automatically set pair name

adium Mingle card (upon commit)

slide-43
SLIDE 43

co-location rocks software is more about communication than technology use information radiators have fun pairing really rocks

lessons learned

slide-44
SLIDE 44

automate everything

When playing with someone who is not experienced at the RPS, look out for double runs

  • r, in other words, the

same throw twice.

slide-45
SLIDE 45

1-click deploy to any environment

using cc.rb as easy deployment tool

slide-46
SLIDE 46

verification (language keys) run all unit tests run all functional tests commit

rake commit

http://github.com/pgr0ss/rake_commit_tasks

slide-47
SLIDE 47

canonical pairing station maintenance

cap pairing_stations

slide-48
SLIDE 48

radmind

http://rsug.itd.umich.edu/software/radmind/

slide-49
SLIDE 49

strict rules for advanced language features

Tell your opponent what you are going to throw and then actually throw what you said.

slide-50
SLIDE 50

monkey patches all live in extensions folder

slide-51
SLIDE 51

modularize extensions

extend (or include) into real class

slide-52
SLIDE 52

ancestors

slide-53
SLIDE 53

where did you come from again?

slide-54
SLIDE 54

test the extensions

duh!

slide-55
SLIDE 55

include a version test to break upon upgrade

slide-56
SLIDE 56

use meta-names somewhere

ack is your friend

slide-57
SLIDE 57

background processing

Try playing the throw that would have lost to your opponents last throw

slide-58
SLIDE 58

run at a certain time CRON-like behavior progress bars image downloading Asynchronous behavior counts updating cached values continually run

3 kinds

slide-59
SLIDE 59

evolution of async messaging

slide-60
SLIDE 60

do work inline

gets slower over time traffic goes up

slide-61
SLIDE 61

use backgroundrb for simple message queue backed by database

slide-62
SLIDE 62

switch to a real messaging queue (Starling)

slide-63
SLIDE 63

YAGNI

emergent design around async messaging

slide-64
SLIDE 64

don’t use databases as message queues (for too long anyway) avoid anticipatory design gradually add complexity DBA’s can sometimes get grumpy

lessons learned

slide-65
SLIDE 65

external dependencies

When playing against someone who asks you to remind them about the rules, take the opportunity to subtly "suggest a throw" as you explain to them by physically showing them the throw you want them to play.

slide-66
SLIDE 66

make well defined boundaries

slide-67
SLIDE 67

mock and stub boundaries

slide-68
SLIDE 68

external builds to test service changes

we often catch bugs & downtime in other services

slide-69
SLIDE 69

tests to validate WSDLs haven’t changed

slide-70
SLIDE 70

tests to call services

check that responses haven’t changed

slide-71
SLIDE 71

tests to check against content & html editors

non-printable characters duplicate ids

slide-72
SLIDE 72

performance &

  • ptimization

When all else fails, go with paper: Statistically, in competition play, it has been observed that scissors is thrown the least often.

slide-73
SLIDE 73

not that many page views...

...really complex pages!

slide-74
SLIDE 74

custom hand-tuned SQL

slide-75
SLIDE 75

Memcache sessions & many database lookups

slide-76
SLIDE 76

MySQL replication

slide-77
SLIDE 77

use separate boxes for ETL schemas

write priority

slide-78
SLIDE 78

challenges

For tournament play, learn the Great Eight Gambits.

slide-79
SLIDE 79

scaling is hard

no matter the technology

slide-80
SLIDE 80

rails can scale!

slide-81
SLIDE 81

daily web trends

slide-82
SLIDE 82

monthly web trends

slide-83
SLIDE 83

upgrading is hard

1 pair => 6 weeks to upgrade from 1.2.3 to 2.2

slide-84
SLIDE 84

back port fixes & improvements

rails

  • ther plugins
slide-85
SLIDE 85

we did not replicate a freakin’ type system!

# of is_a?, kind_of? instance_of? / Total LoC 32/32379 => code (0.09%) 60/103421 => tests (0.06%)

slide-86
SLIDE 86

why all the rochambeau stuff?

slide-87
SLIDE 87

view builds are slow => s e p a r a t e c c . r b b u i l d = > 1 p a i r a s s i g n e d a s v i e w m a s t e r s view builds are fragile =>

slide-88
SLIDE 88

worst ...job ...ever

slide-89
SLIDE 89

today’s view master assigned by yesterday’s...

...or play RPS

slide-90
SLIDE 90

http://www.worldrps.com/

slide-91
SLIDE 91

would we do it again?

hell yeah!

slide-92
SLIDE 92

ThoughtWorks

?’s

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 License. http://creativecommons.org/licenses/by-sa/3.0/us/ NEAL FORD software architect / meme wrangler

ThoughtWorks

nford@thoughtworks.com 3003 Summit Boulevard, Atlanta, GA 30319 www.nealford.com www.thoughtworks.com blog: memeagora.blogspot.com twitter: neal4d PAUL GROSS software developer / consultant

ThoughtWorks

pgross@thoughtworks.com 200 E. Randolph St, 25th Floor, Chicago, IL 60601-6501 pgross@thoughtworks.com www.pgrs.net www.thoughtworks.com