Advanced Performance Optimization of Rails Applications - - PowerPoint PPT Presentation

▶
advanced performance optimization of rails applications
SMART_READER_LITE
LIVE PREVIEW

Advanced Performance Optimization of Rails Applications - - PowerPoint PPT Presentation

Oct 06, 2022 193 likes •1.05k views

Project Management and Scrum Software Alexander Dymo RailsConf 2009 Advanced Performance Optimization of Rails Applications http://en.oreilly.com/rails2009/public/schedule/detail/8615 www.acunote.com What Am I Optimizing? Project Management

slide-1
SLIDE 1

Advanced Performance Optimization

  • f Rails Applications

Alexander Dymo RailsConf 2009

www.acunote.com

http://en.oreilly.com/rails2009/public/schedule/detail/8615

Project Management and Scrum Software

slide-2
SLIDE 2

2 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

What Am I Optimizing?

Acunote www.acunote.com

Online project management and Scrum software Ruby on Rails application since inception in 2006

  • ~3700 customers
  • Hosted on Engine Yard
  • Hosted on Customers' Servers
  • nginx + mongrel
  • PostgreSQL

http://en.oreilly.com/rails2009/public/schedule/detail/8615

slide-3
SLIDE 3

3 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Performance Degradation Over Time

100 150 200 250 300

April 2008 May 2008 June 2008 July 2008 Request Time (on development box), %

Actually Happens: O(nc) Best Case: O(log n)

slide-4
SLIDE 4

4 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Solutions?

Throw Some Hardware at it!

slide-5
SLIDE 5

5 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Solutions?

Performance Optimization!

slide-6
SLIDE 6

6 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

What to optimize?

slide-7
SLIDE 7

7 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

What To Optimize?

Development?

slide-8
SLIDE 8

8 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

What To Optimize?

Development AND Production

slide-9
SLIDE 9

9 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

How to optimize?

slide-10
SLIDE 10

10 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

How To Optimize?

Three rules of performance optimization

slide-11
SLIDE 11

11 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Three Rules Of Performance Optimization

  • 1. Measure!

...the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail... Knuth

slide-12
SLIDE 12

12 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Three Rules Of Performance Optimization

  • 2. Optimize only what's slow!
slide-13
SLIDE 13

13 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Three Rules Of Performance Optimization

  • 3. Optimize for the user!
slide-14
SLIDE 14

14 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-15
SLIDE 15

15 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Ruby: Date Class

What's wrong with Date?

> puts Benchmark.realtime { 1000.times { Time.mktime(2009, 5, 6, 0, 0, 0) } } 0.005 > puts Benchmark.realtime { 1000.times { Date.civil(2009, 5, 6) } } 0.080

16x slower than Time! Why?

%self total self wait child calls name 7.23 0.66 0.18 0.00 0.48 18601 <Class::Rational>#reduce 6.83 0.27 0.17 0.00 0.10 5782 <Class::Date>#jd_to_civil 6.43 0.21 0.16 0.00 0.05 31528 Rational#initialize 5.62 0.23 0.14 0.00 0.09 18601 Integer#gcd

slide-16
SLIDE 16

16 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Ruby: Date Class

Fixing Date: Use C, Luke! Date::Performance gem with Date partially rewritten in C by Ryan Tomayko (with my patches in 0.4.7)

> puts Benchmark.realtime { 1000.times { Time.mktime(2009, 5, 6, 0, 0, 0) } } 0.005 > puts Benchmark.realtime { 1000.times { Date.civil(2009, 5, 6) } } 0.080 > require 'date/performance' puts Benchmark.realtime { 1000.times { Date.civil(2009, 5, 6) } } 0.006

git clone git://github.com/rtomayko/date-performance.git rake package:build cd dist && gem install date-performance-0.4.7.gem

slide-17
SLIDE 17

17 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Ruby: Date Class

Real-world impact of Date::Performance: Before: 0.95 sec After: 0.65 sec 1.5x!

slide-18
SLIDE 18

18 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Ruby: Misc

Use String::<< instead of String::+=

> long_string = "foo" * 100000 > Benchmark.realtime { long_string += "foo" } 0.0003 > Benchmark.realtime { long_string << "foo" } 0.000004

Avoid BigDecimal comparisons with strings and integers

> n = BigDecimal("4.5") > Benchmark.realtime { 10000.times { n <=> 4.5 } } 0.063 > Benchmark.realtime { 10000.times { n <=> BigDecimal("4.5") } } 0.014

in theory: 4.5x in practice: 1.15x in theory: 75x in practice: up to 70x

slide-19
SLIDE 19

19 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-20
SLIDE 20

20 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Rails: Preloading Associations

Preloading associations is always a good idea... except when Rails can't do the job:

class Foo belongs_to :bar end foos = Foo.find_by_sql('select * from foos inner join bar') foos.first.bar #extra SQL query!

slide-21
SLIDE 21

21 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Rails: Preloading Associations

Virtual Attributes plugin: http://github.com/acunote/virtual_attributes/

class Bar end class Foo belongs_to :bar preloadable_association :bar end foos = Foo.find_by_sql(' select * from foos left outer join (select id as preloaded_bar_id, name as preloaded_bar_name from bars) as bars

  • n foos.bar_id = bars.preloaded_bar_id')

foos.first.bar #no extra SQL query!

slide-22
SLIDE 22

22 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Rails: String Callbacks

What can be wrong with this code?

class Task < ActiveRecord::Base before_save "some_check()" end ... 100.times { Task.create attributes }

Kernel#binding is called to eval() the string callback That will duplicate your execution context in memory! More memory taken => More time for GC

slide-23
SLIDE 23

23 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Rails: Partial Rendering

Not too uncommon, right?

<% for object in objects %> #1000 times <%= render :partial => 'object', :locals => { :object => object } %> <% end %>

We create 1000 View instances for each object here! Why?

slide-24
SLIDE 24

24 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

list.rhtml

Optimizing Rails: Partial Rendering

Template inlining for the resque:

<% for object in objects %> #1000 times <%= render :partial => 'object', :locals => { :object => object }, :inline => true %> <% end %>

list.rhtml _object.rhtml _object.rhtml _object.rhtml _object.rhtml _object.rhtml _object.rhtml _object.rhtml _object.rhtml

slide-25
SLIDE 25

25 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Rails: Partial Rendering

Template Inliner plugin: http://github.com/acunote/template_inliner/ Real world effect from template inlining: Rendering of 300 objects, 5 partials for each object without inlining: 0.89 sec with inlining: 0.75 sec

1.2x

slide-26
SLIDE 26

26 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-27
SLIDE 27

27 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Database

How to optimize PostgreSQL: explain analyze explain analyze explain analyze ...

slide-28
SLIDE 28

28 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Database: PostgreSQL Tips

EXPLAIN ANALYZE explains everything, but... ... run it also for the "cold" database state! Example: complex query which works on 230 000 rows and does 9 subselects / joins: cold state: 28 sec, hot state: 2.42 sec Database server restart doesn't help Need to clear disk cache: sudo echo 3 | sudo tee /proc/sys/vm/drop_caches (Linux)

slide-29
SLIDE 29

29 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing Database: PostgreSQL Tips

Use any(array ()) instead of in() to force subselect and avoid join

explain analyze select * from issues where id in (select issue_id from tags_issues); QUERY PLAN

  • Merge IN Join (actual time=0.096..576.704 rows=55363 loops=1)

Merge Cond: (issues.id = tags_issues.issue_id)

  • > Index Scan using issues_pkey on issues (actual time=0.027..270.557 rows=229991 loops=1)
  • > Index Scan using tags_issues_issue_id_key on tags_issues (actual time=0.051..73.903 rows=70052loops=1)

Total runtime: 605.274 ms explain analyze select * from issues where id = any( array( (select issue_id from tags_issues) ) ); QUERY PLAN

  • Bitmap Heap Scan on issues (actual time=247.358..297.932 rows=55363 loops=1)

Recheck Cond: (id = ANY ($0)) InitPlan

  • > Seq Scan on tags_issues (actual time=0.017..51.291 rows=70052 loops=1)
  • > Bitmap Index Scan on issues_pkey (actual time=246.589..246.589 rows=70052 loops=1)

Index Cond: (id = ANY ($0)) Total runtime: 325.205 ms

2x!

slide-30
SLIDE 30

30 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Database Optimization: PostgreSQL Tips

Push down conditions into subselects and joins PostgreSQL often won't do that for you

select *, (select notes.author from notes where notes.issue_id = issues.id ) as note_authors from issues where org_id = 1 select *, (select notes.author from notes where notes.issue_id = issues.id and org_id = 1 ) as note_authors from issues where org_id = 1

Issues

id serial name varchar

  • rg_id

integer

Notes

id serial name varchar issue_id integer

  • rg_id

integer

slide-31
SLIDE 31

31 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

What To Do?

  • Optimize For Development Box

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Optimize For Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Optimize For The User

– HTTP – Javascript – Internet Explorer

slide-32
SLIDE 32

32 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

Everybody says "JRuby and Ruby 1.9 are faster" Is that true in production?

slide-33
SLIDE 33

33 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

In short, YES!

= Acunote Benchmarks = MRI JRuby 1.9.1 Date/Time Intensive Ops 1.23 0.58 0.53 Rendering Intensive Ops 0.61 0.44 0.30 Calculations Intensive Ops 2.57 1.79 1.33 Database Intensive Ops 5.58 4.63 3.29

slide-34
SLIDE 34

34 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

In short, YES!

= Acunote Benchmarks = MRI JRuby 1.9.1 Date/Time Intensive Ops 1x 2.1x 2.3x Rendering Intensive Ops 1x 1.4x 2.0x Calculations Intensive Ops 1x 1.4x 1.9x Database Intensive Ops 1x 1.2x 1.7x

JRuby: 1.5x faster Ruby 1.9: 2.0x faster

slide-35
SLIDE 35

35 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

What is faster?

slide-36
SLIDE 36

36 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

This is where all the improvement went:

Acunote Copy Tasks Benchmark MRI JRuby 1.9.1 Request Time 5.52 4.45 3.24 Template Rendering Time 0.53 0.21 0.21 Database Time 0.70 1.32 0.69 GC Time 1.07 N/A 0.62

Much faster template rendering! Less GC! JDBC database driver performance issue with JRuby?

slide-37
SLIDE 37

37 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

Why faster?

slide-38
SLIDE 38

38 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

Things I usually see in the profiler after optimizing:

%self self calls name 2.73 0.05 351 Range#each-1 2.73 0.05 33822 Hash#[]= 2.19 0.04 4 Acts::AdvancedTree::Tree#walk_tree 2.19 0.04 44076 Hash#[] 1.64 0.03 1966 Array#each-1 1.64 0.03 378 Org#pricing_plan 1.64 0.03 1743 Array#each 1.09 0.02 1688 ActiveRecord::AttributeMethods#respond_to? 1.09 0.02 1311 Hash#each 1.09 0.02 6180 ActiveRecord::AttributeMethods#read_attribute_before_typecast 1.09 0.02 13725 Fixnum#== 1.09 0.02 46736 Array#[] 1.09 0.02 15631 String#to_s 1.09 0.02 24330 String#concat 1.09 0.02 916 ActiveRecord::Associations#association_instance_get 1.09 0.02 242 ActionView::Helpers::NumberHelper#number_with_precision 1.09 0.02 7417 Fixnum#to_s

slide-39
SLIDE 39

39 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

# of method calls during one request: 50 000 - Array 35 000 - Hash 25 000 - String Slow classes written in Ruby: Date Rational

slide-40
SLIDE 40

40 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

Alternative Rubys optimize mostly:

  • the cost of function call
  • complex computations in pure Ruby
  • memory by not keeping source code AST
slide-41
SLIDE 41

41 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

Alternative Rubys optimize mostly:

  • the cost of function call
  • complex computations in pure Ruby
  • memory by not keeping source code AST
slide-42
SLIDE 42

42 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Alternative Ruby

So, shall I use alternative Ruby? Definitely Yes!... but

JRuby: if your application works with it (run requests hundreds of times to check) Ruby 1.9: if all gems you need are ported

slide-43
SLIDE 43

43 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-44
SLIDE 44

44 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

Issues we experienced deploying on Engine Yard: 1) VPS is just too damn slow 2) VPS may have too little memory to run the request! 3) shared database server is a problem 4) network filesystem may cause harm as well

slide-45
SLIDE 45

45 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

VPS may have too little memory to run the request Think 512M should be enough? Think again. We saw requests that took 1G of memory! Solutions:

  • buy more memory
  • optimize memory
  • set memory limits for mongrels (with monit)
slide-46
SLIDE 46

46 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

You're competing for memory cache on a shared server:

  • 1. two databases with equal load share the cache
slide-47
SLIDE 47

47 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

You're competing for memory cache on a shared server:

  • 2. one of the databases gets more load and wins the cache
slide-48
SLIDE 48

48 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

As a result, your database can always be in a "cold" state and you read data from disk, not from memory! complex query which works on 230 000 rows and does 9 subselects / joins: from disk: 28 sec, from memory: 2.42 sec Solutions:

  • ptimize for the cold state

push down SQL conditions

sudo echo 3 | sudo tee /proc/sys/vm/drop_caches

slide-49
SLIDE 49

49 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

fstat() is slow on network filesystem (GFS) Request to render list of tasks in Acunote:

  • n development box: 0.50 sec
  • n production box:

0.50 - 2.50 sec

slide-50
SLIDE 50

50 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

fstat() is slow on network filesystem (GFS) Couldn't figure out why until we ran strace We used a) filesystem store for fragment caching b) expire_fragment(regexp) Later looked through all cache directories even though we knew the cache is located in only one specific subdir

slide-51
SLIDE 51

51 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimizing For Shared Environment

fstat() is slow on network filesystem (GFS) Solution: memcached instead of filesystem if filesystem is ok, here's a trick: http://blog.pluron.com/2008/07/hell-is-paved-w.html

slide-52
SLIDE 52

52 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-53
SLIDE 53

53 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Live Debugging

To see what's wrong on "live" application: For Linux: strace and oprofile For Mac and Solaris: dtrace For Windows: uhm... about time to switch ;) To monitor for known problems: monit nagios

  • wn scripts to analyze application logs
slide-54
SLIDE 54

54 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-55
SLIDE 55

55 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Load Balancing

The problem of round-robin and fair load balancing

Rails App 1 Rails App 2 Rails App 3 1 1 3 2 1 3 per-process queues 2 2

slide-56
SLIDE 56

56 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Load Balancing

Solution: the global queue

Rails App 1 Rails App 2 Rails App 3 2 1 4 5 3

slide-57
SLIDE 57

57 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Load Balancing

Think you need mod_rails / Passenger for this? You're right, but... you can emulate this with nginx and mongrels

slide-58
SLIDE 58

58 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Load Balancing

Dedicated queues for long-running requests

Rails App 1 Rails App 2 Rails App 3 1 1 2 1 3 queue for long-running requests 2 regular per-process queues

slide-59
SLIDE 59

59 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Load Balancing

nginx configuration for dedicated queues

upstream mongrel { server 127.0.0.1:5000; server 127.0.0.1:5001; } upstream rss_mongrel { server 127.0.0.1:5002; } server { location / { location ~ ^/feeds/(rss|atom) { if (!-f $request_filename) { proxy_pass http://rss_mongrel; break; } } if (!-f $request_filename) { proxy_pass http://mongrel; } } }

slide-60
SLIDE 60

60 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-61
SLIDE 61

61 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize For The User: HTTP

Network and Frontend Backend

Things to consider:

  • Gzip HTML, CSS and JS
  • Minify JS
  • Collect JS and CSS

(javascript_include_tag :all, :cache => true)

  • Far future expires headers for JS, CSS, images
  • Sprites
  • Cache-Control: public
  • everything else YSlow tells you

5% 95%

slide-62
SLIDE 62

62 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-63
SLIDE 63

63 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: Javascript

The worst enemy of every web developer is...

slide-64
SLIDE 64

64 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: Javascript

slide-65
SLIDE 65

65 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: Javascript

Things you don't want to hear from your users:

"...Your server is slow..."

said the user after clicking

  • n the link to show a form

with plain javascript (no AJAX)

slide-66
SLIDE 66

66 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: Javascript

Known hotspots in Javascript:

  • eval()
  • all DOM operations - avoid if possible, for example
  • use element.className instead of element.readAttribute('class')
  • use element.id instead of element.readAttirbute('id')
  • $$() selectors, especially attribute selectors
  • may be expensive, measure first
  • $$('#some .listing td a.popup[accesslink]'
  • use getElementsByTagName() and iterate results instead
  • element.style.* changes
  • change class instead
  • $() and getElementById on large (~20000 elements) pages
slide-67
SLIDE 67

67 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Things To Optimize

  • Development

– Ruby code – Rails code – Database queries – Alternative Ruby

  • Production

– Shared filesystems and databases – Live debugging – Load balancing

  • Frontend

– HTTP – Javascript – Internet Explorer

slide-68
SLIDE 68

68 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: IE

Unsurprisingly... IE is also slow! Slow things that are especially slow in IE:

  • $() and $$(), even on small pages
  • getElementsByName()
  • style switching
slide-69
SLIDE 69

69 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Optimize Frontend: IE

Good things about IE:

profiler in IE8 fast in IE => fast everywhere else!

slide-70
SLIDE 70

70 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast!

So, you've optimized your application. How to keep it fast?

slide-71
SLIDE 71

71 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast!

Measure, measure and measure... Use profiler Optimize CPU and Memory

slide-72
SLIDE 72

72 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Measure

Keep a set of benchmarks for most frequent user requests. For example:

Benchmark Burndown 120 0.70 ± 0.00 Benchmark Inc. Burndown 120 0.92 ± 0.01 Benchmark Sprint 20 x (1+5) (C) 0.45 ± 0.00 Benchmark Issues 100 (C) 0.34 ± 0.00 Benchmark Prediction 120 0.56 ± 0.00 Benchmark Progress 120 0.23 ± 0.00 Benchmark Sprint 20 x (1+5) 0.93 ± 0.00 Benchmark Timeline 5x100 0.11 ± 0.00 Benchmark Signup 0.77 ± 0.00 Benchmark Export 0.20 ± 0.00 Benchmark Move Here 20/120 0.89 ± 0.00 Benchmark Order By User 0.98 ± 0.00 Benchmark Set Field (EP) 0.21 ± 0.00 Benchmark Task Create + Tag 0.23 ± 0.00 ... 30 more ...

slide-73
SLIDE 73

73 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Measure

Benchmarks as a special kind of tests:

class RenderingTest < ActionController::IntegrationTest def test_sprint_rendering login_with users(:user), "user" benchmark :title => "Sprint 20 x (1+5) (C)", :route => "projects/1/sprints/3/show", :assert_template => "tasks/index" end end Benchmark Sprint 20 x (1+5) (C) 0.45 ± 0.00

slide-74
SLIDE 74

74 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Measure

Benchmarks as a special kind of tests:

def benchmark(options = {}) (0..100).each do |i| GC.start pid = fork do begin

  • ut = File.open("values", "a")

ActiveRecord::Base.transaction do elapsed_time = Benchmark::realtime do request_method = options[:post] ? :post : :get send(request_method, options[:route]) end

  • ut.puts elapsed_time if i > 0
  • ut.close

raise CustomTransactionError end rescue CustomTransactionError exit end end Process::waitpid pid ActiveRecord::Base.connection.reconnect! end values = File.read("values") print "#{mean(values).to_02f} ± #{sigma(values).to_02f}\n" end

slide-75
SLIDE 75

75 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Query Testing

Losing 10ms in benchmark might seem OK Except that it's sometimes not because you're running one more SQL query

slide-76
SLIDE 76

76 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Query Testing

def test_queries queries = track_queries do get :index end assert_equal queries, [ "Foo Load", "Bar Load", "Event Create" ] end

slide-77
SLIDE 77

77 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Query Testing

module ActiveSupport class BufferedLogger attr_reader :tracked_queries def tracking=(val) @tracked_queries = [] @tracking = val end def debug_with_tracking(message) @tracked_queries << $1 if @tracking && message =~ /3[56]\;1m(.* (Load|Create| Update|Destroy)) \(/ debug_without_tracking(message) end alias_method_chain :debug, :tracking end end class ActiveSupport::TestCase def track_queries(&block) RAILS_DEFAULT_LOGGER.tracking = true yield result = RAILS_DEFAULT_LOGGER.tracked_queries RAILS_DEFAULT_LOGGER.tracking = false result end end

slide-78
SLIDE 78

78 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Use Profiler

Profiler will always tell you what's wrong:

%self total self child calls name 8.39 0.54 0.23 0.31 602 Array#each_index 7.30 0.41 0.20 0.21 1227 Integer#gcd 6.20 0.49 0.17 0.32 5760 Timecell#date 5.11 0.15 0.14 0.01 1 Magick::Image#to_blob

gem install ruby-prof use "performance" tests for Rails 2.x KCachegrind to visualize the results http://kcachegrind.sourceforge.net

slide-79
SLIDE 79

79 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Use Profiler

slide-80
SLIDE 80

80 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Optimize CPU and Memory

Memory profiler will explain the missing details: Example benchmark: 5.52 sec request time, 1.07 sec GC time Consumed memory: 55M Ruby runs GC after allocating 8M memory or doing 10000 allocations Simple math: 55 / 8 = 6 GC calls Each GC call takes 0.18 sec!

slide-81
SLIDE 81

81 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Keep It Fast: Optimize CPU and Memory

How to use memory profiler: Recompile Ruby with GC patch http://www.acunote.com/system/ruby186-p287-gc.patch Reinstall ruby-prof Use RUBY_PROF_MEASURE_MODE=memory when running ruby-prof http://blog.pluron.com/2008/02/memory-profilin.html

slide-82
SLIDE 82

82 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Remember!

Measure, measure, measure... (with ruby-prof) Optimize only what's slow Optimize not only CPU, but memory Optimize for user experience Keep a set of performance regression tests Monitor performance

slide-83
SLIDE 83

83 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Help Us Make Ruby and Rails Faster!

We need your help!

  • we'd like to see GC patch applied to mainstream Ruby

(to profile memory without recompiling Ruby)

  • we'd like to see people porting their gems to Ruby 1.9
  • we'd like to see more real-world benchmarks
slide-84
SLIDE 84

84 / 84 Alexander Dymo • Advanced Performance Optimization of Rails Applications • RailsConf 2009

Project Management and Scrum Software

Thanks!

Slides for this talk: http://en.oreilly.com/rails2009/public/schedule/detail/8615 Rails performance articles and more: http://blog.pluron.com