4x High Performance for Drupal - Step by Step Fabian Franz - - PowerPoint PPT Presentation

▶
4x high performance for drupal step by step
SMART_READER_LITE
LIVE PREVIEW

4x High Performance for Drupal - Step by Step Fabian Franz - - PowerPoint PPT Presentation

Feb 04, 2024 123 likes •1.89k views

4x High Performance for Drupal - Step by Step Fabian Franz (Fabianx) - @fabianfranz Performance Track 4x High Performance for Drupal Presented by Fabian Franz Step by Step Your BOSS is calling! It happens to the best of us

slide-1
SLIDE 1
slide-2
SLIDE 2

4x High Performance for Drupal - Step by Step

Fabian Franz (Fabianx) - @fabianfranz

Performance Track
slide-3
SLIDE 3

4x High Performance for Drupal

Presented by Fabian Franz

Step by Step

slide-4
SLIDE 4

Your BOSS is calling!

slide-5
SLIDE 5

It happens to the best

  • f us …
slide-6
SLIDE 6

Especially during DrupalCon …

slide-7
SLIDE 7

… or during elections.

slide-8
SLIDE 8

The site goes down, the site is slow, grab a tut’, and make it grow!

slide-9
SLIDE 9

But first …

slide-10
SLIDE 10

Lets start with a little story …

slide-11
SLIDE 11

„Where is the power

  • f Drupal ... I hate

Drupal!“

slide-12
SLIDE 12

„... I hate Drupal!“

„[...] and it always overload the database (mysql) some times the load reach 200 and it never reach it before ... I enabled core cache and the server got down again next day ..."

  • groups.drupal.org/node/158244
slide-13
SLIDE 13

My site is so slow ... HEEEEELP!

  • "Where is the power of

Drupal ... I hate Drupal!"

  • This is really sad ...
slide-14
SLIDE 14

My site is so slow ... HEEEEELP!

  • There is a need for High Performance

Drupal!

  • Faster sites earn more money
  • Faster sites get ranked higher by

Google

slide-15
SLIDE 15

My site is so slow ... HEEEEELP!

  • Visitors love fast sites
  • Mentioned in the media? What if your

server goes down exactly then?

slide-16
SLIDE 16

BUT ...

slide-17
SLIDE 17

… THE QUESTION IS …

slide-18
SLIDE 18

How do I get a blazingly fast site?

slide-19
SLIDE 19

Doing it Wrong

slide-20
SLIDE 20

Doing it Wrong

"Okay, I have now tweaked my Sauerkraut* settings, but the site is still

  • slow. What Sauerkraut settings do I

need to tweak so that it is as fast as xyz.com?"

* Sauerkraut was APC in this case, but we come to that later
slide-21
SLIDE 21

Doing it Wrong

„I have setup 10 Slave DB Servers, but

  • nce I test the site it is sooo slooooow!“
slide-22
SLIDE 22

Doing it Wrong

„I have setup NGINX with AdvAgg and Varnish combined with Entitycache and Views_Opt_Cache - still the performance remains the same. :-(“

slide-23
SLIDE 23

„Have you setup Memcache?

slide-24
SLIDE 24

???

slide-25
SLIDE 25

Doing it Wrong

“I have setup static page caching for all the pages. The high traffic day can come! What could possibly go wrong?”

slide-26
SLIDE 26

We all wish ...

slide-27
SLIDE 27

... we had ...

slide-28
SLIDE 28

THE MAGIC PILL!

slide-29
SLIDE 29

Optimization is a process

  • not a pill

„Just one pill and the site is fast!“

slide-30
SLIDE 30

Optimization is a process

  • not a pill

4 common ways to fail

  • 1. Optimizing one part to death while

neglecting all the others: "If you build the house on one pillar, it'll not hold long ..."

slide-31
SLIDE 31

Optimization is a process

  • not a pill

4 common ways to fail

  • 2. Optimizing things without knowing

where the pain is: "Is the bottleneck MySQL, PHP, Apache or something else?"

slide-32
SLIDE 32

Optimization is a process

  • not a pill

4 common ways to fail

  • 3. Optimizing things with new methods

without really understanding them: "Reinvent the wheel or stand on the shoulder of giants?""

slide-33
SLIDE 33

Optimization is a process

  • not a pill

4 common ways to fail

  • 4. Optimizing things without testing it'll

hold the load: "You are featured by BigNews.com - Your server goes down."

slide-34
SLIDE 34

4 Common Ways to Fail

  • 1. Optimizing one part to death
  • 2. Optimizing just random parts
  • 3. Optimizing parts with <BuzzWord>
  • 4. Optimizing without testing
slide-35
SLIDE 35

"Ouch, that are lots of ways to fail ..."

slide-36
SLIDE 36

„That is all so complicated.“ *sigh*

slide-37
SLIDE 37

This is all so

  • complicated. *sigh*

"Is there nothing I can do to make this easier and have a fast site?"

slide-38
SLIDE 38

This is all so

  • complicated. *sigh*
  • The easy answer:

Hire a performance consultant.

slide-39
SLIDE 39

„This is all so complicated.“ *sigh* Hire a performance consultant NOW. CALL NOW IN THIS SECOND:

  • and enjoy blazingly fast sites.

0800 - DRUPAL PERFORMANCE

slide-40
SLIDE 40

The End

Now you know Performance is really difficult to get right and that you should hire a Performance Consultant.

Remember this number: 0800 - DRUPAL PERFORMANCE QUESTIONS?

slide-41
SLIDE 41

JUST KIDDING ...

  • Okay, okay. You got me ...

"Hiring a performance consultant can be really useful at times, but even more useful is learning and spreading the knowledge."

slide-42
SLIDE 42

JUST KIDDING ...

4x HIGH PERFORMANCE FOR DRUPAL - STEP BY STEP

  • „Lets stand on the shoulders of Giants

and walk the paths of our ancestors!“

slide-43
SLIDE 43

Your Mission

slide-44
SLIDE 44

Loading your mission ...

slide-45
SLIDE 45

The Mission

  • Drupal 7 / 8
  • Several Performance Problems
  • ... real life problems!
slide-46
SLIDE 46

Lets meet some friends …

slide-47
SLIDE 47

… and help them in their need …

slide-48
SLIDE 48

Your Mission

  • D. Pages feel

slow, sluggish and big ... ... and are totally unhappy :-( „This is sooooo heavy load.“

slide-49
SLIDE 49

Your Mission

  • Mrs. MySQL is

exhausted and needs a time out :-( *sigh* :-( „I just need a SELECT break.“

slide-50
SLIDE 50

Your Mission

  • Mr. Apache is

sweating under the load :-(( „I give 100% all the time, but this is just too much.“

slide-51
SLIDE 51

Your Mission

  • Mr. Code is

buggy and a real trouble maker ;-) „Yeah! He he he!“

slide-52
SLIDE 52

Your Mission

  • D. Pages feel slow and are unhappy :-(
  • Mrs. MySQL is exhausted and needs a

time out :-(

  • Mr. Apache is sweating under the

load :-((

  • Mr. Code is buggy and a real trouble

maker ;-)

slide-53
SLIDE 53

The Task

  • Investigate

and Fix!

  • Let‘s go!
slide-54
SLIDE 54
  • 1. Server

Performance

slide-55
SLIDE 55

Measuring Server Performance....

slide-56
SLIDE 56

Measuring Server Performance....

  • System Load: 4.14
  • Page Load Time: 20 sec
  • Apache Load: 100%
slide-57
SLIDE 57

How to measure Performance on Server?

  • top command
slide-58
SLIDE 58

How to measure Performance on Server?

  • Handy Drush Command for page

generation time of any page (Drupal 7):

time drush php-eval '
 $path="node/17"; menu_set_active_item($path); menu_execute_active_handler($path, TRUE);' > /dev/null

slide-59
SLIDE 59

Uhm, really?

slide-60
SLIDE 60

Yes!

slide-61
SLIDE 61

Why would I need that?

slide-62
SLIDE 62

Production debugging!

slide-63
SLIDE 63

Sometimes problems

  • nly show up on

production.

slide-64
SLIDE 64

So now we know there is a problem.

slide-65
SLIDE 65

How do we solve them?

slide-66
SLIDE 66

The 4 Shoulders of the Giants

slide-67
SLIDE 67

While I have said …

slide-68
SLIDE 68

…. that you should know your pain points first …

slide-69
SLIDE 69

… there is a “stack”

slide-70
SLIDE 70

…. that many high performance sites use.

slide-71
SLIDE 71

Pressflow / Good code APC / Opcache Varnish / NGINX / CDN Memcache / Redis

slide-72
SLIDE 72

Pressflow / Good code APC / Opcache Varnish / NGINX / CDN Memcache / Redis PHP-FPM (!)

Hey there, I am new!

slide-73
SLIDE 73

Pressflow / Good code APC / Opcache Varnish / NGINX / CDN Memcache / Redis BigPipe + Streaming

Hey there, I am also new!

slide-74
SLIDE 74

Uhm, BigPipe, you have already been handled yesterday!

slide-75
SLIDE 75

Pressflow / Good code APC / Opcache Varnish / NGINX / CDN Memcache / Redis PHP-FPM (!) + PHP 7

Hey there, I am new!

slide-76
SLIDE 76

So how can those help me?

slide-77
SLIDE 77

Pressflow

  • Only really relevant for Drupal 6 sites
  • Drupal 7 already includes most

Pressflow patches / approaches

  • Drupal 8 has performance best

practices all around.

slide-78
SLIDE 78

Inofficial Pressflow

  • https://groups.drupal.org/node/

210683 (Wiki)

  • All collected performance patches that

are relevant for Drupal 7.

  • Drupal 8: All committed already :D.
slide-79
SLIDE 79

PHP 7

  • Up to 50% faster

  • Drupal 8 supports PHP 7 since 8.0.0
  • Drupal 7 supports PHP 7 since 7.50
slide-80
SLIDE 80

PHP 7

  • Just use it and start transforming your

stack to it.

  • PHP 5.6



 End-of-Life 31/12/2016
 LTS 31/12/2018

slide-81
SLIDE 81

PHP 7: Is it worth it?

  • Seriously: It is

really really worth it!
 
 Admin performance: Image from:
 https://pantheon.io/blog/measuring-php-7-performance-new-relic- nobsbenchmarks by Josh Koenig

slide-82
SLIDE 82

PHP 7

  • Most hosters already support it,

because it is just that much better.
 
 Fun fact:
 
 Drupal 8 was the reason why PHP 7.0.0 release was delayed by 2 weeks.

slide-83
SLIDE 83

APC

  • Alternative PHP Cache
  • Highly recommended (easy to install)
  • PHP PECL Extension
  • Speeds up PHP execution by caching

pre-compiled PHP objects

slide-84
SLIDE 84

Opcache (PHP >= 5.5)

  • Opcode Cache
  • Highly recommended (easy to install)
  • In PHP >=5.5 by default (Yeah!!!)
  • Speeds up PHP execution by caching

pre-compiled PHP objects

slide-85
SLIDE 85

Opcache: Is it worth it?

slide-86
SLIDE 86

OpCache: Is it worth it?

  • Yes
  • Saves:
  • ~ 100 ms
  • ~ 20 MB
slide-87
SLIDE 87

OpCache: What can go wrong?

  • opcache_reset() buggy



 => in all released PHP versions.

  • opcache is disabled during restart; can

take up to two minutes.

slide-88
SLIDE 88

OpCache: What can go wrong?

  • Size it properly (it its full => restart)

  • Use php-fpm graceful restart.

  • Blog Post about systemd + php-fpm for

graceful restart coming soon on tag1consulting.com

slide-89
SLIDE 89

OpCache: What can go wrong?

  • Look at the

stats!

  • Opcache going
  • ffline can lead

to huge spikes!

slide-90
SLIDE 90

Memcache/Redis

  • Replaces caching in MySQL Database
  • Key/Value Store in Main Memory





store(Apple, 10)
 get(Apple) == 10

  • Very Fast!
slide-91
SLIDE 91

Memcache/Redis: What does it get me?

slide-92
SLIDE 92

Memcache/Redis: What does it get me?

  • Way less load on the database
  • Overall faster caches
  • Much easier to scale up


( Distributed key-value storage)

slide-93
SLIDE 93

LCache (d.go.to/lcache)

  • Newest kid on the block (alpha release)
  • Key/Value Store within PHP (APCu)





store(Apple, 10)
 get(Apple) == 10

  • Even faster than memcache!
slide-94
SLIDE 94

LCache: What does it get me?

  • Transaction log + consistent store in
  • ne
  • No additional service needed (!) -

except for APC/APCu extension.


  • Presentation online already.
slide-95
SLIDE 95

SuperCache

  • Similar to lcache / ChainedFast in core

  • Especially optimized for write

performance and cache tags


  • => Caveat: Might have race conditions.
slide-96
SLIDE 96

Varnish

  • Save whole response to memory
  • Serve response from memory
  • Like a “shield” for your server
  • Varnish is crucial for high throughput.


(unless you use NGINX)

slide-97
SLIDE 97

Varnish: That sounds pretty complicated!

slide-98
SLIDE 98

Varnish: That sounds pretty complicated!

  • Best practice configurations:
  • https://fourkitchens.atlassian.net/wiki/

display/TECH/Configure+Varnish+3+for +Drupal+7

  • https://www.lullabot.com/blog/article/

configuring-varnish-high-availability- multiple-web-servers

slide-99
SLIDE 99

Varnish: What does it get me?

  • 50 ms response times!
slide-100
SLIDE 100

Your Mission: UPDATE!

slide-101
SLIDE 101

Your Mission: UPDATE!

  • Anonymous Pages: „We are

blazingly fast, still big, but quite happy“.

slide-102
SLIDE 102

Your Mission: UPDATE!

  • Mrs. MySQL: „I have less to do

now, but if I have it is still too

  • much. Those authenticated

users ...“

slide-103
SLIDE 103

Your Mission: UPDATE!

  • Mr. Apache: „I have much less to

do, but when those authenticated users come, I still sweat.“

  • „And I hate those anonymous

utm_ requests!“

slide-104
SLIDE 104

Quick Fix for GA Problem

  • VCL (Varnish 2.1.x) Rule:
# Strip out Google Analytics campaign variables.
 # They are only needed by the javascript 
 # running on the page.
 # utm_source, utm_medium, utm_campaign, gclid if(req.url ~ "(\?|&)(gclid|utm_[a-z]+)=") { set req.url = regsuball(req.url, "(gclid|utm_[a-z]+)=[^ \&]+&?", ""); set req.url = regsub(req.url, "(\?|&)$", ""); }
slide-105
SLIDE 105
  • 2. Client

Performance

slide-106
SLIDE 106

Measuring Client Performance....

slide-107
SLIDE 107

Measuring Client Performance....

  • Page Load Size: 300 kB
  • Page Load Time: 20 sec
slide-108
SLIDE 108

How to measure Performance on Client?

  • Use Google Chrome

Developer Toolbar

➡Network Tab

slide-109
SLIDE 109

How to measure Performance on Client?

  • http://webpagetest.org
slide-110
SLIDE 110

Why are those pages so big?

slide-111
SLIDE 111

Need Compression of CSS and JS!

slide-112
SLIDE 112

Compression of CSS and JS

  • Very easy to setup
  • In Drupal 7 Core:
  • Administration Menu

➡Configuration ➡Development ➡Performance

slide-113
SLIDE 113

Compression of CSS and JS

  • Very easy to setup
  • In Drupal 8 Core:
  • Enabled by default!
slide-114
SLIDE 114

Compression of CSS and JS

  • Aggregation and

Compression

  • Do this before Go-Live!
  • Users will thank you

for it!

  • Needs: mod_rewrite

and mod_headers

slide-115
SLIDE 115

Compression of CSS and JS (Drupal 6)

  • Can be also done in Apache (e.g.)
  • mod_deflate
  • But: Can put high load on the server!

✓Combine with: Varnish

slide-116
SLIDE 116

Minimize CSS and JS source files!

slide-117
SLIDE 117

AdvAgg (Drupal 7)

  • => Way less

aggregates

slide-118
SLIDE 118

Alternatives to AdvAgg:

  • http://drupal.org/project/agrcache





  • goes hopefully into core for Drupal 8 (https://

www.drupal.org/node/1014086)
 
 +

  • https://www.drupal.org/project/simple_aggregation





  • very simple, but effective module to reduce number
  • f aggregates, very low risk
slide-119
SLIDE 119

Set proper caching headers

slide-120
SLIDE 120

Set proper caching headers

  • Goal: Cache for some

time on Client machine

  • Drupal 7 sets the

headers for you

  • Just need to adjust the

numbers.

slide-121
SLIDE 121

Client Performance: What did we achieve?

  • Only 4 HTTP requests.
  • Much faster page load time.
slide-122
SLIDE 122

Your Mission: UPDATE!

slide-123
SLIDE 123

Your Mission: UPDATE!

✓Anonymous Pages: „We are

blazingly fast, really slick, and really really happy :-)“.

slide-124
SLIDE 124

Your Mission: UPDATE!

  • Anonymous Pages: „We are

blazingly fast, really slick, and really really happy :-)“.

slide-125
SLIDE 125

Additional techniques

  • CDN: Content Delivery

Network

  • Caches files close to

the users location

  • Useful for images,

CSS / JS files

  • http://drupal.org/

project/cdn

slide-126
SLIDE 126

Additional techniques

  • CDN: Content Delivery

Network

  • Fastly
  • CloudFlare
  • => Put your whole site

behind a CDN!
 
 Nicely integrated with tags for Drupal 8!

slide-127
SLIDE 127

Additional techniques

  • AJAX/PJAX: Only re-

load the content you need

  • Useful for pagers,

image galleries

  • http://drupal.org/

project/pjax

slide-128
SLIDE 128

Additional techniques

  • Refreshless: Only re-

load the content you need

  • Drupal 8 only
  • Out of the box!
  • http://drupal.org/

project/refreshless

slide-129
SLIDE 129

Quick-Tip: Fix slow JS!

  • Unresponsive script error on loading of

page?

  • Workaround -- Wrap Code in:



 setTimeout(function() {
 // Old code
 }, 100);

slide-130
SLIDE 130
  • 3. Module

Performance

slide-131
SLIDE 131

Measuring Module Performance....

slide-132
SLIDE 132

Measuring Module Performance....

  • Drupal Bootstrap: 240 ms
  • menu_execute_active_handler: 6 sec
  • Memory Usage: 104 MB
slide-133
SLIDE 133

How to measure Module Performance?

  • Use xhprof PHP

extension

  • Integration via xhprof

module

➡admin/config/

development/devel

slide-134
SLIDE 134

How to measure Module Performance?

  • Drupal 8: webprofiler
  • Drupal 7: block_timer
  • Want insights about

your caches?
 
 heisencache

slide-135
SLIDE 135

Common Pitfalls

slide-136
SLIDE 136

Common Pitfalls

  • variable_set on each page request
  • ➡Can bring your DB server to its knees!
slide-137
SLIDE 137

Common Pitfalls

  • Generally: Writes on each page

request

  • ➡Can bring your DB server to its knees!
slide-138
SLIDE 138

Common Pitfalls

  • Anonymous $SESSION set for saving

simple data

  • Example: low_bw flag

➡Disables anonymous caching ➡Solution: Use Javascript to set/receive

cookies directly and change page.

slide-139
SLIDE 139

Common Pitfalls

  • Having installed way

toooooooooooooooooooooooooooo many modules

➡Adds a little to the page request every

  • time. (Drupal 6: worse)
slide-140
SLIDE 140

Interesting Pitfalls

  • views loading 5000 nodes in one page

request for openlayers

  • ➡Can easily exceed memory and takes

quite some time to attach fields.

➡Adv. Solution: openlayers_quick_query

sandbox

slide-141
SLIDE 141

Improve performance

  • Use block caching
  • Use render caching

(render_cache module)

slide-142
SLIDE 142

Improve performance

  • Use

blockcache_alter module

➡Lots of more

  • pportunities for

caching then by default

slide-143
SLIDE 143

Improve performance

  • Setup views caching
  • Setup panels caching
  • => Click those

checkboxes and see what is acceptable for your users!

slide-144
SLIDE 144

Improve performance

  • Drupal 8:

dynamic_page_cache enabled - out of the box!

  • Ensure it is effective!



 Pages are still slow => something might be disabling your caching.

  • Use a placeholder!
slide-145
SLIDE 145

Improve performance

  • Drupal 7: authcache 

  • Needs to be configured;

much easier to use now.

  • Don’t split too much!
slide-146
SLIDE 146

Module Performance: What did we achieve?

  • We found the bad code and removed it!
  • Page is much faster now!
slide-147
SLIDE 147

Your Mission: UPDATE!

slide-148
SLIDE 148

Your Mission: UPDATE!

  • Mrs. MySQL: Almost Happy. She

feels still kinda slow sometimes

✓Apache: Really happy! ✓D. Pages: Really Happy!

slide-149
SLIDE 149
  • 4. Database

Performance

slide-150
SLIDE 150

Measuring MySQL Performance....

slide-151
SLIDE 151

Measuring MySQL Performance....

  • Slow SQL Query: 10 sec
slide-152
SLIDE 152

Measuring MySQL Performance....

  • Tip: Use Percona’s mysql slow query log

analyzer

  • https://www.percona.com/doc/percona-

toolkit/2.2/pt-query-digest.html

slide-153
SLIDE 153

How to measure MySQL Performance?

  • Enable slow query log
  • dbtuner module (6.x
  • nly) / mysqltuner

script for 7.x

  • EXPLAIN queries
slide-154
SLIDE 154

Common Tweaks

slide-155
SLIDE 155

Common Tweaks

  • Use Inno DB
  • ✓Default: Most current MySQL

configuratios

slide-156
SLIDE 156

Common Tweaks

  • Be aware of the barrier!
  • nobarrier=1 for ext3/ext4 file systems
  • Newer Linux kernels / Ubuntu
  • Many many threads by Developers
slide-157
SLIDE 157

Common Tweaks

  • Be aware of the barrier!

➡Attention: Needs to be used with special

hardware for production usage.

➡Useful Guide: Red Hat Handbook

slide-158
SLIDE 158

Common Tweaks

  • Use XFS file system
  • Good and proven file system for MySQL

databases

  • Size appropriately to the use-case
slide-159
SLIDE 159

Fix Slow Queries

  • EXPLAIN queries

✓Add indexes where necessary

  • Run EXPLAIN again
slide-160
SLIDE 160

MySQL Performance: What did we achieve?

  • No more slow queries! (YEAH!)
slide-161
SLIDE 161

Your Mission: UPDATE!

slide-162
SLIDE 162

Werbung / Commercials

slide-163
SLIDE 163

Recap: Best Practices

slide-164
SLIDE 164

Best Practices

  • Setup Base Performance
slide-165
SLIDE 165

Best Practices

  • You want to have your own



 „High Performance Stack“

slide-166
SLIDE 166

Best Practices

  • Having your own stack is not

difficult ...

  • ... as you have seen
slide-167
SLIDE 167

Best Practices

  • Analyze Pain Points first
slide-168
SLIDE 168

Where is the problem?

  • Server based?
  • Client based?
  • Modules?
  • Database?
slide-169
SLIDE 169

Best Practices

  • Optimize Pain Points
slide-170
SLIDE 170

Your Mission: UPDATE!

slide-171
SLIDE 171

Your Mission: UPDATE!

✓Mrs. MySQL: Very Happy! ✓Apache: Really happy! ✓D. Pages: Really Happy!

slide-172
SLIDE 172

Mission: Completed!

Wake up, Neo!

slide-173
SLIDE 173

QUESTIONS?

slide-174
SLIDE 174

FOLLOW ME!

  • Fabian Franz
  • @Fabianx
  • twitter: @fabianfranz
  • github: LionsAd
  • Tag1 Consulting

Performance & Scalability Experts

slide-175
SLIDE 175

JOIN US FOR CONTRIBUTION SPRINTS

First Time Sprinter Workshop - 9:00-12:00 - Room Wicklow2A Mentored Core Sprint - 9:00-18:00 - Wicklow Hall 2B General Sprints - 9:00 - 18:00 - Wicklow Hall 2A

slide-176
SLIDE 176

Evaluate This Session THANK YOU!

events.drupal.org/dublin2016/schedule

WHAT DID YOU THINK?