Session Slides: Improving Web Performance with Dynamic Compression - - PowerPoint PPT Presentation

Session Slides: Improving Web Performance with Dynamic Compression by Slava Bizyayev <slava@apache.org>

ApacheCon US 2005 Wednesday, December 14 2005 San Diego, CA

Slava Bizyayev Session Slides: Improving Web Performance with Dynamic Compression Slide 1

Last modified Wed Oct 19 13:11:03 2005 GMT

Slava Bizyayev Session Slides: Improving Web Performance with Dynamic Compression Slide 2

Slava Bizyayev Basics of Content Compression Slide 3

1 Basics of Content Compression

Slava Bizyayev Basics of Content Compression Slide 4

1.1 Introduction

shorter response times for clients lower consumption of bandwidth for content providers

Slava Bizyayev Basics of Content Compression Slide 5

Slava Bizyayev Basics of Content Compression Slide 6

1.2 A Bit of the Pure Theory

gzip encoding deflate compression

Slava Bizyayev Basics of Content Compression Slide 7

1.2.1 LZ77 Algorithm with Huffman Coding

LZ77 is a lossless data compression algorithm published by Abraham Lempel and Jacob Ziv in 1977 it was never patented LZ77 is the sliding window compression algorithm of the class of dictionary coders.

Slava Bizyayev Basics of Content Compression Slide 8

1.2.2 Green Eggs and Ham

That Sam-I-am! That Sam-I-am! I do not like that Sam-I-am! Do you like green eggs and ham? I do not like them, Sam-I-am. I do not like green eggs and ham. Compressed this becomes: |That Sam|-I-am! []|I do not| like t[]| Do you[]|green eg|gs and h|am? []em,|[].[][].

Slava Bizyayev Basics of Content Compression Slide 9

After applying dynamic Huffman coding:

• rw-rw-r-- 1 bon bon 182 May 17 04:39 poem.txt
• rw-rw-r-- 1 bon bon 110 May 17 04:39 poem.gz

Slava Bizyayev Basics of Content Compression Slide 10

1.3 Compression as a Filter

It always takes its input as plain text data that it converts to another binary form, and outputs to some destination. Content compression should usually be the last executable component in any chain of filters. Only encryption can be optionally applied after compression.

Slava Bizyayev Basics of Content Compression Slide 11

1.4 How it Works over HTTP

C05 --> S06 GET /html/coolfile.html HTTP/1.1 C05 --> S06 Host: devl4.outlook.net C05 --> S06 Accept: */* C05 --> S06 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98) C05 --> S06 Accept-Encoding: gzip, deflate == Body was 0 bytes == C05 <-- S06 HTTP/1.1 200 OK C05 <-- S06 Server: Apache C05 <-- S06 Content-Type: text/plain C05 <-- S06 Content-Encoding: gzip C05 <-- S06 Content-Length: 110 C05 <-- S06 Vary: Accept-Encoding == Transmission: text gzip == == Restored Body was 182 bytes == Slava Bizyayev Basics of Content Compression Slide 12

Slava Bizyayev Server Configuration for Content Compression Slide 13

2 Server Configuration for Content Compression

Slava Bizyayev Server Configuration for Content Compression Slide 14

2.1 Content Filters on Apache

There are version specific content compression software for Vanilla Apache-1.3 mod_perl enabled Apache-1.3 Apache-2 The major differences between the compression components are

• riginating from the differences in content filtering concepts.

Slava Bizyayev Server Configuration for Content Compression Slide 15

2.2 Compression on Vanilla Apache-1.3

module chain two compression modules written in C: mod_deflate by Igor Sysoev mod_gzip by Kevin Kiley

Slava Bizyayev Server Configuration for Content Compression Slide 16

2.2.1 Sample config for mod_gzip

<IfModule mod_gzip.c> ######################## ### responsibilities ### ######################## mod_gzip_on Yes ###################################### ### statically precompressed files ### ###################################### mod_gzip_can_negotiate Yes # extension (suffix) for statically precompressed files mod_gzip_static_suffix .gz AddEncoding gzip .gz mod_gzip_update_static No ################### ### bureaucracy ### ################### mod_gzip_command_version ’/mod_gzip_status’ ####################### ### data management ### ####################### mod_gzip_keep_workfiles No ##################

Slava Bizyayev Server Configuration for Content Compression Slide 17

### file sizes ### ################## # minimum size (in bytes) for files to be compressed mod_gzip_minimum_file_size 500 # maximum size (in bytes) for files to be compressed mod_gzip_maximum_file_size 500000 # maximum size (in bytes) for files to be compressed in memory mod_gzip_maximum_inmem_size 60000 #################### ### requirements ### #################### mod_gzip_min_http 1000 # Possible values: ’GET’, ’POST’ or a list of both values. mod_gzip_handle_methods GET POST ############### ### filters ### ############### mod_gzip_item_exclude reqheader "User-agent: Mozilla/4.0[678]"

mod_gzip_item_include file \.html$ mod_gzip_item_exclude file \.js$ mod_gzip_item_exclude file \.css$ mod_gzip_item_include file \.pl$ mod_gzip_item_include handler ^cgi-script$ mod_gzip_item_include mime ^text/html$ mod_gzip_item_include mime ^text/plain$

Slava Bizyayev Server Configuration for Content Compression Slide 18

mod_gzip_item_include mime ^httpd/unix-directory$ mod_gzip_item_exclude mime ^image/ ########################## ### transfer encodings ### ########################## mod_gzip_dechunk Yes ############### ### logging ### ############### . . . CustomLog logs/mod_gzip.log common_with_mod_gzip_info2 mod_gzip_add_header_count Yes ############### ### proxies ### ############### mod_gzip_send_vary On </IfModule>

Slava Bizyayev Server Configuration for Content Compression Slide 19

2.3 Compression on mod_perl enabled Apache-1.3

Apache::Filter Apache::OutputChain PerlModule Apache::Filter <Files ~ "*\.blah"> SetHandler perl-script PerlSetVar Filter On PerlHandler Filter1 Filter2 Filter3 </Files>

Slava Bizyayev Server Configuration for Content Compression Slide 20

PerlModule Apache::OutputChain PerlModule Apache::GzipChain PerlModule Apache::SSIChain PerlModule Apache::PassHtml <Files *.html> SetHandler perl-script PerlHandler Apache::OutputChain Apache::GzipChain Apache::SSIChain Apache::PassHtml </Files>

Slava Bizyayev Server Configuration for Content Compression Slide 21

2.3.1 Content Compression Handlers

Apache::Compress by Ken Williams Apache::Dynagzip by Slava Bizyayev Apache::Dynagzip’s features include: Support for both HTTP/1.0 and HTTP/1.1 Control over the chunk size Support for Perl, Java, or C/C++ CGI applications. Advanced Vary control over the proxy cache

Slava Bizyayev Server Configuration for Content Compression Slide 22

Optional Expires control over content lifetime Optional support for server-side caching Optional extra-light compression Apache::Gzip by Lincoln Stein and Doug MacEachern Apache::GzipChain by Andreas Koenig

Slava Bizyayev Server Configuration for Content Compression Slide 23

2.3.1.1 sample configuration for Apache::Dynagzip

PerlModule Apache::Filter PerlModule HTML::Mason::ApacheHandler PerlModule Apache::Dynagzip <Directory /path/to/subdirectory> <FilesMatch "\.html$"> SetHandler perl-script PerlHandler HTML::Mason::ApacheHandler Apache::Dynagzip PerlSetVar Filter On </FilesMatch> </Directory>

Slava Bizyayev Server Configuration for Content Compression Slide 24

2.4 Compression on Apache-2

input filters

• utput filters.

The set of output filters that apply to data can be manipulated with the directives: SetOutputFilter AddOutputFilter RemoveOutputFilter

Slava Bizyayev Server Configuration for Content Compression Slide 25

The following user-selectable output filters are currently provided with the Apache-2 server distribution: INCLUDES Server-Side Includes processing by mod_include DEFLATE Compress output before sending it to the client using mod_deflate In addition, module mod_ext_filter allows for external programs to be defined as filters.

Slava Bizyayev Server Configuration for Content Compression Slide 26

2.4.1 mod_deflate for Apache-2

by Ian Holsman provides gzip-encoded content. is capable of gzipping outbound traffic from any content generator supports flushing an output filter itself has sufficient configuration options

Slava Bizyayev Server Configuration for Content Compression Slide 27

2.4.2 A sample configuration of mod_deflate :

<Location /> SetOutputFilter DEFLATE # Netscape 4.x has some problems... BrowserMatch ^Mozilla/4 gzip-only-text/html # Netscape 4.06-4.08 have some more problems BrowserMatch ^Mozilla/4\.0[678] no-gzip # Don’t compress images SetEnvIfNoCase Request_URI \ \.(?:gif|jpe?g|png)$ no-gzip dont-vary # Make sure proxies don’t deliver the wrong content Header append Vary User-Agent env=!dont-vary </Location>

Slava Bizyayev Server Configuration for Content Compression Slide 28

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 29

3 Dynamic Compression

• vs. Static Compression

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 30

3.1 Response Latency

It apears important to know that the streaming compression can be performed over HTTP/1.1 as well as over HTTP/1.0.

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 31

3.2 Google vs. Yahoo in the Content Compression Contest

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 32

3.2.1 Yahoo

C05 --> S07 GET / HTTP/1.1 C05 --> S07 Host: www.yahoo.com C05 --> S07 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) C05 --> S07 Accept: */* C05 --> S07 Accept-Language: en-us C05 --> S07 Accept-Encoding: gzip, deflate, compress;q=0.9 == Body was 0 bytes == C05 <-- S07 HTTP/1.1 200 OK C05 <-- S07 Transfer-Encoding: chunked C05 <-- S07 Content-Type: text/html C05 <-- S07 Cache-Control: private C05 <-- S07 Vary: User-Agent C05 <-- S07 Content-Encoding: gzip == Latency = 0.419 seconds, Extra Delay = 0.299 seconds == Chunk Log == 29d6 (hex) = 10710 (dec) 0 (hex) = 0 (dec)

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 33

3.2.2 Google

C05 --> S07 GET /search?hl=en&q=Dynagzip&btnG=Google+Search&meta= HTTP/1.1 C05 --> S07 Host: www.google.co.uk C05 --> S07 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) C05 --> S07 Accept: */* C05 --> S07 Accept-Encoding: gzip, deflate, compress;q=0.9 == Body was 0 bytes == C05 <-- S07 HTTP/1.1 200 OK C05 <-- S07 Transfer-Encoding: chunked C05 <-- S07 Content-Type: text/html C05 <-- S07 Server: GWS/2.1 C05 <-- S07 Content-Encoding: gzip == Latency = 0.040 seconds, Extra Delay = 0.614 seconds == Chunk Log == 632 (hex) = 1586 (dec) 8a9 (hex) = 2217 (dec) 8a (hex) = 138 (dec) 0 (hex) = 0 (dec)

Slava Bizyayev Dynamic Compression vs. Static Compression Slide 34

Slava Bizyayev Real World of Web Content Compression Slide 35

4 Real World of Web Content Compression

Slava Bizyayev Real World of Web Content Compression Slide 36

4.1 Why it is Important

noticeably increases delivery speed to clients allows providers to serve higher content volumes

Slava Bizyayev Real World of Web Content Compression Slide 37

Slava Bizyayev Real World of Web Content Compression Slide 38

4.2 How Much is Improvement

transmission efficiency from 3 to 20 times. Image files normally employ their own compression techniques that reduce the advantage of further compression.

Slava Bizyayev Real World of Web Content Compression Slide 39

4.3 How Hard it is to Implement

typically no more than installing and configuring an appropriate Apache software component no client side changes or settings are required

Slava Bizyayev Real World of Web Content Compression Slide 40

4.4 It is Easy to Combine Compression with Encryption

Compressed content can be securely transmitted over SSL

Slava Bizyayev Real World of Web Content Compression Slide 41

HTTP data compression does not conflict with any physical-level data compression like V.42bis, V.44, and/or NNP5 used in the current generation of modems. Indeed, they complement each other.

Slava Bizyayev Real World of Web Content Compression Slide 42

4.5 Compressing the Last Mile to the Client

Slava Bizyayev Real World of Web Content Compression Slide 43

The Last Mile ISP project is based on the Dynagzip project.

Slava Bizyayev Real World of Web Content Compression Slide 44

4.6 Compressing XML Communications

Basically, this is just another content body at the HTTP level. All things like native XML, RSS, etc. can be very effectively compressed at the HTTP level with the same software components. The use of the gzip dictionary in this case might result in additional benefits..

Slava Bizyayev Real World of Web Content Compression Slide 45

Slava Bizyayev That’s all for now folks. Slide 46

5 That’s all for now folks.

Slava Bizyayev That’s all for now folks. Slide 47

5.1 Thanks

Kevin Kiley, Igor Sysoev, Michel Schroepl, Henrik Nordstrom all subscribers of the mod_perl, mod_gzip, and squid users mailing lists Stas Bekman Dan Hansen Slav Company, Ltd.

Slava Bizyayev That’s all for now folks. Slide 48

5.2 Further Questions?

Take a look at Web Content Compression FAQ at

http://perl.apache.org/docs/tutorials/client/compression/compression.html

Grab me at the corridor and demand answers Ask at modperl@perl.apache.org

Slava Bizyayev That’s all for now folks. Slide 49