Choosing T omcat Connectors Jean-Frederic Clere What I will cover - - PowerPoint PPT Presentation

▶
choosing t omcat connectors
SMART_READER_LITE
LIVE PREVIEW

Choosing T omcat Connectors Jean-Frederic Clere What I will cover - - PowerPoint PPT Presentation

Jun 26, 2023 1.02k likes •1.52k views

Choosing T omcat Connectors Jean-Frederic Clere What I will cover Who I am. Connectors JIO, NIO, NIO2, APR AJP/HTTP Proxy AJP/HTTP Performances tests With ab or customized client load generator. Questions?

slide-1
SLIDE 1

Choosing T

  • mcat Connectors

Jean-Frederic Clere

slide-2
SLIDE 2

What I will cover

  • Who I am.
  • Connectors
  • JIO, NIO, NIO2, APR
  • AJP/HTTP
  • Proxy AJP/HTTP
  • Performances tests
  • With ab or customized client load generator.
  • Questions?

11/18/14 2

slide-3
SLIDE 3

Who I am

Jean-Frederic Clere Red Hat Responsible of JWS product. Years writjng JAVA code and server sofuware Tomcat commituer since 2001 Doing OpenSource since 1999 Cyclist/Runner etc Lived 15 years in Spain (Barcelona) Now in Neuchâtel (CH)

11/18/14 3

slide-4
SLIDE 4

Remote location

11/18/14 4

slide-5
SLIDE 5

Red Hat Offjce Neuchâtel

11/18/14 5

slide-6
SLIDE 6

Protocol basic

  • HTTP/1.1 request
  • Responses:
  • Normal
  • Chunked
  • Upgrade (to websocket for example)
  • Proxy AJP/HTTP

11/18/14 6

slide-7
SLIDE 7

Request HTTP/1.1

POST /comet/CometServletT est1 HTTP/1.1\n User-Agent: testclient\n Host: localhost\n T ransfer-Encoding: chunked\n

11/18/14 7

slide-8
SLIDE 8

Response for example

Chunked:

HTTP/1.1 200 OK\n Server: Apache-Coyote/1.1\n Set-Cookie: JSESSIONID=obcoR30qlz7DMJfZmsVTt+Uv; Path=/comet\n T ransfer-Encoding: chunked\n Date: Mon, 07 Nov 2011 22:09:33 GMT\n

Upgrade:

HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat 11/18/14 8

slide-9
SLIDE 9

What is a Connector?

  • Tomcat's interface to the world
  • Binds to a port
  • Understands a protocol
  • Dispatches requests
  • protocol="org.apache.coyote.http11.Http11Protocol"
  • protocol="org.apache.coyote.http11.Http11AprProtocol"
  • protocol="org.apache.coyote.http11.Http11NioProtocol"
  • protocol="org.apache.coyote.http11.Http11Nio2Protocol"
slide-10
SLIDE 10

T

  • mcat Connectors
  • Java Blocking I/O (BIO or sometimes JIO)
  • Java Non-blocking I/O (NIO)
  • Native / Apache Portable Runtime (APR)
  • Java NIO.2

Technically, there are combinations of all of the above with HTTP and AJP protocols. We’ll discuss those a bit later.

slide-11
SLIDE 11

11/18/14

Types of I/O

  • Polling
  • Straightforward API: peek()
  • CPU-inefficient
  • Thread loops while waiting for data
  • Blocking
  • Straightforward API: read()
  • CPU-efficient (blocks)
  • Thread stalls while waiting for data
slide-12
SLIDE 12

11/18/14

Types of I/O

  • Non-blocking
  • Complicated API (registration, event callbacks)
  • Channel
  • Buffer
  • Selector
  • CPU-efficient
  • Thread not required to: execution continues
  • When data is ready, the selector notifies observers
slide-13
SLIDE 13

11/18/14

Common Connector Features

  • Support for all protocols
  • HTTP, AJP, Websocket
  • Support for all dispatch methods
  • Standard, Comet, Servlet 3.0 async
  • Support for HTTPS (SSL/TLS)
  • Acceptor thread(s) call accept() and hand-off
  • Request processor thread pool
slide-14
SLIDE 14

11/18/14

Blocking I/O Connector (1)

  • All I/O operations are blocking in processor thread
  • SSL handshake
  • Read request line (e.g. GET, POST, etc.)
  • Read request body
  • Write response
  • Read next request (HTTP keep-alive)
  • Simple, stable, mature
slide-15
SLIDE 15

11/18/14

Blocking I/O Connector (2)

  • Single thread handles request straight-through, after

accept

  • Uses Java Secure Sockets Extension (JSSE) for SSL/TLS
slide-16
SLIDE 16

11/18/14

Blocking I/O Connector (3)

  • Request throughput limited by thread count
  • Clients can waste threads
  • Slow request line (mobile)
  • Aborted keep-alive stalls thread (default=20sec!)
  • Unfair: accepted connections get priority for keep-alive

requests

slide-17
SLIDE 17

11/18/14

NON-Blocking I/O Connector (1)

  • Single thread handles request after request-line
  • Poller thread(s) manage non-blocking Selector
  • Read SSL handshake
  • Read request line
  • Wait for next keep-alive request
slide-18
SLIDE 18

11/18/14

NON-Blocking I/O Connector (2)

  • Block poller simulates blocking
  • Request header/body reads
  • Response writes
  • Processor thread sleeps during sim-blocking
  • Uses JSSE for SSL/TLS
  • Supports sendFile
slide-19
SLIDE 19

11/18/14

NON-Blocking I/O Connector (3)

  • Allows huge number of parallel requests
  • Not limited by request-processor threads
  • Slow clients do not stall threads
  • Aborted keep-alives die in the poller queue
  • Simulated blocking adds overhead
slide-20
SLIDE 20

11/18/14

Native Connector (APR) (1)

  • Single thread handles request after accept()
  • Poller thread(s) handle certain I/O reads
  • Wait for next keep-alive request
  • Some I/O operations block processor thread
  • SSL handshake
  • Read request line
  • Read request body
  • Write response
slide-21
SLIDE 21

11/18/14

Native Connector (APR) (2)

  • Uses OpenSSL for SSL/TLS
  • Supports sendFile
slide-22
SLIDE 22

11/18/14

Native Connector (APR) (3)

  • Request throughput limited by thread count
  • Slow clients can stall threads
  • Aborted keep-alives die in the poller queue
  • OpenSSL offers performance advantage
  • Native code risks JVM instability
slide-23
SLIDE 23

11/18/14

“Non-blocking” I/O Connector NIO.2 (1)

  • Single thread handles request after request-line
  • The thread are handled via an

AsynchronousChannelGroup and completion call backs

  • Read SSL handshake
  • Read request line and headers
  • Wait for next keep-alive request
slide-24
SLIDE 24

11/18/14

“Non-blocking” I/O Connector NIO.2 (2)

  • NIO2 implementation takes care of blocking using Future
  • bjects waiting for IO
  • Request body reads
  • Response writes
  • Processor thread sleeps during blocking
  • Uses JSSE for SSL/TLS
  • It emulates sendFile (NIO1 transferTo doesn't work with

NIO2)

slide-25
SLIDE 25

11/18/14

“Non-blocking” I/O Connector NIO.2 (3)

  • Allows huge number of parallel requests
  • Not limited by request-processor threads
  • Slow clients do not stall threads
  • High level of abstraction and blocking over async adds
  • verhead
  • NIO 2 provides blocking capabilities over its async IO. The

tomcat code is simpler (good) but an overhead still exists.

slide-26
SLIDE 26

11/18/14

T echnical constraints

  • Don’t try bother using non-blocking protocols with blocking

connectors (BIO+Websocket = bad)

  • AJP can be thought of as 100% keep-alive
  • AJP doesn’t support HTTP upgrade
  • Use of sendFile is highly recommended for any static-

content (APR or NIO.1)

slide-27
SLIDE 27

11/18/14

Connector Performance

  • Compare connector throughput against each other
  • Only static content was compared, varying file sizes
  • Run on fast machines, 10 Gbps local network
  • Tests:
  • Compare the connectors (tc8.0.14) with httpd (2.2.22) no

SSL.

  • Same with SSL
  • What about using a proxy: compare proxies.
slide-28
SLIDE 28

11/18/14

Connector Throughput (c4)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-29
SLIDE 29

11/18/14

Connector CPU Use (c4)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 5 10 15 20 25 30 35 40 45 50 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-30
SLIDE 30

11/18/14

Connector Throughput (c40)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-31
SLIDE 31

11/18/14

Connector CPU Use (c40)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 10 20 30 40 50 60 70 80 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-32
SLIDE 32

11/18/14

Connector Throughput (c80)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-33
SLIDE 33

11/18/14

Connector CPU Use (c80)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 10 20 30 40 50 60 70 80 coyote_apr coyote_nio coyote_nio_ns coyote_apr_ns coyote httpd

slide-34
SLIDE 34

11/18/14

Connector Performance

  • Intermediate conclusion:
  • Using sendfile helps a little. (but just emulated in NIO2!)
  • So using JIO/BIO is probably an “old” idea.
slide-35
SLIDE 35

11/18/14

SSL Connector Throughput (c4)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 coyote_apr_ns coyote_nio2_ns coyote coyote_nio_ns httpd_ssl

slide-36
SLIDE 36

11/18/14

SSL Connector CPU Use (c4)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 10 20 30 40 50 60 coyote_apr_ns coyote_nio2_ns coyote httpd_ssl

slide-37
SLIDE 37

11/18/14

SSL Connector Throughput (c40)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 coyote_apr_ns coyote_nio2_ns coyote httpd_ssl

slide-38
SLIDE 38

11/18/14

SSL Connector CPU Use (c40)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 20 40 60 80 100 120 coyote_apr_ns coyote_nio2_ns coyote httpd_ssl

slide-39
SLIDE 39

11/18/14

Connector Performance

  • Intermediate conclusion:
  • OpenSSL performs better that JSSE
  • JIO/BIO and NIO(2) give similar results.
slide-40
SLIDE 40

11/18/14

Proxy Throughput (c4)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 700000 httpd_ssl httpd mod_jk proxy_ajp proxy_http ssl_mod_jk ssl_proxy_ajp ssl_proxy_http

slide-41
SLIDE 41

11/18/14

HTTPD CPU Use (c4)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 10 20 30 40 50 60 70 httpd_ssl httpd mod_jk proxy_ajp proxy_http ssl_mod_jk ssl_proxy_ajp ssl_proxy_http

slide-42
SLIDE 42

11/18/14

SSL Proxy CPU Use (c4)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 10 20 30 40 50 60 70 httpd_ssl ssl_mod_jk ssl_proxy_ajp ssl_proxy_http

slide-43
SLIDE 43

11/18/14

Proxy Throughput (c40)

4KiB.bin 16KiB.bin 64KiB.bin 128KiB.bin 512KiB.bin 2MiB.bin 8MiB.bin 32MiB.bin 100000 200000 300000 400000 500000 600000 mod_jk proxy_ajp proxy_http ssl_mod_jk ssl_proxy_ajp ssl_proxy_http

slide-44
SLIDE 44

11/18/14

Proxy CPU Use (c40)

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB 20 40 60 80 100 120 mod_jk proxy_ajp proxy_http ssl_mod_jk ssl_proxy_ajp ssl_proxy_http

slide-45
SLIDE 45

11/18/14

Connector/Proxy What use?

  • Conclusion:
  • If you need SSL better use a proxy
  • Basically any “httpd proxy” will do the work.
  • Use mod_jk if you need a Swiss knife configuration.
  • Use http otherwise
  • WebSocket
  • Use httpd-2.4.x for mod_proxy_wstunnel.
slide-46
SLIDE 46

11/18/14

Questions? Thank you!

  • jfclere@gmail.com
  • users@tomcat.apache.org
  • Repo with the scripts for the tests:
  • https://github.com/jfclere/AC2014scripts
slide-47
SLIDE 47

Choosing tomcat connectors