Node.js: Asynchronous I/O for Fun and Profit Stefan Tilkov @ QCon - - PowerPoint PPT Presentation

▶
node js asynchronous i o for fun and profit
SMART_READER_LITE
LIVE PREVIEW

Node.js: Asynchronous I/O for Fun and Profit Stefan Tilkov @ QCon - - PowerPoint PPT Presentation

Dec 08, 2022 129 likes •1.48k views

Node.js: Asynchronous I/O for Fun and Profit Stefan Tilkov @ QCon London 2011 Friday, March 11, 2011 Stefan Tilkov @stilkov stefan.tilkov@innoq.com http:/ /www.innoq.com Friday, March 11, 2011 Concurrent Request Processing Friday, March

slide-1
SLIDE 1

Node.js: Asynchronous I/O for Fun and Profit

Stefan Tilkov @ QCon London 2011

Friday, March 11, 2011

slide-2
SLIDE 2

Stefan Tilkov @stilkov stefan.tilkov@innoq.com

http:/ /www.innoq.com

Friday, March 11, 2011

slide-3
SLIDE 3

Concurrent Request Processing

Friday, March 11, 2011

slide-4
SLIDE 4

Friday, March 11, 2011

slide-5
SLIDE 5

Friday, March 11, 2011

slide-6
SLIDE 6

Friday, March 11, 2011

slide-7
SLIDE 7

Friday, March 11, 2011

slide-8
SLIDE 8

Friday, March 11, 2011

slide-9
SLIDE 9

Friday, March 11, 2011

slide-10
SLIDE 10

read request

Friday, March 11, 2011

slide-11
SLIDE 11

read request parse request

Friday, March 11, 2011

slide-12
SLIDE 12

read request parse request process

Friday, March 11, 2011

slide-13
SLIDE 13

read request parse request process send backend request

Friday, March 11, 2011

slide-14
SLIDE 14

read request parse request process send backend request read backend answer

Friday, March 11, 2011

slide-15
SLIDE 15

read request parse request process send backend request read backend answer process

Friday, March 11, 2011

slide-16
SLIDE 16

read request parse request process send backend request read backend answer process format response

Friday, March 11, 2011

slide-17
SLIDE 17

read request parse request process send backend request read backend answer process format response send response

Friday, March 11, 2011

slide-18
SLIDE 18

read request parse request process send backend request read backend answer process format response send response

Friday, March 11, 2011

slide-19
SLIDE 19

Blocking I/O Problems

Friday, March 11, 2011

slide-20
SLIDE 20

Blocking I/O Problems

Thread starvation

Friday, March 11, 2011

slide-21
SLIDE 21

Blocking I/O Problems

Thread starvation Memory utilization

Friday, March 11, 2011

slide-22
SLIDE 22

Blocking I/O Problems

Thread starvation Memory utilization External dependencies

Friday, March 11, 2011

slide-23
SLIDE 23

Blocking I/O Problems

Thread starvation Memory utilization External dependencies Cascading problems

Friday, March 11, 2011

slide-24
SLIDE 24

Blocking I/O Problems

Thread starvation Memory utilization External dependencies Cascading problems Non-streaming approach

Friday, March 11, 2011

slide-25
SLIDE 25

Kernel User Space

Friday, March 11, 2011

slide-26
SLIDE 26

Event Loop

while (true) ready_channels = select(io_channels) for (channel in ready_channels) performIO(channel)

Friday, March 11, 2011

slide-27
SLIDE 27

Async I/O Characteristics

Friday, March 11, 2011

slide-28
SLIDE 28

Async I/O Characteristics

Program always running

Friday, March 11, 2011

slide-29
SLIDE 29

Async I/O Characteristics

Program always running I/O-bound calls never block

Friday, March 11, 2011

slide-30
SLIDE 30

Async I/O Characteristics

Program always running I/O-bound calls never block Kernel handles I/O

Friday, March 11, 2011

slide-31
SLIDE 31

Async I/O Characteristics

Program always running I/O-bound calls never block Kernel handles I/O Notification via events

Friday, March 11, 2011

slide-32
SLIDE 32

Async I/O Characteristics

Program always running I/O-bound calls never block Kernel handles I/O Notification via events Used for timers, file I/O, net I/O, ...

Friday, March 11, 2011

slide-33
SLIDE 33

requests/second

http://blog.webfaction.com/a-little-holiday-present

Friday, March 11, 2011

slide-34
SLIDE 34

memory

http://blog.webfaction.com/a-little-holiday-present

Friday, March 11, 2011

slide-35
SLIDE 35

select() poll() epoll() kqueue() /dev/poll aio_*()

Friday, March 11, 2011

slide-36
SLIDE 36

java.nio .NET I/O Completion Ports

Friday, March 11, 2011

slide-37
SLIDE 37

Async I/O Perception

Friday, March 11, 2011

slide-38
SLIDE 38

Async I/O Perception

Not widely known

Friday, March 11, 2011

slide-39
SLIDE 39

Async I/O Perception

Not widely known Low level

Friday, March 11, 2011

slide-40
SLIDE 40

Async I/O Perception

Not widely known Low level Hard to use

Friday, March 11, 2011

slide-41
SLIDE 41

Async I/O Perception

Not widely known Low level Hard to use Exception rather than rule

Friday, March 11, 2011

slide-42
SLIDE 42

JavaScript

Friday, March 11, 2011

slide-43
SLIDE 43

JavaScript Perception

Friday, March 11, 2011

slide-44
SLIDE 44

JavaScript Perception

“Toy language”

Friday, March 11, 2011

slide-45
SLIDE 45

JavaScript Perception

“Toy language” Incompatible

Friday, March 11, 2011

slide-46
SLIDE 46

JavaScript Perception

“Toy language” Incompatible Inherent design problems

Friday, March 11, 2011

slide-47
SLIDE 47

JavaScript Perception

“Toy language” Incompatible Inherent design problems Low Performance

Friday, March 11, 2011

slide-48
SLIDE 48

http://commons.wikimedia.org/wiki/File:Audi_S5_V8_FSI_engine.jpg

Friday, March 11, 2011

slide-49
SLIDE 49

Friday, March 11, 2011

slide-50
SLIDE 50

http://commons.wikimedia.org/wiki/File:Ateles_paniscus_-Brazil-8.jpg

Friday, March 11, 2011

slide-51
SLIDE 51

Friday, March 11, 2011

slide-52
SLIDE 52

The JavaScript Arms Race

Friday, March 11, 2011

slide-53
SLIDE 53

Friday, March 11, 2011

slide-54
SLIDE 54

Friday, March 11, 2011

slide-55
SLIDE 55

http://oreilly.com/catalog/9780596517748

Friday, March 11, 2011

slide-56
SLIDE 56

Friday, March 11, 2011

slide-57
SLIDE 57

JavaScript Today

Friday, March 11, 2011

slide-58
SLIDE 58

JavaScript Today

Popular & widely used

Friday, March 11, 2011

slide-59
SLIDE 59

JavaScript Today

Popular & widely used Ofen mandatory

Friday, March 11, 2011

slide-60
SLIDE 60

JavaScript Today

Popular & widely used Ofen mandatory Fast

Friday, March 11, 2011

slide-61
SLIDE 61

JavaScript Today

Popular & widely used Ofen mandatory Fast Compatible

Friday, March 11, 2011

slide-62
SLIDE 62

JavaScript Today

Popular & widely used Ofen mandatory Fast Compatible Best practices

Friday, March 11, 2011

slide-63
SLIDE 63

Node.js

Friday, March 11, 2011

slide-64
SLIDE 64

libev libeio v8 http_parser c_ares

Node.js Architecture

Friday, March 11, 2011

slide-65
SLIDE 65

libev libeio v8 http_parser c_ares

Node.js Architecture

Network/Platform layer (C)

Friday, March 11, 2011

slide-66
SLIDE 66

libev libeio v8 http_parser c_ares

Node.js Architecture

Network/Platform layer (C) API (JavaScript)

Friday, March 11, 2011

slide-67
SLIDE 67

High-performance network runtime, using JavaScript as a high-level DSL

Friday, March 11, 2011

slide-68
SLIDE 68

echo.js

var net = require('net'); var server = net.createServer(function (socket) { socket.write("Echo server\r\n"); socket.pipe(socket); }) server.listen(8124, "127.0.0.1");

Code samples: http://github.com/stilkov/node-samples

Friday, March 11, 2011

slide-69
SLIDE 69

echo-upcase.js

var net = require('net'); var server = net.createServer(function (socket) { socket.write("Echo server\r\n"); socket.setEncoding('ascii'); socket.on('data', function(data) { socket.write(data.toUpperCase()); }); }); server.listen(8124, "127.0.0.1");

Friday, March 11, 2011

slide-70
SLIDE 70

var sys = require("sys"), http = require("http"), url = require("url"), path = require("path"), fs = require("fs"); var dir = process.argv[2] || './public'; var port = parseFloat(process.argv[3]) || 8080; sys.log('Serving files from ' + dir + ', port is ' + port); http.createServer(function(request, response) { var uri = url.parse(request.url).pathname; var filename = path.join(process.cwd(), dir, uri); path.exists(filename, function(exists) { if(exists) { fs.readFile(filename, function(err, data) { response.writeHead(200); response.end(data); }); } else { sys.log('File not found: ' + filename); response.writeHead(404); response.end(); } }); }).listen(port);

file-server.js

Friday, March 11, 2011

slide-71
SLIDE 71

Concurrency Level: 100 Time taken for tests: 6.000 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 710781 bytes HTML transferred: 150165 bytes Requests per second: 1666.72 [#/sec] (mean) Time per request: 59.998 [ms] (mean) Time per request: 0.600 [ms] (mean, across all concurrent requests) Transfer rate: 115.69 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 8 8.3 5 57 Processing: 1 51 44.4 40 307 Waiting: 0 43 43.5 30 302 Total: 1 59 44.8 50 316 Percentage of the requests served within a certain time (ms) 50% 50 66% 58 75% 68 80% 73 90% 112 95% 174 98% 206 99% 224 100% 316 (longest request)

Friday, March 11, 2011

slide-72
SLIDE 72

file-server-md5.js

http.createServer(function(request, response) { var uri = url.parse(request.url).pathname; var filename = path.join(process.cwd(), dir, uri); sys.log('Serving file ' + filename); path.exists(filename, function(exists) { if(exists) { fs.readFile(filename, function(err, data) { var hash = crypto.createHash('md5'); hash.update(data); response.writeHead(200, { 'Content-Type': 'text/plain', 'Content-MD5': hash.digest('base64') } ); response.end(data); }); } else { response.writeHead(404); response.end(); } }); }).listen(port);

Friday, March 11, 2011

slide-73
SLIDE 73

HTTP Chunking

Friday, March 11, 2011

slide-74
SLIDE 74

HTTP/1.0

Client Server

connect

Friday, March 11, 2011

slide-75
SLIDE 75

HTTP/1.0

Client Server

connect send request

Friday, March 11, 2011

slide-76
SLIDE 76

HTTP/1.0

Client Server

connect send request send response

Friday, March 11, 2011

slide-77
SLIDE 77

HTTP/1.0

Client Server

connect close connection send request send response

Friday, March 11, 2011

slide-78
SLIDE 78

HTTP/1.0

Client Server

connect connect close connection send request send response

Friday, March 11, 2011

slide-79
SLIDE 79

HTTP/1.0

Client Server

connect send request connect close connection send request send response

Friday, March 11, 2011

slide-80
SLIDE 80

HTTP/1.0

Client Server

connect send request send response data connect close connection send request send response

Friday, March 11, 2011

slide-81
SLIDE 81

HTTP/1.0

Client Server

connect send request send response data ... connect close connection send request send response

Friday, March 11, 2011

slide-82
SLIDE 82

HTTP/1.0

Client Server

connect send request send response data send response data ... connect close connection send request send response

Friday, March 11, 2011

slide-83
SLIDE 83

HTTP/1.0

Client Server

connect send request send response data send response data close connection ... connect close connection send request send response

Friday, March 11, 2011

slide-84
SLIDE 84

HTTP/1.1: Content-length

Client Server

Friday, March 11, 2011

slide-85
SLIDE 85

HTTP/1.1: Content-length

Client Server

connect

Friday, March 11, 2011

slide-86
SLIDE 86

HTTP/1.1: Content-length

Client Server

connect Connection: keep-alive

Friday, March 11, 2011

slide-87
SLIDE 87

HTTP/1.1: Content-length

Client Server

connect send request Connection: keep-alive

Friday, March 11, 2011

slide-88
SLIDE 88

HTTP/1.1: Content-length

Client Server

connect send request send response Connection: keep-alive

Friday, March 11, 2011

slide-89
SLIDE 89

Content-length: xxx

HTTP/1.1: Content-length

Client Server

connect send request send response Connection: keep-alive

Friday, March 11, 2011

slide-90
SLIDE 90

... Content-length: xxx

HTTP/1.1: Content-length

Client Server

connect send request send response Connection: keep-alive

Friday, March 11, 2011

slide-91
SLIDE 91

... Content-length: xxx

HTTP/1.1: Content-length

Client Server

connect send request send response send request Connection: keep-alive

Friday, March 11, 2011

slide-92
SLIDE 92

... Content-length: xxx

HTTP/1.1: Content-length

Client Server

connect send request send response send response send request Connection: keep-alive

Friday, March 11, 2011

slide-93
SLIDE 93

... Content-length: xxx Content-length: yyy

HTTP/1.1: Content-length

Client Server

connect send request send response send response send request Connection: keep-alive

Friday, March 11, 2011

slide-94
SLIDE 94

... Content-length: xxx Content-length: yyy

HTTP/1.1: Content-length

Client Server

connect send request send response send response close connection send request Connection: keep-alive

Friday, March 11, 2011

slide-95
SLIDE 95

HTTP/1.1: Transfer-encoding: chunked

Client Server

Friday, March 11, 2011

slide-96
SLIDE 96

HTTP/1.1: Transfer-encoding: chunked

Client Server

connect Connection: keep-alive

Friday, March 11, 2011

slide-97
SLIDE 97

HTTP/1.1: Transfer-encoding: chunked

Client Server

send request connect Connection: keep-alive

Friday, March 11, 2011

slide-98
SLIDE 98

HTTP/1.1: Transfer-encoding: chunked

Client Server

send request send response data connect Connection: keep-alive

Friday, March 11, 2011

slide-99
SLIDE 99

HTTP/1.1: Transfer-encoding: chunked

Client Server

send request send response data connect Connection: keep-alive Transfer-encoding: chunked

Friday, March 11, 2011

slide-100
SLIDE 100

HTTP/1.1: Transfer-encoding: chunked

xxx↵[data]

Client Server

send request send response data connect Connection: keep-alive Transfer-encoding: chunked

Friday, March 11, 2011

slide-101
SLIDE 101

HTTP/1.1: Transfer-encoding: chunked

xxx↵[data]

Client Server

send request send response data connect Connection: keep-alive xxx↵[data] send response data Transfer-encoding: chunked

Friday, March 11, 2011

slide-102
SLIDE 102

HTTP/1.1: Transfer-encoding: chunked

xxx↵[data]

Client Server

send request send response data connect Connection: keep-alive xxx↵[data] send response data 0↵ send response data Transfer-encoding: chunked

Friday, March 11, 2011

slide-103
SLIDE 103

HTTP/1.1: Transfer-encoding: chunked

xxx↵[data]

Client Server

send request send response data close connection connect Connection: keep-alive xxx↵[data] send response data 0↵ send response data Transfer-encoding: chunked

Friday, March 11, 2011

slide-104
SLIDE 104

stream-file-server.js

http.createServer(function(request, response) { var uri = url.parse(request.url).pathname; var filename = path.join(process.cwd(), dir, uri); path.exists(filename, function(exists) { if(exists) { f = fs.createReadStream(filename); f.on('open', function() { response.writeHead(200); }); f.on('data', function(chunk) { response.write(chunk); }); f.on('error', function(err) { // ... }); f.on('end', function() { response.end(); }); } else { response.writeHead(404);

  • response.end();

} }); }).listen(port);

Friday, March 11, 2011

slide-105
SLIDE 105

hash-file-stream.js (see stream-file-server-md5.js)

var hashFile = function(filename, cb) { path.exists(filename, function(exists) { if(exists) { r = fs.createReadStream(filename); var hash = crypto.createHash('md5'); r.on('data', function(data) { hash.update(data); }); r.on('end', function() { cb(hash.digest('base64')); }); } else { throw 'File ' + filename + ' does not exist or can not be read'; } }); } var filename = path.join(process.argv[2]); hashFile(filename, function(hash) { console.log(filename + ': ' + hash); });

Friday, March 11, 2011

slide-106
SLIDE 106

proxy.js

var options = function(request) { // ... } http.createServer(function(request, response) { sys.log("--> " + request.url); var remoteRequest = http.request(options(request), function (remoteResponse) { response.writeHead(remoteResponse.statusCode, remoteResponse.headers); remoteResponse.on('data', function (chunk) { response.write(chunk); }); remoteResponse.on('end', function () { sys.log("<-- " + response.statusCode + " " + request.url); response.end(); }); }); request.on('data', function (chunk) { remoteRequest.write(chunk); }); request.on('end', function () { remoteRequest.end(); }); }).listen(port);

Friday, March 11, 2011

slide-107
SLIDE 107

proxy-pump.js

http.createServer(function(request, response) { sys.log("--> " + request.url); var remoteRequest = http.request(options(request), function (remoteResponse) { response.writeHead(remoteResponse.statusCode, remoteResponse.headers); remoteResponse.on('end', function () { sys.log("<-- " + response.statusCode + " " + request.url); }); util.pump(remoteResponse, response); }); util.pump(request, remoteRequest); }).listen(port);

Friday, March 11, 2011

slide-108
SLIDE 108

Asynchronous Programming Challenges

Friday, March 11, 2011

slide-109
SLIDE 109
  • r:

Why Programming with Callbacks Sucks

Friday, March 11, 2011

slide-110
SLIDE 110

Friday, March 11, 2011

slide-111
SLIDE 111

var bold = function(text) { return text.bold(); }; var capitalize = function(text) { return text.toUpperCase(); }; console.log("Synchronous:"); var result1 = capitalize("Hello, synchronous world."); var result2 = bold(result1); console.log("Sync result is " + result2);

async1.js

Friday, March 11, 2011

slide-112
SLIDE 112

var boldAsync = function(text, callback) { setTimeout(function (text) { callback(text.bold()); }, 100, text); }; var capitalizeAsync = function(text, callback) { setTimeout(function (text) { callback(text.toUpperCase()); }, 100, text); };

async1.js

Friday, March 11, 2011

slide-113
SLIDE 113

var boldAsync = function(text, callback) { setTimeout(function (text) { callback(text.bold()); }, 100, text); }; var capitalizeAsync = function(text, callback) { setTimeout(function (text) { callback(text.toUpperCase()); }, 100, text); };

async1.js

console.log("Asynchronous:"); capitalizeAsync("Hello, asynchronous world.", function(result1) { boldAsync(result1, function(result2) { console.log("Async result is " + result2); }); });

Friday, March 11, 2011

slide-114
SLIDE 114

async2.js

try { console.log("Synchronous:"); var result1 = capitalize(null); var result2 = bold(result1); console.log("Sync result is " + result2); } catch (exception) { console.log("Sync exception caught: " + exception); }

Friday, March 11, 2011

slide-115
SLIDE 115

async2.js

try { console.log("Asynchronous:"); capitalizeAsync(text, function(result1) { boldAsync(result1, function(result2) { console.log("Async result is " + result2); }); }); } catch (exception) { console.log("Async exception caught: " + exception); }

Friday, March 11, 2011

slide-116
SLIDE 116

async2.js

try { console.log("Asynchronous:"); capitalizeAsync(text, function(result1) { boldAsync(result1, function(result2) { console.log("Async result is " + result2); }); }); } catch (exception) { console.log("Async exception caught: " + exception); } // bad, don't do this

Friday, March 11, 2011

slide-117
SLIDE 117

async3.js

var boldAsync = function(text, callback) { setTimeout(function (text) { try { callback(null, text.bold()); } catch (exception) { callback(exception); } }, 100, text); }; var capitalizeAsync = function(text, callback) { setTimeout(function (text) { try { callback(null, text.toUpperCase()); } catch (exception) { callback(exception); } }, 100, text); };

Friday, March 11, 2011

slide-118
SLIDE 118

async3.js

capitalizeAsync(text, function(err, result1) { if (!err) { boldAsync(result1, function(err, result2) { if (!err) { console.log("Async result is " + result2); } else { console.log("Handling async error: " + err); } }); } else { console.log("Handling async error: " + err); } });

Friday, March 11, 2011

slide-119
SLIDE 119

async3.js

var handleError = function(err, fn) { if (err) { console.log("Handling async error: " + err); } else { fn(); } } capitalizeAsync(text, function(err, result1) { handleError(err, function () { boldAsync(result1, function(err, result2) { handleError(err, function () { console.log("Async result is " + result2); }); }); }); });

Friday, March 11, 2011

slide-120
SLIDE 120

async3.js

var step = require("step"); step( function () { capitalizeAsync(text, this); }, function (err, result) { if (err) throw err; boldAsync(result, this); }, function(err, result) { if (err) { console.log("Handling async error: " + err); } else { console.log("Async result is " + result); } } );

Friday, March 11, 2011

slide-121
SLIDE 121

parallel1.js

var words = ['one', 'two', 'three', 'four', 'five']; var upcasedWords = []; words.forEach(function (word) { capitalize(word, function(err, word) { upcasedWords.push(word); }); }); console.log('Done, upcased words: <' + upcasedWords.join(' ') + '>');

Friday, March 11, 2011

slide-122
SLIDE 122

parallel1.js

var words = ['one', 'two', 'three', 'four', 'five']; var upcasedWords = []; words.forEach(function (word) { capitalize(word, function(err, word) { upcasedWords.push(word); }); }); console.log('Done, upcased words: <' + upcasedWords.join(' ') + '>'); // bad, don't do this

Friday, March 11, 2011

slide-123
SLIDE 123

parallel1.js

var count = words.length; words.forEach(function (word) { capitalize(word, function(err, word) { upcasedWords.push(word); if (--count === 0) { console.log('Done, upcased words: <' + upcasedWords.join(' ') + '>'); } }); });

Friday, March 11, 2011

slide-124
SLIDE 124

parallel2.js

var words = ['one', 'two', 'three', 'four', 'five']; step( function () { var i, length; for (i = 0, length = words.length; i < length; i++) { capitalize(words[i], this.parallel()); } }, function (err) { if (err) throw err; var upcasedWords = Array.prototype.slice.call(arguments); upcasedWords.shift(); console.log('Done, upcased words: <' + upcasedWords.join(' ') + '>'); } );

Friday, March 11, 2011

slide-125
SLIDE 125

Tools & Ecosystem

Friday, March 11, 2011

slide-126
SLIDE 126

Friday, March 11, 2011

slide-127
SLIDE 127

npm node package manager Connect Asynchronous, low-level HTTP handler framework inspired by Rack/WSGI Express Sinatra-inspired Web framework on top

  • f Connect

multi-node Spawns child processes sharing listeners node-inspector Visual debugger for Node.js >700 more modules see https:/ /github.com/joyent/node/ wiki/modules

Friday, March 11, 2011

slide-128
SLIDE 128

multi-file-server.js

var multi = require("multi-node"); var server = http.createServer(function(request, response) { var uri = url.parse(request.url).pathname; var filename = path.join(process.cwd(), dir, uri); path.exists(filename, function(exists) { if(exists) { fs.readFile(filename, function(err, data) { if (err) { sys.log('Error serving file ' + filename + ' ' + err); sys.log('request: ' + uri); } response.writeHead(200, { 'X-Node-Id': process.pid }); response.end(data); }); } else { response.writeHead(404); response.end(); } }); }); var nodes = multi.listen({ port: port, nodes: 10 }, server); sys.log("Server " + process.pid + " running at http://localhost:" + port);

Friday, March 11, 2011

slide-129
SLIDE 129

Summary

Friday, March 11, 2011

slide-130
SLIDE 130

Node.js popularizes the “right way”

  • f network programming

Friday, March 11, 2011

slide-131
SLIDE 131

JavaScript doesn’t suck as much as you think

Friday, March 11, 2011

slide-132
SLIDE 132

There’s a smart and active community

Friday, March 11, 2011

slide-133
SLIDE 133

Node.js is fun to use!

Friday, March 11, 2011

slide-134
SLIDE 134

Thank you!

Q&A

Friday, March 11, 2011