JavaScript on the Server Niels Olof Bouvin 1 Overview The HTTP - - PowerPoint PPT Presentation

JavaScript on the Server

Niels Olof Bouvin

1

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

2

Client/server communication

The Web is an example of a client/server architecture

clients make requests servers return resources communication is always initiated by the client

In the context of the Web, clients are (usually) Web browsers, though they can be anything that speaks http and https

http and https are the protocols with which Web clients and servers communicate

Server Client Client request resource request resource

3

The Uniform Resource Locator

Combines protocol

the method with which computers communicate, e.g., http, https, ftp, rtp, …

with host or server name

usually resolved using DNS

resource name

could just be a fjle name, but could also something more general (we’ll return to this)

and fragment identifjer

used to point into a resource—usually a text-based one

protocol

z }| { https ://

server

z }| { users−cs.au.dk/

resource

z }| { bouvin/dBIoTP2PC/2017/exam.html#

fragment

z }| { topics

4

The HyperText Transfer Protocol

At its simplest, a request and response for a resource

but in reality, it is a bit more complex and far richer: http://127.0.0.1:8080/hello.txt

GET /hello.txt HTTP/1.1 Host: 127.0.0.1:8080 User-Agent: curl/7.54.0 Accept: */* HTTP/1.1 200 OK server: ecstatic-2.2.1 last-modified: Tue, 02 Jan 2018 11:54:58 GMT etag: "77865272-13-"2018-01-02T11:54:58.000Z"" cache-control: max-age=3600 content-length: 13 content-type: text/plain; charset=UTF-8 Date: Tue, 02 Jan 2018 12:01:52 GMT Connection: keep-alive Hello World!

request: (from client) response: (from server) headers explaining what the client wants and what the server delivers the actual resource (A blank line)

5

Operating on resources

HTTP supports four main methods: GET

retrieve the state of a resource — don’t modify it

POST

create new resource, do not specify identifjer

PUT

update existing resource, or create new resource with an identifjer

DELETE

remove a resource

6

A selection of HTTP status codes

200 OK

Standard response for successful HTTP requests.

201 Created

The request has been fulfjlled and resulted in a new resource being created

202 Accepted

The request has been accepted, but is not yet fulfjlled

301 Moved Permanently

This and all future requests should be directed to the given URI

404 Not Found

The requested resource could not be found

500 Internal Server Error

The server has experienced an error, and could not fulfjl the request

7

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

8

What should a Web server do?

Receive a Request

extract information from the Request’s header and body, such as resource identifjcation (‘/min/test.txt’), type of operation (GET, PUT, …), acceptable data formats (‘text/html’), etc.

Create a Response

return a suitable resource for the Request of the appropriate data type and with the correct information in the header

This is all

9

Back to Hello World

Now that we are starting server programming, we should begin using proper tools and structure in our code

that means using NPM to handle the basics, such as starting the program, and keeping track of necessary modules, which is done through the package.json fjle as well as having a systematic approach to folder structure

A new project can be initialised with npm init -fy

10

./package.json

I have fjlled out a few fjelds, including start and author

. ├── app │ └── index.js ├── index.js └── package.json { "name": "basic-hello-world", "version": "1.0.0", "main": "basic-hello-world.js", "scripts": { "start": "node index.js", "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "Niels Olof Bouvin", "license": "ISC", "description": "A simple hello world server" }

11

./index.js

It is considered good coding style to let the central index.js be sparse, and let the actual functionality reside in appropriately named folders and fjles

. ├── app │ └── index.js ├── index.js └── package.json 'use strict' require('./app/index')

12

./app/index.js

http.createServer() requires a callback function that gets a request and a response object

the request object is read (quite simply, here) and the response object has data added to it, and is fjnalised with .end(), which returns the response to the Web browser but fjrst, the server must be set to listen to incoming requests on a specifjc port

. ├── app │ └── index.js ├── index.js └── package.json 'use strict' const http = require('http') const port = 3000 let counter = 0 const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.end(`Hello World! times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

13

Running the server

14

Taking a closer look

15

Adding a bit of HTML style

This should work, right?

. ├── app │ └── index.js ├── index.js └── package.json 'use strict' const http = require('http') const port = 3000 let counter = 0 const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.end(`<i>Hello World!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

16

Probably not as intended

17

Correct header info is crucial

If we do not set an explicit Content-Type using the setHeader() method, the browser will assume it is ‘text/ plain’ , and that is rarely what we want ‘text/html’ is right for HTML documents

. ├── app │ └── index.js ├── index.js └── package.json 'use strict' const http = require('http') const port = 3000 let counter = 0 const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.setHeader('Content-Type', 'text/html') response.end(`<i>Hello World!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

18

Taking a closer look again

19

And in the Web browser…

20

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

21

Input and Output on the Web

We have seen several example of getting data from a server to a Web browser But how do you get data from a Web page back to the server?

this was solved, in the simplest form, in the early Web around HTML 2.0

There are two basic methods

GET POST

22

Input through GET

Very simple:

http://…/foo.html?arg1=value&arg2=value&arg3=value (and so on)

Advantage

you can make links that contain arguments to the server

Disadvantage

there is a space limitation cannot be used for, e.g., fjle upload you would not want a password to be visible in the browser bar

23

A more specifjc greeting

The standard Node ‘url’ module can parse the query

'use strict' const http = require('http') const url = require('url') const port = 3000 let counter = 0 let recipient const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) const query = url.parse(request.url, true).query if ('recipient' in query) { recipient = query['recipient'] } else { recipient = 'World' } response.setHeader('Content-Type', 'text/html') response.end(`<i>Hello ${recipient}!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

24

Input through POST

Requires a different HTTP method: POST Data is transferred in the body of the request rather than the URL

you can send much more data because this data can be split up across several network packages, it is necessary to assemble the data at the server, before it can be processed the standard Node module ‘querystring’ can parse this kind of data

The data must be input through a <form> element on a Web page

25

A HTML form

A form has a method and an action There are many different kinds of input types

<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Greetings for you</title> </head> <body> <form method="POST" action="http://localhost:3000/"> <div> <label for="recipient">Recipient:</label> <input type="text" id="recipient" name="recipient"> </div> <div> <label for="message">Message:</label> <textarea id="message" name="message"></textarea> </div> <div> <button type="submit">Submit</button> </div> </form> </body> </html>

26

Handling the POST data

'use strict' const http = require('http') const querystring = require('querystring') const port = 3000 const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) if (request.method === 'POST') { let body = '' request.on('data', (data) => { // got some data body += data }) request.on('end', () => { // all data received console.log(`request.body=${body}`) const post = querystring.parse(body) const recipient = post['recipient'] || 'World' const message = post['message'] || '' response.setHeader('Content-Type', 'text/html') response.end(`<div>Hello ${recipient}!</div> <div>${message}</div>\n`) }) } }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

27

Filling a form

28

Building a server with the http module

It is completely feasible to build a Web server based

• n the standard Node.js modules

However, as servers become more sophisticated, it requires a fair amount of manual labour This is where a Web server framework can come in handy There are many for Node.js, but the most popular is Express

29

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

30

Adding Express to your project

npm install express --save installs the required modules in node_modules/ and updates package.json

add node_modules/ to .gitignore (you can install all the modules with ‘npm install’)

31

Hello World in Express

The basic case is not much shorter than with http, but the structure quickly becomes clearer, and we do not have to set the Content-Type header ourselves

'use strict' const express = require('express') const app = express() const port = 3000 app.get('/', (request, response) => { response.send('Hello World!') }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

32

Chaining middleware in Express

Express functions by chaining middleware functions that take (request, response, next) as arguments, and each end by call the next() in line

'use strict' const express = require('express') const app = express() const port = 3000 app.use((request, response, next) => { request.timer = Date.now() next() }) app.use((request, response, next) => { console.log(`request.url=${request.url}`) next() }) app.get('/', (request, response) => { response.send('Hello World!') console.log(`Processing took: ${Date.now() - request.timer} ms`) }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

33

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

34

Using a template engine: handlebars

It is extremely tedious and error prone to write HTML directly inside JavaScript

what if the design changes? no tool support for syntax etc

Most Web sites, while having many pages, will have relatively few layouts, wherein the content will vary

it is also tedious to have to repeat the same bits over and over in a static Web site

Template frameworks are made to support this kind of scenario:

defjne the layout once, mark the places where content should be inserted, confjgure your application, and everything works

35

The layout: main.hbs

Standard HTML, except

things inside {{{}}} will be replaced directly (HTML will not be escaped)

You can develop a Web page until satisfjed, and then replace the changing bits with {{{}}}

app └── index.js views ├── hello.hbs └── layouts └── main.hbs <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Express handlebars </title> </head> <body> {{{body}}} </body> </html>

36

The view: hello.hbs

This is what replaces {{{body}}} in main.hbs Things inside {{}} will have HTML escaped

app └── index.js views ├── hello.hbs └── layouts └── main.hbs <h2>Hello {{recipient}}</h2>

37

Confjguring Express: index.js

'use strict' const express = require('express') const exphbs = require('express-handlebars') const path = require('path') const app = express() const port = 3000 app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views')) app.get('/', (request, response) => { response.render('hello', { recipient: ‘World’// remember {{recipient}} in hello.hbs? }) }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) }) app └── index.js views ├── hello.hbs └── layouts └── main.hbs

38

Handling input with Express

Express handles get input directly

request.query.recipient (assuming http://…/somePage?recipient=Johnny)

POST data requires an additional module, such as body-parser (there are quite a few others)

npm install body-parser --save app.use(bodyParser.urlencoded({ extended: false })) // to add it to express

39

The greeting: hello.hbs

Ought to be familiar from earlier

app └── index.js views ├── form.hbs ├── hello.hbs └── layouts └── main.hbs <h2>Hello {{recipient}}</h2> <p>{{message}}</p>

40

The form: form.hbs

Ought to be familiar from earlier

app └── index.js views ├── form.hbs ├── hello.hbs └── layouts └── main.hbs <form method="POST" action="/"> <div> <label for="recipient">Recipient:</label> <input type="text" id="recipient" name="recipient"> </div> <div> <label for="message">Message:</label> <textarea id="message" name="message"></textarea> </div> <div> <button type="submit">Submit</button> </div> </form>

41

The code: index.js 1/2

All this confjgures Express with body-parser and handlebars

app └── index.js views ├── form.hbs ├── hello.hbs └── layouts └── main.hbs 'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const app = express() const port = 3000 app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views')) app.use(bodyParser.urlencoded({ extended: false }))

42

The code: index.js 2/2

This site handles both POST and GET data

app └── index.js views ├── form.hbs ├── hello.hbs └── layouts └── main.hbs app.get('/', (request, response) => { response.render('form') }) app.post('/', (request, response) => { response.render('hello', { recipient: request.body.recipient, message: request.body.message }) }) app.get('/hello', (request, response) => { response.render('hello', { recipient: request.query.recipient, message: request.query.message }) }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

43

In action, handling POST and GET

44

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

45

Data drives the Web

In practice, most Websites are database-driven Layout, stylesheets, images, etc., are typically static resources and handled separately The actual contents resides in databases, and are added into the Web pages as they are generated by the server

we will later be looking at Web pages that “build themselves” by retrieving data from servers

46

Static resources

Everything found in the specifjed folder is served as http-server would have served it

the fjles in public/ are just ordinary HTML and CSS fjles

You can call app.use(express.static()) multiple times, if you need to serve static resources from different paths

app └── index.js public/ ├── index.html └── style └── main.css 'use strict' const express = require('express') const path = require('path') const app = express() const port = 3000 app.use(express.static(path.join(__dirname, '../public'))) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

47

Static resources

Note how the css fjle is down in a folder, but is correctly served nonetheless by express.static

app └── index.js public/ ├── index.html └── style └── main.css

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <title>Static pages!</title> <link rel="stylesheet" type="text/css" media="screen" href="style/main.css" /> </head> <body> <h1>Hello World!</h1> <p>This is just a static page, served up by Express, and referencing a stylesheet, served in a similar manner.</p> </body> </html>

body { background-color: palegreen; }

48

Databases!

I believe you have heard of them? In a Web context, databases largely come in two kinds:

SQL databases, such as MySQL, PostgreSQL, SQLite, DB2, Oracle, … NoSQL databases, such as MongoDB, CouchDB, Cassandra, Redis, …

Given your expertise with SQL, we will stick with SQL…

49

The data driven Greetings site

Let’s create a site, where users can add their own greeting to a list of greetings

50

What should the site support?

GET /form

the form with which to create a greeting

POST /greetings

create a new greeting

GET /greetings/:id

get a specifjc greeting

GET /greetings

get all greetings

DELETE /greeting/:id

delete a specifjc greeting

51

Redirecting

A Web server can instruct a Web browser to another page through the Location: fjeld in the HTTP header This can be done from Express with the method redirect() We will use that to automatically direct users from / to /form

52

Let’s start with a simpler version

When starting out, it can be simpler to take things step by step, e.g., starting out with an in-memory array before building the full database-backed version Once that works, it will provide us with the skeleton for the full version

53

The code: index.js 1/2

app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const app = express() const port = 3000 const greetings = [] app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views')) app.use(bodyParser.urlencoded({ extended: true })) app.use(express.static(path.join(__dirname, '../public')))

54

The code: index.js 1/2

app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs app.get('/', (request, response) => { response.redirect('/form') }) app.get('/form', (request, response) => { response.render('form') }) app.post('/greetings', (request, response) => { const greeting = {recipient: request.body.recipient, message: request.body.message} greetings.push(greeting) response.redirect('/greetings') }) app.get('/greetings', (request, response) => { response.render('greeting', { greetings: greetings }) }) app.get('/greetings/:id', (request, response) => { const id = request.params.id const greeting = greetings[id] response.render('greeting', { greetings: [greeting] }) }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

55

Greetings with Handlebars

With #each, Handlebars can iterate through arrays of

• bjects, accessing the fjelds within the objects

app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

{{#each greetings}} <h2>Hello {{recipient}}</h2> <p>{{message}}</p> {{/each}} <div><a href="/form">Add a greeting</a></div>

56

The site in action

57

Moving on to a proper database

We need a proper database You already know MySQL, so let’s try something new! By far the most widespread SQL database system in the world is SQLite

it is very compact (~700 kB), fast, and implements most of SQL-92 it is not very good with concurrent writes, but concurrent reads are fjne it is found almost everywhere, from desktop computers (handles, e.g., Mail on macOS) to Web browsers to mobile phones (your contact list is almost certainly in SQLite) some idiot lecturer forgot to install it on the Raspberry Pi image—see Resources for instructions (and the node-sqlite3 module installer is slow on RPi, sorry about that)

But, once installed it is easy to administer

58

SQLite3 in action

59

The database: db.js 1/2

We open the database fjle, and if our table does not exist, we create it using standard SQL

app └── index.js db ├── db.js └── greetings.sqlite public ├── js │ └── main.js └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

const sqlite3 = require('sqlite3').verbose() const path = require('path') const dbName = path.join(__dirname, 'greetings.sqlite') const db = new sqlite3.Database(dbName) db.serialize(() => { const sql = ` CREATE TABLE IF NOT EXISTS greetings (id integer primary key, recipient, message TEXT) ` db.run(sql) })

60

The database: db.js 2/2

class Greetings { static all (callback) { db.all('SELECT * FROM greetings', callback) } static find (id, callback) { db.get('SELECT * FROM greetings WHERE id = ?', id, callback) } static create (greeting, callback) { const sql = 'INSERT INTO greetings(recipient, message) VALUES (?, ?)' db.run(sql, greeting.recipient, greeting.message, callback) } static delete (id, callback) { if (!id) { return callback(new Error('Please provide an id')) } db.run('DELETE FROM greetings WHERE id = ?', id, callback) } } module.exports = db module.exports.Greetings = Greetings

61

The code: index.js 1/2

'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const Greetings = require('../db/db').Greetings const app = express() const port = 3000 app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views')) app.use(bodyParser.urlencoded({ extended: true })) app.use(express.static(path.join(__dirname, '../public'))) app.get('/', (request, response, next) => { response.redirect('/form') }) app.get('/form', (request, response, next) => { response.render('form') })

62

The code: index.js 2/2

app.post('/greetings', (request, response, next) => { const greeting = {recipient: request.body.recipient, message: request.body.message} Greetings.create(greeting, (err, greeting) => { if (err) return next(err) response.redirect('/greetings') }) }) app.get('/greetings', (request, response, next) => { Greetings.all((err, greetings) => { if (err) return next(err) response.render('greeting', { greetings: greetings }) }) }) app.get('/greetings/:id', (request, response, next) => { const id = request.params.id Greetings.find(id, (err, greeting) => { if (err) return next(err) response.render('greeting', { greetings: [greeting] }) }) }) app.listen(port, (err) => { if (err) return console.error(`An error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

63

The site in action

64

Note on the use of databases

You should always let the module handle data that you have no control over

e.g., something that has been entered into a form and sent over the network

This is correct, because it allows the module to sanitise the input This is wrong and dangerous, do not do this:

const sql = 'INSERT INTO greetings(recipient, message) VALUES (?, ?)' db.run(sql, greeting.recipient, greeting.message, callback) const bad = `INSERT INTO greetings(recipient, message) VALUES (${greeting.recipient}, ${greeting.message})` db.run(bad)

65

Wrap-up

Node.js comes into its own, once we start server programming With Express and its associates modules, it is possible to create functional Web sites in relatively little code Templates alone will save you a lot of time Proper databases of course makes everything better

66