Neues in Open-Source-SQL-Datenbanken @MarkusWinand @ModernSQL - - PowerPoint PPT Presentation

neues in open source sql datenbanken
SMART_READER_LITE
LIVE PREVIEW

Neues in Open-Source-SQL-Datenbanken @MarkusWinand @ModernSQL - - PowerPoint PPT Presentation

Neues in Open-Source-SQL-Datenbanken @MarkusWinand @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf Neues in Open-Source-SQL-Datenbanken @MarkusWinand @ModernSQL


slide-1
SLIDE 1

Neues in
 Open-Source-SQL-Datenbanken

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-2
SLIDE 2

Neues in
 Open-Source-SQL-Datenbanken

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-3
SLIDE 3
slide-4
SLIDE 4

SQL-92

slide-5
SLIDE 5

CHECK Constraints

slide-6
SLIDE 6

CHECK Constraints Since SQL-92

CREATE TABLE order_lines ( … qty INTEGER NOT NULL CHECK (qty > 0), … )

slide-7
SLIDE 7

CHECK Constraints Since SQL-92

CREATE TABLE order_lines ( … qty INTEGER NOT NULL CHECK (qty > 0), … ) INSERT … (…, qty, …) VALUES (…, 1, …)

slide-8
SLIDE 8

CHECK Constraints Since SQL-92

CREATE TABLE order_lines ( … qty INTEGER NOT NULL CHECK (qty > 0), … ) INSERT … (…, qty, …) VALUES (…, 1, …) INSERT … (…, qty, …) VALUES (…, 3, …)

slide-9
SLIDE 9

CHECK Constraints Since SQL-92

CREATE TABLE order_lines ( … qty INTEGER NOT NULL CHECK (qty > 0), … ) INSERT … (…, qty, …) VALUES (…, 1, …) INSERT … (…, qty, …) VALUES (…, 3, …) INSERT … (…, qty, …) VALUES (…, 0, …)

slide-10
SLIDE 10

CHECK Constraints Since SQL-92

CREATE TABLE order_lines ( … qty INTEGER NOT NULL CHECK (qty > 0), … ) INSERT … (…, qty, …) VALUES (…, 1, …) INSERT … (…, qty, …) VALUES (…, 3, …) INSERT … (…, qty, …) VALUES (…, 0, …)

Before MySQL 8.0.16
 and MariaDB 10.2:

Syntax accepted,
 Constraint ignored

slide-11
SLIDE 11

CHECK Constraints Since SQL-92

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

10.2

MariaDB

8.0.16 MySQL 8.3

PostgreSQL

3.5.7

SQLite

9.7

DB2 LUW

11gR1

Oracle

2008R2

SQL Server

slide-12
SLIDE 12

INTERSECT and EXCEPT

slide-13
SLIDE 13

Since SQL-92 INTERSECT & EXCEPT

UNION [ALL] Concatenates
 two results

slide-14
SLIDE 14

Since SQL-92 INTERSECT & EXCEPT

UNION [ALL] Concatenates
 two results INTERSECT [ALL] Common rows from
 two results

slide-15
SLIDE 15

Since SQL-92 INTERSECT & EXCEPT

UNION [ALL] Concatenates
 two results INTERSECT [ALL] Common rows from
 two results EXCEPT [ALL] Remove rows
 from first result

slide-16
SLIDE 16

INTERSECT & EXCEPT Since SQL-92

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

10.3

[0]

MariaDB MySQL

8.3

PostgreSQL

3.5.7

[0]

SQLite

9.7

DB2 LUW

11gR1[0]

Oracle

2008R2[0]

SQL Server

[0]Not [all]
slide-17
SLIDE 17
slide-18
SLIDE 18

SQL:1999

slide-19
SLIDE 19

LATERAL

slide-20
SLIDE 20

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar
slide-21
SLIDE 21

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar

, column_2 More than


  • ne column?

⇒Syntax error

slide-22
SLIDE 22

Select-list sub-queries must be scalar[0]:

LATERAL Before SQL:1999

SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(an atomic quantity that can hold only one value at a time[1])

[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar

, column_2 More than


  • ne column?

⇒Syntax error

}

More than


  • ne row?

⇒Runtime error!

slide-23
SLIDE 23

SELECT * FROM t1 CROSS JOIN LATERAL (SELECT * FROM t2 WHERE t2.x = t1.x ) derived_table ON (true)

LATERAL Since SQL:1999

Lateral derived queries can see table names defined before:

slide-24
SLIDE 24

SELECT * FROM t1 CROSS JOIN LATERAL (SELECT * FROM t2 WHERE t2.x = t1.x ) derived_table ON (true)

LATERAL Since SQL:1999

V a l i d d u e t

  • L

A T E R A L k e y w

  • r

d

Lateral derived queries can see table names defined before:

slide-25
SLIDE 25

SELECT * FROM t1 CROSS JOIN LATERAL (SELECT * FROM t2 WHERE t2.x = t1.x ) derived_table ON (true)

LATERAL Since SQL:1999

V a l i d d u e t

  • L

A T E R A L k e y w

  • r

d Useless, but still required

Lateral derived queries can see table names defined before:

slide-26
SLIDE 26

SELECT * FROM t1 CROSS JOIN LATERAL (SELECT * FROM t2 WHERE t2.x = t1.x ) derived_table ON (true)

LATERAL Since SQL:1999

V a l i d d u e t

  • L

A T E R A L k e y w

  • r

d

Lateral derived queries can see table names defined before:

Use CROSS JOIN
 to omit the ON clause

slide-27
SLIDE 27

But WHY?

slide-28
SLIDE 28

Use-Cases LATERAL

slide-29
SLIDE 29
  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

Use-Cases LATERAL

slide-30
SLIDE 30

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

Use-Cases LATERAL

slide-31
SLIDE 31

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

Use-Cases LATERAL

Add proper index
 for Top-N query

http://use-the-index-luke.com/sql/partial-results/top-n-queries
slide-32
SLIDE 32

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

Use-Cases LATERAL

Add proper index
 for Top-N query

http://use-the-index-luke.com/sql/partial-results/top-n-queries
slide-33
SLIDE 33

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

  • Table function arguments


(TABLE often implies LATERAL)


Use-Cases LATERAL

FROM t JOIN TABLE (your_func(t.c))

slide-34
SLIDE 34

LATERAL is the "for each" loop of SQL LATERAL plays well with outer and cross joins LATERAL is great for Top-N subqueries LATERAL can join table functions (unnest!)


LATERAL In a Nutshell

slide-35
SLIDE 35

LATERAL Availability

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0.14 MySQL 9.3

PostgreSQL SQLite

9.1

DB2 LUW

11gR1[0] 12cR1

Oracle

2005[1]

SQL Server

[0]Undocumented. Requires setting trace event 22829. [1]LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.
slide-36
SLIDE 36

SQL:2003

slide-37
SLIDE 37

BOOLEAN Tests

slide-38
SLIDE 38

Before we start

SQL uses a three-valued logic. Boolean values are either
 true, false or unknown(=null).

See: http://modern-sql.com/concept/three-valued-logic

BOOLEAN Aggregates

slide-39
SLIDE 39

BOOLEAN Tests

Similar to is null, there are tests for each Boolean value
 (of which there are three: true, false, unknown/null) IS [NOT] [TRUE|FALSE|UNKNOWN]

Since SQL:2003

slide-40
SLIDE 40

CREATE TABLE prices ( … valid_from DATE NOT NULL, valid_to DATE, -- null: open end … CHECK (valid_from < valid_to), ); WHERE valid_from < CURRENT_DATE AND (valid_to <= CURRENT_DATE) IS NOT FALSE

BOOLEAN Tests Since SQL:2003

slide-41
SLIDE 41

CREATE TABLE prices ( … valid_from DATE NOT NULL, valid_to DATE, -- null: open end … CHECK (valid_from < valid_to), ); WHERE valid_from < CURRENT_DATE AND (valid_to <= CURRENT_DATE) IS NOT FALSE

BOOLEAN Tests Since SQL:2003

slide-42
SLIDE 42

CREATE TABLE prices ( … valid_from DATE NOT NULL, valid_to DATE, -- null: open end … CHECK (valid_from < valid_to), ); WHERE valid_from < CURRENT_DATE AND (valid_to <= CURRENT_DATE) IS NOT FALSE

BOOLEAN Tests Since SQL:2003

UNKNOWN if
 VALID_TO is NULL

slide-43
SLIDE 43

CREATE TABLE prices ( … valid_from DATE NOT NULL, valid_to DATE, -- null: open end … CHECK (valid_from < valid_to), ); WHERE valid_from < CURRENT_DATE AND (valid_to <= CURRENT_DATE) IS NOT FALSE

BOOLEAN Tests Since SQL:2003

UNKNOWN if
 VALID_TO is NULL Takes TRUE and UNKNOWN

slide-44
SLIDE 44

BOOLEAN Tests Since SQL:2003

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

5.0.51a

MySQL

8.3

PostgreSQL

3.23.0

[0]

SQLite DB2 LUW Oracle SQL Server

[0]No IS [NOT] UNKNOWN. Use IS [NOT] NULL instead.
slide-45
SLIDE 45

BOOLEAN Type

slide-46
SLIDE 46

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

slide-47
SLIDE 47

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Without NOT NULL
 it is a
 three-valued Boolean

slide-48
SLIDE 48

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

SELECT … FROM … WHERE NOT(deleted)

slide-49
SLIDE 49

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

slide-50
SLIDE 50

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

slide-51
SLIDE 51

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

INSERT … (…, deleted, …) VALUES (…, true, …)

slide-52
SLIDE 52

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

INSERT … (…, deleted, …) VALUES (…, true, …) INSERT … (…, deleted, …) VALUES (…, false, …)

slide-53
SLIDE 53

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

INSERT … (…, deleted, …) VALUES (…, true, …) INSERT … (…, deleted, …) VALUES (…, false, …) INSERT … (…, deleted, …) VALUES (…, 42, …)

slide-54
SLIDE 54

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

INSERT … (…, deleted, …) VALUES (…, true, …) INSERT … (…, deleted, …) VALUES (…, false, …) INSERT … (…, deleted, …) VALUES (…, 42, …) UNIQUE,

slide-55
SLIDE 55

BOOLEAN Type

CREATE TABLE … ( … deleted BOOLEAN NOT NULL, … )

Since SQL:2003

Alias for int

MySQL MariaDB
 SQLite

INSERT … (…, deleted, …) VALUES (…, true, …) INSERT … (…, deleted, …) VALUES (…, false, …) INSERT … (…, deleted, …) VALUES (…, 42, …) UNIQUE, +----+ | de | +----+ | 1 | | 0 | | 42 | +----+

slide-56
SLIDE 56

BOOLEAN Type Since SQL:2003

Note that boolean in base tables is often questionable:


slide-57
SLIDE 57

BOOLEAN Type Since SQL:2003

Note that boolean in base tables is often questionable:


  • deleted flags are a poor mans temporal database model

slide-58
SLIDE 58

BOOLEAN Type Since SQL:2003

Note that boolean in base tables is often questionable:


  • deleted flags are a poor mans temporal database model

  • States often need more than two (or three) values


consider using an enum instead (can be evolved)

slide-59
SLIDE 59

BOOLEAN Type Since SQL:2003

Note that boolean in base tables is often questionable:


  • deleted flags are a poor mans temporal database model

  • States often need more than two (or three) values


consider using an enum instead (can be evolved) 
 See: 3 Reasons I Hate Booleans In Databases by Jeff Potter
 https://medium.com/@jpotts18/646d99696580

slide-60
SLIDE 60

BOOLEAN Type Since SQL:2003

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

[0]

MariaDB

5.0.51a

[0]

MySQL

8.4

PostgreSQL

3.23.0

[0]

SQLite DB2 LUW Oracle SQL Server

[0]BOOLEAN, TRUE, FALSE are aliases for TINYINT(1), 1, 0 respectivley.
slide-61
SLIDE 61

OVER

and

PARTITION BY

slide-62
SLIDE 62

OVER (PARTITION BY) The Problem

Two distinct concepts could not be used independently:

slide-63
SLIDE 63

OVER (PARTITION BY) The Problem

Two distinct concepts could not be used independently:

  • Merge rows with the same key properties
  • GROUP BY to specify key properties
  • DISTINCT to use full row as key
slide-64
SLIDE 64

OVER (PARTITION BY) The Problem

Two distinct concepts could not be used independently:

  • Merge rows with the same key properties
  • GROUP BY to specify key properties
  • DISTINCT to use full row as key
  • Aggregate data from related rows
  • Requires GROUP BY to segregate the rows
  • COUNT, SUM, AVG, MIN, MAX to aggregate grouped rows
slide-65
SLIDE 65

OVER (PARTITION BY) The Problem

slide-66
SLIDE 66

OVER (PARTITION BY) The Problem

SELECT c1 , c2 FROM t SELECT c1 , c2 FROM t

slide-67
SLIDE 67

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No SELECT c1 , c2 FROM t SELECT c1 , c2 FROM t

slide-68
SLIDE 68

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

slide-69
SLIDE 69

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

slide-70
SLIDE 70

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

slide-71
SLIDE 71

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

slide-72
SLIDE 72

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

JOIN ( ) ta ON (t.c1=ta.c1)

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

slide-73
SLIDE 73

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

JOIN ( ) ta ON (t.c1=ta.c1)

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1
slide-74
SLIDE 74

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

JOIN ( ) ta ON (t.c1=ta.c1)

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

, tot

slide-75
SLIDE 75

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) The Problem

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t

JOIN ( ) ta ON (t.c1=ta.c1)

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

, tot

slide-76
SLIDE 76

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) Since SQL:2003

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t FROM t

slide-77
SLIDE 77

SELECT c1 , SUM(c2) tot FROM t GROUP BY c1

OVER (PARTITION BY) Since SQL:2003

Yes ⇠ Merge rows ⇢ No No ⇠ Aggregate ⇢ Yes SELECT c1 , c2 FROM t SELECT DISTINCT c1 , c2 FROM t SELECT c1 , c2 FROM t FROM t , SUM(c2) OVER (PARTITION BY c1)

slide-78
SLIDE 78

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

slide-79
SLIDE 79

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

Look here

slide-80
SLIDE 80

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

slide-81
SLIDE 81

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

slide-82
SLIDE 82

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

slide-83
SLIDE 83

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000

OVER (PARTITION BY) How it works

slide-84
SLIDE 84

dep salary 1 1000 6000 22 1000 6000 22 1000 6000 333 1000 6000 333 1000 6000 333 1000 6000 SELECT dep, salary, SUM(salary) OVER() FROM emp

OVER (PARTITION BY) How it works

)

slide-85
SLIDE 85

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 1000 22 1000 2000 22 1000 2000 333 1000 3000 333 1000 3000 333 1000 3000

OVER (PARTITION BY) How it works

)

slide-86
SLIDE 86

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 1000 22 1000 2000 22 1000 2000 333 1000 3000 333 1000 3000 333 1000 3000

OVER (PARTITION BY) How it works

) PARTITION BY dep

slide-87
SLIDE 87

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 1000 22 1000 2000 22 1000 2000 333 1000 3000 333 1000 3000 333 1000 3000

OVER (PARTITION BY) How it works

) PARTITION BY dep

slide-88
SLIDE 88

SELECT dep, salary, SUM(salary) OVER() FROM emp dep salary 1 1000 1000 22 1000 2000 22 1000 2000 333 1000 3000 333 1000 3000 333 1000 3000

OVER (PARTITION BY) How it works

) PARTITION BY dep

slide-89
SLIDE 89

OVER

and

ORDER BY

(Framing & Ranking)

slide-90
SLIDE 90

acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

OVER (ORDER BY) The Problem

SELECT id, value, FROM transactions t

slide-91
SLIDE 91

acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

OVER (ORDER BY) The Problem

SELECT id, value, FROM transactions t

slide-92
SLIDE 92

acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

OVER (ORDER BY) The Problem

SELECT id, value, FROM transactions t

slide-93
SLIDE 93

acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

OVER (ORDER BY) The Problem

SELECT id, value, (SELECT SUM(value) FROM transactions t2 WHERE t2.id <= t.id) FROM transactions t

slide-94
SLIDE 94

acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

OVER (ORDER BY) The Problem

SELECT id, value, (SELECT SUM(value) FROM transactions t2 WHERE t2.id <= t.id) FROM transactions t

Range segregation (<=)
 not possible with
 GROUP BY or
 PARTITION BY

slide-95
SLIDE 95

OVER (ORDER BY) Since SQL:2003

SELECT id, value, (SELECT SUM(value) FROM transactions t2 WHERE t2.id <= t.id) FROM transactions t acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

slide-96
SLIDE 96

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

slide-97
SLIDE 97

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20

slide-98
SLIDE 98

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id

slide-99
SLIDE 99

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING

slide-100
SLIDE 100

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-101
SLIDE 101

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-102
SLIDE 102

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-103
SLIDE 103

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-104
SLIDE 104

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-105
SLIDE 105

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-106
SLIDE 106

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-107
SLIDE 107

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +30 22 3

  • 10

+20 333 4 +50 +70 333 5

  • 30

+40 333 6

  • 20

+20 ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-108
SLIDE 108

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 22 2 +20 22 3

  • 10

333 4 +50 333 5

  • 30

333 6

  • 20

ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW

slide-109
SLIDE 109

OVER (ORDER BY) Since SQL:2003

SELECT id, value, FROM transactions t SUM(value) OVER ( ) acnt id value balance 1 1 +10 +10 22 2 +20 +20 22 3

  • 10

+10 333 4 +50 +50 333 5

  • 30

+20 333 6

  • 20
. 0

ORDER BY id ROWS BETWEEN
 UNBOUNDED PRECEDING AND CURRENT ROW PARTITION BY acnt

slide-110
SLIDE 110
  • Aggregates without GROUP BY

Use Cases OVER (SQL:2003)

slide-111
SLIDE 111
  • Aggregates without GROUP BY
  • Running totals,


moving averages

Use Cases

AVG(…) OVER(ORDER BY … ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg

OVER (SQL:2003)

slide-112
SLIDE 112
  • Aggregates without GROUP BY
  • Running totals,


moving averages

  • Ranking

Use Cases

AVG(…) OVER(ORDER BY … ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg

OVER (SQL:2003)

slide-113
SLIDE 113
  • Aggregates without GROUP BY
  • Running totals,


moving averages

  • Ranking
  • Top-N per Group

Use Cases

SELECT * FROM (SELECT ROW_NUMBER() OVER(PARTITION BY … ORDER BY …) rn , t.* FROM t) numbered_t WHERE rn <= 3 AVG(…) OVER(ORDER BY … ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg

OVER (SQL:2003)

slide-114
SLIDE 114
  • Aggregates without GROUP BY
  • Running totals,


moving averages

  • Ranking
  • Top-N per Group
  • Avoiding self-joins

Use Cases

SELECT * FROM (SELECT ROW_NUMBER() OVER(PARTITION BY … ORDER BY …) rn , t.* FROM t) numbered_t WHERE rn <= 3 AVG(…) OVER(ORDER BY … ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg

OVER (SQL:2003)

slide-115
SLIDE 115
  • Aggregates without GROUP BY
  • Running totals,


moving averages

  • Ranking
  • Top-N per Group
  • Avoiding self-joins

[… many more …]

Use Cases

SELECT * FROM (SELECT ROW_NUMBER() OVER(PARTITION BY … ORDER BY …) rn , t.* FROM t) numbered_t WHERE rn <= 3 AVG(…) OVER(ORDER BY … ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg

OVER (SQL:2003)

slide-116
SLIDE 116

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0

MySQL

8.4

PostgreSQL

3.25.0

SQLite

7.0

DB2 LUW

8i

Oracle

2005[0] 2012

SQL Server

[0]No framing

OVER (SQL:2003) Availability

slide-117
SLIDE 117

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0

MySQL

8.4

PostgreSQL

3.25.0

SQLite

7.0

DB2 LUW

8i

Oracle

2005[0] 2012

SQL Server

[0]No framing

OVER (SQL:2003) Availability

Impala Spark NuoDB BigQuery Hive

slide-118
SLIDE 118

OVER

SQL:2003 frame exclusion

slide-119
SLIDE 119

Since SQL:2003

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] )

OVER (frame exclusion)

slide-120
SLIDE 120

Since SQL:2003

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default

OVER (frame exclusion)

no others

slide-121
SLIDE 121

Since SQL:2003

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default

OVER (frame exclusion)

no others

slide-122
SLIDE 122

current row

Since SQL:2003

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default

OVER (frame exclusion)

current row

slide-123
SLIDE 123

group

x = current_row

current row

Since SQL:2003

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default

OVER (frame exclusion)

group

slide-124
SLIDE 124

ties group

x = current_row

current row

Since SQL:2003

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default

OVER (frame exclusion)

ties

slide-125
SLIDE 125

OVER (frame exclusion) Since SQL:2011

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

11

PostgreSQL

3.28.0 SQLite

DB2 LUW Oracle SQL Server

slide-126
SLIDE 126

FILTER

slide-127
SLIDE 127

FILTER Before we start

In SQL, most aggregate functions*
 drop null arguments
 prior to the aggregation.

*Exceptions: Some aggregate functions that return structured data: array_agg, json_objectagg, xmlagg See: http://modern-sql.com/concept/null#aggregates

slide-128
SLIDE 128

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X:

slide-129
SLIDE 129

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X:

slide-130
SLIDE 130

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X:

slide-131
SLIDE 131

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X: Optional:
 ELSE NULL is default

slide-132
SLIDE 132

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X: Optional:
 ELSE NULL is default Aggregates
 ignore NULL*

*Exceptions:array_agg, json_objectagg, xmlagg See: https://modern-sql.com/concept/null#aggregates

slide-133
SLIDE 133

SELECT YEAR, SUM(CASE WHEN MONTH = 1 THEN revenue
 ELSE 0
 END) JAN, SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB, … FROM sales GROUP BY YEAR

FILTER The Problem

Pivot table: Years on the Y axis, month on X:

slide-134
SLIDE 134

SELECT YEAR, SUM(revenue) FILTER (WHERE MONTH = 1) JAN, SUM(revenue) FILTER (WHERE MONTH = 2) FEB, … FROM sales GROUP BY YEAR;

FILTER Since SQL:2003

SQL:2003 allows FILTER (WHERE…) after aggregates:

slide-135
SLIDE 135

FILTER Since SQL:2003

Year 2016 2016 2016 2016 2016 Month 1 2 3 ... 12 Revenue 1 23 345 ... 1234 Year 2016 Jan 1 Feb 23 Mar 345 ... ... Dec 1234 S U M ( … ) F I L T E R ( W H E R E … ) S U M ( … ) F I L T E R ( W H E R E m

  • n

t h = 2 ) S U M ( r e v e n u e ) F I L T E R ( W H E R E m

  • n

t h = 3 ) S U M ( r e v e n u e ) F I L T E R ( W H E R E m

  • n

t h = … ) S U M ( r e v e n u e ) F I L T E R ( W H E R E m

  • n

t h = 1 2 )

Pivot in SQL

  • 1. Use GROUP BY

to combine rows

  • 2. Use FILTER to pick

rows per column

See: https://modern-sql.com/use-case/pivot

slide-136
SLIDE 136

FILTER Availability

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

9.4

PostgreSQL

3.25.0

[0]

SQLite DB2 LUW Oracle SQL Server

[0]Only with OVER clause
slide-137
SLIDE 137

Inverse Distribution Functions (percentiles)

slide-138
SLIDE 138

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

slide-139
SLIDE 139

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

slide-140
SLIDE 140

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

Number rows

slide-141
SLIDE 141

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

Number rows Pick middle one

slide-142
SLIDE 142

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

Number rows Pick middle one

slide-143
SLIDE 143

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Inverse Distribution Functions

The Problem

Grouped rows cannot be ordered prior to aggregation.

(how to get the middle value (median) of a set)

Number rows Pick middle one

slide-144
SLIDE 144

Since SQL:2003

Inverse Distribution Functions

slide-145
SLIDE 145

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

slide-146
SLIDE 146

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median

Since SQL:2003

Inverse Distribution Functions

slide-147
SLIDE 147

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median Which value?

Since SQL:2003

Inverse Distribution Functions

slide-148
SLIDE 148

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

slide-149
SLIDE 149

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-150
SLIDE 150

1 2 3 4

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-151
SLIDE 151

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-152
SLIDE 152

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-153
SLIDE 153

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-154
SLIDE 154

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-155
SLIDE 155

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-156
SLIDE 156

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT(0.5) PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Since SQL:2003

Inverse Distribution Functions

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

slide-157
SLIDE 157

Inverse Distribution Functions

Availability

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.3

[0]

MariaDB MySQL

9.4

PostgreSQL SQLite

11.1

DB2 LUW

9iR1

Oracle

2012[0]

SQL Server

[0]Only as window function (requires OVER clause)
slide-158
SLIDE 158

SQL:2011

slide-159
SLIDE 159

OVER

SQL:2011 groups option

slide-160
SLIDE 160

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following

slide-161
SLIDE 161

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

slide-162
SLIDE 162

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

x 1 3 3.5 3.5 4

CURRENT ROW

slide-163
SLIDE 163

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

slide-164
SLIDE 164

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

range

x between current_row - 1
 and current_row + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

slide-165
SLIDE 165

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

New in
 SQL:2011


groups

range

x between current_row - 1
 and current_row + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

slide-166
SLIDE 166

groups

count(distinct x)

OVER (groups option) Since SQL:2011

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

New in
 SQL:2011


groups

range

x between current_row - 1
 and current_row + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

slide-167
SLIDE 167

Since SQL:2011 OVER (groups option)

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

11

PostgreSQL

3.28.0 SQLite

DB2 LUW Oracle SQL Server

slide-168
SLIDE 168

System Versioning

(Time Traveling)

slide-169
SLIDE 169

INSERT UPDATE DELETE are

DESTRUCTIVE

System Versioning The Problem

slide-170
SLIDE 170

System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-171
SLIDE 171

CREATE TABLE t (...,

System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-172
SLIDE 172

CREATE TABLE t (..., start_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW START,

System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-173
SLIDE 173

CREATE TABLE t (..., start_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW START, end_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW END,


System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-174
SLIDE 174

CREATE TABLE t (..., start_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW START, end_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW END,
 PERIOD FOR SYSTEM_TIME (start_ts, end_ts)

System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-175
SLIDE 175

CREATE TABLE t (..., start_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW START, end_ts TIMESTAMP(9) GENERATED
 ALWAYS AS ROW END,
 PERIOD FOR SYSTEM_TIME (start_ts, end_ts) ) WITH SYSTEM VERSIONING

System Versioning Since SQL:2011

Table can be system versioned, application versioned or both.

slide-176
SLIDE 176

ID Data start_ts end_ts 1 X 10:00:00

UPDATE ... SET DATA = 'Y' ...

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00

DELETE ... WHERE ID = 1 INSERT ... (ID, DATA) VALUES (1, 'X')

System Versioning Since SQL:2011

slide-177
SLIDE 177

ID Data start_ts end_ts 1 X 10:00:00

UPDATE ... SET DATA = 'Y' ...

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00

DELETE ... WHERE ID = 1 INSERT ... (ID, DATA) VALUES (1, 'X')

System Versioning Since SQL:2011

slide-178
SLIDE 178

ID Data start_ts end_ts 1 X 10:00:00

UPDATE ... SET DATA = 'Y' ...

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00

DELETE ... WHERE ID = 1 INSERT ... (ID, DATA) VALUES (1, 'X')

System Versioning Since SQL:2011

slide-179
SLIDE 179

ID Data start_ts end_ts 1 X 10:00:00

UPDATE ... SET DATA = 'Y' ...

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00

DELETE ... WHERE ID = 1

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00

System Versioning Since SQL:2011

slide-180
SLIDE 180

Although multiple versions exist, only the “current”

  • ne is visible per default.

After 12:00:00, SELECT * FROM t doesn’t return anything anymore.

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00

System Versioning Since SQL:2011

slide-181
SLIDE 181

ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00

With FOR … AS OF you can query anything you like: SELECT * FROM t FOR SYSTEM_TIME AS OF TIMESTAMP '2019-05-02 10:30:00'

ID Data start_ts end_ts 1 X 10:00:00 11:00:00

System Versioning Since SQL:2011

slide-182
SLIDE 182

System Versioning Since SQL:2011

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.3

MariaDB MySQL PostgreSQL SQLite

10.1

DB2 LUW

10gR1[0] 11gR1[1]

Oracle

2016

SQL Server

[0]Short term using Flashback. [1]Flashback Archive. Proprietery syntax.
slide-183
SLIDE 183

FOR PORTION OF

(UPDATE, DELETE)

slide-184
SLIDE 184

CREATE TABLE t ( ...,

Since SQL:2011

Table can be system versioned, application versioned or both.

FOR PORTION OF

slide-185
SLIDE 185

CREATE TABLE t ( ..., start_ts TIMESTAMP(9),

Since SQL:2011

Table can be system versioned, application versioned or both.

FOR PORTION OF

slide-186
SLIDE 186

CREATE TABLE t ( ..., start_ts TIMESTAMP(9), end_ts TIMESTAMP(9),


Since SQL:2011

Table can be system versioned, application versioned or both.

FOR PORTION OF

slide-187
SLIDE 187

CREATE TABLE t ( ..., start_ts TIMESTAMP(9), end_ts TIMESTAMP(9),
 PERIOD FOR app (start_ts, end_ts) )

Since SQL:2011

Table can be system versioned, application versioned or both.

FOR PORTION OF

slide-188
SLIDE 188

ID Data start_ts end_ts 1 X 10:00:00 12:00:00

UPDATE t FOR PORTION OF app FROM '10:30:00' TO '11:30:00' SET DATA = 'Y'

ID Data start_ts end_ts 1 X 10:00:00 10:30:00 1 Y 10:30:00 11:30:00 1 X 11:30:00 12:00:00

INSERT t (ID, DATA, start_ts , end_ts) 
 VALUES ( 1, 'X', '10:00:00', '12:00:00')

Since SQL:2011 FOR PORTION OF

slide-189
SLIDE 189

ID Data start_ts end_ts 1 X 10:00:00 12:00:00

UPDATE t FOR PORTION OF app FROM '10:30:00' TO '11:30:00' SET DATA = 'Y'

ID Data start_ts end_ts 1 X 10:00:00 10:30:00 1 Y 10:30:00 11:30:00 1 X 11:30:00 12:00:00

INSERT t (ID, DATA, start_ts , end_ts) 
 VALUES ( 1, 'X', '10:00:00', '12:00:00')

Since SQL:2011 FOR PORTION OF

slide-190
SLIDE 190

ID Data start_ts end_ts 1 X 10:00:00 12:00:00

UPDATE t FOR PORTION OF app FROM '10:30:00' TO '11:30:00' SET DATA = 'Y'

ID Data start_ts end_ts 1 X 10:00:00 10:30:00 1 Y 10:30:00 11:30:00 1 X 11:30:00 12:00:00

INSERT t (ID, DATA, start_ts , end_ts) 
 VALUES ( 1, 'X', '10:00:00', '12:00:00')

Since SQL:2011 FOR PORTION OF

slide-191
SLIDE 191

FOR PORTION OF Since SQL:2011

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL PostgreSQL SQLite

10.5

DB2 LUW Oracle SQL Server

10.4

slide-192
SLIDE 192

FOR PORTION OF Since SQL:2011

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL PostgreSQL SQLite

10.5

DB2 LUW Oracle SQL Server

10.4

slide-193
SLIDE 193

Since SQL:2011

Application Versioning

For a useful application versioning, WITHOUT OVERLAPS constraints are required.

Apparently this was moved to MariaDB 10.5: https://jira.mariadb.org/browse/MDEV-16978

slide-194
SLIDE 194
slide-195
SLIDE 195

https://commons.wikimedia.org/wiki/File:Tamias_striatus_CT.jpg

Mühsam
 ernährt
 sich das
 Eichhörnchen

slide-196
SLIDE 196

@ModernSQL modern-sql.com

My other website:

This is the training
 you are looking for:
 https://winand.at/