More Than a Query Language: SQL in the 21 st Century @MarkusWinand - - PowerPoint PPT Presentation

more than a query language sql in the 21 st century
SMART_READER_LITE
LIVE PREVIEW

More Than a Query Language: SQL in the 21 st Century @MarkusWinand - - PowerPoint PPT Presentation

More Than a Query Language: SQL in the 21 st Century @MarkusWinand @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf More Than a Query Language: SQL in the 21 st Century @MarkusWinand @ModernSQL


slide-1
SLIDE 1

More Than a Query Language: SQL in the 21st Century

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-2
SLIDE 2

More Than a Query Language: SQL in the 21st Century

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-3
SLIDE 3
slide-4
SLIDE 4

1974 1992

slide-5
SLIDE 5

SQL-92 — Tied to the Relational Idea

slide-6
SLIDE 6

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
slide-7
SLIDE 7

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

slide-8
SLIDE 8

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

A B C

Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

slide-9
SLIDE 9

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

A B C

slide-10
SLIDE 10

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

A B C

slide-11
SLIDE 11

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

A B C C D B E

slide-12
SLIDE 12

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E A B C D E

slide-13
SLIDE 13

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E

A B E

slide-14
SLIDE 14

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E A B E

C D E

slide-15
SLIDE 15

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E A B E C D E

slide-16
SLIDE 16

1992 1999

slide-17
SLIDE 17

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-18
SLIDE 18

SQL:1999 — Escaping the Relational Cage

To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear.

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-19
SLIDE 19

SQL:1999 — Escaping the Relational Cage

To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear. With SQL/99 you can get the best of both worlds and


  • f course, you can get the worst of both worlds.


It’s up to the database practitioners to do the right thing.

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-20
SLIDE 20

SQL:1999 — Escaping the Relational Cage

slide-21
SLIDE 21

Relational Model?

SQL:1999 — Escaping the Relational Cage

slide-22
SLIDE 22

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

I was as confused as anyone else

slide-23
SLIDE 23

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

?

I was as confused as anyone else

slide-24
SLIDE 24

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

slide-25
SLIDE 25

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-26
SLIDE 26

Relational Model?

  • Introduced rich types

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-27
SLIDE 27

Relational Model?

  • Introduced rich types

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-28
SLIDE 28

Relational Model?

  • Introduced rich types
  • arrays

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B

[ , ] [ ] []

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-29
SLIDE 29

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B

[ , ] [ ] []

C

C D C D C D

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-30
SLIDE 30

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)
  • composite types (objects)

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B C D

[ , ]

{x: , y: }

[ ]

{x: , y: }

[]

{x: , y: }

C D C D C D

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-31
SLIDE 31

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)
  • composite types (objects)

Non-Relational Operations

  • Introduced recursive


queries that process
 their own output

  • Transitive closure

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-32
SLIDE 32

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-33
SLIDE 33

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-34
SLIDE 34

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-35
SLIDE 35

SQL:1999 — Recursion

slide-36
SLIDE 36

SQL:1999 — Recursion

slide-37
SLIDE 37

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ?

slide-38
SLIDE 38

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-39
SLIDE 39

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-40
SLIDE 40

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-41
SLIDE 41

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-42
SLIDE 42

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-43
SLIDE 43

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-44
SLIDE 44

SQL:1999 — Recursion

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0 MySQL 8.4

PostgreSQL

1.0 3.8.3

SQLite

7.0

DB2 LUW

11gR2

Oracle

2005

SQL Server

slide-45
SLIDE 45
slide-46
SLIDE 46

LATERAL

slide-47
SLIDE 47

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

slide-48
SLIDE 48

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

, column_2 More than


  • ne column?

⇒Syntax error

slide-49
SLIDE 49

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

, column_2 More than


  • ne column?

⇒Syntax error

}

More than


  • ne row?

⇒Runtime error!

slide-50
SLIDE 50

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt …

slide-51
SLIDE 51

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause

slide-52
SLIDE 52

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated”

slide-53
SLIDE 53

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated” Regular join semantics

slide-54
SLIDE 54

SQL:1999 — LATERAL

slide-55
SLIDE 55

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

slide-56
SLIDE 56

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-57
SLIDE 57

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-58
SLIDE 58

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-59
SLIDE 59

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

  • Table function arguments


(TABLE often implies LATERAL)


Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN TABLE (your_func(t.c)) FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-60
SLIDE 60

SQL:1999 — LATERAL

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0.14

MySQL

9.3

PostgreSQL

1.0

SQLite

9.1

DB2 LUW

11gR1

[0]

12cR1

Oracle

2005[1]

SQL Server

[0]Undocumented. Requires setting trace event 22829. [1]LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.

slide-61
SLIDE 61

SQL:1999 — LATERAL

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0.14

MySQL

9.3

PostgreSQL

1.0

SQLite

9.1

DB2 LUW

11gR1

[0]

12cR1

Oracle

2005[1]

SQL Server

[0]Undocumented. Requires setting trace event 22829. [1]LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.

MDEV-6373: Fix Version: none

slide-62
SLIDE 62

1999 2003

slide-63
SLIDE 63

http://www.acm.org:80/sigmod/record/issues/0206/standard.pdf (via Wayback machine)

slide-64
SLIDE 64

SQL:2003 — Schemaless & Analytical

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

slide-65
SLIDE 65

SQL:2003 — Schemaless & Analytical

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-66
SLIDE 66

SQL:2003 — Schemaless & Analytical

Analytical

  • Introduced


window functions

  • Accessing other rows

  • f the current result

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-67
SLIDE 67

SQL:2003 — Schemaless & Analytical

Analytical

  • Introduced


window functions

  • Accessing other rows

  • f the current result

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • Extended in SQL:2011
  • Popular among “New SQLs”
  • 2013: BigQuery, Hive
  • 2014: Impala
  • 2015: Spark SQL
  • 2016: NuoDB, MemSQL,

Cockroach DB, VoltDB

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-68
SLIDE 68

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

SQL:2003 — Analytical

slide-69
SLIDE 69

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

slide-70
SLIDE 70

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

+10

slide-71
SLIDE 71

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

+10 +30 +20 +70 +40 +20

slide-72
SLIDE 72

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

, SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-73
SLIDE 73

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-74
SLIDE 74

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-75
SLIDE 75

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-76
SLIDE 76

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-77
SLIDE 77

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-78
SLIDE 78

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-79
SLIDE 79

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-80
SLIDE 80

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-81
SLIDE 81

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-82
SLIDE 82

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-83
SLIDE 83

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-84
SLIDE 84

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-85
SLIDE 85

SQL:2003 — Analytical

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0 MySQL 8.4

PostgreSQL

1.0 3.25.0

SQLite

7.0

DB2 LUW

8i

Oracle

2005

[0]

2012

SQL Server

[0]Without framing

slide-86
SLIDE 86

Inverse Distribution Functions (percentiles)

slide-87
SLIDE 87

SQL:2003 — Analytical (Median)

slide-88
SLIDE 88

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

SQL:2003 — Analytical (Median)

slide-89
SLIDE 89

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows

SQL:2003 — Analytical (Median)

slide-90
SLIDE 90

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-91
SLIDE 91

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-92
SLIDE 92

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-93
SLIDE 93

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

Notice

All employees must

wash hands

after using

self-joins

slide-94
SLIDE 94

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

Notice

All employees must

wash hands

after using

self-joins

Grab stickers
 and coasters!

slide-95
SLIDE 95

SQL:2003 — Analytical (Median)

slide-96
SLIDE 96

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

SQL:2003 — Analytical (Median)

slide-97
SLIDE 97

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median

SQL:2003 — Analytical (Median)

slide-98
SLIDE 98

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median Which value?

SQL:2003 — Analytical (Median)

slide-99
SLIDE 99

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

SQL:2003 — Analytical (Median)

slide-100
SLIDE 100

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-101
SLIDE 101

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-102
SLIDE 102

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-103
SLIDE 103

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-104
SLIDE 104

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-105
SLIDE 105

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-106
SLIDE 106

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT(0.5) PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-107
SLIDE 107

SQL:2003 — PERCENTILE_DISC

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.3.7

[0]

MariaDB MySQL

9.4

PostgreSQL

1.0

SQLite DB2 LUW

9iR1

Oracle

2012

[0]

SQL Server

[0]Only as window function (OVER required).

slide-108
SLIDE 108

2003 2016

slide-109
SLIDE 109

SQL:2016 — JSON

http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

slide-110
SLIDE 110

SQL:2016 — JSON

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

slide-111
SLIDE 111

SQL:2016 — JSON

id a1 42 foo 43 bar [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

slide-112
SLIDE 112

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

SQL:2016 — JSON_TABLE

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

slide-113
SLIDE 113

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

SQL:2016 — JSON_TABLE

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

Bind Parameter

slide-114
SLIDE 114

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

SQL:2016 — JSON_TABLE

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

SQL/JSON Path

  • Query language to

select elements from a JSON document

  • Defined in the


SQL standard

Bind Parameter

slide-115
SLIDE 115

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

SQL:2016 — JSON_TABLE

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

SQL/JSON Path

  • Query language to

select elements from a JSON document

  • Defined in the


SQL standard

Bind Parameter

slide-116
SLIDE 116

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

SQL:2016 — JSON_TABLE

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

SQL/JSON Path

  • Query language to

select elements from a JSON document

  • Defined in the


SQL standard

Bind Parameter

slide-117
SLIDE 117

SQL:2016 — JSON_TABLE — Use Case

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

slide-118
SLIDE 118

SQL:2016 — JSON_TABLE — Use Case

SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r INSERT INTO target_table

[ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ]

id a1 42 foo 43 bar

slide-119
SLIDE 119

SQL:2016 — JSON_TABLE

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0 MySQL

PostgreSQL

1.0

SQLite

11.1.4.4

DB2 LUW

12cR1

Oracle

2016

[0]

SQL Server

[0]OPENJSON provides similar functionality

slide-120
SLIDE 120

SQL:2016 — JSON_TABLE

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0 MySQL

PostgreSQL

1.0

SQLite

11.1.4.4

DB2 LUW

12cR1

Oracle

2016

[0]

SQL Server

[0]OPENJSON provides similar functionality

MDEV-17399: Fix Version: 10.5

slide-121
SLIDE 121

2011 2016

slide-122
SLIDE 122

SQL:2011 — Time Travelling

http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf

slide-123
SLIDE 123

SQL:2011 — Time Travelling

Application Versioning

  • Dedicated syntax added
  • When did something


happen in the
 real world?

http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf

slide-124
SLIDE 124

SQL:2011 — Time Travelling

Application Versioning

  • Dedicated syntax added
  • When did something


happen in the
 real world?

New syntax (excerpt)

  • FOR PORTION OF in


UPDATE and DELETE

  • WITHOUT OVERLAPS in UNIQUE

constraints & PRIMARY KEYS

  • [IMMEDIATELY] PRECEDES,

OVERLAPS in WHERE,HAVING,…

slide-125
SLIDE 125

SQL:2011 — Time Travelling

System Versioning

  • Fully automatic and


(almost) transparent

  • When did we learn


about something Application Versioning

  • Dedicated syntax added
  • When did something


happen in the
 real world?

New syntax (excerpt)

  • FOR PORTION OF in


UPDATE and DELETE

  • WITHOUT OVERLAPS in UNIQUE

constraints & PRIMARY KEYS

  • [IMMEDIATELY] PRECEDES,

OVERLAPS in WHERE,HAVING,…

slide-126
SLIDE 126

SQL:2011 — Time Travelling

System Versioning

  • Fully automatic and


(almost) transparent

  • When did we learn


about something Application Versioning

  • Dedicated syntax added
  • When did something


happen in the
 real world?

New syntax (excerpt)

  • FOR PORTION OF in


UPDATE and DELETE

  • WITHOUT OVERLAPS in UNIQUE

constraints & PRIMARY KEYS

  • [IMMEDIATELY] PRECEDES,

OVERLAPS in WHERE,HAVING,… Transparent changes, new syntax for queries

  • INSERT, UPDATE & DELETE


use the system time automatically

  • SELECT can use FOR

SYSTEM_TIME AS OF

slide-127
SLIDE 127

SQL:2011 — System Versioning

CREATE TABLE t ( ... , from TIMESTAMP(9) GENERATED ALWAYS
 AS ROW START , till TIMESTAMP(9) GENERATED ALWAYS
 AS ROW END
 , PERIOD FOR SYSTEM_TIME (from, till) ) WITH SYSTEM VERSIONING

slide-128
SLIDE 128

SQL:2011 — System Versioning

slide-129
SLIDE 129

SQL:2011 — System Versioning

INSERT INTO t (id, data)
 VALUES (1 , 'X' )

id data from till 1 X 10:00

slide-130
SLIDE 130

SQL:2011 — System Versioning

INSERT INTO t (id, data)
 VALUES (1 , 'X' )

id data from till 1 X 10:00

UPDATE t SET data = 'Y' WHERE id = 1

id data from till 1 X 10:00 11:00 1 Y 11:00

slide-131
SLIDE 131

SQL:2011 — System Versioning

INSERT INTO t (id, data)
 VALUES (1 , 'X' )

id data from till 1 X 10:00

UPDATE t SET data = 'Y' WHERE id = 1

id data from till 1 X 10:00 11:00 1 Y 11:00

DELETE FROM t WHERE id = 1

id data from till 1 X 10:00 11:00 1 Y 11:00 12:00

slide-132
SLIDE 132

SQL:2011 — System Versioning

id data from till 1 X 10:00 11:00 1 Y 11:00 12:00

slide-133
SLIDE 133

SQL:2011 — System Versioning

id data from till 1 X 10:00 11:00 1 Y 11:00 12:00

SELECT * FROM t

id data from till

slide-134
SLIDE 134

SQL:2011 — System Versioning

id data from till 1 X 10:00 11:00 1 Y 11:00 12:00

SELECT * FROM t

id data from till

SELECT * FROM t FOR SYSTEM_TIME AS OF
 TIMESTAMP'…10:30:00'

id data from till 1 X 10:00 11:00

slide-135
SLIDE 135

SQL:2011 — System Versioning

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.3

MariaDB MySQL PostgreSQL

1.0

SQLite

10.1

DB2 LUW

10gR1

[0]

11gR1

[1]

Oracle

2016

SQL Server

[0]Short term using Flashback. [1]Flashback Archive. Proprietery syntax.

slide-136
SLIDE 136

SQL:2011 — Application Time Periods

slide-137
SLIDE 137

SQL:2011 — Application Time Periods

CREATE TABLE t ( ... , from TIMESTAMP(9) , till TIMESTAMP(9) , PERIOD FOR a (from, till) )

slide-138
SLIDE 138

SQL:2011 — Application Time Periods

ID Data From Till 1 X 10:00:00 12:00:00

INSERT t (id, data, from , till ) VALUES ( 1, 'X', '10:00:00', '12:00:00')

slide-139
SLIDE 139

SQL:2011 — Application Time Periods

UPDATE t
 FOR PORTION OF a FROM '10:30:00' TO '11:30:00' SET DATA = 'Y'

ID Data From Till 1 X 10:00:00 10:30:00 1 Y 10:30:00 11:30:00 1 X 11:30:00 12:00:00 ID Data From Till 1 X 10:00:00 12:00:00

INSERT t (id, data, from , till ) VALUES ( 1, 'X', '10:00:00', '12:00:00')

slide-140
SLIDE 140

SQL:2011 — Application Time Periods

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.4 MariaDB

MySQL PostgreSQL

1.0

SQLite

10.5

DB2 LUW Oracle SQL Server

slide-141
SLIDE 141

https://webstore.iec.ch/publication/59685

slide-142
SLIDE 142

A lot has
 happened since SQL-92

slide-143
SLIDE 143

SQL has evolved
 beyond the relational idea A lot has
 happened since SQL-92

slide-144
SLIDE 144

SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92

slide-145
SLIDE 145

SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92

https://modern-sql.com


@ModernSQL by @MarkusWinand

Training:
 https://winand.at/