More Than a Query Language: SQL in the 21 st Century @MarkusWinand - - PowerPoint PPT Presentation

more than a query language sql in the 21 st century
SMART_READER_LITE
LIVE PREVIEW

More Than a Query Language: SQL in the 21 st Century @MarkusWinand - - PowerPoint PPT Presentation

More Than a Query Language: SQL in the 21 st Century @MarkusWinand @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf More Than a Query Language: SQL in the 21 st Century @MarkusWinand @ModernSQL


slide-1
SLIDE 1

More Than a Query Language: SQL in the 21st Century

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-2
SLIDE 2

More Than a Query Language: SQL in the 21st Century

@MarkusWinand • @ModernSQL

http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

slide-3
SLIDE 3

Safe Harbour Statement …

slide-4
SLIDE 4

Safe Harbour Statement … Instead of a

slide-5
SLIDE 5

Safe Harbour Statement … Take this Safe the Planet Statement:

I’m traveling a lot for business and always aim for the most environmental friendly way to do so. If I cannot avoid flying, I offset its climate impact at a transparent climate protection company. (currently atmosfair.de)

Picture: https://upload.wikimedia.org/wikipedia/commons/2/2c/North_America_from_low_orbiting_satellite_Suomi_NPP .jpg

Instead of a

slide-6
SLIDE 6
slide-7
SLIDE 7

1974 1992

slide-8
SLIDE 8

SQL-92 — Tied to the Relational Idea

slide-9
SLIDE 9

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
slide-10
SLIDE 10

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

slide-11
SLIDE 11

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

A B C

Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

slide-12
SLIDE 12

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)

A B C

slide-13
SLIDE 13

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

A B C

slide-14
SLIDE 14

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

A B C C D B E

slide-15
SLIDE 15

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E A B C D E

slide-16
SLIDE 16

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E

A B E

slide-17
SLIDE 17

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E A B E

C D E

slide-18
SLIDE 18

SQL-92 — Tied to the Relational Idea

Relational Data Model

  • “Atomic” types (domain)
  • Schema independent of


processing purposes

  • “Normalization”

Relational Operations

  • Transform data for


each particular
 processing purposes

  • JOIN, UNION, nesting, …

A B C C D B E

A B C D E A B E C D E

slide-19
SLIDE 19

1992 1999

slide-20
SLIDE 20

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-21
SLIDE 21

SQL:1999 — Escaping the Relational Cage

To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear.

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-22
SLIDE 22

SQL:1999 — Escaping the Relational Cage

To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear. With SQL/99 you can get the best of both worlds and


  • f course, you can get the worst of both worlds.


It’s up to the database practitioners to do the right thing.

https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

slide-23
SLIDE 23

SQL:1999 — Escaping the Relational Cage

slide-24
SLIDE 24

Relational Model?

SQL:1999 — Escaping the Relational Cage

slide-25
SLIDE 25

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

I was as confused as anyone else

slide-26
SLIDE 26

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

?

I was as confused as anyone else

slide-27
SLIDE 27

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

slide-28
SLIDE 28

Relational Model?

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-29
SLIDE 29

Relational Model?

  • Introduced rich types

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-30
SLIDE 30

Relational Model?

  • Introduced rich types

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-31
SLIDE 31

Relational Model?

  • Introduced rich types
  • arrays

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B

[ , ] [ ] []

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-32
SLIDE 32

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B

[ , ] [ ] []

C

C D C D C D

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-33
SLIDE 33

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)
  • composite types (objects)

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

A B C D

[ , ]

{x: , y: }

[ ]

{x: , y: }

[]

{x: , y: }

C D C D C D

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-34
SLIDE 34

Relational Model?

  • Introduced rich types
  • arrays
  • Nested tables (multiset)
  • composite types (objects)

Non-Relational Operations

  • Introduced recursive


queries that process
 their own output

  • Transitive closure

Chris Date

Date on Database: Writings 2000-2006

SQL:1999 — Escaping the Relational Cage

? I was as confused as anyone else By the early 1990s, however,
 I’d seen the light

Domains Can Contain Anything!

slide-35
SLIDE 35

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-36
SLIDE 36

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-37
SLIDE 37

SQL:1999 — Recursion

CREATE TABLE t ( id INTEGER, parent INTEGER, )

slide-38
SLIDE 38

SQL:1999 — Recursion

slide-39
SLIDE 39

SQL:1999 — Recursion

slide-40
SLIDE 40

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ?

slide-41
SLIDE 41

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-42
SLIDE 42

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-43
SLIDE 43

SQL:1999 — Recursion

SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?

slide-44
SLIDE 44

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-45
SLIDE 45

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-46
SLIDE 46

SQL:1999 — Recursion

WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev

slide-47
SLIDE 47

SQL:1999 — Recursion

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0 MySQL 8.4

PostgreSQL

1.0 3.8.3

SQLite

7.0

DB2 LUW

11gR2

Oracle

2005

SQL Server

slide-48
SLIDE 48
slide-49
SLIDE 49

LATERAL

slide-50
SLIDE 50

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

slide-51
SLIDE 51

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

, column_2 More than


  • ne column?

⇒Syntax error

slide-52
SLIDE 52

SQL:1999 — LATERAL

Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 …

(return no more than one domain value)

, column_2 More than


  • ne column?

⇒Syntax error

}

More than


  • ne row?

⇒Runtime error!

slide-53
SLIDE 53

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt …

slide-54
SLIDE 54

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause

slide-55
SLIDE 55

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated”

slide-56
SLIDE 56

SQL:1999 — LATERAL

Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated” Regular join semantics

slide-57
SLIDE 57

SQL:1999 — LATERAL

slide-58
SLIDE 58

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

slide-59
SLIDE 59

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-60
SLIDE 60

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-61
SLIDE 61

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-62
SLIDE 62

SQL:1999 — LATERAL

  • Top-N per group


inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.

  • Also useful to find most recent

news from several subscribed topics (“multi-source top-N”).

  • Table function arguments


(TABLE often implies LATERAL)


Add proper index
 for Top-N query

https://use-the-index-luke.com/sql/partial-results/top-n-queries

FROM t JOIN TABLE (your_func(t.c)) FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table

slide-63
SLIDE 63

SQL:1999 — LATERAL

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB

8.0.14

MySQL

9.3

PostgreSQL

1.0

SQLite

9.1

DB2 LUW

11gR1[0] 12cR1

Oracle

2005[1]

SQL Server

[0]Undocumented. Requires setting trace event 22829. [1]LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.

slide-64
SLIDE 64

1999 2003

slide-65
SLIDE 65

http://www.acm.org:80/sigmod/record/issues/0206/standard.pdf (via Wayback machine)

slide-66
SLIDE 66

SQL:2003 — Schemaless & Analytical

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

slide-67
SLIDE 67

SQL:2003 — Schemaless & Analytical

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-68
SLIDE 68

SQL:2003 — Schemaless & Analytical

Analytical

  • Introduced


window functions

  • Accessing other rows

  • f the current result

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-69
SLIDE 69

SQL:2003 — Schemaless & Analytical

Analytical

  • Introduced


window functions

  • Accessing other rows

  • f the current result

Schemaless

  • Introduced XML
  • Non-uniform


documents in
 a single column

Later:

  • Extended in SQL:2011
  • Popular among “New SQLs”
  • 2013: BigQuery, Hive
  • 2014: Impala
  • 2015: Spark SQL
  • 2016: NuoDB, MemSQL,

Cockroach DB, VoltDB

Later:

  • JSON added with SQL:2016
  • Proprietary JSON support:
  • 2012: PostgreSQL
  • 2014: Oracle
  • 2015: MySQL
  • 2016: SQL Server
slide-70
SLIDE 70

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

SQL:2003 — Analytical

slide-71
SLIDE 71

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

slide-72
SLIDE 72

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

+10

slide-73
SLIDE 73

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

+10 +30 +20 +70 +40 +20

slide-74
SLIDE 74

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

, SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-75
SLIDE 75

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-76
SLIDE 76

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-77
SLIDE 77

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-78
SLIDE 78

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-79
SLIDE 79

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-80
SLIDE 80

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-81
SLIDE 81

SELECT id, value FROM t

id 1 2 3 4 5 6 value +10 +20

  • 10

+50

  • 30
  • 20

bal

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-82
SLIDE 82

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-83
SLIDE 83

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-84
SLIDE 84

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-85
SLIDE 85

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-86
SLIDE 86

SELECT id, value FROM t

id value bal 1 +10 2 +20 3

  • 10

4 +50 5

  • 30

6

  • 20

SQL:2003 — Analytical

ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal

+10 +30 +20 +70 +40 +20

slide-87
SLIDE 87

SQL:2003 — Analytical

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.2

MariaDB

8.0 MySQL 8.4

PostgreSQL

1.0 3.25.0

SQLite

7.0

DB2 LUW

8i

Oracle

2005[0] 2012

SQL Server

[0]Without framing

slide-88
SLIDE 88

OVER

SQL:2011 groups option

slide-89
SLIDE 89

ORDER BY x <frame unit> between 1 preceding and 1 following

SQL:2011 — OVER GROUPS option

slide-90
SLIDE 90

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

SQL:2011 — OVER GROUPS option

slide-91
SLIDE 91

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

x 1 3 3.5 3.5 4

CURRENT ROW

SQL:2011 — OVER GROUPS option

slide-92
SLIDE 92

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

SQL:2011 — OVER GROUPS option

slide-93
SLIDE 93

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

range

x between current_row.x - 1
 and current_row.x + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

SQL:2011 — OVER GROUPS option

slide-94
SLIDE 94

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

New in
 SQL:2011


groups

range

x between current_row.x - 1
 and current_row.x + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

SQL:2011 — OVER GROUPS option

slide-95
SLIDE 95

groups

count(distinct x)

ORDER BY x <frame unit> between 1 preceding and 1 following rows, range

New in
 SQL:2011


groups

range

x between current_row.x - 1
 and current_row.x + 1

rows


count(*)

x 1 3 3.5 3.5 4

CURRENT ROW

SQL:2011 — OVER GROUPS option

slide-96
SLIDE 96

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

11

PostgreSQL

3.28.0 SQLite

DB2 LUW Oracle SQL Server

SQL:2011 — OVER GROUPS option

slide-97
SLIDE 97

OVER

SQL:2003 frame exclusion

slide-98
SLIDE 98

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] )

SQL:2003 — frame exclusion

slide-99
SLIDE 99

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default no others

SQL:2003 — frame exclusion

slide-100
SLIDE 100

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default no others

SQL:2003 — frame exclusion

slide-101
SLIDE 101

current row

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default current row

SQL:2003 — frame exclusion

slide-102
SLIDE 102

group

x = current_row

current row

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default group

SQL:2003 — frame exclusion

slide-103
SLIDE 103

ties group

x = current_row

current row

x 1 2 2 2 3

OVER (ORDER BY … BETWEEN … exclude [ no others | current row | group | ties ] ) default ties

SQL:2003 — frame exclusion

slide-104
SLIDE 104

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

11

PostgreSQL

3.28.0 SQLite

DB2 LUW Oracle SQL Server

SQL:2003 — frame exclusion

slide-105
SLIDE 105

Inverse Distribution Functions (percentiles)

slide-106
SLIDE 106

SQL:2003 — Analytical (Median)

slide-107
SLIDE 107

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

SQL:2003 — Analytical (Median)

slide-108
SLIDE 108

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows

SQL:2003 — Analytical (Median)

slide-109
SLIDE 109

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-110
SLIDE 110

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-111
SLIDE 111

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

slide-112
SLIDE 112

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

Notice

All employees must

wash hands

after using

self-joins

slide-113
SLIDE 113

SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3)

Number rows Pick middle one

SQL:2003 — Analytical (Median)

Notice

All employees must

wash hands

after using

self-joins

Grab stickers
 and coasters!

slide-114
SLIDE 114

SQL:2003 — Analytical (Median)

slide-115
SLIDE 115

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

SQL:2003 — Analytical (Median)

slide-116
SLIDE 116

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median

SQL:2003 — Analytical (Median)

slide-117
SLIDE 117

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Median Which value?

SQL:2003 — Analytical (Median)

slide-118
SLIDE 118

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

SQL:2003 — Analytical (Median)

slide-119
SLIDE 119

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-120
SLIDE 120

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-121
SLIDE 121

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-122
SLIDE 122

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-123
SLIDE 123

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-124
SLIDE 124

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-125
SLIDE 125

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5)

0.25 0.5 0.75 1

1 2 3 4 PERCENTILE_CONT(0.5) PERCENTILE_DISC(0.5)

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data

Two variants:

  • for discrete values


(categories)

  • for continuous values


(linear interpolation)

SQL:2003 — Analytical (Median)

slide-126
SLIDE 126

SQL:2003 — PERCENTILE_DISC

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1 10.3.7

[0]

MariaDB MySQL

9.4

PostgreSQL

1.0

SQLite DB2 LUW

9iR1

Oracle

2012[0]

SQL Server

[0]Only as window function (OVER required).

slide-127
SLIDE 127

2003 2006

slide-128
SLIDE 128

XMLTABLE

slide-129
SLIDE 129

SQL:2006 — XMLTABLE

slide-130
SLIDE 130

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

slide-131
SLIDE 131

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

slide-132
SLIDE 132

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

slide-133
SLIDE 133

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

slide-134
SLIDE 134

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

XQuery (XPath)

  • Each match is

turned into a row

slide-135
SLIDE 135

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

XQuery (XPath)

  • Each match is

turned into a row

In preparation for JSON_TABLE & co, PG 12 introduced SQL/JSON Path

slide-136
SLIDE 136

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

XQuery (XPath)

  • Each match is

turned into a row

In preparation for JSON_TABLE & co, PG 12 introduced SQL/JSON Path

slide-137
SLIDE 137

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

Bind Parameter

XQuery (XPath)

  • Each match is

turned into a row

In preparation for JSON_TABLE & co, PG 12 introduced SQL/JSON Path

slide-138
SLIDE 138

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r

slide-139
SLIDE 139

id c1 42 foo 43 bar

<d> <e id="42"> <c1>foo</c1> </e> <e id="43"> <c1>bar</c1> </e> </d>

SQL:2006 — XMLTABLE

SELECT * FROM XMLTABLE ( '/d/e' PASSING ? COLUMNS id INT PATH '@id' , c1 VARCHAR(…) PATH 'c1' ) r INSERT INTO target_table

slide-140
SLIDE 140

SQL:2006 — XMLTABLE

1999 2001 2003 2005 2007 2009 2011 2013 2015 2017

5.1

MariaDB MySQL

10

[0] PostgreSQL

1.0

SQLite

9.7

DB2 LUW

11gR1

Oracle SQL Server

[0]No XQuery (only XPath). No default namespace declaration.

slide-141
SLIDE 141

https://webstore.iec.ch/publication/59685

slide-142
SLIDE 142

A lot has
 happened since SQL-92

https://webstore.iec.ch/publication/59685

slide-143
SLIDE 143

SQL has evolved
 beyond the relational idea A lot has
 happened since SQL-92

https://webstore.iec.ch/publication/59685

slide-144
SLIDE 144

SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92

slide-145
SLIDE 145

SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92

https://modern-sql.com


@ModernSQL by @MarkusWinand

Training:
 https://winand.at/

slide-146
SLIDE 146

SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92

https://modern-sql.com


@ModernSQL by @MarkusWinand

Training:
 https://winand.at/ Feedback:
 https://2019.pgconf.eu/f