XML and XQuery 5DV120 Database System Principles Ume a University - - PowerPoint PPT Presentation

XML and XQuery

5DV120 — Database System Principles Ume˚ a University Department of Computing Science Stephen J. Hegner hegner@cs.umu.se http://www.cs.umu.se/~hegner

XML and XQuery 20151114 Slide 1 of 54

The XML Context

• XML was designed for data exchange, not for large, homogeneous

databases.

• Its primary usage is thus quite different from that of the relational model.
• In terms of potential application, XML and the relational model are, for

the most part, complementary rather than competitive frameworks.

• Nevertheless, there are similarities.
• In particular, the XML query language XQuery has many similarities with

SQL.

• In these slides, some basic ideas surrounding XML and XQuery are

presented.

• The goal it to provide a brief introduction, not a comprehensive

presentation.

XML and XQuery 20151114 Slide 2 of 54

The Components Surrounding XML

Data: XML is the language for representing data. DML: There are many query languages for XML, among them: XPath: A language for expressing queries as operations on paths. XQuery: A more comprehensive query language, in many ways similar to SQL, containing XPath as a subset. XSLT: An imperative document translation language which may also be used to express queries. SQL/XML: Used for data exchange and storage between relational and XML databases. DDL: There is no true DDL for XML; any well-formed XML expression qualifies as data. Constraints: However, there are at least two languages for expressing constraints on XML data. DTD: An old language, inherited from SGML, with limited expressive power. XML Schema: A newer and very comprehensive language, but relatively complex.

XML and XQuery 20151114 Slide 3 of 54

The Family Tree of XML

• XML, at least originally, stood for eXtended Markup Language.
• XML is a descendant of SGML (Standard Generalized Markup Language).
• In this sense, it is a cousin of HTML.
• All of these languages are characterized by nested blocks with tag pairs
• f the form <foo> and </foo>.
• In HTML, the tags and their semantics are fixed in the definition of the

language.

• In XML, on the other hand, even the tags themselves may be freely

chosen, and are limited only by constraints expressed in an appropriate language such as DTD or XML Schema.

XML and XQuery 20151114 Slide 4 of 54

A Simple Example

• Here is a simple way (but not the only way) to represent the tuple

(’Biology’,’Watson’,’90000’) of the department relation of the university schema.

<department dept_name="Biology"> <building >Watson </building > <budget >90000 </budget > </ department >

• It may be regarded as a simple tree representation.

department dept name Biology building Watson budget 90000

XML and XQuery 20151114 Slide 5 of 54

Types of Vertices in the Tree Representation

<department dept_name="Biology"> <building >Watson </building > <budget >90000 </budget > </ department > department dept name Biology building Watson budget 90000

• Each yellow box is a tag vertex.
• Each green box is an attribute vertex, and dept name is called an

attribute of the tag department with value Biology.

• Each blue box is a text vertex.

Warning: XPath uses a somewhat different and more complex classification (not covered in detail here).

• For now, this one will suffice.

XML and XQuery 20151114 Slide 6 of 54

Attributes versus Elements with Text Values

Question: Why not represent dept name as the value enclosed in tags, as shown on the right, as opposed to an attribute of department?

<department dept_name="Biology"> <building >Watson </building > <budget >90000 </budget > </department > <department > <dept_name >Biology </dept_name > <building >Watson </building > <budget >90000 </budget > </ department >

Answer: It is a design decision, and both work.

• There is an advantage to attributes for representing key constraints in

DTD.

• More later on this.
• It is also possible to have several attributes:

<department dept_name="Biology" budget="90000"> <building >Watson </building > </ department >

XML and XQuery 20151114 Slide 7 of 54

Order of Children

• The order of tag children is significant.
• Thus, the following two expressions are different.

<department dept_name="Biology"> <building >Watson </building > <budget >90000 </budget > </department > <department dept_name="Biology"> <budget >90000 </budget > <building >Watson </building > </ department >

• However, the order of attributes is not significant.
• Thus, the following two expressions are equivalent.

<department dept_name="Biology" budget="90000"> <building >Watson </building > </department > <department budget="90000" dept_name="Biology"> <building >Watson </building > </ department >

• Hence, in the tree representation, the order of tag vertices matters, but

the order of attribute vertices does not.

XML and XQuery 20151114 Slide 8 of 54

Deeper Nesting and Multiple Occurrences

• There is no limit to the depth of nesting, and tags may be repeated.
• Here is part of an example from a nested representation of the university

database.

<department dept_name="Biology"> <building >Watson </building > <budget >90000 </budget > <instructor iid="76766"> <name >Crick </name > <salary >72000 </salary > <teaches > ... </teaches > <teaches > ... </teaches > </instructor > <course cid="BIO -101"> <title >Intro. to Biology </title > <credits >4</credits > <section > ... </section > <section > ... </section > </course > <course cid="BIO -301"> ... </course > <student > <sid >98988 </sid > <name >Tanaka </name > <tot_cred >120 </tot_cred > <takes > ... </takes > <takes > ... </takes > <advisor > <iid >76766 </iid > </advisor > </student > </ department >

• The only requirement is that the tags be nested properly.

XML and XQuery 20151114 Slide 9 of 54

Document Structure

• An entire document must be a well-formed XML expression.
• Thus, there must be an encompassing pair of begin-end block markers.

<university_flat > <!-- The university database

• f the

textbook in XML

• ->

<department dept_name="Biology"> ... </ department > ... <department dept_name=" Basketweaving "> ... </department > <instructor > ... </ instructor > ... <instructor > ... </ instructor > ... </ university_flat >

• Note that linebreaks are just whitespace.
• Layout is as freeform as with HTML.
• Comments are represented just as in HTML.

XML and XQuery 20151114 Slide 10 of 54

Accessing and Querying XML Databases

• Since an XML document is just a text file, it may be created and

accessed with any text editor.

• However, to query it properly, to check it against a DTD or XML Schema

specification, and even to access it at all if it is very large, an XML DBMS, or a relational DBMS with XML support is necessary.

• In this course, the XML DBMS eXist-db will be used for that purpose.
• It uses a Web interface.
• However, it is also very easy to install on your own computer.
• It is written in Java, and requires only JDK.

XML and XQuery 20151114 Slide 11 of 54

The Departmental Installations of eXist-db

• There are two installations of eXist-db on the departmental servers.

Version 2.2: http://exist-db.cs.umu.se Version 2.1: http://exist-db-old.cs.umu.se

• You should have received a user-ID and password. The same user-ID and

password work for each.

• The systems will be demonstrated during a lecture.
• These slides provide just some basic points.
• It is not necessary to log in or even to have an account in order to

execute queries on the (world-readable) databases which are installed on these systems.

• An account is necessary in order to upload your own private databases.

XML and XQuery 20151114 Slide 12 of 54

Access to the Departmental Installations of eXist-db

Versions 2.1 and 2.2 Access to a particular feature is obtained by clicking on the appropriate icon on the main page.

• If a login is required, a prompt for username and password will

appear.

• Not all features require login.
• Access to collection directories is via the Collections icon.

XML and XQuery 20151114 Slide 13 of 54

Databases on the Departmental Installations of eXist-db

Directories for both installations: The main directory is db.

• The databases to be used in this course are:

University databases (of the textbook): /db/university Mondial databases (for an obligatory exercise): /db/mondial

• These are world readable and do not require login for access via the

query browser.

• Other example databases may be found via browsing.
• Individual user directories are under /db/home and are initially

empty.

• Uploads are possible by using the appropriate fields and buttons.

Please do not try to change the permissions on your user directory.

• If just one directory is not world readable, access is not possible

to any home directory for anyone.

• It is fine to protect individual files within your personal directory.

XML and XQuery 20151114 Slide 14 of 54

Issuing Queries on the Departmental Installations of eXist-db

XQuery IDE (eXide): The IDE for executing and developing XQuery programs.

• Not really a full IDE, but still offers some nice features.
• Move the cursor over the red X or warning sign to see a message

about what is wrong.

• Requires special editor initialization in order to talk to Emacs and

similar tools (different clipboards).

• May produce a “version note” popup when it starts, stating that

another version of eXide may have used the same browser, and suggesting to empty the cache.

• This message seems harmless and may be ignored.

XML and XQuery 20151114 Slide 15 of 54

XQuery

• As noted previously, there are several possible languages for querying

XML databases.

• The two “pure” languages are XPath and XQuery.
• XPath is a relatively simple language, based upon paths through the tree

representing an XML expression.

• Joins cannot be expressed in XPath, for example.
• XQuery is a more comprehensive query language with a flavor similar to

that of SQL.

• XPath 1.0 is essentially a subset of XQuery 2.0.
• Thus, while only XQuery will be considered here, the presentation will

include aspects of XPath.

XML and XQuery 20151114 Slide 16 of 54

XQuery: Selecting the Document and Simple Paths

• Every query must give the path to the document.
• This one finds the entire university flat database:
• The answer to this query is the whole document.

doc("/db/ university / university_flat .xml")

• Items in paths are separated by forward slashes, as in Unix.
• This query returns the whole document as well, since the

<university flat> . . . </university flat>

block is at the top level.

doc("/db/ university / university_flat .xml")/ university_flat

• This query returns just the departments.

doc("/db/ university / university_flat .xml")/ university_flat /department

XML and XQuery 20151114 Slide 17 of 54

XQuery: Simple Paths 2

• /*/ acts as a wildcard for one level.
• The following two queries are equivalent, since there is only one

university flat element.

doc("/db/ university / university_flat .xml")/ university_flat /department doc("/db/ university / university_flat .xml")/*/ department

• The following query returns all department names.

doc("/db/ university / university_flat .xml")/*/ department / @dept_name

However, the eXist-db query evaluator displays only well-formed XML

statements, and since the result is a set of strings, it will show an empty answer, although it will provide a correct count of the number of values.

• It will be shown how to overcome this shortly.

XML and XQuery 20151114 Slide 18 of 54

XQuery: Conditions in Paths

• To select specific values, use the square condition brackets.

doc("/db/ university / university_flat .xml") /*/ department[ @dept_name =’Biology ’]/ building doc("/db/ university / university_flat .xml") /*/ department[ @dept_name =’Biology ’ or @dept_name=’Music ’]

• The following query finds all buildings which house a department which

has a dept name attribute.

doc("/db/ university/ university_flat .xml")/*/ department [ @dept_name ]/ building

• The following query finds all departments which have a budget of less

than 90000

doc("/db/ university/ university_flat .xml")/*/ department [budget <90000]

• Conditions need not be at the end of the path.

doc("/db/ university / university_flat .xml") /*/ department[budget <90000]/ building

XML and XQuery 20151114 Slide 19 of 54

XQuery: Going Uphill and Way Downhill in Paths

• It is even possible to go backwards in paths to the parent.
• The following query finds all departments which have a budget of less

than 90000 and which have a building.

doc("/db/ university / university_flat .xml") /*/ department[budget <’90000 ’]/ building /..

• The pair // goes as far down as necessary in a path.
• The following query finds all buildings regardless of how deeply buried by

nesting.

doc("/db/ university / university_flat .xml")// building

XML and XQuery 20151114 Slide 20 of 54

XQuery: FLWOR Expressions

• Path expressions, by themselves, are limited in power.
• FLWOR expressions are more powerful, and often much easier to use

even when a solution with using a path expression is possible. FLWOR = For Let Where Order Return, pronounced “flower”.

• FLWOR expressions have a flavor similar in some ways to SQL.
• The following two queries are equivalent.

doc("/db/ university / university_flat .xml")/*/ department for ✩x in doc("/db/ university / university_flat .xml")/*/ department return ✩x

• But much more can be done.

for $dept in doc("/db/university / university_flat .xml")/*/ department

• rder by ✩dept/name

return ✩dept

XML and XQuery 20151114 Slide 21 of 54

XQuery: Where Clauses and Evaluation

• The following two queries do the same thing:

doc("/db/ university / university_flat .xml") /*/ department[ @dept_name =’Biology ’]/ building for $dept in doc("/db/university / university_flat .xml")/*/ department where ✩dept[ @dept_name="Biology"] return ✩dept/building

• To view results which are strings in the eXist-db result window, wrap

them in tags using the element directive.

for $dept in doc("/db/ university / university_flat .xml")/*/ department where ✩dept/building=<building >Watson </building > return element foo { ✩dept/ @dept_name }

• The following does the same thing.

for ✩dept in doc("/db/ university / university_flat .xml") /*/ department[building=’Watson ’] return <foo > {✩ dept/ @dept_name} </foo >

• In the above, the set brackets {...} indicate that the enclosed

expression should be evaluated and not returned literally.

XML and XQuery 20151114 Slide 22 of 54

XQuery: Enclosing Several Elements in Tags

• Wrapping in tags is also useful for returning several items as a single

expression.

for ✩dept in doc("/db/ university / university_flat .xml")/*/ department where ✩dept[ @dept_name="Biology"] return element foo { ✩dept/building , ✩dept/budget } for ✩dept in doc("/db/ university / university_flat .xml")/*/ department where ✩dept[ @dept_name="Biology"] return <foo > {✩ dept/building} {✩ dept/budget} </foo >

• Attributes may also be included in this fashion.

for ✩dept in doc("/db/ university / university_flat .xml")/*/ department where ✩dept[ @dept_name="Biology"] return element foo { attribute greeting {"hello"}, ✩dept/building , ✩dept/budget }

• As may nested element directives.

for ✩dept in doc("/db/ university / university_flat .xml")/*/ department where ✩dept[ @dept_name="Biology"] return element foo { attribute greeting {"hello"}, element Gruss {"Hallo"}, ✩dept/building , ✩dept/budget }

XML and XQuery 20151114 Slide 23 of 54

XQuery: Joins

• Joins may be realized via a double for loop.
• The following query finds the building of the department of each

instructor.

for ✩dept in doc("/db/ university / university_flat .xml")/*/ department for ✩instr in doc("/db/ university / university_flat .xml")/*/ instructor where ✩dept/ @dept_name = ✩instr/dept_name return element instr_bldg { ✩instr/name , ✩dept/building }

• This may also be written as:

for ✩dept in doc("/db/ university / university_flat .xml")/*/ department for ✩instr in doc("/db/ university / university_flat .xml")/*/ instructor where ✩dept/ @dept_name = ✩instr/dept_name return <instr_bldg > {✩ instr/name} {✩ dept/building} </instr_bldg >

XML and XQuery 20151114 Slide 24 of 54

XQuery: Nested Queries

• Find all colleagues of Srinivasan; that is, all instructors who work in the

same department as Srinivasan.

for ✩dept in doc("/db/ university/ university_flat .xml") // instructor [name=" Srinivasan"]/ dept_name return element colleagues { doc("/db/ university / university_flat .xml")// instructor [dept_name =✩ dept] }

• Exclude Srinivasan.

for ✩dept in doc("/db/ university/ university_flat .xml") // instructor [name=" Srinivasan"]/ dept_name return element colleagues { doc("/db/ university / university_flat .xml")// instructor [dept_name =✩ dept] except doc("/db/ university / university_flat .xml")// instructor [name=" Srinivasan"] }

XML and XQuery 20151114 Slide 25 of 54

XQuery: Let and Aggregation

• The let clause returns the sequence of all values which are matched.
• It is particularly useful in aggregation queries, since the aggregation

functions operate on lists.

• Here is a simple query which counts the departments.

let ✩depts := doc("/db/ university / university_flat .xml")/ university_flat /department return element answer {attribute n_depts {count (✩ depts )} }

XML and XQuery 20151114 Slide 26 of 54

XQuery: Let and Aggregation 2

• Here is a query which returns, for each department, an element with

attributes giving its name, the number of instructors, and the maximum, minimum, average, and total salary of all of its instructors.

for $dept in doc("/db/ university / university_flat .xml")// department let ✩instrs := doc("/db/ university / university_flat .xml") // instructor [dept_name =✩ dept/ @dept_name]

• rder by count (✩ instrs) descending , sum (✩ instrs/salary) descending

return element dept_summary { attribute dept_name {✩ dept/ @dept_name}, attribute instr_count {count (✩ instrs )}, attribute salary_max {max (✩ instrs/salary )}, attribute salary_min {min (✩ instrs/salary )}, attribute salary_avg {avg (✩ instrs/salary )}, attribute salary_sum {sum (✩ instrs/salary )} }

XML and XQuery 20151114 Slide 27 of 54

XQuery Updates: Replace

• XQuery has recently added support for updates.
• A few of the most basic operations are illustrated here.

Reminder: You do not have update privileges on doc("/db/university/university flat.xml"), so try these on your

• wn copy.

Replace: Change the building of the Comp. Sci. department to MIT-huset.

for $bldg in doc("/db/ university / university_flat .xml")/*/ department /building where ✩bldg /../ @dept_name = "Comp.Sci." return update replace ✩bldg with <building >MIT -huset </building >

• It is possible to achieve the same result by replacing the value while

retaining the tags.

for $bldg in doc("/db/ university / university_flat .xml")/*/ department /building where ✩bldg /../ @dept_name = "Comp.Sci." return update value ✩bldg with "MIT -huset"

XML and XQuery 20151114 Slide 28 of 54

XQuery: If-Then-Else

• This is a convenient point to sneak in an example with if-then-else.

Replace: Toggle the building of the Comp. Sci. department between Taylor and MIT-huset.

for ✩bldg in doc("/db/ university / university_flat .xml")/*/ department /building where ✩bldg /../ @dept_name = "Comp.Sci." return if (✩ bldg = element building {"Taylor"}) then update value ✩bldg with "MIT -huset" else if (✩ bldg = element building {"MIT -huset"}) then update replace ✩bldg with element building {"Taylor"} else ()

• The else clause is mandatory, but it may be empty, as shown.

XML and XQuery 20151114 Slide 29 of 54

XQuery Updates: Insert

Insert: Add <foo>bar</foo> to the top-level database.

for ✩top in doc("/db/ university / university_flat .xml")/ university_flat return update insert <foo >bar </foo > into ✩top

• Of course, think of adding something more interesting, such as a new

department, instructor, student, or course.

• Insert <foo>bar</foo> just after the Comp. Sci. department.

for ✩cs in doc("/db/ university / university_flat .xml") /*/ department[ @dept_name ="Comp.Sci."] return update insert <foo >bar </foo > following ✩cs

• To insert it just before instead of just after, replace following with

preceding in the above.

• Add the attribute language to the Comp. Sci. department with value

Swedish.

for ✩cs in doc("/db/ university / university_flat .xml") /*/ department[ @dept_name ="Comp.Sci."] return update insert attribute language {"Swedish"} into ✩cs

XML and XQuery 20151114 Slide 30 of 54

XQuery Updates: Delete

Delete: Delete all occurrences of <foo>bar</foo> from the top-level database.

for ✩foo in doc("/db/ university / university_flat .xml")/ university_flat /foo return update delete ✩foo

• Remove the language attribute from the Comp. Sci. department.

for ✩cs in doc("/db/ university / university_flat .xml") /*/ department[ @dept_name ="Comp.Sci."] return update delete ✩cs/@language

XML and XQuery 20151114 Slide 31 of 54

XQuery Sequences

• A major data type of XQuery is the sequence.
• It has already appeared implicitly in many examples.
• Results of queries are often sequences of elements..
• Sequences are flat; nesting is ignored.

let ✩x := ((1, 2), (3, (4, 5))) return ✩x

returns (1, 2, 3, 4, 5)

• Equality is overlap of values.

let ✩x := (1, 2, 3, 4, 5) return if (3 = ✩x) then "yes" else "no"

returns yes, but with 6 replacing 3 it returns no.

• The following returns yes

let ✩x := (1, 2, 3, 4, 5) let ✩y := (5, 6, 7, 8) return (if ✩x = ✩y) then "yes" else "no"

but if the 5 is removed from either list it returns no.

XML and XQuery 20151114 Slide 32 of 54

XQuery Sequences — 2

• The following selects 9 from the sequence.

let ✩x := (1, 4, 9, 16, 25) return ✩x[3]

• There are also many built-in functions for more complex manipulation.
• Here is one example.

index -of( ("a", "b", "c", "d"), "a") (: returns 1 :) index -of( ("a", "b", "c", "d"), "e") (: returns () :) index -of( ("a", "b", "c", "a"), "a") (: returns (1, 4) :) index -of( (<a>x</a>, <b>x</b>, <c>y</c>), <d>x</d>) (: returns (1, 2) :)

XML and XQuery 20151114 Slide 33 of 54

XQuery: Tokenization of Strings

• To extract the “words” of a string, with “words” separated by

whitespace, use the tokenize operation.

tokenize("ab1 c2drm","\s+")

• The above call returns the sequence ("a","b1","c2","drm").
• The function tokenize takes two arguments, a string and a regular

expression (RE).

• The string is broken into “words” separated by strings matching the RE.
• \s matches any whitespace character; \s+ matches the longest sequence
• f such characters.

XML and XQuery 20151114 Slide 34 of 54

XQuery: Normalizing Whitespace of Strings

• If there is whitespace at the beginning and/or end of the string, the

tokenize function will interpret that whitespace as a token.

• The following returns six tokens a sequence of six tokens:

("","a","b1","c2","drm","")

tokenize("ab1 c2drm","\s+")

• To remove these, use the function normalize-space.

tokenize(normalize -space("ab1 c2drm"),"\s+")

• The above function returns ("a","b1","c2","drm").

XML and XQuery 20151114 Slide 35 of 54

XQuery: Set Operations on Sequences of Atoms

• The XQuery operations union, intersection, and except operate only
• n sequences of nodes, not sequences of atoms.
• Set operations on sequences of atoms may be realized using the

distinct-values operator. Union :

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") return distinct -values ((✩x,✩y))

Intersection:

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") return distinct -values (✩x[.=✩y])

Set difference:

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") return distinct -values (✩x[not (.=✩y)])

XML and XQuery 20151114 Slide 36 of 54

XQuery: Set Operations on Sequences of Atoms 2

• Set operations on sequences of atoms may be realized using more basic

functions. Union : (with duplicates not removed)

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") return (✩x,✩y)

Union : (with duplicates removed)

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") let ✩u := ( for ✩z in ✩y where (every ✩w in ✩x satisfies (✩z != ✩w)) return ✩z) return (✩x,✩u)

XML and XQuery 20151114 Slide 37 of 54

XQuery: Set Operations on Sequences of Atoms 3

• Set operations on sequences of atoms may be realized using more basic

functions (continued). Intersection:

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") for ✩z in ✩x where (some ✩w in ✩y satisfies (✩z = ✩w)) return ✩z

Set difference:

let ✩x := tokenize(normalize -space("ab1c2drm"),"\s+") let ✩y := tokenize(normalize -space("c2drmef"),"\s+") for ✩z in ✩x where (every ✩w in ✩y satisfies (✩z != ✩w)) return ✩z

XML and XQuery 20151114 Slide 38 of 54

XQuery: Quantification Operations on Sequences of Atoms

Universal quantification: The following expression returns the members of $x which are smaller than every element of $y: (1,2).

let ✩x := (1, 2, 3, 4, 5, 6, 7, 8) let ✩y := (3, 4, 5) for ✩z in ✩x where (every ✩w in ✩y satisfies (✩z < ✩w)) return ✩z

• The members of the result are returned one at a time.

Existential quantification: The following expression returns the members of $x which are smaller than some element of $y: (1,2,3,4).

let ✩x := (1, 2, 3, 4, 5, 6, 7, 8) let ✩y := (3, 4, 5) for ✩z in ✩x where (some ✩w in ✩y satisfies (✩z < ✩w)) return $z

XML and XQuery 20151114 Slide 39 of 54

XQuery: User-Defined Functions

• XQuery allows user-defined functions:

declare function local:square (✩x as xs:int) as xs:int {✩x*✩x};

• The prefix local: puts the function in the local namespace.
• The prefix xs: on the data types indicates that the type lies in the

standard xs namespace.

• Multiple arguments are allowed and functions may use other functions.

declare function local:sqdiff (✩x as xs:int , ✩y as xs:int) as xs:int { local:square (✩x) - local:square (✩y)};

• A function may take a sequence as its argument.

declare function local:first (✩x as xs:anyAtomicType *) xs:anyAtomicType {✩x[1]};

XML and XQuery 20151114 Slide 40 of 54

XQuery: User-Defined Functions — 2

• Functions may be recursive.

declare function local:fibonacci (✩x as xs:int) as xs:int {if (✩x<2) then ✩x else local:fibonacci ((✩x)-1) + local:fibonacci ((✩x) -2)};

• In the above function definition, local:fibonacci(✩x-1) will not work, but

local:fibonacci(✩x - 1) and local:fibonacci((✩x)-1) will.

• The types of arguments and even the return type may be omitted.

declare function local:square (✩x) {✩x*✩x};

• This has the advantage of making the function more general.
• There are many different number types.
• But it also has the disadvantage of bypassing type checking.
• Functions may return lists.

declare function local:increment -list (✩ list) {for ✩x in ✩list return ✩x + 1};

XML and XQuery 20151114 Slide 41 of 54

XQuery: the Mapping Directive

• Suppose that it is desired to compute the sum of the squares of the

members of a list.

• The following will not work.

let ✩list := (2, 4, 6, 7, 9) for ✩x in ✩list return sum (✩x*✩x)

• The result consists of the square of each member: (4, 16, 36, 49, 81).
• One way to achieve the desired result is to put the for within the sum.

let ✩list := (2, 4, 6, 7, 9) return sum(for ✩x in ✩list return ✩x*✩x)

• Another way is to use the mapping directive !, introduced in XQuery 3.0.

declare function local:square (✩x) {✩x*✩x}; let ✩list := (2, 4, 6, 7, 9) ✩list ! local:square (.)

• In the above code, the function local:square is applied to each element of

the sequence, resulting in the new sequence (4, 16, 36, 49, 81).

• Placing this within the sum operator achieves the desired result.

declare function local:square (✩x) {✩x*✩x}; let ✩list := (2, 4, 6, 7, 9) return sum (✩ list ! local:square (.))

XML and XQuery 20151114 Slide 42 of 54

XQuery: Comments

• Comments in XQuery are delimited by (: and :).

(: This is a comment which is split

• ver

two

• lines. :)

element bar { let ✩x := (1, 2, 3, 4, 5, 6, 7, 8) let ✩y := (3, 4, 5) for ✩z in ✩x where (some ✩w in ✩y satisfies (✩z < ✩w)) return element foo {$z} }

XML and XQuery 20151114 Slide 43 of 54

Namespaces

• A primary use of XML is data exchange.
• The problems associated with having a common, global namespace for all

sources of data are obvious.

• For example, an XML database for Ume University would very well many

names in common with those for Yale University, although the substructure and hence semantics would well be different.

• This would make meaningful data exchange difficult at best.
• For this reason, there is a way in XML to identify namespaces and to

qualify a name with a namespace.

• The general format is shown on the next slide.

XML and XQuery 20151114 Slide 44 of 54

Namespaces 2

<university_flat xmlns="http: // www.yale.edu" xmlns:umu="http: // www.umu.se" xmlns:hsv="http: // www.hsv.se" > <department > <!-- This department name belongs to the default namespace

• ->

<!-- XML data statements

• ->

</department > <umu:department > <!-- This department name belongs to the umu namespace

• ->

<!-- XML data statements

• ->

</ umu:department > <hsv:department > <!-- This department name belongs to the hsv namespace

• ->

<!-- XML data statements

• ->

</ hsv:department > </ university_flat >

Default namespace: The attribute xmlns identifies the default namespace.

• Names in the default namespace do not need to be qualified.

Other namespaces: These need to be declared with a prefix (which is actually the suffix following xlmns:), and that prefix used with all instances of names from that space.

• Namespace declarations may be nested.

All names beginning with xml are potentially reserved and should not be

user declared.

XML and XQuery 20151114 Slide 45 of 54

Namespaces 3 — the Values of Namespaces

<university_flat xmlns="http: // www.yale.edu" xmlns:umu="http: // www.umu.se" xmlns:hsv="http: // www.hsv.se" > <department > <!-- This department name belongs to the default namespace

• ->

<!-- XML data statements

• ->

</department > <umu:department > <!-- This department name belongs to the umu namespace

• ->

<!-- XML data statements

• ->

</ umu:department > <hsv:department > <!-- This department name belongs to the hsv namespace

• ->

<!-- XML data statements

• ->

</ hsv:department > </ university_flat >

• The URLs in the above example are just strings.
• There is no requirement that they point to any useful information about

the document structure.

• There is no requirement that they point to anything at all.

XML and XQuery 20151114 Slide 46 of 54

Namespace Prefixes of XQuery

• XQuery has many predefined namespace prefixes, a few of which have

already been seen. local: identifies the namespaces for locally defined functions, bound to http://www.w3.org/2005/xquery-local-functions. xs: identifies the schema namespace of XML, bound to http://www.w3.org/2001/XMLSchema.

• Some others include the following.

xml: identifies another XML namespace, bound to http://www.w3.org/XML/1998/namespace xmlns: bound to http://www.w3.org/XML/2000/xmlns. err: for error messages, bound to http://www.w3.org/2005/xqt-errors. fn: for functions, bound to http://www.w3.org/2005/xpath-functions.

• These will not be considered further in this course.

XML and XQuery 20151114 Slide 47 of 54

DTD — Document Type Definition

DTDs Document Type Definitions are inherited from SGML and predate XML.

• They are written in SGML, not XML.
• They may be used to specify some basic constraints, but many useful

types of constraints cannot be expressed.

• They are nevertheless still in fairly wide use because they are easy to

write and understand.

• They use a form of regular expression to express certain things.
• The general format is as follows.

<!DOCTYPE name [ ... declarations ... ]>

XML and XQuery 20151114 Slide 48 of 54

DTD 2

<!DOCTYPE university [ ... <!ELEMENT university_flat ( department*,instructor *,course*,classroom*, section*,teaches*,student*,takes*,advisor*, time_slot*,prereq *)> <!ELEMENT department (building ,budget)> <!ELEMENT building (# PCDATA)> <!ELEMENT budget (# PCDATA)> <!ATTLIST department dept_name ID #REQUIRED > <!ELEMENT instructor (name ,dept_name ,salary)> <!ATTLIST instructor iid ID #REQUIRED > <!ELEMENT name (# PCDATA)> <!ELEMENT salary (# PCDATA)> ... ]>

• Above is a (partial) example of a DTD for university flat.
• An !ELEMENT declaration gives the required list of tagged elements under

the identified tag.

• * means zero or more occurrences.
• + means one or more occurrences.
• Order is significant.
• #PCDATA means that it is a string (Parsed Character DATA).
• An element may also be declared #ANY, meaning that there are no

restrictions on its value.

XML and XQuery 20151114 Slide 49 of 54

DTD 3

• The same schema continued:

<!DOCTYPE university [ ... <!ELEMENT university_flat ( department*,instructor *,course*,classroom*, section*,teaches*,student*,takes*,advisor*, time_slot*,prereq *)> <!ELEMENT department (building ,budget)> <!ELEMENT building (# PCDATA)> <!ELEMENT budget (# PCDATA)> <!ATTLIST department dept_name ID #REQUIRED > <!ELEMENT instructor (name ,dept_name ,salary)> <!ATTLIST instructor iid ID #REQUIRED > <!ELEMENT name (# PCDATA)> <!ELEMENT salary (# PCDATA)> ... ]>

• An !ATTLIST declaration gives the attributes under the identified tag.
• ID means that the value is unique in the entire document.
• #REQUIRED means that it is mandatory to have the attribute.
• Order is not significant.

XML and XQuery 20151114 Slide 50 of 54

DTD 4

<!DOCTYPE university2 [ ... <!ELEMENT university_flat ( department*,instructor *,course*,classroom*, section*,teaches*,student*,takes*,advisor*, time_slot*,prereq *)> <!ELEMENT department (building ,budget)> <!ATTLIST department dept_name ID #REQUIRED > <!ELEMENT instructor (name ,salary)> <!ATTLIST instructor iid ID #REQUIRED dept_name IDREF #REQUIRED > ... ]>

• The above DTD differs slightly from the previous one in that dept name

is now an attribute of instructor, instead of a subelement.

• IDREF means that the value must occur as the ID somewhere else in the

document.

• A weak form of referential integrity (foreign key).
• An attribute which is neither ID or IDREF may be declared to be CDATA,

a character string.

• Attributes other than IDs may be declared to be #IMPLIED instead of

#REQUIRED, which means that they are optional.

XML and XQuery 20151114 Slide 51 of 54

Shortcomings of DTDs

• DTDs have the following significant limitations:
• They are not aware of namespaces.
• The only basic types are (glorified) strings.
• Support for database constraints is very limited.
• Keys must be global over all attributes.
• Referential integrity is existential.
• They impose order when none is wanted.
• All element definitions are global; there is no scoping.
• Nevertheless, they are simple to write and better than nothing, so they

are still used.

XML and XQuery 20151114 Slide 52 of 54

XML Schema

• Many of the shortcomings of DTDs are addressed in XML Schema, a

language which is explicitly designed to express complex constraints on XML documents.

• Key and referential integrity constraints may be expressed, even when the

attributes used form a complex structure.

• Unfortunately, the language is also relatively complex.
• For reasons of limited time, it will not be considered further in this course.

XML and XQuery 20151114 Slide 53 of 54

XQuery Programming

XQJ: A standard API for issuing XQuery queries to an XML database from Java.

• Somewhat analogous to JDBC for SQL.
• Many native XML database systems, including eXist-db, support the

XQJ API. XQC: A developing standard API for C.

• Newer than XQJ and still very much under development.
• Current implementations include Zorba and XQuilla.
• Remember that XML is used much more for data exchange and much

less for database management than the relational model.

• Much of the use of XML is not primarily focused upon formal querying in

general or XQuery in particular.

• Therefore, programming surrounding XML has a very different flavor,

which is very important and useful, but beyond the scope of this course.

XML and XQuery 20151114 Slide 54 of 54