Introduction III Radu Nicolescu Department of Computer Science - - PowerPoint PPT Presentation

A#1R Vers Perf A#1R

Introduction III

Radu Nicolescu Department of Computer Science University of Auckland 19 July 2018

1 / 25

A#1R Vers Perf A#1R

1 A#1R Short FAQ 2 FP LINQ versatility 3 Performance issues 4 Assignment #1R F#

2 / 25

A#1R Vers Perf A#1R

Outline

1 A#1R Short FAQ 2 FP LINQ versatility 3 Performance issues 4 Assignment #1R F#

3 / 25

A#1R Vers Perf A#1R

A#1R Short FAQ

• Read regex format? Please see the revised/extended handout

#2:

@”\s+” or ”\\s+”

• Output file? Write to stdout, i.e. using C#:

Console.WriteLine() , F#: printfn , node.js: console.log .

The markers will redirect such outputs to actual files:

1 jbon007 . exe b . t x t 7 > jbon007 . log 2> jbon007 . e r r 2 3 jbon007 . exe b . t x t 7 1> jbon007 . log 2> jbon007 . e r r 4 5 jbon007 . exe b . t x t 7 1> jbon007 . log 2>&1

• PS. In Unix parlance, everything is a file, including stdout...

4 / 25

A#1R Vers Perf A#1R

A#1R Short FAQ

• Select first?
• Pipeline of fluent method chaining: LINQ-like high-order

methods can be arranged in any order, as long as the output

• n one is still compatible with the input of the next.
• Alternate query syntax: The order is relevant, but we haven’t

discussed this yet .

5 / 25

A#1R Vers Perf A#1R

Outline

1 A#1R Short FAQ 2 FP LINQ versatility 3 Performance issues 4 Assignment #1R F#

6 / 25

A#1R Vers Perf A#1R

FP LINQ versatility

Essentially the high-level same code can be used for:

• Processing in memory sequences, e.g. arrays, lists, ...
• In LINQ, all sequences are covered by the super-interface

IEnumerable

• Processing data base tables, views, ...
• SQL Server, Azure SQL, MySQL, NoSQL, MongoDB,

CosmosDB, ...

• Processing web services
• SOAP based (XML)
• REST based (XML/ATOM or JSON)

7 / 25

A#1R Vers Perf A#1R

FP LINQ versatility

Essentially the high-level same code can be used for:

• Processing in memory sequences, e.g. arrays, lists, ...
• In LINQ, all sequences are covered by the super-interface

IEnumerable

• Processing data base tables, views, ...
• SQL Server, Azure SQL, MySQL, NoSQL, MongoDB,

CosmosDB, ...

• Processing web services
• SOAP based (XML)
• REST based (XML/ATOM or JSON)

7 / 25

A#1R Vers Perf A#1R

FP LINQ versatility

Essentially the high-level same code can be used for:

• Processing in memory sequences, e.g. arrays, lists, ...
• In LINQ, all sequences are covered by the super-interface

IEnumerable

• Processing data base tables, views, ...
• SQL Server, Azure SQL, MySQL, NoSQL, MongoDB,

CosmosDB, ...

• Processing web services
• SOAP based (XML)
• REST based (XML/ATOM or JSON)

7 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?
• REST = software architectural style that allows you to build

highly scalable distributed systems

• REST API is simply an Application Programming Interface

that adheres to the above principles

• How many APIs for Web programming? Almost 20,000

registered!

• Sorry! We are only able to look at a few, such as: Web API,

ODATA, ...

• OData is just one of the many ways to build a REST API –

but quite high-level, close to SQL

• We use several languages and platforms, so hopefully concepts

will surface

8 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – Demo code

Look, ma, no SQL, no HTTP, no URL, ...

1 var r e s = Orders 2 . OrderBy ( o = > o . OrderID ) 3 . Take (5) 4 . S e l e c t ( o = > new { 5

• . OrderID ,

6

• . Customer . CompanyName ,

7 EmployeeName = 8

• . Employee . FirstName + ” ” + o . Employee . LastName

9 } ) ; 10 11 r e s .Dump( ” r e s ” ) ;

9 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – Demo code

Same result from both a local SQl database and from a remote ODATA service:

10 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – URL

Magic: FP, LINQ, Lambdas ⇒ Auto-generated URL – GET request

1 ” http :// s e r v i c e s . odata . org / Northwind / Northwind . svc /” + 2 ” Orders () ?” + 3 ”$orderby=OrderID &” + 4 ”$top=5 &” + 5 ”$expand=Customer , Employee &” + 6 ”$ s e l e c t=OrderID , Customer/CompanyName , Employee/FirstName

11 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – SQL code

Magic: FP, LINQ, Lambdas ⇒ Auto-generated SQL code

1 −− Region Parameters 2 DECLARE @p0 NVarChar (1000) = ’ ’ 3 −− EndRegion 4 SELECT [ t1 ] . [ OrderID ] , [ t2 ] . [ CompanyName ] , ( [ t3 ] . [ FirstNa 5 FROM ( 6 SELECT TOP (5) [ t0 ] . [ OrderID ] , [ t0 ] . [ CustomerID ] , [ t0 7 FROM [ Orders ] AS [ t0 ] 8 ORDER BY [ t0 ] . [ OrderID ] 9 ) AS [ t1 ] 10 LEFT OUTER JOIN [ Customers ] AS [ t2 ] ON [ t2 ] . [ CustomerID ] 11 LEFT OUTER JOIN [ Employees ] AS [ t3 ] ON [ t3 ] . [ EmployeeID ] 12 ORDER BY [ t1 ] . [ OrderID ]

12 / 25

A#1R Vers Perf A#1R

Outline

1 A#1R Short FAQ 2 FP LINQ versatility 3 Performance issues 4 Assignment #1R F#

13 / 25

A#1R Vers Perf A#1R

Data Parallelism – First, sequential versions

• Classical for loop: sum the “worked” results over a given array

1 long For ( long [ ] a ) { 2 var s = 0L ; 3 foreach ( var n in a ) { 4 s = s + Work (n ) ; 5 } 6 return s ; 7 }

• Equivalent (still) sequential C# LINQ:

1 long Seq ( long [ ] a ) { 2 return 3 a . S e l e c t (n = > Work(n ) ) . Sum ( ) ; 4 }

• Which one is faster?

14 / 25

A#1R Vers Perf A#1R

Data Parallelism – Look, ma, no threads!

• Equivalent parallel C# LINQ:

1 long Par ( long [ ] a ) { 2 return 3 a . A s P a r a l l e l ( ) . S e l e c t (n = > Work(n ) ) . Sum ( ) ; 4 }

• Just insert . AsParallel () and the code will automatically scale

to use all available cores!

• No worries about semaphores, monitors, synchronised regions,

locks, deadlocks, ...

• All this critical stuff is well encapsulated under the high-order

functions...

• How fast are all these?

15 / 25

A#1R Vers Perf A#1R

Data Parallelism – Look, ma, no threads!

• Linqpad results on a lab machine, with 4 cores – results are

about twice faster with optimised compiled code

1 . . . For duration : 15905 ms for ( ; ; ) 2 3 . . . Seq duration : 15802 ms LINQ 4 5 . . . Par duration : 4104 ms LINQ parallel

• Your conclusions?

16 / 25

A#1R Vers Perf A#1R

JS – reputed very slow

• Classical for loop

1 function For ( a ) { 2 var s = 0; 3 var l e n = a . length ; 4 for ( var i = 0; i < l e n ; i++) { 5 s = s + Work ( a [ i ] ) ; 6 } 7 return s ; 8 }

• Equivalent sequential JS LINQ:

1 function Seq ( a ) { 2 var s = 3 a . S e l e c t (n = > Work(n ) ) . Sum ( ) ; 4 return s ; 5 }

17 / 25

A#1R Vers Perf A#1R

node.js performance

• Node.js results on the same lab machine, with 4 cores:

1 . . . For duration : 16601.501ms for ( ; ; ) 2 3 . . . Seq duration : 16573.154ms LINQ JS

• Well, JS is not for number crunching, but not too bad...
• Your conclusions?

18 / 25

A#1R Vers Perf A#1R

How to measure performance, the FP way

• Encapsulate all measurement details in one single generic

function – once for all!

1 s t a t i c T Duration<T> ( string cap , Func<T> f ) { 2

• GC. C o l l e c t ( ) ;

3 Console . WriteLine ( // NB: string interpolation 4 $”\ r \n . . . {cap} s t a r t ” ) ; 5 var timer = new Stopwatch ( ) ; 6 timer . S t a r t ( ) ; 7 var r e s = f ( ) ; 8 timer . Stop ( ) ; 9 Console . WriteLine ( 10 $” . . . {cap} duration : ” + 11 ”{ timer . E l a p s e d M i l l i s e c o n d s } ms” ) ; 12 return r e s ; 13 }

19 / 25

A#1R Vers Perf A#1R

How to invoke Duration

• Problem: Duration’s parameter f is a generic function (ok!)...

but one that takes no arguments (empty parameter list)

• How to call Duration to measure a function that needs

arguments, such as Seq (a), etc?

• Solution: call Duration to measure a no-arg lambda that in

turn calls the required function!

1 var f = Duration ( ”For” , () = > For ( a ) ) ; 2 var s = Duration ( ”Seq” , () = > Seq ( a ) ) ; 3 var p = Duration ( ”Par” , () = > Par ( a ) ) ;

• For example, given () => For (a), Duration will call

(() => For (a)) (), and finally For (a)!

• Thus, courtesy of lambdas and closures, our Duration

function is really general purpose and applicable in all cases!

20 / 25

A#1R Vers Perf A#1R

Outline

1 A#1R Short FAQ 2 FP LINQ versatility 3 Performance issues 4 Assignment #1R F#

21 / 25

A#1R Vers Perf A#1R

Assignment #1R F#

• Structure defined by indentation: Strict offside rules! Spaces

not tabs!

• Very powerful type inference: F# is statically/strongly typed,

but most types are inferred!

• Usual exceptions: types of the C# API may still need to be

specified

• True pipelines |> (cf. Unix shell) – no method chaining

surrogates

• F# lambdas:

fun a −> v cf. C#: a => v

• No function return: the last expression of the logical branch is

the return value

• Parentheses required only where really needed

22 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Command-line

• From Environment

1 l e t args = Environment . GetCommandLineArgs ()

• If you want a main entry point – you chose its name

1 [< EntryPoint >] 2 l e t main args = 3 . . .

• [<EntryPoint>] is an F# attribute

23 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Read

1

• pen System

2

• pen System . IO

3

• pen System . Text . R e g u l a r E x p r e s s i o n s

4 5 try 6 l e t fname = ” read −t e s t . t x t ” 7 l e t t e x t = F i l e . ReadAllText fname 8 l e t rgx = Regex @”\ s+” 9 l e t words = 10 rgx . Replace ( text , ” ” ) . Trim ( ) . S p l i t ( ’ ’) 11 12 words |> Array . i t e r ( fun w −> printfn ”%s ” w) 13 14 with ex −> 15 printfn ”∗∗∗ Error : %s ” ex . Message 16 Environment . ExitCode <− 1

24 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Frequencies (LP)

1 l e t Frequencies ( words : l i s t <string >) = 2 words 3 |> Seq . groupBy ( fun w −> w. ToUpper ( ) ) 4 |> Seq . map ( fun (k , s ) −> ( Seq . length s , k )) 5 |> Seq . sortByDescending ( fun ( c , k ) −> c ) 6 // . . . 7 8 l e t Main () = 9 l e t words = [ ”dd” ; ”DD” ; ” ccc ” ; . . . ] 10 l e t f r s = Frequencies words 11 f r s .Dump ” f r s ” 12 13 Main ()

• F#: powerful pattern matching, everywhere, even for the

function parameters (see map, sortBy lambdas) –

• cf. more limited C#: deconstructions, JS: destructuring.

25 / 25