INF4820: Algorithms for Artificial Intelligence and Natural - - PowerPoint PPT Presentation

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing More Common Lisp

Stephan Oepen & Murhaf Fares

Language Technology Group (LTG)

September 6, 2017

Agenda

Previously

◮ Common Lisp essentials ◮ S-expressions (= atoms or lists of s-expressions) ◮ Recursion ◮ Quote ◮ List processing ◮ Identity vs. Equality

2

Agenda

Previously

◮ Common Lisp essentials ◮ S-expressions (= atoms or lists of s-expressions) ◮ Recursion ◮ Quote ◮ List processing ◮ Identity vs. Equality

Today

◮ More Common Lisp ◮ Higher-order functions ◮ Argument lists ◮ Iteration: (the mighty) loop ◮ Additional data structures

2

Conditional Evaluation

Examples ? (defparameter foo 42) ? (if (numberp foo) "number" "something else")

3

Conditional Evaluation

Examples ? (defparameter foo 42) ? (if (numberp foo) "number" "something else") → "number"

3

Conditional Evaluation

Examples ? (defparameter foo 42) ? (if (numberp foo) "number" "something else") → "number" ? (cond ((< foo 3) "less") ((> foo 3) "more") (t "equal"))

3

Conditional Evaluation

Examples ? (defparameter foo 42) ? (if (numberp foo) "number" "something else") → "number" ? (cond ((< foo 3) "less") ((> foo 3) "more") (t "equal")) → "more"

3

Conditional Evaluation

Examples ? (defparameter foo 42) ? (if (numberp foo) "number" "something else") → "number" ? (cond ((< foo 3) "less") ((> foo 3) "more") (t "equal")) → "more" General Form (if predicate

then clause else clause)

(cond (predicate1 clause1+) (predicate2 clause2+) (predicatei clausei+) (t default clause+))

3

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000))

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) →

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) → 42000

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) → 42000 ? foo → 42

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) → 42000 ? foo → 42 ? #’foo → #<Interpreted Function FOO>

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) → 42000 ? foo → 42 ? #’foo → #<Interpreted Function FOO> ? (funcall #’foo foo) → 42000

4

Rewind: A Note on Symbol Semantics

◮ Symbols can have values as functions and variables at the same time. ◮ #’ (sharp-quote) gives us the function object bound to a symbol.

? (defun foo (x) (* x 1000)) ? (defparameter foo 42) → 42 ? (foo foo) → 42000 ? foo → 42 ? #’foo → #<Interpreted Function FOO> ? (funcall #’foo foo) → 42000

◮ #’ and funcall (as well as apply) are useful when passing around

functions as arguments.

4

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test))))

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test))))

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55))

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp)

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

◮ Functions

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

◮ Functions, recursion

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

◮ Functions, recursion, conditionals

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

◮ Functions, recursion, conditionals, predicates

5

Higher-Order Functions

◮ Functions that accept functions as arguments or return values. ◮ Functions in Lisp are first-class objects.

◮ Can be created at run-time, passed as arguments, returned as values,

stored in variables . . . just like any other type of data.

? (defun filter (list test) (cond ((null list) nil) ((funcall test (first list)) (cons (first list) (filter (rest list) test))) (t (filter (rest list) test)))) ? (defparameter foo ’(11 22 33 44 55)) ? (filter foo #’evenp) → (22 44)

◮ Functions, recursion, conditionals, predicates, lists for code and data.

5

Anonymous Functions

◮ We can also pass function arguments without first binding them to a

name, using lambda expressions: (lambda (parameters) body)

◮ A function definition without the defun and symbol part.

? (filter foo #’(lambda (x) (and (> x 20) (< x 50)))) → (22 33 44)

6

Anonymous Functions

◮ We can also pass function arguments without first binding them to a

name, using lambda expressions: (lambda (parameters) body)

◮ A function definition without the defun and symbol part.

? (filter foo #’(lambda (x) (and (> x 20) (< x 50)))) → (22 33 44)

◮ Typically used for ad-hoc functions that are only locally relevant and

simple enough to be expressed inline.

6

Anonymous Functions

◮ We can also pass function arguments without first binding them to a

name, using lambda expressions: (lambda (parameters) body)

◮ A function definition without the defun and symbol part.

? (filter foo #’(lambda (x) (and (> x 20) (< x 50)))) → (22 33 44)

◮ Typically used for ad-hoc functions that are only locally relevant and

simple enough to be expressed inline.

◮ Or, when constructing functions as return values.

6

Returning Functions

◮ We have seen how to create anonymous functions using lambda and

pass them as arguments.

◮ So we can combine that with a function that itself returns another

function (which we then bind to a variable).

7

Returning Functions

◮ We have seen how to create anonymous functions using lambda and

pass them as arguments.

◮ So we can combine that with a function that itself returns another

function (which we then bind to a variable). ? (defparameter foo ’(11 22 33 44 55)) ? (defun make-range-test (lower upper) #’(lambda (x) (and (> x lower) (< x upper))))

7

Returning Functions

◮ We have seen how to create anonymous functions using lambda and

pass them as arguments.

◮ So we can combine that with a function that itself returns another

function (which we then bind to a variable). ? (defparameter foo ’(11 22 33 44 55)) ? (defun make-range-test (lower upper) #’(lambda (x) (and (> x lower) (< x upper)))) ? (filter foo (make-range-test 10 30)) → (11 22)

7

Parameter Lists: Variable Arities and Naming

Optional Parameters

? (defun foo (x &optional y (z 42)) (list x y z)) ? (foo 1) → (1 nil 42) ? (foo 1 2 3) → (1 2 3)

8

Parameter Lists: Variable Arities and Naming

Optional Parameters

? (defun foo (x &optional y (z 42)) (list x y z)) ? (foo 1) → (1 nil 42) ? (foo 1 2 3) → (1 2 3)

Keyword Parameters

? (defun foo (x &key y (z 42)) (list x y z)) ? (foo 1) → (1 nil 42) ? (foo 1 :z 3 :y 2) → (1 2 3)

8

Parameter Lists: Variable Arities and Naming

Optional Parameters

? (defun foo (x &optional y (z 42)) (list x y z)) ? (foo 1) → (1 nil 42) ? (foo 1 2 3) → (1 2 3)

Keyword Parameters

? (defun foo (x &key y (z 42)) (list x y z)) ? (foo 1) → (1 nil 42) ? (foo 1 :z 3 :y 2) → (1 2 3)

Rest Parameters

? (defun avg (x &rest rest) (let ((numbers (cons x rest))) (/ (apply #’+ numbers) (length numbers)))) ? (avg 3) → 3 ? (avg 1 2 3 4 5 6 7) → 4

8

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent]

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t ? (eql 42 42.0) → nil

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t ? (eql 42 42.0) → nil ? (equalp 42 42.0) → t

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t ? (eql 42 42.0) → nil ? (equalp 42 42.0) → t ? (equal "foo" "foo") → t

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t ? (eql 42 42.0) → nil ? (equalp 42 42.0) → t ? (equal "foo" "foo") → t ? (equalp "FOO" "foo") → t

9

Recap: Equality for One and All

◮ eq tests object identity; it is not useful for numbers or characters. ◮ eql is like eq, but well-defined on numbers and characters. ◮ equal tests structural equivalence ◮ equalp is like equal but insensitive to case and numeric type.

? (eq (list 1 2 3) ’(1 2 3)) → nil ? (equal (list 1 2 3) ’(1 2 3)) → t ? (eq 42 42) → ? [implementation-dependent] ? (eql 42 42) → t ? (eql 42 42.0) → nil ? (equalp 42 42.0) → t ? (equal "foo" "foo") → t ? (equalp "FOO" "foo") → t

◮ Also many type-specialized tests like =, string=, etc.

9

Suggested Home Exercise

From the 2013 Final Exam Write two versions of a function swap; one based on recursion and

• ne based on iteration. The function should take three

parameters—x, y and list— where the goal is to replace every element matching x with y in the list list. Here is an example of the expected behavior: ? (swap "foo" "bar" ’("zap" "foo" "foo" "zap" "foo")) → ("zap" "bar" "bar" "zap" "bar") Try to avoid using destructive operations if you can. [7 points]

10

A Brief Detour: Macros

◮ Elevator Pitch: programs that generate programs. ◮ Macros provide a way for our code to manipulate itself (before it is

passed to the compiler).

◮ Can implement transformations that extend the syntax of the language. ◮ Allows us to control (or even prevent) the evaluation of arguments. ◮ We have already encountered some built-in Common Lisp macros:

and, or, if, cond, defun, setf, etc.

11

A Brief Detour: Macros

◮ Elevator Pitch: programs that generate programs. ◮ Macros provide a way for our code to manipulate itself (before it is

passed to the compiler).

◮ Can implement transformations that extend the syntax of the language. ◮ Allows us to control (or even prevent) the evaluation of arguments. ◮ We have already encountered some built-in Common Lisp macros:

and, or, if, cond, defun, setf, etc.

◮ Although macro writing is out of the scope of this course, we will look

at perhaps the best example of how macros can redefine the syntax of the language—for good or for worse, depending on who you ask:

◮ loop 11

Iteration

◮ While recursion is a powerful

control structure,

◮ sometimes iteration comes

more natural.

◮ dolist and dotimes are fine

for simple iteration.

(let ((result nil)) (dolist (x ’(0 1 2 3 4 5)) (when (evenp x) (push x result))) (reverse result)) → (0 2 4)

12

Iteration

◮ While recursion is a powerful

control structure,

◮ sometimes iteration comes

more natural.

◮ dolist and dotimes are fine

for simple iteration.

(let ((result nil)) (dolist (x ’(0 1 2 3 4 5)) (when (evenp x) (push x result))) (reverse result)) → (0 2 4) (let ((result nil)) (dotimes (x 6) (when (evenp x) (push x result))) (reverse result)) → (0 2 4)

12

Iteration

◮ While recursion is a powerful

control structure,

◮ sometimes iteration comes

more natural.

◮ dolist and dotimes are fine

for simple iteration.

◮ But (the mighty) loop is much

more general and versatile.

(let ((result nil)) (dolist (x ’(0 1 2 3 4 5)) (when (evenp x) (push x result))) (reverse result)) → (0 2 4) (let ((result nil)) (dotimes (x 6) (when (evenp x) (push x result))) (reverse result)) → (0 2 4) (loop for x below 6 when (evenp x) collect x) → (0 2 4)

12

Iteration with loop

(loop for i from 10 to 50 by 10 collect i) → (10 20 30 40 50)

◮ Illustrates the power of syntax extension through macros; ◮ loop is basically a mini-language for iteration.

13

Iteration with loop

(loop for i from 10 to 50 by 10 collect i) → (10 20 30 40 50)

◮ Illustrates the power of syntax extension through macros; ◮ loop is basically a mini-language for iteration. ◮ Reduced uniformity: different syntax based on special keywords. ◮ Paul Graham on loop: “one of the worst flaws in Common Lisp”.

13

Iteration with loop

(loop for i from 10 to 50 by 10 collect i) → (10 20 30 40 50)

◮ Illustrates the power of syntax extension through macros; ◮ loop is basically a mini-language for iteration. ◮ Reduced uniformity: different syntax based on special keywords. ◮ Paul Graham on loop: “one of the worst flaws in Common Lisp”. ◮ But non-Lispy as it may be, loop is extremely general and powerful!

13

loop: A Few More Examples

? (loop for i below 10 when (oddp i) sum i) → 25

14

loop: A Few More Examples

? (loop for i below 10 when (oddp i) sum i) → 25 ? (loop for x across "foo" collect x) → (#\f #\o #\o)

14

loop: A Few More Examples

? (loop for i below 10 when (oddp i) sum i) → 25 ? (loop for x across "foo" collect x) → (#\f #\o #\o) ? (loop with foo = ’(a b c d) for i in foo for j from 0 until (eq i ’c) do (format t "~a: ~a ~%" j i)) ❀ 0: A 1: B

14

loop: Even More Examples

? (loop for i below 10 if (evenp i) collect i into evens else collect i into odds finally (return (list evens odds))) → ((0 2 4 6 8) (1 3 5 7 9))

15

loop: Even More Examples

? (loop for i below 10 if (evenp i) collect i into evens else collect i into odds finally (return (list evens odds))) → ((0 2 4 6 8) (1 3 5 7 9)) ? (loop for value being each hash-value of *dictionary* using (hash-key key) do (format t "~&~a -> ~a" key value))

15

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp ◮ Control: { while | until | repeat | when | unless | . . . } sexp

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp ◮ Control: { while | until | repeat | when | unless | . . . } sexp ◮ Local variables: with symbol = sexp

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp ◮ Control: { while | until | repeat | when | unless | . . . } sexp ◮ Local variables: with symbol = sexp ◮ Initialization and finalization: { initially | finally } sexp+

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp ◮ Control: { while | until | repeat | when | unless | . . . } sexp ◮ Local variables: with symbol = sexp ◮ Initialization and finalization: { initially | finally } sexp+ ◮ All of these can be combined freely, e.g. iterating through a list,

counting a range, and stepwise computation, all in parallel.

16

loop: The Swiss Army Knife of Iteration

◮ Iteration over lists or vectors: for symbol { in | on | across } list ◮ Counting through ranges:

for symbol [ from number ] { to | downto } number [ by number ]

◮ Iteration over hash tables:

for symbol being each { hash-key | hash-value } in hash table

◮ Stepwise computation: for symbol = sexp then sexp ◮ Accumulation: { collect | append | sum | minimize | count | . . . } sexp ◮ Control: { while | until | repeat | when | unless | . . . } sexp ◮ Local variables: with symbol = sexp ◮ Initialization and finalization: { initially | finally } sexp+ ◮ All of these can be combined freely, e.g. iterating through a list,

counting a range, and stepwise computation, all in parallel.

◮ Note: without at least one accumulator, loop will only return nil.

16

Input and Output

◮ Reading and writing is mediated through streams. ◮ The symbol t indicates the default stream, the terminal.

? (format t "~a is the ~a.~%" 42 "answer") ❀ 42 is the answer. → nil

17

Input and Output

◮ Reading and writing is mediated through streams. ◮ The symbol t indicates the default stream, the terminal.

? (format t "~a is the ~a.~%" 42 "answer") ❀ 42 is the answer. → nil

◮ (read-line stream nil) reads one line of text from stream,

returning it as a string.

◮ (read stream nil) reads one well-formed s-expression. ◮ The second reader argument asks to return nil on end-of-file.

17

Input and Output

◮ Reading and writing is mediated through streams. ◮ The symbol t indicates the default stream, the terminal.

? (format t "~a is the ~a.~%" 42 "answer") ❀ 42 is the answer. → nil

◮ (read-line stream nil) reads one line of text from stream,

returning it as a string.

◮ (read stream nil) reads one well-formed s-expression. ◮ The second reader argument asks to return nil on end-of-file.

(with-open-file (stream "sample.txt" :direction :input) (loop for line = (read-line stream nil) while line do (format t "~a~%" line)))

17

More Data Structures: Arrays

◮ Integer-indexed container (indices count from zero)

? (setf array (make-array 5)) → #(nil nil nil nil nil) ? (setf (aref array 0) 42)→ 42 ? array → #(42 nil nil nil nil)

18

More Data Structures: Arrays

◮ Integer-indexed container (indices count from zero)

? (setf array (make-array 5)) → #(nil nil nil nil nil) ? (setf (aref array 0) 42)→ 42 ? array → #(42 nil nil nil nil)

◮ Can be fixed-sized (default) or dynamically adjustable. ◮ Can also represent ‘grids’ of multiple dimensions:

? (defparameter array (make-array ’(2 5) :initial-element 0)) → #((0 0 0 0 0) (0 0 0 0 0)) ? (incf (aref array 1 2)) → 1

1 2 3 4 1 1

18

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) →

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar"

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha"

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha" ? (remove ’a ’(a b b a)) → (b b)

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha" ? (remove ’a ’(a b b a)) → (b b) ? (some #’listp ’(1 a "2" 3 (b))) →

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha" ? (remove ’a ’(a b b a)) → (b b) ? (some #’listp ’(1 a "2" 3 (b))) → T

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha" ? (remove ’a ’(a b b a)) → (b b) ? (some #’listp ’(1 a "2" 3 (b))) → ? (sort ’(1 2 1 3 1 0) #’<) → (0 1 1 1 2 3)

19

Arrays: Specializations and Generalizations

◮ Vectors ≡ specialized type of arrays: one-dimensional. ◮ Strings ≡ specialized type of vectors (similarly: bit vectors). ◮ Vectors and lists are subtypes of the abstract data type sequence. ◮ Large number of built-in sequence functions, e.g.:

? (length "foo") → 3 ? (elt "foo" 0) → #\f ? (count-if #’numberp ’(1 a "2" 3 (b))) → 2 ? (subseq "foobar" 3 6) → "bar" ? (substitute #\a #\o "hoho") → "haha" ? (remove ’a ’(a b b a)) → (b b) ? (some #’listp ’(1 a "2" 3 (b))) → ? (sort ’(1 2 1 3 1 0) #’<) → (0 1 1 1 2 3)

◮ Others: position, every, count, remove-if, find, merge, map, reverse,

concatenate, reduce, . . .

19

In Conclusion

http://xkcd.com/297/ 20

More Practically: Programming in INF4820

◮ In the IFI Linux environment, we have available Allegro Common Lisp,

a commercial Lisp interpreter and compiler.

◮ We provide a pre-configured, integrated setup with emacs and the

SLIME Lisp interaction mode.

◮ First-time users, please spend some time studying basic keyboard

commands, for example: C-h t and M-x doctor RET.

◮ Several open-source Lisp implementation exist, e.g. Clozure or SBCL,

compatible with SLIME, so feel free to experiment (on your own time).

◮ We have posted a Getting Started guide and Emacs Cheat Sheet on the

course pages last week.

21

Good Lisp Style

Bottom-Up Design

◮ Instead of trying to solve everything with one large function: Build your

program with layers of smaller functions.

◮ Eliminate repetition and patterns.

◮ Related; define abstraction barriers.

◮ Separate the code that uses a given data abstraction from the code that

implement that data abstraction.

◮ Promotes code re-use:

◮ Makes the code shorter and easier to read, debug and maintain. 22

Good Lisp Style

Bottom-Up Design

◮ Instead of trying to solve everything with one large function: Build your

program with layers of smaller functions.

◮ Eliminate repetition and patterns.

◮ Related; define abstraction barriers.

◮ Separate the code that uses a given data abstraction from the code that

implement that data abstraction.

◮ Promotes code re-use:

◮ Makes the code shorter and easier to read, debug and maintain.

◮ Somewhat more mundane:

◮ Adhere to the time-honored 80 column rule. ◮ Close multiple parens on the same line. ◮ Use auto-indentation (TAB) in Emacs. 22

If you can’t see the forest for the trees. . .

. . . or can’t even see the trees for the parentheses. A Lisp specialty: Uniformity

◮ Lisp beginners can sometimes find the syntax overwhelming. What’s

with all the parentheses?

◮ For seasoned Lispers the beauty lies in the fact that there’s hardly any

syntax at all (beyond the abstract data type of lists).

◮ Lisp code is a Lisp data structure. ◮ Lisp programs are trees of sexps (comparable to the abstract syntax

trees created internally by the parser/compiler for other languages).

◮ Makes it easier to write code that generates code: macros.

23

Next week

◮ Can we automatically infer the meaning of words? ◮ Distributional semantics ◮ Vector spaces: Spatial models for representing data ◮ Semantic spaces

24