What is an Algorithm? Quite generally, an algorithm is a set of - - PDF document

1

Fundamentals: All About Algorithms

Computer Literacy Lecture 23

What is an Algorithm?

• Quite generally, an algorithm is a set of

instructions that lead to some desired result

• Must first decompose a problem, and

then express it’s method of solution as a series of well defined steps to be followed.

• So an algorithm is a recipe for achieving a

certain goal

Pancake Recipe

• 1. Mix 1 egg and 11/2 cups of milk
• 2. Add to 2 cups of sifted flour
• 3. Stir until smooth
• 4. Mix in 2 tablespoons of melted butter
• 5. Pour some of the mixture into a hot pan

and fry

• 6. Serve with maple syrup, sugar or jam

It’s All Relative

• Whether or not a list of instructions counts

as an ‘algorithm’ is relative to a set of background assumptions about what kind

• f device will be following the instructions,

what its basic capacities are, etc.

• Can fill in steps between steps as the level
• f analysis becomes more and more fine

grained

Where to begin

• For example, the Pancake recipe doesn’t

include an initial step:

• 0. place an adequately sized mixing bowl
• n a flat surface

And it doesn’t tell you how to sift the flour or melt the butter, etc.

• these steps between steps are simply

assumed

Effective Procedures

In the most rigorous context, algorithms are the recipes that a computer can follow

• In this sense they can be thought of as

effective procedures (we’ve seen these before): – a finitary set of instructions that can be followed ‘mechanically’, i.e. with no understanding or interpretation required

2

World Knowledge

In this most precise sense of an effective procedure, there can be no missing steps or ambiguities

• ‘Common sense’ or ‘world knowledge’ is a central

aspect of human intelligence not shared by computers

• One of the most difficult aspects of designing

algorithms is to break down a problem in such a way that the solution really is mechanical, relative to its level of execution

Pancake Bugs

• So even though the average

(contemporary western?) human could follow the pancake recipe, it assumes too much world knowledge for a computer

• Has a number of bugs as a potential

program: e.g. step 1. says to mix 1 egg with milk, but it doesn’t say not to include the shell

More Bugs

Step 2. Add to 2 cups of sifted flour But add what – more flour, red wine, marmite? Step 3. Stir until smooth But with what, and exactly how smooth – can the stirring go on for 3 hours? Step 5. Pour some of the mixture into a hot pan and fry - until when?

Numerical Example

• Here’s a simple recipe for manipulating

numbers:

• 1. Pick a number between 1 and 10
• 2. Multiply by 9
• 3. Add the digits together
• 4. Subtract 5

Your number is …..? This simple algorithm can be thought of as calculating a function on the numbers between 1 and 10

Functions

• Since ancient times, mathematicians have

been concerned with giving algorithmic methods for calculating various functions

• It’s one thing to give a high-level definition
• f a function, but it can be quite another

thing to specify a method for calculating the function on any given values

Functions

• One of the best known ancient examples

is Euclid’s algorithm for calculating the function f(x,y)= the greatest common divisor of x and y

• The function f(x,y) is perfectly well

defined with this description, but what procedure should we follow to

• btain specific values?

3

Euclid’s Recipe (3rd century B.C.)

For illustration let x = 53,667 and y = 25,527 to calculate f(x,y): 53,667 = 25,527 · 2 + 2,613 25,527 = 2,613 · 9 + 2,010 2,613 = 2,010 · 1 + 603 2,010 = 603 · 3 + 201 603 = 201 · 3 + 0 f(x,y) = 201 So the general concept of an algorithm has been around for a long time before computers!

Equations

• Within a normal mathematical context,

equations constitute algorithms for calculating basic values, e.g. areas in Euclidean geometry: for a triangle with base b and height h, A = (bh) / 2 for a circle with radius r A = π r2

Arithmetical Functions

• The equations A = (bh)/2 and A = π r2 both

assume the two-place numerical function

• f product or multiplication
• This is because the normal context is

human beings, who can perform this

• peration without even stopping to think
• We’re trained to multiply by first

memorizing a ‘times table’ of basic values

Arithmetical Functions

• But what if you wanted to fill in some steps

between the steps specified in the equations for computing areas?

• The next level of analysis is to give an

algorithm for calculating the function x·y

• a recipe that could be followed by

someone who didn’t know how to multiply

Decomposition

• An obvious way to decompose

multiplication is in terms of repeated addition

• So x·y can be seen as adding x to itself y

many times, e.g. 3·6 = 3+3+3+3+3+3 In this manner you could, in principle, eliminate multiplication and define it in terms of +

More Decomposition

• As with pancakes, we’re familiar with the

basic concept of addition and don’t need to analyze the operation in more explicit detail to perform everyday manipulations

• But what if you wanted to give a recipe for

calculating X+Y, a general equation that could be followed even by a mindless device that didn’t understand the abstract concept??

4

Plus One

• Addition is a two place function, using

variables X and Y

• An obvious way to decompose addition is

in terms of a repeated one place

• peration of adding 1
• This was utilized by the logician Kurt

Gödel, to give recursive axioms for arithmetic

Artificial Languages

• Here’s an example of a recursive definition to

specify the simple artificial language L:

• Vocabulary of L:
• logical vocabulary: ¬, v, &, →, ),(
• statement letters: P1, P2, P3, …

Recursive definition of formulas of L: base step: all statement letters are formulas of L recursion step: if φ, ψ are formulas of L, then ¬φ, (φvψ), (φ&ψ), (φ→ψ) are formulas of L

Recursive Definitions

• The definition of ‘formula of L’ has 2 parts:
• the base step specifies a stock of initial

ingredients (in this case, the atomic formulas)

• the recursion step may appear circular:

‘formula of L’ occurs in the definition of ‘formula

• f L’. But only formulas that have already been

defined in a prior application of the step

• Note that the definition will yield infinitely many

(non-atomic) formulas of L

• Now back to arithmetic …

The Successor Function

• Let the one place function s(X) = the next larger

natural number after X, often called the successor of X, e.g. s(6)=7, s(9)=10

• We can denote all natural numbers in terms of 0

and the successor function: 0=0, 1=s(0), 2=s(s(0)), 3=s(s(s(0))), 4=s(s(s(s(0)))) … Although tedious for humans, this notation reduces numbers to their structural core, and is well suited for purely mechanical manipulations

Axioms for Addition

• With the successor function and the

foregoing version of monadic notation we can give a

• Recursive definition of the two-place +

function in terms of the successor function: i) X + 0 = X ii) X + s(Y) = s(X + Y)

Recursive Definitions

• The two axioms for addition have the

standard recursive form used by many algorithms

• The first equation, X+0 = X, specifies the

initial value

• The second equation, X+s(Y) = s(X+Y)

may appear circular - it refers to ‘+’ when defining ‘+’. But this is OK - it only uses a value that has already been computed

5

Algorithmic Addition

• With the recursive axioms it’s possible to

mechanically calculate the sum of any two natural numbers, without understanding the basic concept of two place addition

• Method: if you want to calculate A+B

recursively, it will take B+1 steps. Let X=A throughout and begin by applying equation i): this is the base step

Algorithmic Addition

Then apply equation ii), X+s(Y) = s(X+Y)

• The base step has already given us the

value of X+Y when Y=0, so now we take the successor of this value to calculate X+s(0) = s(X+0)

• Keep reapplying equation ii) until you get

to the step X+s(Y), where s(Y)=B

Add 2+3 Algorithmically

• 1. set X=2=s(s(0)) and apply equation i):

s(s(0))+0 = s(s(0))

• 2. apply equation ii) and get

s(s(0))+s[0] = s[s(s(0))+0] = s[s(s(0))]

• 3. reapply equation ii) and get

s(s(0))+s[s(0)] = s[s(s(0))+s(0)] = s[s(s(s(0)))]

• 4. reapply equation ii) and get

s(s(0))+s[s(s(0))] = s[s(s(0))+s(s(0))] = s[s(s(s(s(0))))] = 5

Axioms for Multiplication

• As mentioned before, multiplication can be

decomposed as iterated addition

• There are 2 corresponding equations that

give a precise, recursive definition of multiplication in terms of addition i) X·0 = 0 ii) X·s(Y) = X+(X·Y)

2·3 Algorithmically

• 1. s(s(0))·0 = 0
• 2. s(s(0))·s[0] = s(s(0))+0 = s(s(0))
• 3. s(s(0))·s[s(0)] = s(s(0))+[s(s(0))·s(0)]

= s(s(0))+s(s(0)) = s(s(s(s(0))))

• 4. s(s(0))·s[s(s(0))] = s(s(0))+[s(s(0))·s(s(0))]

= s(s(0))+s(s(s(s(0)))) = s(s(s(s(s(s(0)))))) = 6

More Functions

• So now we can define multiplication with

just assuming the one-place successor function s(x) as primitive

• In turn we can use multiplication to define

exponentiation, etc.

• Gödel exploited this method to formally

define the class of recursive functions, which is another framework for expressing the notion of an effective procedure

6

Back to the Stoneage

• The successor function s(x) should remind

you of the caveman adding a stone to some box (and to a TM printing a 1 on a tape)

• Gödel’s recursive framework can compute

exactly the same set of functions as the Lambek Abacus Machines or a UTM (the Church-Turing Thesis revisited)

Computers

• Everything an actual computer can do is

the result of extremely simple arithmetic and logical operations

• At the ground level, even these simple
• perations must be specified

algorithmically

• In this respect computers are exceptionally

stupid – but they make up for it by being extremely fast and accurate

Programs

• Most of the time we’re concerned with problems

and their procedural solutions at a much higher level of analysis

• In such contexts, the solution might first be

expressed as a set of step-by-step instructions in a natural language such as English

• Then the algorithm must be refined as a set of

instructions appropriate to some high level programming language –

• This gives the context relative to which the

instructions must be ‘mechanical’

Back to Pancakes

• 1. Mix 1 egg and 11/2 cups of milk
• 2. Add to 2 cups of sifted flour
• 3. Stir until smooth
• 4. Mix in 2 tablespoons of melted butter
• 5. Pour some of the mixture into a hot pan

and fry

• 6. Serve with maple syrup, sugar or jam

Three Concepts in the Recipe

• Ingredients
• Eggs, flour, milk, butter
• How much, e.g. 2 cups of flour
• Cooking actions
• Frying, mixing, sifting
• The recipe
• Instructions describing what to do with the

ingredients and methods

• Includes the instruction to perform a step

‘until’ some condition is satisfied

Three Concepts in a Program

• Variables
• Eggs, flour, milk, butter
• Functions
• Frying, mixing, etc.
• The algorithm
• Instructions describing what to do with

the variables and functions

• ‘Control flow’ allows variations in the number
• f steps or sequential order of steps

7

Syntax versus Semantics

• As we saw in an early lecture, language has 2

primary levels: syntax – sign, symbol, notation semantics – meaning or interpretation, what the symbols are about

• Computation is the rule-governed manipulation
• f syntax
• A computer may be able to manipulate the

English strings ‘e-g-g-s’, ‘f-l-o-u-r’, etc., but there aren’t any eggs stored in its registers!

Computers Can’t Cook

• So even though the variables in a program

may be intended to range over things in the outside world, like eggs and flour, it still can’t perform functions on these

• bjects –

it can process syntax, but it can’t fry pancakes

• And full-scale domestic robots are a very

long way off!