[PPT] - Computer aided assessment of mathematics: the current state of the PowerPoint Presentation

SLIDE 1

Computer aided assessment of mathematics: the current state of the art and a look to the future.

Chris Sangwin

School of Mathematics University of Edinburgh

October 2018

Chris Sangwin (University of Edinburgh) STACK October 2018 1 / 30

SLIDE 2

Question level

Computer algebra systems (CAS).

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 3

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 4

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored?

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 5

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored? Multi-part questions.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 6

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored? Multi-part questions. Student “enters steps”.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 7

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored? Multi-part questions. Student “enters steps”. Unit testing of questions.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 8

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored? Multi-part questions. Student “enters steps”. Unit testing of questions. Special libraries: e.g. support for scientific units.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 9

Question level

Computer algebra systems (CAS). Include CAS calculations within feedback. Formative feedback. Test for properties, e.g. is this factored? Multi-part questions. Student “enters steps”. Unit testing of questions. Special libraries: e.g. support for scientific units. But every system is still different.

Chris Sangwin (University of Edinburgh) STACK October 2018 2 / 30

SLIDE 10

Student Interface

(1) Typed expressions: validity/correctness.

Chris Sangwin (University of Edinburgh) STACK October 2018 3 / 30

SLIDE 11

Student Interface

(1) Typed expressions: validity/correctness. Validity: Students know what is required. Students not penalised on a technicality. Increases robustness of marking.

Chris Sangwin (University of Edinburgh) STACK October 2018 3 / 30

SLIDE 12

Student Interface

(1) Typed expressions: validity/correctness. Validity: Students know what is required. Students not penalised on a technicality. Increases robustness of marking. (2) Handwriting recognition

Chris Sangwin (University of Edinburgh) STACK October 2018 3 / 30

SLIDE 13

Integration with other systems

Chris Sangwin (University of Edinburgh) STACK October 2018 4 / 30

SLIDE 14

Integration with other systems

LTI API

Chris Sangwin (University of Edinburgh) STACK October 2018 4 / 30

SLIDE 15

Material banks

High quality materials are expensive.

Chris Sangwin (University of Edinburgh) STACK October 2018 5 / 30

SLIDE 16

Material banks

High quality materials are expensive. Design level: Freudenthal institute

Chris Sangwin (University of Edinburgh) STACK October 2018 5 / 30

SLIDE 17

Material banks

High quality materials are expensive. Design level: Freudenthal institute Options Community led

◮ WebWork (open) ◮ Abacus (bring to the party)

Commercial, via publishers

Chris Sangwin (University of Edinburgh) STACK October 2018 5 / 30

SLIDE 18

Line by line reasoning

Chris Sangwin (University of Edinburgh) STACK October 2018 6 / 30

SLIDE 19

Chris Sangwin (University of Edinburgh) STACK October 2018 7 / 30

SLIDE 20

Expert systems and AI

Excellent examples.

Chris Sangwin (University of Edinburgh) STACK October 2018 8 / 30

SLIDE 21

Expert systems and AI

Excellent examples. BUT still require large expert projects. Bespoke areas (not general systems).

Chris Sangwin (University of Edinburgh) STACK October 2018 8 / 30

SLIDE 22

The nature of the subject

Is 2 log(x) ≡ log(x2)?

Chris Sangwin (University of Edinburgh) STACK October 2018 9 / 30

SLIDE 23

The nature of the subject

Is 2 log(x) ≡ log(x2)? (0, ∞) = R/{0}.

Chris Sangwin (University of Edinburgh) STACK October 2018 9 / 30

SLIDE 24

The nature of the subject

Is 2 log(x) ≡ log(x2)? (0, ∞) = R/{0}. We are still arguing about what algebraic steps are legitimate!

Chris Sangwin (University of Edinburgh) STACK October 2018 9 / 30

SLIDE 25

The nature of the subject

Is 2 log(x) ≡ log(x2)? (0, ∞) = R/{0}. We are still arguing about what algebraic steps are legitimate! Online assessment can (and should) drive this argument.

Chris Sangwin (University of Edinburgh) STACK October 2018 9 / 30

SLIDE 26

Current State of Proof Assessment

{}

Chris Sangwin (University of Edinburgh) STACK October 2018 10 / 30

SLIDE 27

Current State of Proof Assessment

{} Professional automatic reasoning systems. (COQ)

Chris Sangwin (University of Edinburgh) STACK October 2018 10 / 30

SLIDE 28

Current State of Proof Assessment

{} Professional automatic reasoning systems. (COQ) But professional mathematicians use L

AT

EX.

Chris Sangwin (University of Edinburgh) STACK October 2018 10 / 30

SLIDE 29

Current State of Proof Assessment

{} Professional automatic reasoning systems. (COQ) But professional mathematicians use L

AT

EX.

1

Educational purpose.

2

Professional practice.

Chris Sangwin (University of Edinburgh) STACK October 2018 10 / 30

SLIDE 30

Babbage and the Analytical Engine

Chris Sangwin (University of Edinburgh) STACK October 2018 11 / 30

SLIDE 31

Technology which looks back

Babbage set out to print log tables!

Chris Sangwin (University of Edinburgh) STACK October 2018 12 / 30

SLIDE 32

Technology which looks back

Babbage set out to print log tables! Knuth set out to replicate movable type!

Chris Sangwin (University of Edinburgh) STACK October 2018 12 / 30

SLIDE 33

What we teach and why

The “big three”. Factor Integrate Solve Currently teach ad hoc methods.

Chris Sangwin (University of Edinburgh) STACK October 2018 13 / 30

SLIDE 34

What we teach and why

The “big three”. Factor Integrate Solve Currently teach ad hoc methods. CAS don’t use these!

1

... foundations for

2

... special cases (e.g. Gauss elimination)

3

... direct methods not guess and check.

Chris Sangwin (University of Edinburgh) STACK October 2018 13 / 30

SLIDE 35

19th Century methods!

Chris Sangwin (University of Edinburgh) STACK October 2018 14 / 30

SLIDE 36

Solving quadratics

x2 + 4x − 12 = x2 + 4x = 12 x2 + 4x + 4 = 16 (x − 2)2 = 42 (x − 2)2 − 42 = (x − 2 − 4)(x − 2 + 4) = (x − 6)(x + 2) = Both “solve” and “factor” are direct methods.

Chris Sangwin (University of Edinburgh) STACK October 2018 15 / 30

SLIDE 37

Proof: Assessment of whole argument

Will require a sea-change in how we write mathematics.

Chris Sangwin (University of Edinburgh) STACK October 2018 16 / 30

SLIDE 38

Proof: Assessment of whole argument

Will require a sea-change in how we write mathematics. “Those who cannot remember the past are condemned to repeat it.” (George Santayana)

Chris Sangwin (University of Edinburgh) STACK October 2018 16 / 30

SLIDE 39

Better interface

Pell (1668) (see Stedall (2002))

Chris Sangwin (University of Edinburgh) STACK October 2018 17 / 30

SLIDE 40

Better interface

Pell (1668) (see Stedall (2002)) The constraints of the interface focus thought.

Chris Sangwin (University of Edinburgh) STACK October 2018 17 / 30

SLIDE 41

Better interface

Pell (1668) (see Stedall (2002)) The constraints of the interface focus thought. The constraints of algebraic symbolism focus thought.

Chris Sangwin (University of Edinburgh) STACK October 2018 17 / 30

SLIDE 42

Reasoning by equivalence demo

Chris Sangwin (University of Edinburgh) STACK October 2018 18 / 30

SLIDE 43

Chris Sangwin (University of Edinburgh) STACK October 2018 19 / 30

SLIDE 44

STACK and RE (2018)

Working Polynomials, rational expressions ±, √ Single variable inequalities over R Simultaneous equations Equating coefficients Not RE: “Let” (previous line only...) Not RE: Calculus operations

Chris Sangwin (University of Edinburgh) STACK October 2018 20 / 30

SLIDE 45

STACK and RE (2018)

Working Polynomials, rational expressions ±, √ Single variable inequalities over R Simultaneous equations Equating coefficients Not RE: “Let” (previous line only...) Not RE: Calculus operations Future |x| Systems of inequalities

Chris Sangwin (University of Edinburgh) STACK October 2018 20 / 30

SLIDE 46

STACK and RE (2018)

Working Polynomials, rational expressions ±, √ Single variable inequalities over R Simultaneous equations Equating coefficients Not RE: “Let” (previous line only...) Not RE: Calculus operations Future |x| Systems of inequalities Distant future Trig (requires infinite solution sets)

Chris Sangwin (University of Edinburgh) STACK October 2018 20 / 30

SLIDE 47

... but what about steps?

... ability to follow the teacher’s model answer

Chris Sangwin (University of Edinburgh) STACK October 2018 21 / 30

SLIDE 48

... but what about steps?

... ability to follow the teacher’s model answer ... better feedback

Chris Sangwin (University of Edinburgh) STACK October 2018 21 / 30

SLIDE 49

Bigger picture

What is a modern “book”?

Chris Sangwin (University of Edinburgh) STACK October 2018 22 / 30

SLIDE 50

Bigger picture

What is a modern “book”? Mastery learning as a practical approach. (Bloom)

Chris Sangwin (University of Edinburgh) STACK October 2018 22 / 30

SLIDE 51

Bigger picture

What is a modern “book”? Mastery learning as a practical approach. (Bloom) Fundamentals of Algebra and Calculus

Chris Sangwin (University of Edinburgh) STACK October 2018 22 / 30

SLIDE 52

Exams

Chris Sangwin (University of Edinburgh) STACK October 2018 23 / 30

SLIDE 53

Online exams

Chris Sangwin (University of Edinburgh) STACK October 2018 24 / 30

SLIDE 54

Online exams

Will happen.

Chris Sangwin (University of Edinburgh) STACK October 2018 24 / 30

SLIDE 55

Online exams

Will happen. (Examples from the last century)

Chris Sangwin (University of Edinburgh) STACK October 2018 24 / 30

SLIDE 56

Comparison with school exams?

(Nadine Köcher & Chris Sangwin, 2014)

International Baccalaureate examinations in STACK? # marks (i) Awarded by STACK (2014) exactly 112 18% (ii) Final answers and implied method marks 227 37% (iii) Reasoning by equivalence 218 36% Total of max of (ii) and (iii) per question 376 61%

Chris Sangwin (University of Edinburgh) STACK October 2018 25 / 30

SLIDE 57

Comparison with school exams?

(Nadine Köcher & Chris Sangwin, 2014)

International Baccalaureate examinations in STACK? # marks (i) Awarded by STACK (2014) exactly 112 18% (ii) Final answers and implied method marks 227 37% (iii) Reasoning by equivalence 218 36% Total of max of (ii) and (iii) per question 376 61% Scottish Highers Repeat analysis with STACK in 2018: Scottish Highers papers from 2015. # marks (i) Awarded by STACK (2018) exactly 47 36% (ii) Of which reasoning by equivalence 35 27%

Chris Sangwin (University of Edinburgh) STACK October 2018 25 / 30

SLIDE 58

Comparison with school exams?

(Nadine Köcher & Chris Sangwin, 2014)

International Baccalaureate examinations in STACK? # marks (i) Awarded by STACK (2014) exactly 112 18% (ii) Final answers and implied method marks 227 37% (iii) Reasoning by equivalence 218 36% Total of max of (ii) and (iii) per question 376 61% Scottish Highers Repeat analysis with STACK in 2018: Scottish Highers papers from 2015. # marks (i) Awarded by STACK (2018) exactly 47 36% (ii) Of which reasoning by equivalence 35 27% The most important single form of reasoning in school mathe- matics is reasoning by equivalence.

Chris Sangwin (University of Edinburgh) STACK October 2018 25 / 30

SLIDE 59

But why teach this at all?

(Photomath)

Chris Sangwin (University of Edinburgh) STACK October 2018 26 / 30

SLIDE 60

Tempora mutantur, nos et mutamur in illis

Chris Sangwin (University of Edinburgh) STACK October 2018 27 / 30

SLIDE 61

Tempora mutantur, nos et mutamur in illis Times are changed, we also are changed with them (Ovid) Will mathematics become like Latin in school?

Chris Sangwin (University of Edinburgh) STACK October 2018 27 / 30

SLIDE 62

Nature of the subject

Polya 1962: Mathematical Discovery: on understanding, learning and teaching problem solving. Patterns of thought for solving problems the pattern of two loci

Chris Sangwin (University of Edinburgh) STACK October 2018 28 / 30

SLIDE 63

Nature of the subject

Polya 1962: Mathematical Discovery: on understanding, learning and teaching problem solving. Patterns of thought for solving problems the pattern of two loci superposition

Chris Sangwin (University of Edinburgh) STACK October 2018 28 / 30

SLIDE 64

Nature of the subject

Polya 1962: Mathematical Discovery: on understanding, learning and teaching problem solving. Patterns of thought for solving problems the pattern of two loci superposition recursion

Chris Sangwin (University of Edinburgh) STACK October 2018 28 / 30

SLIDE 65

Nature of the subject

Polya 1962: Mathematical Discovery: on understanding, learning and teaching problem solving. Patterns of thought for solving problems the pattern of two loci superposition recursion Cartesian pattern

Chris Sangwin (University of Edinburgh) STACK October 2018 28 / 30

SLIDE 66

Nature of the subject

Polya 1962: Mathematical Discovery: on understanding, learning and teaching problem solving. Patterns of thought for solving problems the pattern of two loci superposition recursion Cartesian pattern Legitimate patterns of thought → an acceptable proof.

Chris Sangwin (University of Edinburgh) STACK October 2018 28 / 30

SLIDE 67

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 68

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

2

Reduce any mathematical problem to algebra.

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 69

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

2

Reduce any mathematical problem to algebra.

3

Reduce any algebra problem to a single equation & solve.

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 70

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

2

Reduce any mathematical problem to algebra.

3

Reduce any algebra problem to a single equation & solve. Polya: “The more you know, the more gaps you can see in this project”

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 71

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

2

Reduce any mathematical problem to algebra.

3

Reduce any algebra problem to a single equation & solve. Polya: “The more you know, the more gaps you can see in this project” Solving the equation is only the last step...

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 72

Cartesian pattern

Descartes’ Rules for the Direction of the mind.

1

Reduce any kind of problem to a mathematical problem.

2

Reduce any mathematical problem to algebra.

3

Reduce any algebra problem to a single equation & solve. Polya: “The more you know, the more gaps you can see in this project” Solving the equation is only the last step... Assessment of the whole process is the challenge!

Chris Sangwin (University of Edinburgh) STACK October 2018 29 / 30

SLIDE 73

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30

SLIDE 74

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

We can largely automate the methods-based parts.

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30

SLIDE 75

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

We can largely automate the methods-based parts. (Automate the methods and assessment of the methods!)

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30

SLIDE 76

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

We can largely automate the methods-based parts. (Automate the methods and assessment of the methods!) Increasingly automate proof and reasoning.

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30

SLIDE 77

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

We can largely automate the methods-based parts. (Automate the methods and assessment of the methods!) Increasingly automate proof and reasoning. Disconnects

1

Current teaching with internal CAS methods!

2

Historic proof with emerging ATP

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30

SLIDE 78

Conclusion

Computer aided assessment of mathematics: the current state

f the art and a look to the future.

We can largely automate the methods-based parts. (Automate the methods and assessment of the methods!) Increasingly automate proof and reasoning. Disconnects

1

Current teaching with internal CAS methods!

2

Historic proof with emerging ATP

Change in society will and must change mathematics education.

Chris Sangwin (University of Edinburgh) STACK October 2018 30 / 30