A Survey on Analog Models of Computation Amaury Pouly Joint work - - PowerPoint PPT Presentation

A Survey on Analog Models of Computation

Amaury Pouly Joint work with Olivier Bournez

Université de Paris, IRIF, CNRS, F-75013 Paris, France

30 june 2020 Survey: https://arxiv.org/abs/1805.05729

1 / 24

The meaning of “analog”

Historically: “analog” = by analogy, i.e. same evolution m k b F(t) z R L C q V(t) F = m¨ z + b ˙ z + kz , V = L¨ q + R ˙ q + 1

C q

2 / 24

The meaning of “analog”

Historically: “analog” = by analogy, i.e. same evolution m k b F(t) z R L C q V(t) F = m¨ z + b ˙ z + kz , V = L¨ q + R ˙ q + 1

C q

Nowadays: “analog” = continuous/opposite of digital ) orthogonal concepts ) even continuous/discrete unclear: hybrid exists

2 / 24

Some analog machines

Difference Engine Linear Planimeter Slide Rule Antikythera mechanism

3 / 24

Some analog machines

ENIAC Admiralty Fire Control Table Differential Analyzer Kelvin’s Tide Predicter

4 / 24

Classifying machines/models

space time discrete continuous

5 / 24

Classifying machines/models

space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore

5 / 24

Classifying machines/models

space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Differential Analyzer Analog Circuits Planimeter Antikythera Tide Predicter AFCT

5 / 24

Classifying machines/models

space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Differential Analyzer Analog Circuits Planimeter Antikythera Tide Predicter AFCT Difference Engine Slide Rule

5 / 24

Classifying machines/models

Not general purpose space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Differential Analyzer Analog Circuits Planimeter Antikythera Tide Predicter AFCT Difference Engine Slide Rule

5 / 24

Classifying machines/models

space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Differential Analyzer Analog Circuits

5 / 24

Classifying machines/models

Mathematical model space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Discrete Dynamical System yn+1 = f(yn) Differential Analyzer Analog Circuits Continuous Dynamical System y0 = f(y)

5 / 24

Classifying machines/models

Computability model space time discrete continuous laptop server supercomputer Digital Circuits ENIAC Commodore Turing machine Differential Analyzer Analog Circuits GPAC y0 = p(y)

5 / 24

The many many models

space time discrete continuous

Turing machines Lambda calculus Recursive functions Post systems Cellular automata Finite state automata Population protocols Chemical reaction networks Petri nets

6 / 24

The many many models

space time discrete continuous

Turing machines Lambda calculus Recursive functions Post systems Cellular automata Finite state automata Population protocols Chemical reaction networks Petri nets Neural networks Deep learning models Blum Shub Smale machines Hybrid systems Natural computing influence dynamics Signal machines Continuous Automata

6 / 24

The many many models

space time discrete continuous

Turing machines Lambda calculus Recursive functions Post systems Cellular automata Finite state automata Population protocols Chemical reaction networks Petri nets Neural networks Deep learning models Blum Shub Smale machines Hybrid systems Natural computing influence dynamics Signal machines Continuous Automata Shannon’s GPAC Hopfield’s neural networks Physarum computing Reaction-Diffusion Systems Hybrid Systems Timed automata Large population protocols Black hole models Rrecursive functions Chemical reaction networks

6 / 24

The many many models

space time discrete continuous

Turing machines Lambda calculus Recursive functions Post systems Cellular automata Finite state automata Population protocols Chemical reaction networks Petri nets Neural networks Deep learning models Blum Shub Smale machines Hybrid systems Natural computing influence dynamics Signal machines Continuous Automata Shannon’s GPAC Hopfield’s neural networks Physarum computing Reaction-Diffusion Systems Hybrid Systems Timed automata Large population protocols Black hole models Rrecursive functions Chemical reaction networks Boolean difference equation models

6 / 24

Making sense of all these models

discrete Turing machines boolean circuits logic recursive functions lambda calculus continuous

“Church” thesis

All discrete models are Turing machine-computable.

7 / 24

Making sense of all these models

discrete Turing machines boolean circuits logic recursive functions lambda calculus quantum continuous

“Church” thesis

All discrete models are Turing machine-computable.

7 / 24

Making sense of all these models

discrete Turing machines boolean circuits logic recursive functions lambda calculus quantum analog continuous ?

“Church” thesis ?

All models are Turing machine-computable. Clearly wrong: a single real number (Ω of Chaitin) is super-Turing pow- erful.

7 / 24

Making sense of all these models

discrete Turing machines boolean circuits logic recursive functions lambda calculus quantum analog continuous ?

“Church” thesis ?

All physical machine-based models are Turing machine-computable. Several issues with that statement.

7 / 24

Machine vs mathematical model

physical machine

8 / 24

Machine vs mathematical model

physical machine mathematical model

abstraction

I mathematical model = abstraction of a system

8 / 24

Machine vs mathematical model

physical machine mathematical model

abstraction

computability results

proof

I mathematical model = abstraction of a system I properties of model 6= properties of system

8 / 24

Machine vs mathematical model

physical machine mathematical model

abstraction

computability results

proof

physical truth

interpretation

? I mathematical model = abstraction of a system I properties of model 6= properties of system I conclusion might be quantitatively or qualitatively wrong

8 / 24

Black hole model and hypercomputations

I machine: the universe I model: general relativity

9 / 24

Black hole model and hypercomputations

I machine: the universe I model: general relativity

Informal theorem

If slowly rotating Kerr black holes exists, one can check consistency of ZFC or solve the Turing halting problem in finite time. I conclusion: hypercomputations are possible ?

9 / 24

Black hole model and hypercomputations

I machine: the universe I model: general relativity

Informal theorem

If slowly rotating Kerr black holes exists, one can check consistency of ZFC or solve the Turing halting problem in finite time. I conclusion: hypercomputations are possible ? Common occurrence in analog models: non-computable reals, Zeno phenomena, ...

9 / 24

Back to the Church thesis

Distinguish machines from models:

Actual Church thesis

Every effective computation can be carried out by a Turing machine, and vice versa. ) effective = systematic method in logic/mathematics/CS

10 / 24

Back to the Church thesis

Distinguish machines from models:

Actual Church thesis

Every effective computation can be carried out by a Turing machine, and vice versa. ) effective = systematic method in logic/mathematics/CS

Physical Church Turing thesis/Thesis M

Whatever can be calculated by a machine (with finite data/instructions) is Turing machine-computable. ) machine that conforms to the physical laws

10 / 24

Back to the Church thesis

Distinguish machines from models:

Actual Church thesis

Every effective computation can be carried out by a Turing machine, and vice versa. ) effective = systematic method in logic/mathematics/CS

Physical Church Turing thesis/Thesis M

Whatever can be calculated by a machine (with finite data/instructions) is Turing machine-computable. ) machine that conforms to the physical laws

Alternative thesis

All reasonable models of computations are equivalent to Turing machines.

10 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2

11 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2 Semantics (assuming law of mass action): I discrete I differential I stochastic

11 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2 Semantics (assuming law of mass action): I discrete ! I differential I stochastic yi = molecule count close to population protocols

11 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2 Semantics (assuming law of mass action): I discrete I differential ! I stochastic yi = concentration polynomial ODEs

11 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2 Semantics (assuming law of mass action): I discrete I differential I stochastic ! yi = probability distribution stochastic ODEs

11 / 24

Chemical Reaction Networks (CRNs)

A reaction system is a finite set of I molecular species y1, . . . , yn I reactions of the form P

i aiyi f

• ! P

i biyi

(ai, bi 2 N, f = rate) Example (any resemblance to chemistry is purely coincidental): 2H + O ! H2O C + O2 ! CO2 Semantics (assuming law of mass action): I discrete I differential I stochastic

Observation

A system/machine can have several models, all useful, depending on the level of abstraction.

11 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems

12 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems Let’s do something useful with it!

12 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems Let’s do something useful with it! Something is wrong, change the model.

12 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems Let’s do something useful with it! Something is wrong, change the model. Let’s study it! Especially if it doesn’t exist.

12 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems Let’s do something useful with it! Something is wrong, change the model. Let’s study it! Especially if it doesn’t exist. Even if it exists, it cannot be verified.

12 / 24

Are analog systems capable of hypercomputations?

Examples: Black holes, signal machines, hybrid systems Let’s do something useful with it! Something is wrong, change the model. Let’s study it! Especially if it doesn’t exist. Even if it exists, it cannot be verified.

Possible conclusion

All reasonable models of computations are equivalent to Turing

• machines. Hypercomputability results can help us correct models.

12 / 24

General Purpose Analog Computer (GPAC)

Differential analyzer

13 / 24

General Purpose Analog Computer (GPAC)

Differential analyzer k

k

+

u+v u v

⇥

uv u v

R

R u u

General Purpose Analog Computer, Shannon 1936

13 / 24

General Purpose Analog Computer (GPAC)

Differential analyzer k

k

+

u+v u v

⇥

uv u v

R

R u u

General Purpose Analog Computer, Shannon 1936 y(0) = y0, y0(t) = p(y(t)) Polynomial Differential Equation, Graça 2004 t

y1(t)

13 / 24

General Purpose Analog Computer (GPAC)

Differential analyzer k

k

+

u+v u v

⇥

uv u v

R

R u u

General Purpose Analog Computer, Shannon 1936 y(0) = y0, y0(t) = p(y(t)) Polynomial Differential Equation, Graça 2004 t

y1(t)

13 / 24

Example of dynamical system

✓ `

m

g ¨ ✓ + g

` sin(✓) = 0

14 / 24

Example of dynamical system

✓ `

m

g ¨ ✓ + g

` sin(✓) = 0

8 > > < > > : y0

1 = y2

y0

2 = g l y3

y0

3 = y2y4

y0

4 = y2y3

, 8 > > < > > : y1 = ✓ y2 = ˙ ✓ y3 = sin(✓) y4 = cos(✓)

14 / 24

Example of dynamical system

✓ `

m

g ⇥ R R ⇥ R

g `

⇥ ⇥

1

R y1 y2 y3 y4 ¨ ✓ + g

` sin(✓) = 0

8 > > < > > : y0

1 = y2

y0

2 = g l y3

y0

3 = y2y4

y0

4 = y2y3

, 8 > > < > > : y1 = ✓ y2 = ˙ ✓ y3 = sin(✓) y4 = cos(✓)

14 / 24

Example of dynamical system

✓ `

m

g ⇥ R R ⇥ R

g `

⇥ ⇥

1

R y1 y2 y3 y4 ¨ ✓ + g

` sin(✓) = 0

8 > > < > > : y0

1 = y2

y0

2 = g l y3

y0

3 = y2y4

y0

4 = y2y3

, 8 > > < > > : y1 = ✓ y2 = ˙ ✓ y3 = sin(✓) y4 = cos(✓)

Remark on “analog”

Continuous and analogy between circuits/mechanics/ODEs.

14 / 24

Computing with differential equations

Generable functions ⇢ y(0)= y0 y0(x)= p(y(x)) x 2 R f(x) = y1(x) x

y1(x)

Shannon’s notion sin, cos, exp, log, ...

15 / 24

Computing with differential equations

Generable functions ⇢ y(0)= y0 y0(x)= p(y(x)) x 2 R f(x) = y1(x) x

y1(x)

Shannon’s notion sin, cos, exp, log, ... Considered "weak": not Γ and ⇣ Only analytic functions

15 / 24

Computing with differential equations

Generable functions ⇢ y(0)= y0 y0(x)= p(y(x)) x 2 R f(x) = y1(x) x

y1(x)

Shannon’s notion sin, cos, exp, log, ... Considered "weak": not Γ and ⇣ Only analytic functions Computable ⇢y(0)= q(x) y0(t)= p(y(t)) x 2 R t 2 R+ f(x) = lim

t!1 y1(t)

t

f(x) x y1(t)

Modern notion sin, cos, exp, log, Γ, ⇣, ...

15 / 24

Computing with differential equations

Generable functions ⇢ y(0)= y0 y0(x)= p(y(x)) x 2 R f(x) = y1(x) x

y1(x)

Shannon’s notion sin, cos, exp, log, ... Considered "weak": not Γ and ⇣ Only analytic functions Computable ⇢y(0)= q(x) y0(t)= p(y(t)) x 2 R t 2 R+ f(x) = lim

t!1 y1(t)

t

f(x) x y1(t)

Modern notion sin, cos, exp, log, Γ, ⇣, ... Turing powerful [Bournez et al., 2007]

15 / 24

More formally

t

1 1 Yes No

y1(t) y1(t) y1(t) x

16 / 24

More formally

t

1 1 Yes No

y1(t) y1(t) y1(t) x

Theorem (Bournez et al, 2010)

This is equivalent to a Turing machine.

16 / 24

More formally

t

1 1 Yes No

y1(t) y1(t) y1(t) x

Theorem (Bournez et al, 2010)

This is equivalent to a Turing machine. I analog computability theory I purely continuous characterization of classical computability

16 / 24

Can Analog Machines Compute Faster?

Computability discrete Turing machine boolean circuits logic recursive functions lambda calculus quantum analog continuous

Church Thesis

All reasonable models of computation are equivalent.

17 / 24

Can Analog Machines Compute Faster?

Complexity discrete Turing machine boolean circuits logic recursive functions lambda calculus quantum analog continuous > ? ?

Effective Church Thesis

All reasonable models of computation are equivalent for complexity.

17 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC:

Tentative definition

T(x) = ?? y(0) = x y0 = p(y) t

f(x) x y1(t)

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC:

Tentative definition

T(x, µ) = y(0) = x y0 = p(y) t

f(x) x y1(t)

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC:

Tentative definition

T(x, µ) = first time t so that |y1(t) f(x)| 6 eµ y(0) = x y0 = p(y) t

f(x) x y1(t)

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC:

Tentative definition

T(x, µ) = first time t so that |y1(t) f(x)| 6 eµ y(0) = x y0 = p(y) t

f(x) x y1(t)

; z(t) = y(et) t

f(x) x z1(t)

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC:

Tentative definition

T(x, µ) = first time t so that |y1(t) f(x)| 6 eµ y(0) = x y0 = p(y) t

f(x) x y1(t)

; z(t) = y(et) t

f(x) x z1(t)

w(t) = y(eet) t

f(x) x w1(t)

18 / 24

Complexity of analog systems

I Turing machines: T(x) = number of steps to compute on x I GPAC: time contraction problem ! open problem

Tentative definition

T(x, µ) = first time t so that |y1(t) f(x)| 6 eµ y(0) = x y0 = p(y) t

f(x) x y1(t)

; z(t) = y(et) t

f(x) x z1(t)

Something is wrong...

All functions have constant time complexity. w(t) = y(eet) t

f(x) x w1(t)

18 / 24

Time-space correlation of the GPAC

y(0) = q(x) y0 = p(y) t

f(x) q(x) y1(t)

; z(t) = y(et) t

f(x) ˜ q(x) z1(t)

19 / 24

Time-space correlation of the GPAC

y(0) = q(x) y0 = p(y) t

f(x) q(x) y1(t)

; z(t) = y(et) t

f(x) ˜ q(x) z1(t)

extra component: w(t) = et t

w(t)

19 / 24

Time-space correlation of the GPAC

y(0) = q(x) y0 = p(y) t

f(x) q(x) y1(t)

; z(t) = y(et) t

f(x) ˜ q(x) z1(t)

Observation

Time scaling costs “space”. ; Time complexity for the GPAC must involve time and space ! extra component: w(t) = et t

w(t)

19 / 24

Complexity in the analog world

Complexity measure: length of the curve x y(10) = x y(1) Time acceleration: same curve = same complexity !

20 / 24

Complexity in the analog world

Complexity measure: length of the curve x y(10) = x y(1) Time acceleration: same curve = same complexity ! x y(1) ⌧ x y(1) Same time, different curves: different complexity !

20 / 24

Analog complexity

ANALOG-PTIME ANALOG-PR

`(t)

1 1

poly(|w|) w2L w / 2L y1(t) y1(t) y1(t) (w) `(t) f(x) x y1(t)

Theorem

I L 2 PTIME of and only if L 2 ANALOG-PTIME I f : [a, b] ! R computable in polynomial time , f 2 ANALOG-PR I Analog complexity theory based on length I Time of Turing machine , length of the GPAC I Purely continuous characterization of PTIME

21 / 24

Analog complexity

ANALOG-PTIME ANALOG-PR

`(t)

1 1

poly(|w|) w2L w / 2L y1(t) y1(t) y1(t) (w) `(t) f(x) x y1(t)

Theorem

I L 2 PTIME of and only if L 2 ANALOG-PTIME I f : [a, b] ! R computable in polynomial time , f 2 ANALOG-PR I Analog complexity theory based on length I Time of Turing machine , length of the GPAC I Purely continuous characterization of PTIME I Only rational coefficients needed

21 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y

22 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y x y x = y

22 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y x y x y x = y x > y

22 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y x y x y x y x = y x > y x < y

22 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y x y x y x y x = y x > y x < y Output: sign(x y) ?

22 / 24

Does a balance scale compute a function?

Inputs: x, y 2 [0, +1) x y x y x y x y x = y x > y x < y Output: sign(x y) ? Complexity: ???

22 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout Example: ↵ 2 [0, 1] unknown, x programmable Turing machine x ↵ Queries:

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout Example: ↵ 2 [0, 1] unknown, x programmable Turing machine x ↵ Queries: I x = 1

2, T = 1 second ; Yes

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout Example: ↵ 2 [0, 1] unknown, x programmable Turing machine x ↵ Queries: I x = 1

2, T = 1 second ; Yes

I x = 3

4, T = 1 second ; No

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout Example: ↵ 2 [0, 1] unknown, x programmable Turing machine x ↵ Queries: I x = 1

2, T = 1 second ; Yes

I x = 3

4, T = 1 second ; No

I x = 5

8, T = 1 second ; Timeout

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout Example: ↵ 2 [0, 1] unknown, x programmable Turing machine x ↵ Queries: I x = 1

2, T = 1 second ; Yes

I x = 3

4, T = 1 second ; No

I x = 5

8, T = 1 second ; Timeout

I x = 5

8, T = 2 seconds ; Yes

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout x ↵ x ↵ Wheatstone bridge

• r

Brewster’s angle experiment Experiments types: I Two-sided: time

C |x↵|d , Yes/No/Timeout

I One-sided: time

C |x↵|d , Yes/Timeout

I Vanishing: Yes (if x 6= ↵)/Timeout, can test if |x ↵| 6 |x0 ↵|

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

Model: Turing Machine with “physical oracle” Oracle: performs physical experiments with time limit Outcomes: Yes, No, Timeout x ↵ x ↵ Wheatstone bridge

• r

Brewster’s angle experiment Experiments types: I Two-sided: time

C |x↵|d , Yes/No/Timeout

I One-sided: time

C |x↵|d , Yes/Timeout

I Vanishing: Yes (if x 6= ↵)/Timeout, can test if |x ↵| 6 |x0 ↵| Precision: infinite, unbounded, fixed

23 / 24

Physical Oracles (Beggs, Costa, Poças and Tucker)

x ↵ x ↵ Wheatstone bridge

• r

Brewster’s angle experiment Experiments types: I Two-sided: time

C |x↵|d , Yes/No/Timeout

I One-sided: time

C |x↵|d , Yes/Timeout

I Vanishing: Yes (if x 6= ↵)/Timeout, can test if |x ↵| 6 |x0 ↵| Precision: infinite, unbounded, fixed

Theorem (Beggs, Costa, Poças and Tucker)

For a broad class of oracles + PTIME machine, complexity bounded by1 BPP//log?, or P/poly if non-computable analog-digital interface.

1BPP + non-uniform log advise. 23 / 24

Summary

I analog: analogy/continuous, orthogonal meanings I machines vs models: need to distinguish concepts I “Church” thesis: subtle, several variants I hypercomputability: various interpretations I some reasonable models exists: GPAC, equivalent to TM I complexity: difficult to define in general, several approaches I influence of noise on computations x ↵

24 / 24