Learning Equivalence Structures Luca San Mauro (Vienna University of - - PowerPoint PPT Presentation

Learning Equivalence Structures

Luca San Mauro (Vienna University of Technology) Logic Colloquium 2018

joint work with Ekaterina Fokina and Timo Koetzing

Computational Learning Theory

Computational Learning Theory (CLT) is a vast research program that comprises different models of learning in the limit. It deals with the question of how a learner, provided with more and more data about some environment, is eventually able to achieve systematic knowledge about it.

1

Computational Learning Theory

Computational Learning Theory (CLT) is a vast research program that comprises different models of learning in the limit. It deals with the question of how a learner, provided with more and more data about some environment, is eventually able to achieve systematic knowledge about it.

• (Gold, 1967): language identification.

1

Computational Learning Theory

Computational Learning Theory (CLT) is a vast research program that comprises different models of learning in the limit. It deals with the question of how a learner, provided with more and more data about some environment, is eventually able to achieve systematic knowledge about it.

• (Gold, 1967): language identification.

More recently researchers applied the machinery of CLT to algebraic structures:

1

Computational Learning Theory

Computational Learning Theory (CLT) is a vast research program that comprises different models of learning in the limit. It deals with the question of how a learner, provided with more and more data about some environment, is eventually able to achieve systematic knowledge about it.

• (Gold, 1967): language identification.

More recently researchers applied the machinery of CLT to algebraic structures:

• (Stephan, Ventsov, 2001): Learning ring ideals of

commutative rings.

• (Harizanov, Stephan, 2002): Learning subspaces of V∞.

1

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

2

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

• A learner M is a total function which takes for its inputs finite

substructures of a given structure S from K.

2

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

• A learner M is a total function which takes for its inputs finite

substructures of a given structure S from K.

• If M(Si) ↓= n, for finite Si ⊆ S, then n represents M’s

conjecture as to an index for S in the above enumeration.

2

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

• A learner M is a total function which takes for its inputs finite

substructures of a given structure S from K.

• If M(Si) ↓= n, for finite Si ⊆ S, then n represents M’s

conjecture as to an index for S in the above enumeration.

• M InfEx∼

=-learns S if, for all T ∼

= S, there exists n ∈ ω such that T ∼ = Cn and M(T i) ↓= n, for all but finitely many i.

2

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

• A learner M is a total function which takes for its inputs finite

substructures of a given structure S from K.

• If M(Si) ↓= n, for finite Si ⊆ S, then n represents M’s

conjecture as to an index for S in the above enumeration.

• M InfEx∼

=-learns S if, for all T ∼

= S, there exists n ∈ ω such that T ∼ = Cn and M(T i) ↓= n, for all but finitely many i.

• A family of structures A is InfEx∼

=-learnable if there is M that

learns all A ∈ A.

2

Our framework

• Let K be a class of structures with some uniform effective

enumeration {Ci}i∈ω of the computable structures from K, up to isomorphism.

• A learner M is a total function which takes for its inputs finite

substructures of a given structure S from K.

• If M(Si) ↓= n, for finite Si ⊆ S, then n represents M’s

conjecture as to an index for S in the above enumeration.

• M InfEx∼

=-learns S if, for all T ∼

= S, there exists n ∈ ω such that T ∼ = Cn and M(T i) ↓= n, for all but finitely many i.

• A family of structures A is InfEx∼

=-learnable if there is M that

learns all A ∈ A.

• InfEx∼

=(K) denotes the class of families of K-structures that

are InfEx∼

=-learnable. 2

Our framework, continued

Our notation comes from classical CLT: Inf (short for informant) means that we receive both positive and negative information about S; Ex (short for explanatory) means that M shall converge

• n a single input for S.

At the end, we will discuss learning classes obtained by choosing natural alternatives of Inf, Ex, and ∼ =.

3

Our framework, continued

Our notation comes from classical CLT: Inf (short for informant) means that we receive both positive and negative information about S; Ex (short for explanatory) means that M shall converge

• n a single input for S.

At the end, we will discuss learning classes obtained by choosing natural alternatives of Inf, Ex, and ∼ =. Remark: our model shares many analogies with the First-order Framework introduced in (Martin, Osherson, 1998).

3

Equivalence structures

• Denote by E the class of equivalence structures. Our main

focus is on InfEx∼

=(E). 4

Equivalence structures

• Denote by E the class of equivalence structures. Our main

focus is on InfEx∼

=(E).

• (Downey, Melnikov, Ng, 2016): there is a Friedberg

enumeration of computable equivalence structures.

4

Equivalence structures

• Denote by E the class of equivalence structures. Our main

focus is on InfEx∼

=(E).

• (Downey, Melnikov, Ng, 2016): there is a Friedberg

enumeration of computable equivalence structures.

• A non-Friedberg one is of course much more easy to be

defined, e.g., for all n, let the size of the En-classes be the cardinality of the columns of Wn.

4

Example of learnability, 1

A B

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = A

S

5

Example of learnability, 1 A B

M(S) = B

S

5

Example of learnability, 2

A B

6

Example of learnability, 2 A B S

M(S) = A

6

Example of learnability, 2 A B S

M(S) = A

6

Example of learnability, 2 A B S

M(S) = A

6

Example of learnability, 2 A B S

M(S) = A

6

Example of learnability, 2 A B S

M(S) = B

6

Example of learnability, 2 A B S

M(S) = B

6

Example of learnability, 2 A B S

M(S) = B

6

Example of learnability, 2 A B S

M(S) = A

6

Example of nonlearnability, 1

A B

7

Example of nonlearnability, 1

Strategy

• Assume that M learns {A, B},
• Construct by stages a structure

S ∈ {T : T ∼ = A ∨ T ∼ = B} such that M fails to learn S.

7

Example of nonlearnability, 1 A B S ∼ = B

M(S) = A

7

Example of nonlearnability, 1 A B S ∼ = B

M(S) = A

7

Example of nonlearnability, 1 A B S ∼ = B

M(S) = B

7

Example of nonlearnability, 1 A B S ∼ = B

M(S) = B

7

Example of nonlearnability, 1 A B S ∼ = A

M(S) = B

7

Example of nonlearnability, 1 A B S ∼ = A

M(S) = B

7

Example of nonlearnability, 1 A B S ∼ = A

M(S) = B

7

Example of nonlearnability, 1 A B S ∼ = A

M(S) = A

7

Example of nonlearnability, 1 A B S ∼ = A

M(S) = A

7

Example of nonlearnability, 1 A B S ∼ = B

M(S) = A

7

Example of nonlearnability, 2

Recall that the character cA of A is cA = {k, i : A has i equivalence classes of size k}. Define A = {Ai}i∈ω+ such that, for all Ai’s, cAi = {k, 1 : k = i}. Proposition A / ∈ InfEx∼

=(E). 8

Finite separability

• S is a limit of a finite family A if there is A ∈ A such that

A ֒ →fin S ∧ A ∼ = S,

9

Finite separability

• S is a limit of a finite family A if there is A ∈ A such that

A ֒ →fin S ∧ A ∼ = S, and cS ⊆ cA.

9

Finite separability

• S is a limit of a finite family A if there is A ∈ A such that

A ֒ →fin S ∧ A ∼ = S, and cS ⊆ cA.

• S is a limit of an infinite family A if

(∀A ∈ A)(A ֒ →fin S ∧ A ∼ = S)

9

Finite separability

• S is a limit of a finite family A if there is A ∈ A such that

A ֒ →fin S ∧ A ∼ = S, and cS ⊆ cA.

• S is a limit of an infinite family A if

(∀A ∈ A)(A ֒ →fin S ∧ A ∼ = S) and cS ⊆

• B∈Pinf (A)
• B∈B

cB.

9

Finite separability

• S is a limit of a finite family A if there is A ∈ A such that

A ֒ →fin S ∧ A ∼ = S, and cS ⊆ cA.

• S is a limit of an infinite family A if

(∀A ∈ A)(A ֒ →fin S ∧ A ∼ = S) and cS ⊆

• B∈Pinf (A)
• B∈B

cB.

• A is finitely separable if, for all B ⊆ A, B has no limits in A.

9

Characterization of InfEx∼

=(E)

Theorem If A is a family of equivalence structures, then A is InfEx∼

=-learnable ⇔ A is finitely separable. 10

Characterization of InfEx∼

=(E)

Theorem If A is a family of equivalence structures, then A is InfEx∼

=-learnable ⇔ A is finitely separable.

Corollary

• 1. If A /

∈ InfEx∼

=(E) and A/∼ = is finite, then there exists

(A, B) ⊆ A such that (A, B) / ∈ InfEx∼

=(E). 10

Characterization of InfEx∼

=(E)

Theorem If A is a family of equivalence structures, then A is InfEx∼

=-learnable ⇔ A is finitely separable.

Corollary

• 1. If A /

∈ InfEx∼

=(E) and A/∼ = is finite, then there exists

(A, B) ⊆ A such that (A, B) / ∈ InfEx∼

=(E).

• 2. There is A /

∈ InfEx∼

=(E) such that, for all B ⊆ A, if B/∼ = is

finite, then B ∈ InfEx∼

=(E). 10

Three corollaries

What happens if we restrict to learners of some fixed complexity?

11

Three corollaries

What happens if we restrict to learners of some fixed complexity? Two sources of complexity to learn A:

• 1. enumerating A,
• 2. learning A (given an enumeration of A).

11

Three corollaries

What happens if we restrict to learners of some fixed complexity? Two sources of complexity to learn A:

• 1. enumerating A,
• 2. learning A (given an enumeration of A).

Definition A ∈ 0(α)-InfEx∼

=(K) if there is a Turing functional Ψe such that

Ψ(0(α)⊕d)

e

InfEx∼

=-learns A, for all d that enumerates A. 11

Three corollaries

What happens if we restrict to learners of some fixed complexity? Two sources of complexity to learn A:

• 1. enumerating A,
• 2. learning A (given an enumeration of A).

Definition A ∈ 0(α)-InfEx∼

=(K) if there is a Turing functional Ψe such that

Ψ(0(α)⊕d)

e

InfEx∼

=-learns A, for all d that enumerates A.

Theorem 0-InfEx∼

=(E) InfEx∼ =(E) 11

Three corollaries

What happens if we restrict to learners of some fixed complexity? Two sources of complexity to learn A:

• 1. enumerating A,
• 2. learning A (given an enumeration of A).

Definition A ∈ 0(α)-InfEx∼

=(K) if there is a Turing functional Ψe such that

Ψ(0(α)⊕d)

e

InfEx∼

=-learns A, for all d that enumerates A.

Theorem 0-InfEx∼

=(E) InfEx∼ =(E) = 0′-InfEx∼ =(E) 11

Other learnability classes, I

We obtain different learnability classes by replacing the main ingredients of InfEx∼

= with natural alternatives: 12

Other learnability classes, I

We obtain different learnability classes by replacing the main ingredients of InfEx∼

= with natural alternatives:

• 1. Inf → Txt: in Txt-learning (short for text) the learner

receives only positive information of the structure to be learnt.

12

Other learnability classes, I

We obtain different learnability classes by replacing the main ingredients of InfEx∼

= with natural alternatives:

• 1. Inf → Txt: in Txt-learning (short for text) the learner

receives only positive information of the structure to be learnt.

• 2. ∼

= → E, where E is some nice equivalence relations relation between elements of K, such as bi-embeddability (≈), computable isomorphism (∼ =0), computable bi-embeddability (≈0) – and so forth.

12

Other learnability classes, I

We obtain different learnability classes by replacing the main ingredients of InfEx∼

= with natural alternatives:

• 1. Inf → Txt: in Txt-learning (short for text) the learner

receives only positive information of the structure to be learnt.

• 2. ∼

= → E, where E is some nice equivalence relations relation between elements of K, such as bi-embeddability (≈), computable isomorphism (∼ =0), computable bi-embeddability (≈0) – and so forth.

• 3. Ex → BC: in BC-learning (short for behaviourally correct)

the learner is allowed to change its mind infinitely many times as far as almost all its conjectures lie in the same E-class (with E defined as in 2.).

12

Other learnability classes, II InfEx∼

=0

InfEx∼

=

InfEx≈0 InfEx≈

• 13

Other learnability classes, II TxtEx∼

=0

TxtEx∼

=

TxtEx≈0 TxtEx≈

• 13

Further directions

• 1. Apply the framework to other classes K.
• 2. Investigate BC-learnability.

14

Thank you!