Solving Mathematical Puzzles: a Deep Reasoning Challenge for - - PowerPoint PPT Presentation

Solving Mathematical Puzzles: a Deep Reasoning Challenge for Intelligent Agents

First URANIA Workshop, November 28th, 2016 AIxIA 2016 Conference, Genova, November 28th—December 1st, 2016

• F. Chesani, P. Mello, M. Milano

Premises... (1/2)

• "Understanding the nature of intelligence" is still an open

problem in AI research

• In the past, games intended as symptomps of human

intelligence

• A number of AI challenges have been based on games...

– Turing's imitation game – Chess – Jeopardy – Robot Soccer World Cup – ...

Premises... (2/2)

Recently, a number of achievements in AI

• AlphaGO
• SIRI, Cortana, Echo, etc.
• Deep Blue and Watson
• GoogLeNet for image recognition

Generally speaking, (some) AI techniques are reaching a good maturity level ... ... however, the integration of different techniques toward autonomy is somehow neglected ...

What's next?

An idea for the next AI challenge

Let's focus on mathematical puzzles

• They are still "games"
• Described using different media requires

multimodal comprehension

• They might involve many (if not all) AI

techniques

• They require autonomy
• They require "deep reasoning"

Mathematical Puzzles

Mathematical puzzles are recreational games where:

• a single human player is challenged with a problem;
• the problem is described by text and/or by images.

To solve them, a human player uses:

• understanding
• intuition
• common sense
• simple logical knowledge
• simple mathematical/geometry knowledge
• causal relations and reasoning
• temporal reasoning
• ...

Deep Reasoning (?)

• vs. / with

Deep Learning ?

Mathematical Puzzles – Three Friends

Jacob, Lucy and Frank are three friends. All together they are 28 years old. In how many years they will be together 37 years old?

Explicit knowledge (to be extracted from the description):

• There are 3 friends.
• All together here means the sum operator.
• All together refers to the age of each friend.
• The sum of their ages now is 28.
• Ages and years are natural numbers.
• How many refers to a quantity X of years.
• In X years the sum of their ages will be 37.
• We have to find X.

Hidden knowledge:

• Time will flow in the same manner for all the three friends

Mathematical Puzzles – other examples...

How many triangles are in the Figure? Given the picture, which is bigger: the total area of blue or the total area of white?

Mathematical Puzzles – Knights and Knaves

There is an island where every inhabitants is either a knight

• r a knave (exclusive or). Knights always tell the truth,

while knaves always lie. You are a tourist just arrived in the island, and you met two inhabitant A and B. A says “I am a knave, or B is a knight”. What are A and B?

• Logic-based…
• … which logic? Propositional logic?

A Challenge for AI Research

By the middle of the 21st century, (a team of) fully autonomous agent(s) shall win a mathematical puzzle competition against primary school students, winners of the most recent competitions.

Stress on:

• autonomy
• multi-modal input
• modelling AND reasoning tasks
• integration of different reasoning techniques
• strategy for winning the competition

A Challenge for AI Research

By the middle of the 21st century, (a team of) fully autonomous agent(s) shall win a mathematical puzzle competition against primary school students, winners of the most recent competitions.

• Before solving a puzzle, the autonomous entity should also decide

WHICH solving method/technique should employ

• When taking part to the context, there is also the strategy:

– Puzzles have different points (depending on the difficulty): which one to choose? – The competition is time-limited… again, which puzzle to tackle first?

Few discussion points...

A possible, but debatable, process:

• 1. Understand text and images
• 2. Identify a modeling and a solving technique
• 3. Identify problem components and possible hidden knowledge
• 4. Frame the problem model
• 5. Solve the problem

Few discussion points...

In the human reasoning process the initial phases are merged together, and no clear distinction can be made...

• ... is it the same for an autonomous agent?

To which extent each phase has/might/should influence the other phases?

• ... e.g., NLP understanding should be influenced by the chosen

modeling technique?

Few discussion points...

Should we consider puzzles that are gameable?

• ... e.g., in the puzzle of comparing the blue area vs.

white area, it would suffice to count the number of pixels of each color... no understanding of figures is required...

• ... the same puzzle does not require any geometric

knowledge, and can be solved counted the squares... No sophisticated AI/CS techniques are required to solve the puzzles (indeed, they are usually solved by primary school students)

• ... should we think puzzles more "machine-like"? why?

Understanding of puzzles could open up to novel human machine interaction ways... ?

Few discussion points...

Other discussion points:

• Means for evaluating the challenge… ?
• Going beyond the Turing Test… ?
• Which impact on NLP techniques ?
• Multi-modal information extraction… ?
• Common-sense knowledge and deep understanding
• Machine Learning for Knowledge Extraction and Problem

Solving

• Special vs. General Problem Solvers: the “big switch

statement”…

• Computational Thinking vs. AI Thinking…

Notice that...

A number of similar initiatives are currently being investigated:

• Aristo Project (Allen Institute): focus on passing math and science

elementary tests

• Euclid Project (Allen Institute): high school math and geometry

problems

• GEOS: School Admission Problems (SAT)
• ARIS: Arithmethic Word Problems
• TODAI Robot Project: Getting a high score at the Japan National

Test for University Admission

• At the recent NAACL/TACL conference a number of contributes on

related topics

Federico Chesani, Paola Mello, Michela Milano

Dipartimento di Informatica – Scienza e Ingegneria

Thank you for the attention!!! Questions?