ARTIFICIAL INTELLIGENCE Machine learning: introduction Lecturer: - - PowerPoint PPT Presentation

ARTIFICIAL INTELLIGENCE

Lecturer: Silja Renooij

Machine learning: introduction

Utrecht University The Netherlands

These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html

INFOB2KI 2019-2020

Outline

• Introduction to machine learning
• Important concepts
• Issues with learning
• (semi)‐supervised approaches:

– discussed in several classes…

2

What is learning?

A computer program is said to learn ‐ from experience E ‐ with respect to some class of tasks T ‐ and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

3

Machine learning in practice

How to acquire a model from data / experience (that can be used for reasoning/ decision making.)

‐ experience E: usually data or other input

‐ class of tasks T: classification, regression, clustering, density estimation, … ‐ performance measure P: e.g. accuracy Offline: learn model prior to use Online: (continue to) learn model while in use

4

What can we learn?

• Parameter values (e.g. probabilities)
• Situation classification
• Action decision
• Structure (e.g. BN graph, decision tree)
• Strategy/policy
• Hidden concepts (e.g. user profiles from clustering)
• …

5

Can we learn (reliably)?

• Depends on properties of application domain:

– Enough data?

• Too many parameters
• Too many values
• Too many possible actions/decisions

– Changing environment – Dependencies between actions

(e.g. shooting and running)

• Depends on properties of the learning method:

– Bias vs. variance

6

Why Learning of Game AI?

The process of learning in games generally implies the adaptation of behavior for opponent players in order to improve performance

• Self‐correction

– Automatically fixing exploits

• Creativity

– Responding intelligently to new situations

• Scalability

– Better entertainment for strong players – Better entertainment for weak players (A user only has fun or learns if performing on his/her own level)

7

• Online – during gameplay

– Adapt to player tactics – Avoid repetition of mistakes – Requirements: computationally cheap, effective, robust, fast learning (Spronck 2004)

• Offline ‐ before the game is released

– Devise new tactics – Discover exploits

Offline vs. Online Learning

8

Types of learning algorithms

Most learning can be seen as discovering the representation of a function

• Supervised learning:

– learn from a set of (input, output) examples

• Semi‐supervised learning:

– learn from partial feedback

• Reinforcement learning:

– learn from experience and gained rewards

• Unsupervised learning:

– find regularities based on statistics (Data Mining)

9

strong supervision weak supervision

Supervised vs Reinforcement

Learning System

Input x from environment Output (based on) h(x) Training Info

The general learning task: learn a model or function h, that approximates the true function f, from a training set. Training info is of following form:

• (x,~f(x)) for supervised learning
• (x, reinforcement signal from environment)

for reinforcement learning

10

Important concepts

Required data/input:

• Training set: to learn from
• Held out / validation set: used in learning phase; for tuning

hyperparameters, or to prevent overfitting

• Test set: to determine performance on; independent; not

considered in learning!

Overfitting and generalization:

• Generalization: learned model should do well on unseen

(test) data

• Overfitting = fitting the training data well, but not

generalizing well

11

Learning from examples

Simplest form: inductive inference

(i.e. learn a function from examples)

• f is the target function (= true function)
• An example is a labelled pair (x, f(x))
• Problem: find a hypothesis function h

– such that h approximates f – given a training set of (possibly noisy!) examples (This is a highly simplified model of real learning:

– ignores prior knowledge – assumes examples are given: “supervised”)

12

Inductive learning method

• Construct/adjust h to agree with f on training set
• E.g., curve fitting:

Can we fit a function through these points?

‐ linear? ‐ quadratic? ‐ higher‐order? Each choice has effect on predictive accuracy (error) How?  inspect bias/variance decomposition of the error

13

Error sources: bias & variance

Consider:

• the true function f(x) that generated the (noisy) data/input
• the learned approximation h(x)

Then, the (supervised) learning method

• is biased, if h(x) systematically differs from f(x)

(on average over different training sets!)  erroneous assumptions in method

• has high variance if h(x) strongly depends on the training set

 method sensitive to fluctuations in training set

14

Bias & variance in curve fitting

Curve fitting:

• What if we fit a linear function?

 high bias, low variance; not flexible

15

Curve fitting

• What if we fit a quadratic function?

 Possibly lower bias, but somewhat higher variance

16

Bias & variance in curve fitting

Curve fitting

• What if we fit a higher‐order polynomial?

 low bias, high variance; very flexible

17

Bias & variance in curve fitting

Curve fitting:

• What if we fit an even higher order function?

 Don’t overdo it: prefer the simplest hypothesis consistent with data, i.e. agreeing with f on all examples (Ockham’s razor)

18

Bias & variance in curve fitting

Bias-Variance Trade-off

If model complexity exceeds optimum, we are overfitting.

• In practice, optimum cannot be found analytically

 choose suitable accuracy measure to minimize total error.

19

Overfitting

Hypothesis h overfits the data if there exists a h' with

• greater error than h over training examples (‘seen’ instances):

errortrain(h’) > errortrain(h)

• but less error than h over entire distribution of instances

(including ‘unseen’ instances):

errortrue(h’) < errortrue(h) Overfitting models is serious problem for all inductive learning methods!

20

Some Machine Learning Techniques

21