ARTIFICIAL INTELLIGENCE Machine learning: introduction Lecturer: - PowerPoint PPT Presentation

Utrecht University INFOB2KI 2019-2020 The Netherlands ARTIFICIAL INTELLIGENCE Machine learning: introduction Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html

Outline  Introduction to machine learning  Important concepts  Issues with learning  (semi)‐supervised approaches: – discussed in several classes… 2

What is learning? A computer program is said to learn ‐ from experience E ‐ with respect to some class of tasks T ‐ and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. 3

Machine learning in practice How to acquire a model from data / experience (that can be used for reasoning/ decision making.) ‐ experience E: usually data or other input ‐ class of tasks T: classification, regression, clustering, density estimation, … ‐ performance measure P: e.g. accuracy Offline: learn model prior to use Online: (continue to) learn model while in use 4

What can we learn?  Parameter values (e.g. probabilities)  Situation classification  Action decision  Structure (e.g. BN graph, decision tree)  Strategy/policy  Hidden concepts (e.g. user profiles from clustering)  … 5

Can we learn (reliably)?  Depends on properties of application domain: – Enough data? • Too many parameters • Too many values • Too many possible actions/decisions – Changing environment – Dependencies between actions (e.g. shooting and running)  Depends on properties of the learning method: – Bias vs. variance 6

Why Learning of Game AI? The process of learning in games generally implies the adaptation of behavior for opponent players in order to improve performance  Self‐correction – Automatically fixing exploits  Creativity – Responding intelligently to new situations  Scalability – Better entertainment for strong players – Better entertainment for weak players (A user only has fun or learns if performing on his/her own level) 7

Offline vs. Online Learning  Online – during gameplay – Adapt to player tactics – Avoid repetition of mistakes – Requirements: computationally cheap, effective, robust, fast learning (Spronck 2004)  Offline ‐ before the game is released – Devise new tactics – Discover exploits 8

Types of learning algorithms Most learning can be seen as discovering the representation of a function strong supervision  Supervised learning: – learn from a set of (input, output) examples  Semi‐supervised learning: weak supervision – learn from partial feedback  Reinforcement learning: – learn from experience and gained rewards  Unsupervised learning: – find regularities based on statistics (Data Mining) 9

Supervised vs Reinforcement Training Info Input x from Learning Output (based on) h(x) environment System The general learning task: learn a model or function h, that approximates the true function f , from a training set. Training info is of following form: • (x,~f(x)) for supervised learning • (x, reinforcement signal from environment) for reinforcement learning 10

Important concepts Required data/input:  Training set: to learn from  Held out / validation set: used in learning phase; for tuning hyperparameters, or to prevent overfitting  Test set: to determine performance on; independent; not considered in learning! Overfitting and generalization:  Generalization: learned model should do well on unseen (test) data  Overfitting = fitting the training data well, but not generalizing well 11

Learning from examples Simplest form: inductive inference (i.e. learn a function from examples)  f is the target function (= true function)  An example is a labelled pair ( x , f(x) )  Problem: find a hypothesis function h – such that h approximates f – given a training set of (possibly noisy!) examples (This is a highly simplified model of real learning: – ignores prior knowledge – assumes examples are given: “supervised”) 12

Inductive learning method  Construct/adjust h to agree with f on training set  E.g., curve fitting: Can we fit a function through these points? ‐ linear? ‐ quadratic? ‐ higher‐order? Each choice has effect on predictive accuracy (error) How?  inspect bias/variance decomposition of the error 13

Error sources: bias & variance Consider:  the true function f(x) that generated the (noisy) data/input  the learned approximation h(x) Then, the (supervised) learning method  is biased, if h(x) systematically differs from f(x) (on average over different training sets!)  erroneous assumptions in method  has high variance if h(x) strongly depends on the training set  method sensitive to fluctuations in training set 14

Bias & variance in curve fitting Curve fitting:  What if we fit a linear function?  high bias, low variance; not flexible 15

Bias & variance in curve fitting Curve fitting  What if we fit a quadratic function?  Possibly lower bias, but somewhat higher variance 16

Bias & variance in curve fitting Curve fitting  What if we fit a higher‐order polynomial?  low bias, high variance; very flexible 17

Bias & variance in curve fitting Curve fitting:  What if we fit an even higher order function?  Don’t overdo it: prefer the simplest hypothesis consistent with data, i.e. agreeing with f on all examples (Ockham’s razor) 18

Bias-Variance Trade-off If model complexity exceeds optimum, we are overfitting. • In practice, optimum cannot be found analytically  choose suitable accuracy measure to minimize total error. 19

Overfitting Hypothesis h overfits the data if there exists a h' with • greater error than h over training examples (‘seen’ instances) : error train (h’) > error train (h) • but less error than h over entire distribution of instances (including ‘unseen’ instances): error true (h’) < error true (h) Overfitting models is serious problem for all inductive learning methods! 20

Some Machine Learning Techniques 21