MACHINE LEARNING Liviu Ciortuz Department of CS, University of Ia - - PowerPoint PPT Presentation

▶
machine learning
SMART_READER_LITE
LIVE PREVIEW

MACHINE LEARNING Liviu Ciortuz Department of CS, University of Ia - - PowerPoint PPT Presentation

Apr 03, 2024 99 likes •359 views

0. MACHINE LEARNING Liviu Ciortuz Department of CS, University of Ia si, Rom ania 1. What is Machine Learning? ML studies algorithms that improve with experience. learn from Tom Mitchells Definition of the [

slide-1
SLIDE 1

MACHINE LEARNING

Liviu Ciortuz Department of CS, University of Ia¸ si, Romˆ ania

0.

slide-2
SLIDE 2

What is Machine Learning?

  • ML studies algorithms that improve with
  • learn from

experience.

Tom Mitchell’s Definition of the [general] learning problem: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance

  • n tasks in T, as measured by P, improves with experience E.”
  • Examples of [specific] learning problems (see next slide)
  • [Liviu Ciortuz:] ML is data-driven programming
  • [Liviu Ciortuz:] ML gathers a number of well-defined sub-

domains/disciplines, each one of them aiming to solve in its

  • wn way the above-formulated [general] learning problem.

1.

slide-3
SLIDE 3

What is Machine Learning good for?

  • natural language (text & speech) processing
  • genetic sequence analysis
  • robotics
  • customer (financial risc) evaluation
  • terrorist threat detection
  • compiler optimisation
  • semantic web
  • computer security
  • software engineering
  • computer vision (image processing)
  • etc.

2.

slide-4
SLIDE 4

Related courses at FII

  • Genetic Algorithms
  • Artificial Neural Networks
  • Probabilistic programming
  • Special Chapters of Machine Learning
  • Special Chapters of Artificial Neural Networks
  • Data Mining
  • Nature-inspired computing methods
  • Big Data Analytics
  • Image Processing
  • Computer Vision
  • Bioinformatics

3.

slide-5
SLIDE 5

A multi-domain view

Intelligence Artificial (concept learning) Algorithms Mathematics Statistics (model fitting) Machine Learning Learning Statistical Pattern Recognition Mining Data Engineering Database Systems (Knowledge Discovery in Databases)

4.

slide-6
SLIDE 6

The Machine Learning Undergraduate Course: Plan

  • 0. Introduction to Machine Learning (T. Mitchell, ch. 1)
  • 1. Probabilities Revision (Ch. Manning & H. Sch¨

utze, ch. 2)

  • 2. Decision Trees (T. Mitchell, ch. 3)
  • 3. Bayesian Learning (T. Mitchell, ch. 6)

[and the relationship with Logistic Regression]

  • 4. Instance-based Learning (T. Mitchell, ch. 8)
  • 5. Clustering Algorithms (Ch. Manning & H. Sch¨

utze, ch. 14)

5.

slide-7
SLIDE 7

The Machine Learning Master Course:

Tentative Plan

  • 1. Probabilities Revision (Ch. Manning & H. Sch¨

utze, ch. 2)

  • 2. Parameter estimation for probablistic distributions

(see Estimating Probabilities, additional chapter to T. Mitchell’s book, 2016)

  • 3. Decision Trees: Boosting
  • 4. Gaussian Bayesian Learning
  • 5. The EM algorithmic schemata (T. Mitchell, ch. 6.12)
  • 6. Support Vector Machines (N. Cristianini & J. Shawe-Taylor, 2000)
  • 7. Hidden Markov Models (Ch. Manning & H. Sch¨

utze, ch. 9)

  • 8. Computational Learning Theory (T. Mitchell, ch. 7)

6.

slide-8
SLIDE 8

Bibliography

  • 0. “Exercit

¸ii de ˆ ınv˘ at ¸are automat˘ a”

  • L. Ciortuz, A. Munteanu E. B˘

ad˘ ar˘ au. Ia¸ si, Romania, 2020 www.info.uaic.ro/∼ciortuz/ML.ex-book/book.pdf

  • 1. “Machine Learning”

Tom Mitchell. McGraw-Hill, 1997

  • 2. “The Elements of Statistical Learning”

Trevor Hastie, Robert Tibshirani, Jerome Friedman. Springer, 2nd ed. 2009

  • 3. “Machine Learning – A Probabilistic Perspective”

Kevin Murphy, MIT Press, 2012

  • 4. “Pattern Recognition and Machine Learning”

Christopher Bishop. Springer, 2006

  • 5. “Foundations of Statistical Natural Language Processing”

Christopher Manning, Hinrich Sch¨

  • utze. MIT Press, 2002

7.

slide-9
SLIDE 9

A general schema for machine learning methods

test/generalization data predicted classification algorithm machine learning model data training data

“We are drawning in information but starved for knowledge.”

John Naisbitt, “Megatrends” book, 1982

8.

slide-10
SLIDE 10

Basic ML Terminology

  • 1. instance x, instance set X

concept c ⊆ X, or c : X → {0, 1} example (labeled instance): x, c(x); positive examples, neg. examples

  • 2. hypotheses h : X → {0, 1}

hypotheses representation language hypotheses set H hypotheses consistent with the concept c: h(x) = c(x), ∀ example x, c(x) version space

  • 3. learning = train + test

supervised learning (classification), unsupervised learning (clustering)

  • 4. errorh = | {x ∈ X, h(x) = c(x)} |

training error, test error accuracy, precision, recall

  • 5. validation set, development set

n-fold cross-validation, leave-one-out cross-validation

  • verfitting

9.

slide-11
SLIDE 11

The Inductive Learning Assumption

Any hypothesis found to conveniently approximate the target function over a sufficiently large set of training examples will also conveniently approximate the target function

  • ver other unobserved examples.

10.

slide-12
SLIDE 12

Inductive Bias

Consider

  • a concept learning algorithm L
  • the instances X, and the target concept c
  • the training examples Dc = {x, c(x)}.
  • Let L(xi, Dc) denote the classification assigned to the instance xi by L

after training on data Dc.

Definition:

The inductive bias of L is any minimal set of assertions B such that (∀xi ∈ X)[(B ∨ Dc ∨ xi) ⊢ L(xi, Dc)] for any target concept c and corresponding training examples Dc. (A ⊢ B means A logically entails B)

11.

slide-13
SLIDE 13

Inductive systems can be modelled by equivalent deductive systems

12.

slide-14
SLIDE 14

Evaluation measures in Machine Learning

tn c h fn fp tp

tp − true positives fp − false positives tn − true negatives fn − false negatives accuracy: Acc = tp + tn tp + tn + fp + fn precision: P = tp tp + fp recall (or: sensitivity): R = tp tp + fn F-measure: F = 2 P × R P+R specificity: Sp = tn tn + fp follout: = fp tn + fp Mathew’s Correlation Coefficient: MCC = tp × tn − fp × fn

  • (tp + fp)×(tn + fn)×(tp + fn)×(tn + fp)

13.

slide-15
SLIDE 15

Lazy learning vs. eager learning algorithms

Eager: generalize before seeing query

  • ID3, Backpropagation, Naive Bayes, Radial basis function net-

works, . . .

  • Must create global approximation

Lazy: wait for query before generalizing

  • k-Nearest Neighbor, Locally weighted regression, Case based rea-

soning

  • Can create many local approximations

Does it matter? If they use the same hypothesis space H, lazy learners can represent more complex functions. E.g., a lazy Backpropagation algorithm can learn a NN which is dif- ferent for each query point, compared to the eager version of Back- propagation.

14.

slide-16
SLIDE 16

Who is Liviu Ciortuz?

  • Diploma (maths and CS) from UAIC, Ia¸

si, Romania, 1985 PhD in CS from Universit´ e de Lille, France, 1996

  • programmer:

Bac˘ au, Romania (1985-1987)

  • full-time researcher:

Germany (DFKI, Saarbr¨ ucken, 1997-2001), UK (Univ. of York and Univ. of Aberystwyth, 2001-2003), France (INRIA, Rennes, 2012-2013)

  • assistant, lecturer and then associate professor:
  • Univ. of Iasi, Romania (1990-1997, 2003-2012, 2013-today)

15.

slide-17
SLIDE 17

ADDENDA

“...colleagues at the Computer Science department at Saarland University have a strong conviction, that nothing is as practical as a good theory.”

Reinhard Wilhelm, quoted by Cristian Calude, in The Human Face of Computing, Imperial College Press, 2016

16.

slide-18
SLIDE 18

“Mathematics translates concepts into formalisms and applies those formalisms to derive insights that are usually NOT amenable to a LESS formal analysis.” J¨ urgen Jost, Mathematical Concepts, Springer, 2015

17.

slide-19
SLIDE 19

“Mathematics is a journey that must be shared, and by sharing our own journey with

  • thers, we, together, can change the world.”

“Through the power of mathematics, we can explore the uncertain, the counterintuitive, the invisible; we can reveal order and beauty, and at times transform theories into practi- cal objects, things or solutions that you can feel, touch or use.”

Cedric Villani, winner of the Fields prize, 2010

  • cf. http://www.bbc.com/future/sponsored/story/20170216-inside-the-mind-of-a-mathematician, 15.03.2017

xxx

18.

slide-20
SLIDE 20

ADMINISTRATIVIA

19.

slide-21
SLIDE 21

Teaching assistants for the ML undergraduate course 2020 (fall semester)

  • Conf. dr. Anca Ignat (. . . Image processing)

https://profs.info.uaic.ro/∼ancai/ML/

  • Conf. dr. Adrian Z˘

alinescu (. . . Probabilities and Statistics) https://profs.info.uaic.ro/∼adrian.zalinescu/ML.html

  • Sebastian Ciobanu (PhD student)

www.seminarul.ml

  • S

¸tefan Pant ¸iru (MSc)

  • S

¸tefan Matcovici (MSc)

  • Cosmina Asofiei (MSc)

20.

slide-22
SLIDE 22

Grading standards for the ML undergraduate course 2020 Obiectiv: ˆ Inv˘ at ¸are pe tot parcursul semestrului!

Minim: 2p Minim: 2p Minim: 1.25p Minim: 1.25p Prezenta la seminar: obligatorie! Penalizare: 0.2p pentru fiecare absenta de la a doua incolo! Pentru promovare: Nota >= 4.5 T2 S2 S1 T1

Punctaj

Nota = (8 + S1 + S2 + T1 + T2) / 4 <=> S1 + S2 + T1 + T2 >= 10 Test: 6p Test: 6p Seminar: 10p Seminar: 10p Prezenta la curs: recomandata!

21.

slide-23
SLIDE 23

REGULI generale pentru cursul de ˆ Inv˘ at ¸are automat˘ a de la licent ¸˘ a

Regulile de organizare a cursului de ˆ Inv˘ at ¸are Automat˘ a (engl., Machine Learning, ML),

  • sem. I, sunt specificate ˆ

ın fi¸ sa disciplinei http://profs.info.uaic.ro/∼ciortuz/fisa-disciplinei.RO.pdf

  • Bibliografie minimal˘

a: vezi slide #6

  • Planificarea materiei, pentru fiecare s˘

apt˘ amˆ an˘ a (curs + seminar): http://profs.info.uaic.ro/∼ciortuz/what-you-should-know.pdf

  • Prezent

¸a la curs: recomandat˘ a!

  • Regula 0: Prezent

¸a la seminar: obligatorie! Pentru fiecare absent ¸˘ a la seminar, ˆ ıncepˆ and de la a doua absent ¸˘ a ˆ ıncolo, se aplic˘ a o penalizare/depunctare de 0.1 puncte din S1, respectiv din S2. (Vezi formula de notare.) Regulile se aplic˘ a inclusiv student ¸ilor reˆ ınmatriculat ¸i.

  • Este posibil ca s˘

apt˘ amˆ anal se t ¸inem un seminar suplimentar, destinat pentru acei student ¸i care sunt foarte interesat ¸i de acest domeniu. (Vedet ¸i sect ¸iunile “Advanced issues” din documentul http://profs.info.uaic.ro/∼ciortuz/what-you-should-know.pdf.) 22.

slide-24
SLIDE 24

REGULI generale pentru cursul de ˆ Inv˘ at ¸are automat˘ a (cont.)

Regula 1: Pentru seminarii, nu se admit mut˘

ari ale student ¸ilor de la o grup˘ a la alta, decˆ at ˆ ın cadrul grupelor care au acela¸ si asistent / profesor responsabil de seminar.

Regula 2: Nu se fac echival˘

ari de punctaje pentru student ¸ii care nu au promovat cursul ˆ ın anii precedent ¸i.

Regula 3: Profesorul responsabil pentru acest curs, Liviu Ciortuz,

NU va r˘ aspunde la email-uri care pun ˆ ıntreb˘ ari pentru care raspunsul a fost deja dat – fie ˆ ın aceste slide-uri, – fie pe site-ul Piazza dedicat acestui curs: https://piazza.com/info.uaic.ro/fall2020/fiiml2020f/home, (care este ment ¸ionat ¸ si pe pagina profesorului: https://profs.info.uaic.ro/∼ciortuz/) – fie la curs.

Recomandare important˘ a (1) La fiecare curs ¸

si seminar, student ¸ii vor avea culegerea de Exercit ¸ii de ˆ ınv˘ at ¸are automat˘ a (de L. Ciortuz et al) — v˘ a recomand˘ am s˘ a imprimat ¸i capitolele Clasificare bayesian˘ a, ˆ Inv˘ at ¸are bazat˘ a pe memorare, Arbori de decizie ¸ si Clusterizare — ¸ si eventual slide-urile pe care le-am indicat ˆ ın slide-ul precedent.

Recomandare important˘ a (2) Consultat

¸i s˘ apt˘ amˆ anal documentul http://profs.info.uaic.ro/∼ciortuz/what-you-should-know.pdf 23.

slide-25
SLIDE 25

REGULI generale pentru cursul de ˆ Inv˘ at ¸are automat˘ a (cont.) de la licent ¸˘ a

  • Slide-uri de imprimat (ˆ

ın aceast˘ a ordine ¸ si, de preferat, COLOR): http://profs.info.uaic.ro/∼ciortuz/SLIDES/foundations.pdf https://profs.info.uaic.ro/∼ciortuz/ML.ex-book/SLIDES/ML.ex-book.SLIDES.ProbStat.pdf https://profs.info.uaic.ro/∼ciortuz/ML.ex-book/SLIDES/ML.ex-book.SLIDES.DT.pdf https://profs.info.uaic.ro/∼ciortuz/ML.ex-book/SLIDES/ML.ex-book.SLIDES.Bayes.pdf https://profs.info.uaic.ro/∼ciortuz/ML.ex-book/SLIDES/ML.ex-book.SLIDES.IBL.pdf https://profs.info.uaic.ro/∼ciortuz/ML.ex-book/SLIDES/ML.ex-book.SLIDES.Cluster.pdf (Atent ¸ie: acest set de slide-uri poate fi actualizat pe parcursul semestrului!)

  • De imprimat (ALB-NEGRU):

http://profs.info.uaic.ro/∼ciortuz/SLIDES/ml0.pdf http://profs.info.uaic.ro/∼ciortuz/SLIDES/ml3.pdf http://profs.info.uaic.ro/∼ciortuz/SLIDES/ml6.pdf http://profs.info.uaic.ro/∼ciortuz/SLIDES/ml8.pdf http://profs.info.uaic.ro/∼ciortuz/SLIDES/cluster.pdf

24.