Outlook Overcast Humidity Normal High No Yes Wind Strong Weak No Yes Yes Rain Sunny
❯❀❸ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ➉ ✍✑✏✭✁✝✁➋➊✠✡➍➌➎✏✭✁☎➏➐✆✟✄➑➊➓➒➎➔➃→➅✁☎✄➑➊✒✆☛✡✌☞↔➣↕✆✟✞✠➙ ➛ ✤✧✦✩✣❁✺✭✤✧★P◆❖✣✥✵✩✷➜✷✹✤✧★✒✾❀✴✰✦✩■➝✣❁✤✧✴✶✵✸✣✥★✭✿➞✵✩◆✌❃❊➟✩➟✸➟➡➠✌✵✸✷✑✤✰✺ ➢ ✤✰❏✸✦✩✯✚✾ ❙ ✤●✤✶✻✠✦✩✷✹✮✭■❋✤✧✿➅✦✩✣❁✤✕✪✗➤☛✿❁✤✧✴✶✯✚✾❋✵✸✺✭✿ ➥☛➦➃➧➃➧③➨✗➩✰➫❊➭➃➯✼➲✸➳↔➵✟➦➃➧③➨➸➵✶➫❊➯➃➲ ➺✠➻✸➼✠➽✼➾➃➚✩➪✼➶✼➻✠➹✸➻❆➘✼➼✠➽✸➼✭➴③➷❆➘➮➬ ➫➝➱✃➥☛➦③❐➃❐③➨✗➩✰➫➃➫❊➭✼➲✸➳↔➵✟➦➃➦③➨➸➵✶➫❊❐✼➲ ❒ ➪✼➶✼➻✸❮✭➴③➷❆❰✘➹③➚✩Ï✒➹✸➻✠Ð❆➼✭➴③➷✩➘Ñ➬➓Ò ➱➋➥❶➯➃➭➃➯③➨✗➩✚➦Ó➫✳➲✸➳↔➵✟Ô Ò ➨➸➵✶➫ Ò ➲ ❒✂❒ ➪✼➶✒➴✰Õ①➴✳Ö✒➽③➶✠➷❆❰✭➹✂➬✃Ò ➱➓➥☛➧③Ô➃Ô③➨✗➩✰➫❊➧✼➲✸➳↔➵✟Ô➃➯③➨➸➵ Ò ➧✼➲ ❒✂❒ ➪✼➶✒➴✰Õ①➴✳Ö✒➽③➶✠➷❆❰✭➹✂➬ ➫➝➱➓➥☛➧③➭➃➦③➨✗➩✚➭➃➦✼➲✸➳↔➵✟➦✸×✼➨➸➵✶➫❊➭✼➲ ❒✂❒➓❒ ➺✠➻✸➼✠➽✼➾➃➚✩Ø✭➴③➹❆➼✼➶✼➻✠➹➃➹➓➬✃Ò ➱✂➥☛➧➃➧✸×✼➨✗➩☛×✠➯✼➲✸➳↔➵✟➦➃➦③➨➸➵✶➫❊❐✼➲ ❒✂❒➓❒✂❒PÙ ➴✩➶③➼③Ú✭➚❆Û✒➻✠➴✳Ü③Ú✼➼ÑÝ ➧➃➧✸×✼Ô✗➱✃➥☛❐ Ò ➫✧➨✗➩✰➫ Ò ➵✟➭✼➲✸➳↔➵✟Ô➃Þ③➨➸➵ Ò Þ✼➲ ❒✂❒➓❒✂❒PÙ ➴✩➶③➼③Ú✭➚❆Û✒➻✠➴✳Ü③Ú✼➼Ñß✸➬ ➧➃➧✩×✠Ô✗➱✃➥❴➫❊➧➃➧③➨✗➩✚➧➃➭✗➵❶×✒➲✸➳à➵✟➯➃➦③➨➸➵✟❐➃❐✼➲ ❒✂❒➓❒ ➺✠➻✸➼✠➽✼➾➃➚✩Ø✭➴③➹❆➼✼➶✼➻✠➹➃➹➓➬ ➫➝➱✂➥☛➧✸×✼➨✗➩✚❐Ó➫✳➲✸➳↔➵✟➭➃❐③➨➸➵✟➧➃➦✼➲ ❒ ➪✼➶✼➻✸❮✭➴③➷❆❰✘➹③➚✩Ï✒➹✸➻✠Ð❆➼✭➴③➷✩➘Ñ➬ ➫①➱➋➥❶Þ➃Þ③➨✗➩✚➧➃Þ✼➲✸➳↔➵✟➭Ó➫✧➨➸➵✟➧➃Ô✼➲ ➺✠➻✸➼✠➽✼➾➃➚✩➪✼➶✼➻✠➹✸➻❆➘✼➼✠➽✸➼✭➴③➷❆➘➮➬ ❐✗➱✃➥☛➧✸➨✗➩✚❐➃Ô✼➲✸➳↔➵✶➫➃➫✧➨➸➵✟➦➃Ô✼➲ ➺✠➻✸➼✠➽✼➾➃➚✩➪✼➶✼➻✠➹✸➻❆➘✼➼✠➽✸➼✭➴③➷❆➘➮➬ ➧✗➱✃➥☛➦✸➨✗➩✚❐➃❐✼➲✸➳↔➵✟❐➃➯③➨➸➵✟➯➃➧✼➲ ❯❀á ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ✂✁☎✄✝✆✟✞✠✆☛✡✌☞✎✍✑✏✒✁☎✁☎✞ ❈ ✤✧✴✶✾❀✿✚✾❋✵✸✺â✯✚✣❁✤✰✤❾✣✥✤✰✮✒✣✥✤✧✿✚✤✰✺✼✯✥✦❆✯❁✾❋✵✩✺➝ã ❇❉▲ ✦✩✴✽✬ä✾❋✺✼✯❁✤✰✣❁✺✭✦✩■➝✺✭✵➃★✒✤❾✯✚✤✧✿❁✯❁✿å✦✩✺✫✦✩✯✚✯❁✣❁✾❋æ✭ç✒✯✚✤ ❇❉▲ ✦✩✴✽✬äæ✒✣✽✦✩✺✭✴✽✬◗✴✶✵✸✣❁✣✥✤✧✿✚✮Ó✵✩✺✘★✒✿➞✯❁✵✹✦✩✯✚✯❁✣❁✾❋æ✒ç✭✯✚✤ ❙ ✦❆■❋ç✭✤ ❇❉▲ ✦✩✴✽✬ä■❋✤✧✦❆◆è✺✭✵✼★✠✤●✦✩✿✥✿✚✾❀❏✩✺✘✿å✦➡✴✶■❀✦✸✿✥✿✚✾é❚Ó✴✰✦❆✯❁✾❀✵✸✺ ê ✵✳➠✂➠✌✵✸ç✒■❀★ä➠✌✤❾✣❁✤✰✮✭✣❁✤✧✿❁✤✰✺✼✯✶ã ❇✜ë✗ì❴íîì✭ï✙❘✜ð ❇●ñ➀òóë❑ô✕õ☎íâñ♣ö↕ëä÷ùø✔ëäú✕õ ❇åû ✵❆◆✗ü ❯❀❷ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸High Normal Strong Weak No Yes No Yes - - PDF document
High Normal Strong Weak No Yes No Yes - - PDF document
A1=? A2=? f t f t
[29+,35-] [29+,35-] [21+,5-] [8+,30-] [18+,33-] [11+,2-]
✏❪⑩ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ■ ☞➞➊✒✏✭✡❑❏☞▲Entropy(S) 1.0 0.5 0.0 0.5 1.0 p+
❇✽▼ ✾❀✿✜✦ ✿✥✦❆✷✑✮✒■❋✤➐✵✩◆ù✯✚✣✽✦✩✾❋✺✒✾❋✺✒❏P✤✶✻✠✦✩✷✑✮✒■❋✤✧✿ ❇❖◆✢P ✾❀✿➅✯❁✬✒✤❾✮✒✣✥✵✩✮Ó✵✸✣❁✯✚✾❋✵✸✺✫✵✩◆ù✮Ó✵③✿✚✾❋✯❁✾ ❙ ✤✕✤✶✻✠✦❆✷✑✮✒■❋✤✧✿å✾❋✺ ▼ ❇❖◆✢◗ ✾❀✿➅✯❁✬✒✤❾✮✒✣✥✵✩✮Ó✵✸✣❁✯✚✾❋✵✸✺✫✵✩◆ù✺✭✤✰❏③✦❆✯❁✾ ❙ ✤●✤✶✻✠✦✩✷✑✮✒■❋✤✧✿å✾❋✺ ▼ ❇❉▲ ✺✼✯✚✣✥✵✩✮✼▼✑✷✑✤✧✦✩✿❁ç✒✣✥✤✧✿✌✯✚✬✒✤❾✾❋✷✑✮✒ç✭✣❁✾❋✯☛▼❑✵✩◆ ▼ ú ➆✲❘❚❙✰❂ ◆ ➁ ñ❯▼✗õ❲❱❨❳❲◆ P ■❋✵✸❏✷❩ ◆ P ❳❬◆ ◗ ■❋✵✸❏✷❩ ◆ ◗ ✏✒❭ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ■ ☞➞➊✒✏✭✡❑❏☞▲ ú ➆✲❘❚❙✰❂ ◆ ➁ ñ❯▼✗õ☞❪ ✤✶✻✼✮Ó✤✧✴✶✯✚✤❊★◗✺✼ç✒✷ æ✘✤✰✣➞✵✩◆ùæ✭✾❋✯❁✿å✺✒✤✰✤✧★✠✤✧★◗✯✚✵ ✤✰✺✭✴✶✵✼★✠✤●✴✶■❀✦✸✿✥✿ ñ❯❫ ✵✸✣✕❴ õ ✵✩◆ù✣✽✦❆✺✘★✠✵✸✷✹■❋▼P★✠✣✽✦❊➠➅✺ ✷✹✤✰✷ æ✘✤✰✣✌✵✩◆ ▼➓ñ ç✭✺✭★✠✤✰✣➅✯✚✬✒✤❾✵✸✮✒✯❁✾❋✷✑✦✩■➀❄Ó✿❁✬✒✵✸✣❁✯❁✤✧✿✚✯✟➤❶■❋✤✰✺✭❏✩✯❁✬ ✴✶✵✼★✠✤ õ ❋ ✬✼▼❵● ❍ ✺✠◆➀✵✩✣✥✷P✦❆✯❁✾❋✵✩✺◗✯❁✬✒✤✰✵✸✣❁▼✘ã✗✵✩✮✭✯✚✾❋✷P✦❆■➝■❋✤✰✺✭❏✸✯✚✬ ✴✶✵✼★✒✤ ✦✩✿✥✿✚✾❋❏✸✺✭✿ ❳ ■❀✵✩❏ ❩ ◆ æ✒✾❋✯✥✿➅✯❁✵ ✷✑✤✧✿❁✿✥✦❆❏✸✤➐✬✭✦ ❙ ✾❋✺✒❏ ✮✭✣❁✵✸æ✭✦✩æ✒✾❋■❀✾❋✯☛▼ ◆ ❂ ❈➃✵✒❄✒✤✶✻✼✮Ó✤✧✴✶✯✚✤✧★✫✺✠ç✒✷➡æÓ✤✰✣➞✵✩◆ùæ✒✾❋✯✥✿➅✯❁✵ ✤✰✺✭✴✶✵✼★✒✤ ❫ ✵✸✣✕❴➋✵✩◆ ✣✥✦✩✺✘★✠✵✸✷ ✷✹✤✰✷ æÓ✤✰✣î✵✩◆ ▼ ã ◆ P ñ❛❳ ■❋✵✸❏✷❩ ◆ P õ✲❜✛◆ ◗ ñ❝❳ ■❋✵✸❏✷❩ ◆ ◗ õ ú ➆✲❘❚❙❄❂ ◆ ➁ ñ❯▼✗õ❲❱❨❳❖◆ P ■❀✵✩❏✷❩ ◆ P ❳❬◆ ◗ ■❀✵✩❏✷❩ ◆ ◗ ✏✒❞ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸A1=? A2=? f t f t
[29+,35-] [29+,35-] [21+,5-] [8+,30-] [18+,33-] [11+,2-]
✏é❯ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ✍✑✏✒✖î✆✟☞➐✆✟☞✙✛ ■✛✚ ✖✜✁ ❏✣✢✟✁☎✞ ❈ ✦❊▼ ❘ ç✭✯✚■❋✵➃✵ ✌ ✆ ✤✰✷✹✮Ó✤✰✣✥✦✩✯✚ç✭✣❁✤ ê ç✒✷✑✾❀★✠✾❋✯☛▼ ❋ ✾❋✺✭★ ☛ ■❀✦❊▼ ✆ ✤✰✺✒✺✒✾❀✿ ❈ ❃ ❈✼ç✒✺✒✺✠▼ ê ✵✩✯ ê ✾❋❏✸✬ ❋ ✤✧✦❉✌ ➢ ✵ ❈ ❅ ❈✼ç✒✺✒✺✠▼ ê ✵✩✯ ê ✾❋❏✸✬ ❈➃✯✚✣✥✵✸✺✒❏ ➢ ✵ ❈ ✱ ❘✜❙ ✤✰✣✥✴✰✦✸✿✚✯ ê ✵✩✯ ê ✾❋❏✸✬ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈ ❅ ð ✦✩✾❋✺ ✎ ✾❋■❀★ ê ✾❋❏✸✬ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈ ❊ ð ✦✩✾❋✺ ✪î✵➃✵✸■ ➢ ✵✩✣✥✷✑✦✩■ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈✦✥ ð ✦✩✾❋✺ ✪î✵➃✵✸■ ➢ ✵✩✣✥✷✑✦✩■ ❈➃✯✚✣✥✵✸✺✒❏ ➢ ✵ ❈★✧ ❘✜❙ ✤✰✣✥✴✰✦✸✿✚✯ ✪î✵➃✵✸■ ➢ ✵✩✣✥✷✑✦✩■ ❈➃✯✚✣✥✵✸✺✒❏ ✤î✤✧✿ ❈✦✩ ❈✼ç✒✺✒✺✠▼ ✎ ✾❋■❀★ ê ✾❋❏✸✬ ❋ ✤✧✦❉✌ ➢ ✵ ❈✦✪ ❈✼ç✒✺✒✺✠▼ ✪î✵➃✵✸■ ➢ ✵✩✣✥✷✑✦✩■ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈ ❃❊➟ ð ✦✩✾❋✺ ✎ ✾❋■❀★ ➢ ✵✩✣✥✷✑✦✩■ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈ ❃✸❃ ❈✼ç✒✺✒✺✠▼ ✎ ✾❋■❀★ ➢ ✵✩✣✥✷✑✦✩■ ❈➃✯✚✣✥✵✸✺✒❏ ✤î✤✧✿ ❈ ❃ ❅ ❘✜❙ ✤✰✣✥✴✰✦✸✿✚✯ ✎ ✾❋■❀★ ê ✾❋❏✸✬ ❈➃✯✚✣✥✵✸✺✒❏ ✤î✤✧✿ ❈ ❃❊✱ ❘✜❙ ✤✰✣✥✴✰✦✸✿✚✯ ê ✵✩✯ ➢ ✵✩✣✥✷✑✦✩■ ❋ ✤✧✦❉✌ ✤î✤✧✿ ❈ ❃✧❅ ð ✦✩✾❋✺ ✎ ✾❋■❀★ ê ✾❋❏✸✬ ❈➃✯✚✣✥✵✸✺✒❏ ➢ ✵ ✏✒✏ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ →➅✁✫✢☛✁ù✄➑➊✒✆✟☞✙✛➍➊❵✁✙✁✭✬➋✁ ✚ ➊ ➉ ➊✼➊✒✏✘✆✎✮✰✯å➊✒✁Which attribute is the best classifier?
High Normal Humidity [3+,4-] [6+,1-] Wind Weak Strong [6+,2-] [3+,3-] = .940 - (7/14).985 - (7/14).592 = .151 = .940 - (8/14).811 - (6/14)1.0 = .048 Gain (S, Humidity ) Gain (S, ) Wind =0.940 E =0.940 E =0.811 E =0.592 E =0.985 E =1.00 E [9+,5-] S: [9+,5-] S:
✏❋❱ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸Outlook Sunny Overcast Rain [9+,5−] {D1,D2,D8,D9,D11} {D3,D7,D12,D13} {D4,D5,D6,D10,D14} [2+,3−] [4+,0−] [3+,2−] Yes {D1, D2, ..., D14}
? ? Which attribute should be tested here?
Ssunny = {D1,D2,D8,D9,D11} Gain (Ssunny , Humidity) sunny Gain (S , Temperature) = .970 − (2/5) 0.0 − (2/5) 1.0 − (1/5) 0.0 = .570 Gain (S sunny , Wind) = .970 − (2/5) 1.0 − (3/5) .918 = .019 = .970 − (3/5) 0.0 − (2/5) 0.0 = .970
✏❋❸ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸...
+ + +
A1
+ – + –
A2 A3
+
...
+ – + –
A2 A4
– + – + –
A2
+ – +
... ...
–
- ✏❋á
- ➢
Outlook Overcast Humidity Normal High No Yes Wind Strong Weak No Yes Yes Rain Sunny
❱✒❭ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ✴ ✟✌✁☎✏✁✙➊✼➊✭✆✟☞✙✛ ✪î✵✸✺✭✿❁✾❀★✠✤✰✣å✤✰✣❁✣✥✵✩✣å✵❆◆è✬✼▼➃✮✘✵✸✯✚✬✭✤✧✿✚✾❀✿✌☞ä✵ ❙ ✤✰✣ ❇ ✯✚✣✽✦❆✾❋✺✭✾❋✺✒❏P★✒✦✩✯❁✦✠ã ➄✡❙✰❙✰❂✰❙✎✍✑✏ ☞✓✒✕✔ ñ ☞ õ ❇ ✤✰✺✼✯✚✾❋✣✥✤ ★✠✾❀✿❁✯✚✣✥✾❋æ✒ç✒✯❁✾❋✵✩✺✗✖ ✵❆◆ ★✒✦✩✯❁✦✠ã ➄✡❙✰❙✰❂✰❙✙✘ ñ ☞ õ ê ▼➃✮Ó✵✸✯✚✬✒✤✧✿❁✾❀✿✚☞✜✛✻☞✣✢✥✤✧✦✩★✫✪✭✬✎✮●✯❁✣✥✦✩✾❋✺✒✾❋✺✭❏✑★✭✦❆✯✥✦➡✾é◆ ✯✚✬✭✤✰✣❁✤❾✾❀✿ ✦❆✺✫✦❆■❋✯❁✤✰✣❁✺✭✦✩✯❁✾ ❙ ✤➡✬✼▼➃✮✘✵✸✯✚✬✭✤✧✿✚✾❀✿✌☞✁✯✰✛✻☞↔✿✚ç✘✴✥✬ä✯❁✬✭✦✩✯ ➄✡❙✰❙✰❂✰❙ ✍✑✏ ☞✓✒✕✔ ñ ☞ õ✲✱ ➄✡❙✰❙✰❂✰❙ ✍✑✏ ☞✓✒✕✔ ñ ☞ ✯ õ ✦❆✺✘★ ➄✡❙✰❙✰❂✰❙✙✘ ñ ☞ õ✲✳ ➄✡❙✰❙✰❂✰❙✙✘ ñ ☞ ✯ õ ❱✒❞ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ✴✶✵ ✵✮✼✸✷❲✭ ✭✺✯❯✣✺✹ ✯❯✣✻✚✶✵✷✬✷✯✹✸✺✯✹✔✲✣✻✓ ✼✰✵✷✵✗✻✕✵✰✼ ✼✰✣❖✯❯✣✆✹0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 10 20 30 40 50 60 70 80 90 100 Accuracy Size of tree (number of nodes) On training data On test data
❱é❯ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ✽ ✵ ✔✗✯✹★❲✯✹✣✺✹ ✴✶✵ ✵✮✼✸✷❲✭ ✭✺✯❯✣✺✹ ê ✵✳➠➓✴✰✦✩✺◗➠✌✤●✦ ❙ ✵✩✾❀★◗✵ ❙ ✤✰✣✚❚✘✯✚✯✚✾❋✺✭❏✮● ❇ ✿✚✯❁✵✩✮✫❏✩✣✥✵✳➠❉✾❋✺✭❏ ➠❉✬✭✤✰✺✫★✒✦✩✯❁✦✹✿✚✮✭■❋✾❋✯å✺✭✵✩✯å✿✚✯❁✦✩✯❁✾❀✿✚✯✚✾❀✴✰✦✩■❀■❪▼ ✿✚✾❋❏✸✺✒✾é❚Ó✴✰✦❆✺✠✯ ❇ ❏✩✣✥✵✳➠↕◆➀ç✒■❋■✝✯❁✣❁✤✰✤✩❄✭✯❁✬✒✤✰✺◗✮Ó✵✸✿❁✯✟➤❶✮✒✣✥ç✒✺✭✤ ê ✵✳➠✂✯✚✵✑✿✚✤✰■❋✤✧✴✶✯ ✿✚æ✘✤✧✿❁✯✍❁✹✯✚✣✥✤✰✤✩ã ❇✝✎ ✤✧✦✸✿✚ç✭✣❁✤➐✮Ó✤✰✣✚◆➀✵✩✣✥✷✑✦✩✺✭✴✰✤➐✵ ❙ ✤✧✣❉✯✚✣✽✦❆✾❋✺✭✾❋✺✒❏P★✒✦✩✯❁✦ ❇✝✎ ✤✧✦✸✿✚ç✭✣❁✤➐✮Ó✤✰✣✚◆➀✵✩✣✥✷✑✦✩✺✭✴✰✤➐✵ ❙ ✤✧✣➅✿✚✤✰✮✘✦❆✣✽✦❆✯❁✤ ❙ ✦✩■❋✾❀★✒✦✩✯✚✾❀✵✸✺ ★✒✦✩✯❁✦✹✿✚✤✰✯ ❇✝✎✫❈ ➛ ã✒✷✑✾❋✺✒✾❋✷✑✾ ✲✰✤ ➈✰➇✓✾✸➄ ñ ❘❚❙❆➄❊➄ õ✲❜ ➈✰➇✓✾✸➄ ñ ✝ä➇✟➈✎✿✰❽➀❿✼➈❊➈✰➇✓❀✘➇❁✿✶❿✮❘❶➇❝❂✳➆✝➈ ñ ❘❚❙❆➄✳➄ õ✚õ ❱✒✏ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸- ❿✼❽♥➇❝❃➃❿✮❘❶➇❝❂❆➆
- ❱❋❱
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 10 20 30 40 50 60 70 80 90 100 Accuracy Size of tree (number of nodes) On training data On test data On test data (during pruning)
❱❋❸ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ➣ ✯✰✢☛✁➮➌◗✡✌✞➃➊➃➔✩➌➎✏ ✯✙☞➐✆✟☞✙✛ ❃✸❂✸✪î✵✸✺ ❙ ✤✰✣✥✯❉✯❁✣❁✤✰✤❾✯✚✵✹✤✡✠③ç✒✾ ❙ ✦✩■❋✤✧✺✼✯✙✿✚✤✰✯å✵❆◆è✣❁ç✭■❋✤✧✿ ❅ ❂ ☛ ✣❁ç✒✺✭✤➐✤✧✦✩✴✽✬ä✣✥ç✒■❋✤●✾❋✺✭★✒✤✰✮✘✤✰✺✘★✠✤✰✺✼✯❁■❋▼ä✵❆◆è✵✩✯❁✬✒✤✰✣✽✿ ✱✠❂❉❈➃✵✸✣❁✯❉❚✭✺✭✦✩■①✣❁ç✭■❋✤✧✿❉✾❋✺✼✯❁✵✑★✒✤✧✿✚✾❋✣✥✤✧★◗✿✚✤✡✠③ç✭✤✰✺✭✴✶✤❾◆➀✵✩✣åç✭✿✚✤ ☛ ✤✰✣❁✬✭✦✩✮✭✿❉✷✹✵③✿❁✯✗◆➀✣❁✤✡✠③ç✭✤✰✺✼✯✚■❋▼äç✘✿✚✤✧★ä✷✑✤✰✯✚✬✭✵✼★ ñ ✤✩❂ ❏✒❂❀❄✞✝✠✟☛✡✌☞ õ ❱❋á ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸ ➒ ✡î☞ ✟✌✁☎✏✠➊✒✆✚☞✜✛ ➉ ✍✹✏✭✁✝✁➋➊✒✡ ➣ ✯✰✢☛✁ù✞Outlook Overcast Humidity Normal High No Yes Wind Strong Weak No Yes Yes Rain Sunny
❱❋❷ ❲ ❳ ❨ ❩ ❬❪❭ ❳❴❫ ❲ ❵ ❛❪❳ ❫❁❜ ❝❀❭✟❩ ❳ ❞❪❩ ❡❪❝❪❝❀❢❊❣❆❤❋✐ ❥❀❦ ❧♥♠✽♦♣♠ ❤❋q ❧♥❦ ❧❀r❋s ❨ t✼✉ ❝❀✈①✇③②❪✇❆❵ ❩ ❨ ④❀❳ ❲ ❲ s❖✇❆❨⑥⑤❴❭ ⑦⑥⑧✒⑨❁❵ ❲ ❲ s❶⑩⑥❷❋❷❋❸- ✒
- ✑
- ✒