lecture 5
play

Lecture 5 Jan-Willem van de Meent Conjugate Priors <latexit - PowerPoint PPT Presentation

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 5 Jan-Willem van de Meent Conjugate Priors <latexit


  1. Unsupervised Machine Learning 
 and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 5 Jan-Willem van de Meent

  2. Conjugate Priors

  3. <latexit sha1_base64="EgFw9vO2o+NpxsqpE8DCaQWRAo=">AGzHicjZTLbtQwFIYzpQwlXNrCk3ESKgDoyopVdsNUlVUwQqVqjdpPB05zpnEqnPBcdpLbNC7OCJeBHehCXJIhJHFAdKXJ8v9c7BO7CaOpsO2fnYU7i3e795bumw8ePnq8vL65CSNM07gmMQs5mcuToHRCI4FQzOEg4dBmcuhdvC/vpJfCUxtGRuE5gFGI/ohNKsMiXxiu/kAhA4POXlvnijWUizP0QT8flqvXZQiz2rWSt+kYh9axp30Qop/8BIwYTMdQ0ydq0byFO/UCMbqefltLS2v/j73ZOmrVb2Svjle6dnr9mxY+sSpJj2jGgfj1cUfyItJFkIkCMNpOnTsRIwk5oISBspEWQoJhfYh2E+jXAI6UjOjkhZNeuRM5KTOBIQkZpM4jANsQi0xQJO6skyAMDr4etFkey8OJBSv2ornJDZrIg0neLrPMpOeyDJQ8fLenpD3Yej1wNrZVA+HgVYSzYw/ypwn4HCqkJ3NgbO1ozNJxhMGfyG7wIpsOERwReIwxJEn0SUQNcz3B0GUZhyKQiRyQ9lzlFIaXK5ZmY30bxqREtQTRpZyqJnY9hxWF5tC1Bt20+brRsE9tWNlxLdmLdhpnLWymsZkOcQ3izQyhNSYkKWVxpNUzmaNnfTLRg7I5pjrjwiXL7yAPax6ToB1PAqxh42TOVRFu8wT5b+vJIoT4FjEvPjprqgIGA2pSGVlV7qKRv9X5fZmsH1Vb8ri7bpyX2kcdmsMet7p3co4V6dK6pswXxex8qDawGTBlhtcEHm953TvN30ycnGumOvOx83e7t71c23ZDwznhtrhmNsG7vGe+PAODZI57zpfOt8737oSu6sqtKdKFTaZ4atdH9+hs1eHTv</latexit> <latexit sha1_base64="EgFw9vO2o+NpxsqpE8DCaQWRAo=">AGzHicjZTLbtQwFIYzpQwlXNrCk3ESKgDoyopVdsNUlVUwQqVqjdpPB05zpnEqnPBcdpLbNC7OCJeBHehCXJIhJHFAdKXJ8v9c7BO7CaOpsO2fnYU7i3e795bumw8ePnq8vL65CSNM07gmMQs5mcuToHRCI4FQzOEg4dBmcuhdvC/vpJfCUxtGRuE5gFGI/ohNKsMiXxiu/kAhA4POXlvnijWUizP0QT8flqvXZQiz2rWSt+kYh9axp30Qop/8BIwYTMdQ0ydq0byFO/UCMbqefltLS2v/j73ZOmrVb2Svjle6dnr9mxY+sSpJj2jGgfj1cUfyItJFkIkCMNpOnTsRIwk5oISBspEWQoJhfYh2E+jXAI6UjOjkhZNeuRM5KTOBIQkZpM4jANsQi0xQJO6skyAMDr4etFkey8OJBSv2ornJDZrIg0neLrPMpOeyDJQ8fLenpD3Yej1wNrZVA+HgVYSzYw/ypwn4HCqkJ3NgbO1ozNJxhMGfyG7wIpsOERwReIwxJEn0SUQNcz3B0GUZhyKQiRyQ9lzlFIaXK5ZmY30bxqREtQTRpZyqJnY9hxWF5tC1Bt20+brRsE9tWNlxLdmLdhpnLWymsZkOcQ3izQyhNSYkKWVxpNUzmaNnfTLRg7I5pjrjwiXL7yAPax6ToB1PAqxh42TOVRFu8wT5b+vJIoT4FjEvPjprqgIGA2pSGVlV7qKRv9X5fZmsH1Vb8ri7bpyX2kcdmsMet7p3co4V6dK6pswXxex8qDawGTBlhtcEHm953TvN30ycnGumOvOx83e7t71c23ZDwznhtrhmNsG7vGe+PAODZI57zpfOt8737oSu6sqtKdKFTaZ4atdH9+hs1eHTv</latexit> <latexit sha1_base64="2bpafo5d0NT+MPGghH2LVwOajeA=">AGQ3icfZRLb9QwEIC9pYESHm3hyCViJVTQqkpK1faCVBVcEKl6kuql5XjzO5adR7YTrutZX4ZV34CB34DN8QNIZHsRmITR3WkaDTzcseO8w4k8r3f3QW7iw6d+8t3XcfPHz0eHl9cmJTHNB4ZimPBVnIZHAWQLHikOZ5kAEocTsOLt6X9BKEZGlypK4z6MdklLAho0QVqsFKitUYFPn0ynNfvPEwEaOYTAYzpfFypxbTIeh7mUapkz5sMPrx0MW71wDwd3eI2WOn6/50ebYQVEIXVetgsLr4FUcpzWNIFOVEyvPAz1RfE6EY5WBcnEvICL0gIzgvxITEIPt6ujPGq1mPgr4epomChNbcNIlTNTYUpawrGvpuEgMop62UvZ1GSUCyUZJ3SuMjeviCIbFKU0r01HIczD68N2e0X5v63Uv2Ng2DURAVBHBjt8rviYwEgBJhexs9oKtHZvJcpFx+A/5JVZWIyCBK5rGMUkijS+BmvNifzAkMhdQNqJxGOtuYIyx4Bla+EztLp43TozWuFYgvtQT08Su57Cy0QK6tqCbtlg3Fva5DZtNYEv1qp0meQubW2xuQ8KCRLNCaM0JmWQ8Tax+hnP0dE6GdlI+x1RnXIbkxdWPiBUxG7fj2ZhZ7GHjZA5NOS7zxOzOG43TDARqSgv3RVTY85ipqSu7Mb2YsntXoW9mWzf1Iey/Ieh3jcWSUM+Hcz63tkTSkVU58ouW7CRqGOzg2sBswZYbXBJFu9d0HzdbOFkYz3w14OPm93dverlW0LP0HO0hgK0jXbRe3SAjhF39GfzmLHcb45P51fzu8ZutCpfJ6i2nL+/gMAGkC</latexit> <latexit sha1_base64="2bpafo5d0NT+MPGghH2LVwOajeA=">AGQ3icfZRLb9QwEIC9pYESHm3hyCViJVTQqkpK1faCVBVcEKl6kuql5XjzO5adR7YTrutZX4ZV34CB34DN8QNIZHsRmITR3WkaDTzcseO8w4k8r3f3QW7iw6d+8t3XcfPHz0eHl9cmJTHNB4ZimPBVnIZHAWQLHikOZ5kAEocTsOLt6X9BKEZGlypK4z6MdklLAho0QVqsFKitUYFPn0ynNfvPEwEaOYTAYzpfFypxbTIeh7mUapkz5sMPrx0MW71wDwd3eI2WOn6/50ebYQVEIXVetgsLr4FUcpzWNIFOVEyvPAz1RfE6EY5WBcnEvICL0gIzgvxITEIPt6ujPGq1mPgr4epomChNbcNIlTNTYUpawrGvpuEgMop62UvZ1GSUCyUZJ3SuMjeviCIbFKU0r01HIczD68N2e0X5v63Uv2Ng2DURAVBHBjt8rviYwEgBJhexs9oKtHZvJcpFx+A/5JVZWIyCBK5rGMUkijS+BmvNifzAkMhdQNqJxGOtuYIyx4Bla+EztLp43TozWuFYgvtQT08Su57Cy0QK6tqCbtlg3Fva5DZtNYEv1qp0meQubW2xuQ8KCRLNCaM0JmWQ8Tax+hnP0dE6GdlI+x1RnXIbkxdWPiBUxG7fj2ZhZ7GHjZA5NOS7zxOzOG43TDARqSgv3RVTY85ipqSu7Mb2YsntXoW9mWzf1Iey/Ieh3jcWSUM+Hcz63tkTSkVU58ouW7CRqGOzg2sBswZYbXBJFu9d0HzdbOFkYz3w14OPm93dverlW0LP0HO0hgK0jXbRe3SAjhF39GfzmLHcb45P51fzu8ZutCpfJ6i2nL+/gMAGkC</latexit> ML and MAP estimation Maximum Likelihood Estimation Maximum A Posterior Estimation ML Objective Regularization (same as minimizing a loss function)

  4. <latexit sha1_base64="E59eI3vLjhVQUj5ANDoMnc+Po=">AGCHicfZTdbtMwFIC9scIofxtchPRm1WqRjKmbTdI09AEl6Pan7RUk+OctacxNjO1s7yC/AIPAV3CIkLxB28AW9D3AbRxBGOFJ2c850/+8QRZ1Qq3/+9tHxnpX3ur9oOHjx4/WVt/eiqzXBA4IRnLxHmEJTCawomisE5F4CTiMFZdPXG2s+uQUiapcdqymGQ4FKh5RgVagu17b5RqjGoLAXJjT2Jl3vtc3JvOvuaXr/W63ktr7F6udfxNf7Y8VwhKoYPKdXS5vItjDOSJ5AqwrCUF4HP1UBjoShYNphLoFjcoVHcFGIKU5ADvSsPeNVrMfBQA+zVEFKm4aJzLBauwoLSyrWjIuEoOopi2VA2jxCDpK16RYlpt8MYhsVWzyrTcRyMLr/9sBov7fzqhds7ZoaIiAuiWDP7xVPHRgJgLRE9rZ7wc6ey/BcAb/IN9ithoBKdyQLElwGuvwGoi5KPYnhFTmAmwjOowS3QmMQ48Rwufmb0dLhonRuwUmB4rSemjk0XMNtoAU0d6LYp1q2DfWjC5nPXUL1qpnHewOYOm7uQcCBRrxAacwKXlGWp089wgZ7NydBNyhaY8oxtSFb8vzF2IvJxM87H1GH7tZPpGzsuiwQWowQX5xmHARWmbA/3Q1VY0YTqQu7cb1oun/vQp7PdmhqQ6lfUeRPjQOSI2G8zq3rkTSkRc5WyXDdhIVLH5wTWAvAaWG2zJ4r4L6rebK5xubQb+ZvB+u7N/UN58q+g5eoE2UIB20T56h47QCSLoE/qOfqJfrY+tz60vra9zdHmp9HmGKqv14w+e1jPw</latexit> <latexit sha1_base64="L8j+V7PAJMvZrRHurXhl1Y7ETYE=">AF7nicfZTdbtMwFIC9scIofx3cwU1Fb4ZUTcmYtl1Omya4HNP+pKWaHOe0teY4xna2dpYRT8Ed4g5xBw/Ai/A2xG0kmjCkaIjn+/4/NqxYFTpIPiztHxvpX/werD9qPHT54+6w9P1NZLgmckoxl8iLGChjlcKqpZnAhJOA0ZnAeXx84/fkNSEUzfqKnAgYpHnE6pATrYuq8zLSY9C4G2mYaPNp/SCjvLtPsXpjrzq9YCOYra4vhKXQ+U6ulpb+R0lGclT4JowrNRlGAg9MFhqShjYdpQrEJhc4xFcFiLHKaiBmSVhuxXtSTgw4xr4KRiZnCqUqzH3qaDVXWXjAvHIKtuy82BcackoOiIV63i1LbUQLDoqCzyEwSsxysOX63b03Q37bDzd3bA2RkJREuBv0i68OjCQAL5HdrX64veszIpeCwT8ocJiLRgKHW5KlKeaJiW6A2MuiPhFwlUtwiZgoTk0vtNZ68BwtbGb6drSonFhjokqA0Y2Z2Do2XcBcogU09aC7prPuPOxjEzafwYbodTON8wY29jch6QHyXqE0OgThKIs414+wV6NidD3ylbYMoeuyNZcUsT7J0oxs24GFOPa515ti6cVksByluOhzlAmQWGfSXbpbqseMplQrU+qtb0X5/60Kfd3Zoa0OpfvHsTm0HkliNhvMau38CSUyqXIuywZsJKvYvHENoKiBZYEdWbx3Yf184WzY0w2Ag/bPX29suXbxW9Qq/ROgrRDtpD79EROkUEfUbf0U/0qyVaX1pfW9/m6PJSafMCVbrx18GySwU</latexit> <latexit sha1_base64="hq5cQZqmFVe52rldOGLUfIrapF4=">AF5HicfZRPb9MwFMC9scIo/zo4cqnoZUjVlIxp63EamuA4pnWbtFST47y21mwn2E63zvI34Ia4IW7wKfgifBviNhJNHOFI0dN7v/fPfnacMap0EPxZW3+w0Xr4aPNx+8nTZ89fdLZenqs0lwSGJGWpvIyxAkYFDXVDC4zCZjHDC7im/fOfjEDqWgqzvQ8gxHE0HlGBdqK47nWw70lPQuBtxmnTv3l53esFOsFhdXwhLoYfKdXK9tfE7SlKScxCaMKzUVRhkemSw1JQwsO0oV5BhcoMncFWIAnNQI7Mo3XYr1rNwZMap0CBIxc1grjWU0/pYFXVkmRGQ1bakcGRclAUnouoVc9tuRwmMi21cVGaSmOVgzemHI2uC/v67frh7YGuIhKQkwkHQL746MJEAokQGe/1wf+AzWS4zBv+gwGuGgkCbknKORaJiWZA7FWxPxEIlUtwjZgo5qYXWms9eIkWPgt7O1o13ljokqB0czc2To2X8FcowU096D7plj3Hva5CVvOXUP1upnGeQObe2zuQ9KDZL1CaMwJmaIsFV4/4xV6MSdjPylbYcozdiFZcTcT7EXMps14NqUe1o7mVPrxmWVwHLCcXHOUZqBxDqV7tLdUj1lFOtTGm3vhcV/cq7PVkx7Y6lO4fx+bYeiSJ2WIwq3vnTyiRSZVzXTZgE1nFlgfXAGY1sNxgRxbvXVh/3XzhfHcnDHbCT3u9w6Py5dtEr9EbtI1CdIAO0Ud0goaIoBn6jn6iX61x60vra+vbEl1fK31eocpq/fgLqoAnjw=</latexit> Intuition: Bayesian Posterior Posterior Likelihood Prior MAP estimate after 50 trials

  5. <latexit sha1_base64="z9jbK0C2B0bymVtEWNsdOrtQecg=">AF/XicfZTdbtMwFIC9scIofx1clNRCW1SqeJurOECaRqa4HJM+5OWqnKc09Za4gTb2bpZFo/AU3CHuEPcIHgO3oakDdDUAUeKnHO+8+sT+0nIpHKcn0vLN1ZqN2+t3q7fuXv/oPG2sNjGaeCwhGNw1ic+kRCyDgcKaZCOE0EkMgP4cQ/f5XrTy5ASBbzQ3WVQD8iI86GjBKViQaNjcmg23z6srmOn0GeMPz6p4agyJ/pMUn3hg0Wk5nu/cu27T6XRd7OKtbO82MS9bhN3nOlqoWLtD9ZWvnlBTNMIuKIhkfIMO4nqayIUoyGYupdKSAg9JyM4y7acRCD7elqTaZa0h7ivhzFXwGnJTJNIRkSNLWEOy7KUjrPAIMphC2Ff514CkGzEy1Z+ZOp1L4Bh1t9pZjrwxSMPni9a7T3t5s427PLCACgoLArtPOnkVgJAB4gbhbzt2kySiSEv5CTY3k2Ajhc0jiKCA+0dwHUnGX98YDLVEBeiPb8SLewMcaCZ2hmM9XvXnlxGjtlRL0LvTELGJXc1heaAZdWdB1la9rC3tXhc1GriJ7VU2TtIJNLTa1IWFBYjFDqIwJiWRhzK16hnP0dE6GdtBwjinOHcZj9tQCyPybgaT8bMYg8WTubA5OMyTxAxikh2zl6cgCAqFvlPd8nUOGQRU1IXemNbMf5/q0y/GzPlIcyf/u+3jMWSf1wOpjl3tkTSkVQ5vIqK7CRKGOzg6sAkwWwaHBOZvfd70ut+e/NcbeDnQ5+u9Xa2S1uvlX0GD1B6wijHtpBb9A+OkIUfUBf0Hf0o/a+9rH2qfZ5hi4vFTaPUGnVv4ClWsu1Q=</latexit> <latexit sha1_base64="m49vKTjt/uVv8k07CRg8QZKlH4=">AGnicfZTPb9MwFMe9scIovzY4cqnohUNVxd1Yw20aGnAc035JTakc5zW1ljBcfbL8n/Cn8CfwIkb4oa4IPhjSNJUNHAkSLb7/N979l+thsHLJGW9XNl9dZa6/ad9bvte/cfPHy0sfn4JIlSQeGYRkEkzlySQMA4HEsmAziLBZDQDeDUPX+V208vQCQs4kfyOoZxSHzOpowSmU1Nl4rp3AyEr47VlZ/Z/gC23bP6g9sbOPtrGO93MLDgXbkDCSZ4PfqaoJ1pxwO8uFA68lGd6HtLSdhbaD+1bRuqhsB5PNtc+OF9E0BC5pQJkhK1YjhURktEAdNtJE4gJPSc+jLIuJyEkY1XkqjsV6xEeq2nEJXBakSkSJiGRM2Myh5PqLJ1lgUFUw5aTY5V78SBhPq+q3FC324H02zi8yU5wYpaHX4Zk8rq7ez1cODoa4hArySwLbVy7464AsAXiL2dg/v2CYTpyIO4C9k5ViejQAOlzQKQ8I95VwA1aNsfxzgSogX4hy3FB1sdbagOdopinsbWfZeKVWSqLBJ0LdaXr2PUSli80g64N6KbJ142BfWjC5oXkL1spknawKYGm5qQMCBRzxAaY0KcsCDixnqmS3RJ1MzaLDElGecuwyG+0Rw2M8a8bjGTPYw9rJHOq8XJYJIvyQZOfsRDEIiORX7pLJmcBC5lMVGnXporx/6syez3Yvq4WZf53XbWvDZK6QVGY1b0zK5QKr8rlq2zAfFHF5gfXAMY1sNzgnMzeu8Wj1vl352TQx1Yfv9vu7u6VL986eoqeoecIoyHaRW/RATpGFH1C39Ev9Lv1sfWl9bX1bY6urpSaJ6jSWj/+AI5xOt4=</latexit> Likelihood: Bernoulli (a discrete distribution with outcomes x=0 and x=1) µ x (1 − µ ) 1 − x Bern( x | µ ) = E [ x ] = µ var[ x ] = µ (1 − µ ) � 1 if µ � 0 . 5 , mode[ x ] = 0 otherwise H[ ] = ln (1 ) ln µ ∈ [0 , 1] that ariable x ∈ { 0 , 1 } by a single continuous

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend