learning in one layer networks
play

Learning in One-Layer Networks Psych 209 January 9, 2020 - PowerPoint PPT Presentation

Learning in One-Layer Networks Psych 209 January 9, 2020 Input-output mapping Simplest model of learning: input-output mapping Input-output mapping Pattern associator a1 b1 a2 b2 . . . . . . an bn Input stimulus Input pattern


  1. Learning in One-Layer Networks Psych 209 January 9, 2020

  2. Input-output mapping Simplest model of learning: input-output mapping

  3. Input-output mapping Pattern associator a1 b1 a2 b2 . . . . . . an bn Input stimulus Input pattern Output pattern Output behavior

  4. Input-output mapping How can we learn input-output associations? We know how to store (key, value) pairs on a computer a1 b1 a2 b2 . . hash address . . . . an bn Output pattern Input pattern

  5. Input-output mapping But location-based memory is brittle a1 b1 a2 b2 . . hash address . . . . an bn a1’ z1 a2’ z2 . di ff erent . hash . address . . . an’ zn Similar input Totally di ff erent patterns output pattern

  6. Pattern Associator Content-addressable memory: keys directly “produce” values without location lookup Input pattern Output pattern neuron (cell body) connection (axon) Desirable properties Fully connected

  7. Forward Pass Input pattern Output pattern Input 0 w 00 w 01 Input 1 Output 0 w 02 Input 2 Output 1 w 03 Input 3

  8. <latexit sha1_base64="lLa6sjSOQSA6ZvKbViClURecGQ=">ACAHicbZDNSsNAFIUn9a/Wv6gbwc1gEVyVRAR1IRTduKxgbaEN4WY6aedTMLMRCmhbnwVNy5U3PoY7nwbp20W2npg4OPce7lzT5BwprTjfFuFhcWl5ZXiamltfWNzy97euVNxKgmtk5jHshmAopwJWtdMc9pMJIUo4LQRDK7G9cY9lYrF4lYPE+pF0BUsZAS0sXx7D3yGL3BbpZHfxw9+xvojDAZ9u+xUnInwPLg5lFGum9/tTsxSMqNOGgVMt1Eu1lIDUjnI5K7VTRBMgAurRlUEBElZdNLhjhQ+N0cBhL84TGE/f3RAaRUsMoMJ0R6J6arY3N/2qtVIdnXsZEkmoqyHRmHKsYzyOA3eYpETzoQEgkpm/YtIDCUSb0EomBHf25HmoH1fOK+7NSbl6madRPvoAB0hF52iKrpGNVRHBD2iZ/SK3qwn68V6tz6mrQUrn9lFf2R9/gBTDZW6</latexit> <latexit sha1_base64="lLa6sjSOQSA6ZvKbViClURecGQ=">ACAHicbZDNSsNAFIUn9a/Wv6gbwc1gEVyVRAR1IRTduKxgbaEN4WY6aedTMLMRCmhbnwVNy5U3PoY7nwbp20W2npg4OPce7lzT5BwprTjfFuFhcWl5ZXiamltfWNzy97euVNxKgmtk5jHshmAopwJWtdMc9pMJIUo4LQRDK7G9cY9lYrF4lYPE+pF0BUsZAS0sXx7D3yGL3BbpZHfxw9+xvojDAZ9u+xUnInwPLg5lFGum9/tTsxSMqNOGgVMt1Eu1lIDUjnI5K7VTRBMgAurRlUEBElZdNLhjhQ+N0cBhL84TGE/f3RAaRUsMoMJ0R6J6arY3N/2qtVIdnXsZEkmoqyHRmHKsYzyOA3eYpETzoQEgkpm/YtIDCUSb0EomBHf25HmoH1fOK+7NSbl6madRPvoAB0hF52iKrpGNVRHBD2iZ/SK3qwn68V6tz6mrQUrn9lFf2R9/gBTDZW6</latexit> <latexit sha1_base64="lLa6sjSOQSA6ZvKbViClURecGQ=">ACAHicbZDNSsNAFIUn9a/Wv6gbwc1gEVyVRAR1IRTduKxgbaEN4WY6aedTMLMRCmhbnwVNy5U3PoY7nwbp20W2npg4OPce7lzT5BwprTjfFuFhcWl5ZXiamltfWNzy97euVNxKgmtk5jHshmAopwJWtdMc9pMJIUo4LQRDK7G9cY9lYrF4lYPE+pF0BUsZAS0sXx7D3yGL3BbpZHfxw9+xvojDAZ9u+xUnInwPLg5lFGum9/tTsxSMqNOGgVMt1Eu1lIDUjnI5K7VTRBMgAurRlUEBElZdNLhjhQ+N0cBhL84TGE/f3RAaRUsMoMJ0R6J6arY3N/2qtVIdnXsZEkmoqyHRmHKsYzyOA3eYpETzoQEgkpm/YtIDCUSb0EomBHf25HmoH1fOK+7NSbl6madRPvoAB0hF52iKrpGNVRHBD2iZ/SK3qwn68V6tz6mrQUrn9lFf2R9/gBTDZW6</latexit> <latexit sha1_base64="lLa6sjSOQSA6ZvKbViClURecGQ=">ACAHicbZDNSsNAFIUn9a/Wv6gbwc1gEVyVRAR1IRTduKxgbaEN4WY6aedTMLMRCmhbnwVNy5U3PoY7nwbp20W2npg4OPce7lzT5BwprTjfFuFhcWl5ZXiamltfWNzy97euVNxKgmtk5jHshmAopwJWtdMc9pMJIUo4LQRDK7G9cY9lYrF4lYPE+pF0BUsZAS0sXx7D3yGL3BbpZHfxw9+xvojDAZ9u+xUnInwPLg5lFGum9/tTsxSMqNOGgVMt1Eu1lIDUjnI5K7VTRBMgAurRlUEBElZdNLhjhQ+N0cBhL84TGE/f3RAaRUsMoMJ0R6J6arY3N/2qtVIdnXsZEkmoqyHRmHKsYzyOA3eYpETzoQEgkpm/YtIDCUSb0EomBHf25HmoH1fOK+7NSbl6madRPvoAB0hF52iKrpGNVRHBD2iZ/SK3qwn68V6tz6mrQUrn9lFf2R9/gBTDZW6</latexit> Forward Pass Input pattern Output pattern Input 0 w 00 w 01 Input 1 Output 0 w 02 Input 2 Output 1 w 03 Input 3 X a i = w ij a j j

  9. Forward Pass Input pattern Output pattern Input 0 w 00 w 01 Input 1 Output 0 w 02 Input 2 Output 1 inp 0 inp 1 w 03 inp 2 inp 3 Input 3 out 0 w 00 w 01 w 02 w 03 out 1 w 10 w 11 w 12 w 13 out = W inp

  10. Matrix Representation w 00 w 01 w 02 w 03

  11. How do we learn the weights? w 00 w 01 w 02 w 03

  12. How do we learn the weights? w 00 Learning rules: w 01 1. Hebb’s rule w 02 2. Delta rule w 03

  13. Hebb’s Rule • “Neurons that fire together wire together”

  14. <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> Hebb’s Rule • “Neurons that fire together wire together” target ( v ) 1 1 1 -1 1 -1 input ( u ) u ⊗ v 1 -1

  15. <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> <latexit sha1_base64="ghmEL9Yr4Y58wUGVf3yAcHa8M5U=">ACBHicbVBNS8NAEN34WetX1KMeFovgqSQiqLeiF48VjC20oWy2m3bpZhN2J8UScvHiX/HiQcWrP8Kb/8ZtG0FbHw83pthZl6QCK7Bcb6shcWl5ZXV0lp5fWNza9ve2b3Tcao82gsYtUMiGaCS+YB8GaiWIkCgRrBIOrsd8YMqV5LG9hlDA/Ij3JQ04JGKljH7SB3UMQZmnejoFHTOMfZh37IpTdSbA8QtSAUVqHfsz3Y3pmnEJFBtG65TgJ+RhRwKlhebqeaJYQOSI+1DJXE7POzyRc5PjJKF4exMiUBT9TfExmJtB5FgemMCPT1rDcW/NaKYTnfsZlkgKTdLoTAWGI8jwV2uGAUxMoRQxc2tmPaJIhRMcGUTgjv78jzxTqoXVfmtFK7LNIoX10iI6Ri85QDV2jOvIQRQ/oCb2gV+vRerberPdp64JVzOyhP7A+vgFYUJlR</latexit> Hebb’s Rule • “Neurons that fire together wire together” target ( v ) Storing a memory 1 1 1 -1 1 1 1 1 -1 -1 -1 -1 -1 1 input ( u ) u ⊗ v 1 1 1 1 -1 -1 -1 -1 -1 1

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend