Affect- and Personality-based Recommender Systems Hands-on: Unobtrusive Acquisition of Emotions Marko Tkalčič, Free University of Bozen-Bolzano ACM Summer School on Recommender Systems 2017 Marko Tkalčič, RecSys2017SummerSchool-HandsOn 1/18
Goal • Problem statement: lack of recsys datasets with emotional feedback • Solution: To set-up a web app for acquiring emotions during RecSys sessions, hence enabling large-scale acquisition of recsys interactions. Marko Tkalčič, RecSys2017SummerSchool-HandsOn 2/18
Goal • Problem statement: lack of recsys datasets with emotional feedback • Solution: To set-up a web app for acquiring emotions during RecSys sessions, hence enabling large-scale acquisition of recsys interactions. • How? Using a public API for emotion detection. Marko Tkalčič, RecSys2017SummerSchool-HandsOn 2/18
Pre-requisites In order to be able to implement the solution you should have familiarity with programming. The tutorial will be on web client-side (HTML/DOM/Javascript), although you may use another solution (Android, iOS/macOS, Linux, Unity, WIndows). Before the session starts please make sure that you: • have a laptop with an integrated camera • installed an HTTP server on your laptop • apache • MAMP/XAMPP or some other integrated solution • know where your htdocs folder is Marko Tkalčič, RecSys2017SummerSchool-HandsOn 3/18
Scenario • Affectiva SDK : https://developer.affectiva.com/ • we will build an HTML/Javascript page • the full code is available at http://www.inf.unibz.it/~tkalcic/recsys_summerschool_slides/demo.html.txt Marko Tkalčič, RecSys2017SummerSchool-HandsOn 4/18
Step 1 • run the HTTP server • create a subdirectory in the htdocs directory (e.g. .../htdocs/RecSys_demo ) Marko Tkalčič, RecSys2017SummerSchool-HandsOn 5/18
Step 2 • in a text editor create the blank HTML page (e.g. .../htdocs/RecSys_demo/demo.html ) <!DOCTYPE html> <html> <head> <title> ACM RecSys Summer School 2017 Demo </title> </head> <body> </body> </html> Marko Tkalčič, RecSys2017SummerSchool-HandsOn 6/18
Step 3 • in your web browser open the demo.html page http://localhost/RecSys_demo/demo.html Marko Tkalčič, RecSys2017SummerSchool-HandsOn 7/18
Step 4 • create three buttons • create the three visualization boxes <!DOCTYPE html> <html> <head> </head> <body> <button id="start" onclick="onStart()" > Start </button> <button id="stop" onclick="onStop()" > Stop </button> <button id="reset" onclick="onReset()" > Reset </button> <hr> <div id="content" style="border: 1px solid; width: 400px; height: 620px; float: left; margin: 10px; padding: 0;" > CONTENT </div> <div id="preview" style="border: 1px solid; width: 400px; height: 300px; float: left; margin: 10px; padding: 0;" > </div> <div id="results" style="border: 1px solid; width: 400px; height: 700px; float: left; margin: 10px; padding: 0;" > </div> </body> </html> Marko Tkalčič, RecSys2017SummerSchool-HandsOn 8/18
Step 5 • Check your HTML code in the web browser • after each modification to the HTML file go back to the browser, refresh the page and observe the effects of the change Marko Tkalčič, RecSys2017SummerSchool-HandsOn 9/18
Step 6 • Add the needed SDKs to the head of the HTML <head> <title> ACM RecSys Summer School 2017 Demo </title> <meta http-equiv="content-type" content="text/html; charset=UTF-8" > <script type="text/javascript" src="//code.jquery.com/jquery-3.1.0.js" /></script> <script type="text/javascript" src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js" /></script> <script type="text/javascript" src="https://download.affectiva.com/js/3.1/affdex.js" /></script> </head> For reference, see https://knowledge.affectiva.com/docs/getting-started-with-the- emotion-sdk-for-javascript Marko Tkalčič, RecSys2017SummerSchool-HandsOn 10/18
Step 7 • start a <script> element in the bottom of the HTML ... <div id="results" style="border: 1px solid; width: 400px; height: 700px; float: left; margin: 10px; padding: 0;" > </div> <script type="text/javascript" ></script> ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 11/18
Step 8 • set up the detector ... // Link a div for previewing the camera feed var divRoot = $("#preview")[0]; // Configuration var width = 400; var height = 300; var faceMode = affdex.FaceDetectorMode.LARGE_FACES; //Construct a CameraDetector var detector = new affdex.CameraDetector(divRoot, width, height, faceMode); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 12/18
Step 9 • Choose the classifiers • The next step is to turn on the detection of the metrics needed. • list of classifiers: http://developer.affectiva.com/metrics/ ... detector.detectAllExpressions(); detector.detectAllEmotions(); detector.detectAllEmojis(); detector.detectAllAppearance(); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 13/18
Step 10 • add the callback functions for the three buttons ... //function executes when Start button is pushed. function onStart() { if (detector && !detector.isRunning) { detector.start(); } } //function executes when the Stop button is pushed. function onStop() { if (detector && detector.isRunning) { detector.removeEventListener(); detector.stop(); } }; //function executes when the Reset button is pushed. function onReset() { if (detector && detector.isRunning) { detector.reset(); $('#results').html(""); } }; ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 14/18
Step 11 • Configure the callback functions • The Detectors use callbacks to communicate events and results. • For each action there are two callbacks. • A success callback is called when an action successfully completes, and • a failure callback is called in case of an action failure. • The functions addEventListener and removeEventListener are used to register or deregister a callback. ... //Add a callback to notify when the detector is initialized and ready for runing. detector.addEventListener("onInitializeSuccess", function () { //The following two objects are oart of the Affectiva SDK //Display canvas instead of video feed because we want to draw the feature points on it $("#face_video_canvas").css("display", "block"); $("#face_video").css("display", "none"); }); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 15/18
Step 12 ... //Add a callback to receive the results from processing an image. //The faces object contains the list of the faces detected in an image. //Faces object contains probabilities for all the different expressions, emotions and appearance metrics detector.addEventListener("onImageResultsSuccess", function (faces, image, timestamp) { if (faces.length > 0) { var resultsHtml = ""; resultsHtml += "Number of faces found: " + faces.length; resultsHtml += "<br>Appearances: " + JSON.stringify(faces[0].appearance); resultsHtml += "<br>Emotions: " + JSON.stringify(faces[0].emotions); resultsHtml += "<br>Expressions: " + JSON.stringify(faces[0].expressions); resultsHtml += "<br>Emoji: " + faces[0].emojis.dominantEmoji; var htmlToDisplay = resultsHtml.replace(/,/g,",<br>"); // make output nicer $('#results').html(htmlToDisplay); drawFeaturePoints(image, faces[0].featurePoints); } }); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 16/18
Step 13 • add the drawFeaturePoints method ... //Draw the detected facial feature points on the image function drawFeaturePoints(img, featurePoints) { var contxt = $('#face_video_canvas')[0].getContext('2d'); var hRatio = contxt.canvas.width / img.width; var vRatio = contxt.canvas.height / img.height; var ratio = Math.min(hRatio, vRatio); contxt.strokeStyle = "#FFFFFF"; for ( var id in featurePoints) { contxt.beginPath(); contxt.arc(featurePoints[id].x, featurePoints[id].y, 2, 0, 2 * Math.PI); contxt.stroke(); } } ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 17/18
Check the result in the browser Marko Tkalčič, RecSys2017SummerSchool-HandsOn 18/18
Recommend
More recommend
