We Are Humor Beings: Understanding and Predicting Visual Humor - PowerPoint PPT Presentation

We Are Humor Beings: Understanding and Predicting Visual Humor Shuai Wang University of Toronto March 29, 2016 1 / 31

Intro ◮ An integral part but not understood in detail 2 / 31

Intro ◮ An integral part but not understood in detail ◮ An adult laughs 18 times a day 3 / 31

Intro ◮ An integral part but not understood in detail ◮ An adult laughs 18 times a day ◮ A good sense humor ◮ is related to communication competence ◮ helps raise an individual’s social status & popularity ◮ even helps attract compatible mates ◮ makes yourself happier :) 4 / 31

What makes an image funny? 5 / 31

Humor Techniques ◮ Animal doing something unusual ◮ Person doing something unusual ◮ Somebody getting hurt ◮ Somebody getting scared 6 / 31

Animal doing something unusual 7 / 31

Person doing something unusual 8 / 31

Somebody getting hurt 9 / 31

Somebody getting scared 10 / 31

Changing objects can alter the funniness of a scene 11 / 31

Removing Incongruities An elderly person kicking a football while skateboarding is incongruous, but a young girl doing so is not 12 / 31

Adding Incongruities Add incongruities (and humor) by replacing the expected with the unexpected 13 / 31

Two Tasks to Understand Visual Humor ◮ Predicting how funny a given scene is (scene-level) ◮ Changing the funniness of a scene (object-level) 14 / 31

Object-level Features ◮ Object embedding (150-d): captures the context in which an object usually occurs ◮ Local embedding (150-d): weighted sum of object embeddings of all other instances 15 / 31

Scene-level Features ◮ Cardinality (150-d): bag-of-words representation of how many instances of each object are in the scene ◮ Location (300-d): horizontal and vertical coordinates of every object (closest to the center if multiple instance) ◮ Scene embedding (150-d): sum of object embeddings of all objects in the scene 16 / 31

Predicting Funniness Score ◮ Dataset: 6,400 scenes, with funny score from 1-5 labelled by workers from Amazon Mechanical Turk 17 / 31

Predicting Funniness Score ◮ Dataset: 6,400 scenes, with funny score from 1-5 labelled by workers from Amazon Mechanical Turk ◮ Support Vector Regressor (SVR) on scene-level features 18 / 31

Predicting Funniness Score ◮ Dataset: 6,400 scenes, with funny score from 1-5 labelled by workers from Amazon Mechanical Turk ◮ Support Vector Regressor (SVR) on scene-level features ◮ Metric: average relative error 19 / 31

Predicting Funniness Score: Ablation Analysis Different feature subsets perform about the same: slightly better than baseline (average score of the training scenes) 20 / 31

Alter Funniness of a Scene ◮ Detect the objects that do (or do not) contribute to humor ◮ Identify which objects should replace the objects from step 1 21 / 31

Predicting Objects to be Replaced ◮ On average, the model replaces 3.67 objects (2.54 ground truth) → this bias towards replace ensures a large ‘margin’ 22 / 31

Predicting Objects to be Replaced ◮ On average, the model replaces 3.67 objects (2.54 ground truth) → this bias towards replace ensures a large ‘margin’ ◮ Animate objects like humans and animals are more likely sources of humor → tends to replace these objects 23 / 31

Funny → Unfunny Old man dancing → young boy dancing Hawk stealing meat → baseball 24 / 31

Funny → Unfunny Cute puppy → Insect Watermelon → Ax 25 / 31

Unfunny → Funny Couple having dinner at the table → Puppies having dinner at the table 26 / 31

Unfunny → Funny Cating playing around → Racoon driving motorcycle 27 / 31

Discussion ◮ Style/genre of an image or painting can make a difference 28 / 31

Discussion ◮ Style/genre of an image or painting can make a difference ◮ Dataset is small: 6,400 images 29 / 31

Discussion ◮ Style/genre of an image or painting can make a difference ◮ Dataset is small: 6,400 images ◮ Feature representation can be improved 30 / 31

31 / 31