ADNet: A Deep Network for Detecting Adverts M. Hossari, S. Dev, M. - - PowerPoint PPT Presentation

ADNet: A Deep Network for Detecting Adverts

• M. Hossari, S. Dev, M. Nicholson, K. McCabe, A. Nautiyal, C. Conran, J. Tang, W. Xu, and F. Pitié

The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.

adaptcentre.ie

People consume a lot of content

We Never Wanted to Talk to the Person Next to Us: http://republicofweb.org/weblog/?p=1032.

adaptcentre.ie

People consume a lot of content

• People have an insatiable appetite for media

We Never Wanted to Talk to the Person Next to Us: http://republicofweb.org/weblog/?p=1032.

adaptcentre.ie

Technology Create Opportunity

Millennials in Singapore spend almost 3.4 hours a day on their mobile phones: https://www.straitstimes.com/tech/smartphones/millennials-in-singapore-spend-almost-34-hours-a-day-on-their-mobile-phones-study

adaptcentre.ie

Technology Create Opportunity

• 5 billion people now have mobiles phones
• We consume more content on demand
• Using services that can personalise content to the user

Millennials in Singapore spend almost 3.4 hours a day on their mobile phones: https://www.straitstimes.com/tech/smartphones/millennials-in-singapore-spend-almost-34-hours-a-day-on-their-mobile-phones-study

adaptcentre.ie

Opening the door to personalise marketing

Illustration by M. Nicholson, October 2018.

adaptcentre.ie

• Make it easy to seamlessly integrate advertisements into
• n-demand video content
• Allow user access to uninterrupted content; unlike pre-roll,

mid-roll, and post-roll advertisements

• Allow existing advertisement to be automatically replaced by

personalised ads

• Applications in other sectors: information display, education,

social outreach

Benefits of personalised marketing

Illustration by M. Nicholson, October 2018.

adaptcentre.ie

The Industry Challenge How to accurately identify the video frame that contain an advertisement/billboard?

Manual identification of adverts in a video is cumbersome and time-consuming. No similar system exists for accurate detection

• f advert in a video frame.

Diverse scene illumination and severe occlusion make such tasks challenging.

adaptcentre.ie

The recognition module takes images (video frames) as input. It then classifies those images as containing an advert or not (“ad” or “no ad”). We propose using the VGG based network that we call ADNET. Based on VGG19

• Architecture. Transfer learning using pre-trained "ImageNet" model and froze all

layers apart from last 5 layers. Added 3 fully connected layers with a softmax layer as the output layer.

The Solution: AdNet for detecting adverts

adaptcentre.ie

Dataset

We train our ADNet model on a composite dataset, that consists of both positive- and negative- examples of billboard detection. Sources of images are:

• Mapillary Vistas dataset1
• Microsoft COCO dataset2

[1] Neuhold, G., Ollmann, T., Bulo, S.R., Kontschieder, P.: The mapillary vistas dataset for semantic understanding of street scenes. In: ICCV. pp. 5000–5009 (2017) [2] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)

adaptcentre.ie

Subjective Evaluation

adaptcentre.ie

Subjective Evaluation

adaptcentre.ie

Objective Evaluation

We report the classification accuracy of ADNet. Suppose, TP, FP, TN and FN denote the true positive, false positive, true negative and false negative respectively: Classification Accuracy = (TP+TN)/(TP+TN+FP+FN)

adaptcentre.ie

The Developed System

Illustration by M. Nicholson, October 2018. More details on the advert creation system: https://arxiv.org/pdf/1808.00163.pdf

• Prof. François Pitié, The ADAPT Centre, PITIEF@tcd.ie.

The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.

Info and Contact