Semi-Automated Magazine Layout Using Content-Based Image Features - - PowerPoint PPT Presentation

▶
semi automated magazine layout using content based image
SMART_READER_LITE
LIVE PREVIEW

Semi-Automated Magazine Layout Using Content-Based Image Features - - PowerPoint PPT Presentation

Dec 21, 2022 463 likes •900 views

ACM Multimedia 12 31.10.2012 Semi-Automated Magazine Layout Using Content-Based Image Features Mikko Kuhna Aalto University Ida-Maria Kivel School of Science Pirkko Oittinen Department of Media Technology Contents 1. Introduction

slide-1
SLIDE 1

Semi-Automated Magazine Layout Using Content-Based Image Features

Mikko Kuhna Ida-Maria Kivelä Pirkko Oittinen Aalto University School of Science Department of Media Technology ACM Multimedia ’12 31.10.2012

slide-2
SLIDE 2

Contents

  • 1. Introduction
  • 2. Visual concept examples
  • 3. Content-based image features
  • 4. System comparison and results
  • 5. Conclusion

2 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-3
SLIDE 3

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

3

slide-4
SLIDE 4

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

4

=

T

A T A T A

slide-5
SLIDE 5

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

5

=

T

A

T

A

T

A

T

A

T

A T A

slide-6
SLIDE 6

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

6

=

T

A

T

A

T

A

T

A

T

A T A

X X

slide-7
SLIDE 7

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

7

=

T

A

T

A

T

A

T

A

T

A T A

T

A

T

A

T

A

X X

slide-8
SLIDE 8

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

Introduction

8

=

T

A

T

A

T

A

T

A

T

A T A

T

A

T

A

T

A

X X X X

slide-9
SLIDE 9

Our solution

center of importance face

  • rientation

color palette perspective

Combined image feature maps Image feature maps Image feature values Image feature algorithms

image cropping article color text

  • verlaying

image alignment contents page edge map textureness map skin color map color saliency map face map salient color map importance map 9

Content-based image features

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

CSS3 media queries (responsive web design)

+

slide-10
SLIDE 10

Article color examples

10 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

article color text

  • verlaying
slide-11
SLIDE 11

Article color examples

11 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-12
SLIDE 12

Salient color map

12

saliency = face detection + skin color + color saliency

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-13
SLIDE 13

Thresholded salient color maps

13 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-14
SLIDE 14

Color palette

14 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-15
SLIDE 15

Color palette examples

15 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-16
SLIDE 16

Text overlay examples

16 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

text

  • verlaying

image cropping contents page

slide-17
SLIDE 17

Image importance

17 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-18
SLIDE 18

Image importance

face detection

18 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com Viola & Jones (2001)

slide-19
SLIDE 19

Image importance

face detection textureness

19 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com Viola & Jones (2001) Bae et al. (2006)

slide-20
SLIDE 20

Image importance

face detection textureness color saliency

20 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com Viola & Jones (2001) Bae et al. (2006) Achanta et al. (2009)

slide-21
SLIDE 21

Image importance

importance

importance = face detection + skin color + textureness + edges + color saliency

21 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-22
SLIDE 22

Image features in action

center of importance face

  • rientation

color palette perspective

Combined image feature maps Image feature maps Image feature values Image feature algorithms

image cropping article color text

  • verlaying

image alignment contents page edge map textureness map skin color map color saliency map face map salient color map importance map 22 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-23
SLIDE 23

Test magazine

23 lakercompendium.com tietokone.fi/lehti

center of importance face

  • rientation

color palette perspective

Combined image feature maps Image feature maps Image feature values Image feature algorithms

image cropping article color text

  • verlaying

image alignment contents page edge map textureness map skin color map color saliency map face map salient color map importance map

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-24
SLIDE 24

Color palette

24

article color

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-25
SLIDE 25

Importance map

25

Importance map Article cover page, landscape 1024 x 768 px Article cover page, portrait 768 x 1024 px Contents page 400 x 200 px Navigation bar 100 x 100 px text

  • verlaying

image cropping contents page

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-26
SLIDE 26

Column alignment

26

+ mass_center( importance )

image alignment

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com Fernandes & Oliveira (2008)

slide-27
SLIDE 27

System overview

27

Magazine App

(e.g. Baker or web app)

Stylesheet template (LESS) Document template (HAML)

Authoring tool

Document model

(XML)

Document

(HTML5)

Stylesheet

(CSS3)

Processed images

(JPEG, PNG)

Images

(JPEG, PNG)

Image feature algorithms

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-28
SLIDE 28

System overview

28

Magazine App

(e.g. Baker or web app)

Stylesheet template (LESS) Document template (HAML)

Authoring tool

Document model

(XML)

Document

(HTML5)

Stylesheet

(CSS3)

Processed images

(JPEG, PNG)

Images

(JPEG, PNG)

Image feature algorithms

Manual work

(InDesign-files)

lakercompendium.com bakerframework.com Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-29
SLIDE 29

System comparison

29

Responsive

  • Continuous
  • Semi-automated layout
  • Web-based

Automatic

  • Paginated
  • Automated layout
  • 2-levels

Manual

  • Paginated
  • InDesign-layout
  • Woodwing reader

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-30
SLIDE 30

System comparison, navigation

30

A.1.1 P .1 A.1.2 P .1 A.1.1 P .2 A.1.1 P .3 A.1.2 P .2 S.1 P .1 ... S.2 P .1 A.2.1 P .1 A.2.2 P .1 A.2.1 P .2 A.2.1 P .3 S.1 P .2 CP

Automatic

A.1 P .1 A.2 P .1 A.1 P .2 A.1 P .3 A.2 P .2 CP ... C

Manual

A.1 A.2 CP ...

Responsive

Manual Automatic Responsive C = Cover, CP = Contents Page, S = Section, A = Article, P = Page

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-31
SLIDE 31

Quantitative results

31

Ma Statements Manual Automatic Responsive Aut ”fully disagree” (−3) or ”fully agree” (+3) I like the current section

  • 0.10 (0.57)

0.45 (0.44) 0.43 (0.40) Appearance is interesting 0.58 (0.55) 0.68 (0.53) 0.98 (0.39) Appearance is clear 0.00 (0.65) 0.73 (0.62) 1.30 (0.40) Colors are pleasing 0.53 (0.53) 0.43 (0.47) 1.30 (0.36) Readability is good 0.20 (0.54) 0.30 (0.54) 1.05 (0.41) Font type is good 0.05 (0.50) 0.03 (0.57) 0.65 (0.41) ”too little” (−3), ”neutral” (±0) or ”too much” (+3) Number of images

  • 0.02 (0.28)

0.05 (0.31) 0.05 (0.19) Amount of text 0.33 (0.29)

  • 0.30 (0.33)

0.08 (0.19) Size of images 0.05 (0.31) 0.23 (0.23)

  • 0.06 (0.23)

Amount of whitespace

  • 0.55 (0.29)

0.33 (0.29)

  • 0.05 (0.21)

Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-32
SLIDE 32

Qualitative results

32

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

148 62 150 75 125 277 TOTAL

slide-33
SLIDE 33

Qualitative results

33

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-34
SLIDE 34

Qualitative results

34

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-35
SLIDE 35

Qualitative results

35

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-36
SLIDE 36

Qualitative results

36

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-37
SLIDE 37

Qualitative results

37

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-38
SLIDE 38

Qualitative results

38

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-39
SLIDE 39

Qualitative results

39

U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other

Manual Responsive

9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 %

Automatic

Manual Automatic Responsive

negative user comments positive user comment Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

slide-40
SLIDE 40

Conclusion

  • Content-based image features

had positive impact

  • HTML5 already sufficient for

magazines

  • Qualitative analysis useful for

evaluating tablet magazines

40 Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com

center of importance face
  • rientation
color palette perspective

Combined image feature maps Image feature maps Image feature values Image feature algorithms

image cropping article color text
  • verlaying
image alignment contents page edge map textureness map skin color map color saliency map face map salient color map importance map U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other Manual Responsive 9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 % Automatic
slide-41
SLIDE 41

Thank you

More information: mikkokuhna.com This research was funded by the Finnish Funding Agency for Technology and Innovation TEKES via the Next Media program by the Strategic Centre for Science, Technology and Innovation TIVIT. We would like to thank all those who participated in the experiments as well as the partnering companies (Sanoma Magazines Finland and Anygraaf) for providing the material.

slide-42
SLIDE 42

User experiments

42

  • 1. Instructions (1st system)
  • 2. Free-reading (10 min)
  • 3. SUS-questionnaire (10 questions)
  • 4. Instructions (2nd system)
  • 5. Free-reading (5 min)
  • 6. Screenshot task (4 screenshots, talk-aloud)
  • 7. Layout questionnaire (10 questions / screenshot)

1st 2nd video-recorded talk-aloud qualitative data analysis

atlasti.com Mikko Kuhna mikko.kuhna@aalto.fi http://mikkokuhna.com SUS = System Usability Scale