Image organization, annotation, Image organization, annotation, and - - PowerPoint PPT Presentation

▶
image organization annotation image organization
SMART_READER_LITE
LIVE PREVIEW

Image organization, annotation, Image organization, annotation, and - - PowerPoint PPT Presentation

Feb 22, 2024 113 likes •400 views

Image organization, annotation, Image organization, annotation, and retrieval from a human- - and retrieval from a human centered perspective centered perspective Dr. Oge Marques Associate Professor Computer Science and Engineering Florida

slide-1
SLIDE 1
  • Dr. Oge Marques

Associate Professor Computer Science and Engineering Florida Atlantic University Boca Raton, FL (USA)

May 2008

Image organization, annotation, Image organization, annotation, and retrieval from a human and retrieval from a human-

  • centered perspective

centered perspective

slide-2
SLIDE 2

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 2

Research Team Research Team

Liam M. Mayron, Ph.D. (FAU, Boca Raton, FL - USA)

  • Prof. Humberto R. Gamba, Ph.D.

Gustavo B. Borba (Ph.D. candidate) (UTFPR, Curitiba, PR - Brazil) Dipl.-Ing. Dr. Mathias Lux Arthur Pitman (Klagenfurt University, Austria) Vladimir Nedovic (Ph.D. candidate) (UvA, Amsterdam, NL)

slide-3
SLIDE 3

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 3

Background and Motivation Background and Motivation

Image retrieval is (still) an open research question

– Content-based approaches only work well in narrow domains – Keyword-based solutions have limitations, too – Hybrid (semantically capable) solutions are the ‘holy grail’

The landscape for image annotation has changed

– From ‘expensive’ to ‘fun’

Image organization tools have evolved and become popular

– Picasa, iPhoto, etc. help users organize their ‘digital shoeboxes’

slide-4
SLIDE 4

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 4

Our belief Our belief

We believe that image organization, annotation, and retrieval are highly related tasks.

– Consequently, there is a market (and research potential) for solutions that integrate those tasks.

Such integration must have the human user at the center of the design process. The question, of course, is “How to do it in a principled way?” Our “sources of inspiration”:

– Cognitive (Vision) Science – Web developments and best practices – Social (Web 2.0) interactions – Games!

slide-5
SLIDE 5

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 5

Human Human-

  • Centered Computing (HCC)

Centered Computing (HCC)

Research area or buzzword?

– Representative publication:

  • A. Jaimes, N. Sebe, and ; D.

Gatica-Perez (2006). "Human- Centered Computing: A Multimedia Perspective”, Proc. of the 14th ACM international conference on Multimedia: 855- 864, Santa Barbara, CA, USA: ACM Press.

– Several research groups (in the US):

GA Tech, Auburn, Berkeley, Maryland, […] – Recent (May 2007) IEEE Computer special issue – NSF cluster

slide-6
SLIDE 6

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 6

Putting ideas into practice Putting ideas into practice… …

Using vision science insights and models to advance image segmentation, (region-based) image retrieval, and grouping

  • f similar images.

Representative publications:

– Oge Marques, Liam M Mayron, Gustavo B Borba, and Humberto R Gamba, “An attention-driven model for grouping similar images with image retrieval applications“, EURASIP Journal on Advances in Signal Processing, Special Issue on Image Perception, 2007, Article ID 43450, 17 pages. – Oge Marques, Liam M Mayron, Gustavo B Borba, and Humberto R Gamba, “On the potential of incorporating knowledge of human visual attention into CBIR systems“, Special Session on Perceptual Visual Processing, IEEE International Conference on Multimedia & Expo (ICME) 2006, July 9-12, 2006, Toronto, Canada.

slide-7
SLIDE 7

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 7

An attention An attention-

  • driven model for grouping similar

driven model for grouping similar images for CBIR images for CBIR

slide-8
SLIDE 8

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 8

An attention An attention-

  • driven model for grouping similar

driven model for grouping similar images for CBIR images for CBIR

slide-9
SLIDE 9

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 9

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-10
SLIDE 10

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 10

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-11
SLIDE 11

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 11

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-12
SLIDE 12

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 12

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-13
SLIDE 13

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 13

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-14
SLIDE 14

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 14

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-15
SLIDE 15

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 15

Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention

slide-16
SLIDE 16

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 16

Region Region-

  • based image retrieval

based image retrieval

slide-17
SLIDE 17

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 17

Putting ideas into practice Putting ideas into practice… …

Designing an Ajax-based web solution for image

  • rganization, annotation, (keyword- and content-based)

image retrieval, with collaborative filtering capabilities: PRISM. Representative publications:

– Liam M Mayron, Gustavo B Borba, Vladimir Nedovic, Oge Marques and Humberto R Gamba, “A forward-looking user interface for CBIR and CFIR systems“, IEEE International Symposium on Multimedia (ISM2006), December 11-13, 2006, San Diego, CA. – Liam M Mayron and Oge Marques, “Design of a web-based interface for image retrieval“, International Conference on Web Information Systems and Technologies (WEBIST), March 3-6, 2007, Barcelona, Spain.

Provisional US patent filed through the university’s OTT

slide-18
SLIDE 18

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 18

PRISM PRISM

slide-19
SLIDE 19

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 19

PRISM PRISM

slide-20
SLIDE 20

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 20

PRISM PRISM

slide-21
SLIDE 21

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 21

Putting ideas into practice Putting ideas into practice… …

Testing PRISM with a game metaphor. Representative publication:

– Liam M. Mayron and Oge Marques, “Using a game to evaluate image retrieval, organization, and annotation”, First ICIP Workshop

  • n "Multimedia Information Retrieval: New Trends and Challenges",

co-located with IEEE International Conference on Image Processing (ICIP) 2008, October 12-15, 2008, San Diego, California. (accepted)

slide-22
SLIDE 22

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 22

PRISM Game PRISM Game

slide-23
SLIDE 23

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 23

PRISM Game PRISM Game

slide-24
SLIDE 24

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 24

PRISM Game PRISM Game

slide-25
SLIDE 25

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 25

Putting ideas into practice Putting ideas into practice… …

Using visual analysis techniques to improve tag suggestions in social media sharing sites. Representative publication:

– Mathias Lux, Oge Marques, and Arthur Pitman, “Using Visual Features to Improve Tag Suggestions in Image Sharing Sites”, International Workshop on Knowledge Acquisition from the Social Web (KASW'08), September 3, 2008, Graz, Austria. (submitted)

slide-26
SLIDE 26

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 26

Concluding remarks Concluding remarks

If we keep asking (the right) questions, better human-centered solutions (for image organization, annotation, and retrieval) will emerge…

I’m willing to work with some of you on joint collaborative efforts If you’re interested in any of this, please contact me.

slide-27
SLIDE 27

Image organization, annotation, and retrieval from a human-centered perspective May 2008 - Dr. Oge Marques 27

Thank you! Thank you!

Questions? Contact Information

– Department of Computer Science and Engineering – Florida Atlantic University 777 Glades Road Boca Raton, FL 33431 - USA – Phone: +1-561-297-3857 – Fax: +1-561-297-2800 – Email: omarques@fau.edu – Web: http://www.cse.fau.edu/~oge/