Announcements Reminder: A1 due this Friday Texture Thurs Feb 1, - PDF document

CS 376 : Computer Vision - lecture 5 1/31/2018 Announcements • Reminder: A1 due this Friday Texture Thurs Feb 1, 2018 Kristen Grauman UT Austin Recap: last time Issues • Edge detection: • What to do with “noisy” binary – Filter for gradient outputs? – Threshold gradient magnitude, thin – Holes – Extra small fragments • Chamfer matching – to compare shapes (in terms of edge points) • How to demarcate multiple – Distance transform regions of interest? – Count objects • Binary image analysis – Compute further features per – Thresholding object – Morphological operators to “clean up” – Connected components to find regions Connected components Connectedness • Identify distinct regions of “connected pixels” • Defining which pixels are considered neighbors 8-connected 4-connected Source: Chaitanya Chandra Shapiro and Stockman 1

CS 376 : Computer Vision - lecture 5 1/31/2018 Sequential connected components Connected components • We’ll consider a sequential algorithm that requires only 2 passes over the image. • Input : binary image • Output : “label” image, where pixels are numbered per their component • Note: foreground here is denoted with black pixels. Adapted from J. Neira Sequential connected components Sequential connected components Sequential connected components Connected components Slide credit: Pinar Duygulu 2

CS 376 : Computer Vision - lecture 5 1/31/2018 Binary image analysis: Region properties basic steps (recap) • Given connected components, can compute simple features per blob, such as: • Convert the image into binary form – Area (num pixels in the region) – Thresholding – Centroid (average x and y position of pixels in the region) • Clean up the thresholded image – Bounding box (min and max coordinates) – Circularity (ratio of mean dist. to centroid over std) – Morphological operators • Extract separate blobs – Connected components • Describe the blobs with region properties A2=170 A1=200 Example using binary image analysis: Example using binary image analysis: OCR segmentation of a liver [Luis von Ahn et al. http://recaptcha.net/learnmore.html] Slide credit: Li Shen Example using binary image analysis: Visual hulls Bg subtraction + blob detection … 3

CS 376 : Computer Vision - lecture 5 1/31/2018 Example using binary image analysis: Binary images Bg subtraction + blob detection • Pros – Can be fast to compute, easy to store – Simple processing techniques available – Lead to some useful compact shape descriptors • Cons – Hard to get “clean” silhouettes – Noise common in realistic scenarios – Can be too coarse of a representation – Not 3d University of Southern California http://iris.usc.edu/~icohen/projects/vace/detection.htm Today: Texture Includes: more regular patterns What defines a texture? Includes: more random patterns Scale and texture 4

CS 376 : Computer Vision - lecture 5 1/31/2018 Shape from texture Texture-related tasks • Use deformation of texture from point to point to • Shape from texture estimate surface shape – Estimate surface orientation or shape from image texture Pics from A. Loh: http://www.csse.uwa.edu.au/~angie/phdpics1.html Analysis vs. Synthesis Texture-related tasks • Shape from texture Why analyze texture? – Estimate surface orientation or shape from image texture • Segmentation/classification from texture cues – Analyze, represent texture – Group image regions with consistent texture • Synthesis – Generate new texture patches/images given some examples Images:Bill Freeman, A. Efros 5

CS 376 : Computer Vision - lecture 5 1/31/2018 What kind of response will we get with an edge detector for these images? Images from Malik and Perona, 1990 http://animals.nationalgeographic.com/ Why analyze texture? Importance to perception: • Often indicative of a material’s properties • Can be important appearance cue, especially if shape is similar across objects • Aim to distinguish between shape, boundaries, and texture Technically: • Representation-wise, we want a feature one step above “building blocks” of filters, edges. …and for this image? Image credit: D. Forsyth Psychophysics of texture • Some textures distinguishable with preattentive perception– without scrutiny, eye movements [Julesz 1975] Same or different? 6

CS 376 : Computer Vision - lecture 5 1/31/2018 Texture representation Capturing the local patterns with image measurements • Textures are made up of repeated local [Bergen & patterns, so: Adelson, – Find the patterns Nature 1988] • Use filters that look like patterns (spots, bars, raw patches…) Scale of • Consider magnitude of response patterns – Describe their statistics within each local influences window, e.g., discriminability • Mean, standard deviation Size-tuned • Histogram linear filters • Histogram of “prototypical” feature occurrences Texture representation: example Texture representation: example mean mean mean mean d/dx d/dy d/dx d/dy value value value value Win. #1 4 10 Win. #1 4 10 Win.#2 18 7 original image original image … … statistics to statistics to derivative filter summarize patterns derivative filter summarize patterns in small windows in small windows responses, squared responses, squared 7

CS 376 : Computer Vision - lecture 5 1/31/2018 Texture representation: example Texture representation: example mean mean mean mean d/dx d/dy d/dx d/dy value value value value Win. #1 4 10 Win. #1 4 10 Win.#2 18 7 Win.#2 18 7 … Win.#9 20 20 original image original image … … statistics to statistics to summarize patterns summarize patterns derivative filter derivative filter in small windows in small windows responses, squared responses, squared Texture representation: example Texture representation: example Windows with primarily horizontal Both edges Dimension 2 (mean d/dy value) Dimension 2 (mean d/dy value) mean mean mean mean d/dx d/dy d/dx d/dy value value value value Win. #1 4 10 Win. #1 4 10 Win.#2 18 7 Win.#2 18 7 … … Win.#9 20 20 Win.#9 20 20 Dimension 1 (mean d/dx value) Dimension 1 (mean d/dx value) … … Windows with Windows with statistics to small gradient in primarily vertical statistics to summarize patterns summarize patterns both directions edges in small windows in small windows Texture representation: example Texture representation: example Dimension 2 (mean d/dy value) mean mean d/dx d/dy value value Far: dissimilar Win. #1 4 10 textures Win.#2 18 7 Close: similar … textures Win.#9 20 20 original image Dimension 1 (mean d/dx value) … visualization of the assignment to texture “types” statistics to derivative filter summarize patterns in small windows responses, squared 8

CS 376 : Computer Vision - lecture 5 1/31/2018 Texture representation: example Texture representation: example     D ( a , b ) ( a b ) 2 ( a b ) 2 a a 1 1 2 2 a 2  Dimension 2 Dimension 2   2 D ( a , b ) ( a b ) i i b b  i 1 b Dimension 1 Dimension 1 b Distance reveals how dissimilar texture from window a is from texture in window b. Texture representation: Filter banks window scale • Our previous example used two filters, and • We’re assuming we know the relevant window resulted in a 2-dimensional feature vector to size for which we collect these statistics. describe texture in a window. – x and y derivatives revealed something about local Possible to perform scale structure. selection by looking for • We can generalize to apply a collection of window scale where texture multiple ( d ) filters: a “filter bank” description not changing. • Then our feature vectors will be d -dimensional. – still can think of nearness, farness in feature space Multivariate Gaussian Filter banks orientations scales “Edges” “Bars” “Spots” • What filters to put in the bank? – Typically we want a combination of scales and orientations, different types of patterns.  10 5   9 0    16 0             5 5 0 9   0 9     Matlab code available for these examples: http://www.robots.ox.ac.uk/~vgg/research/texclass/filters.html 9

CS 376 : Computer Vision - lecture 5 1/31/2018 Filter bank Image from http://www.texasexplorer.com/austincap2.jpg You try: Can you match the texture to Showing magnitude of responses the response? Filters A B 1 2 C 3 Mean abs responses Derek Hoiem Representing texture by mean abs response Filters [r1, r2, …, r38] We can form a feature vector from the list of responses at each pixel. Mean abs responses Derek Hoiem 10