COMP 364: Computer Tools for Life Sciences Introduction to image - - PowerPoint PPT Presentation

COMP 364: Computer Tools for Life Sciences

Introduction to image analysis with scikit-image (part one) Christopher J.F. Cameron and Carlos G. Oliver

1 / 27

Key course information

Quiz #9

◮ the penultimate quiz ◮ available Monday, November 27th (closes 11:59:59 pm)

◮ covers topics from the last two weeks

HW5

◮ available early next week ◮ due Thursday, December 7th, 2017 at 11:59:59 pm

◮ shorter than previous assignments

Course evaluations

◮ available now at the following link:

◮ https://horizon.mcgill.ca/pban1/twbkwbis.P_

WWWLogin?ret_code=f

2 / 27

Why perform digital image analysis?

Digital image analysis (DIA) The extraction of useful information from images

◮ important for good feature desgin ◮ emphasizes important traits while diluting noisy ones

For example

◮ in machine vision, image preprocessing plays a huge role ◮ before extracting features from an digital image ◮ it’s extremely useful to be able to augment it ◮ to highlight aspects that are important for the machine

learning task to stand out

3 / 27

DIA in Python

scikit-image module or (skimage)

◮ image processing module in Python ◮ holds a wide library of image processing algorithms: filters,

transforms, point detection

◮ API

◮ http://scikit-image.org/docs/dev/api/api.html

We’ll start with an example image using the io module

◮ basic I/O submodule of scikit-image ◮ API

◮ http:

//scikit-image.org/docs/dev/api/skimage.io.html

4 / 27

5 / 27

Reading an image into memory

1

import skimage.io as io

2 3

# read image into memory

4

image = io.imread("./../images/monkey.jpg")

5

# print top-left pixel RGB values

6

print(image[0,0])

7

# prints: [255 255 255]

8

# write image to disk

9

io.imsave("./../images/monkey_copy.jpg",image) What are RGB values?

6 / 27

RGB colors

Red green blue (RGB) An RGB color value is specified with: rgb(red, green, blue) Each parameter (red, green, and blue) defines the intensity of the color as an integer between 0 and 255 For example, rgb(0, 0, 255)

◮ is rendered as blue ◮ because the blue parameter is set to its highest value (255) ◮ the others are set to 0

RGB color picker/codes chart: http://www.rapidtables.com/web/color/RGB_Color.htm

7 / 27

Handling colors

Let’s make copies of our image and

◮ increase intensity for each color intensity

◮ red, green, blue

◮ note: the format of our image object is

◮ image[ #ycoordinate , #xcoordinate , [red green blue]] ◮ top-left pixel is [0, 0, [red green blue]] 1

red, green, blue = image.copy(),

2

image.copy(), image.copy()

3

red[:,:,(1,2)] = 0 # NumPy indexing

4

green[:,:,(0,2)] = 0

5

blue[:,:,(0,1)] = 0

6

io.imsave("./../images/monkey_red.jpg",red)

7

io.imsave("./../images/monkey_green.jpg",green)

8

io.imsave("./../images/monkey_blue.jpg",blue)

8 / 27

red intensity green intensity blue intensity

9 / 27

Grayscaling

Most image processing algorithms assume a 2D matrix

◮ not an image with a third dimension of color

To bring the image into two dimensions

◮ we need to summarize the three colors into a single value ◮ this process is more commonly know as grayscaling ◮ where the resulting image only holds intensities of gray

◮ with values between 0 and 1

skimage submodule color has useful functions for this task

◮ API

http://scikit-image.org/docs/dev/api/skimage. color.html

10 / 27

1

from skimage.color import rgb2gray

2 3

# read image into memory

4

image = io.imread("./../images/monkey.jpg")

5

# convert to grayscale

6

gray_image = rgb2gray(image)

7

io.imsave("./../images/monkey_grayscale.jpg",gray_image)

8

print(image[0,0])

9

# prints: [255 255 255]

10

print(gray_image[0,0])

11

# prints: 1.0 After we view the grayscale image

◮ let’s find a better way to view the transformation using

histograms

11 / 27

12 / 27

Histogram of RGB intensities

1

import matplotlib.pyplot as plt

2 3

for index,label in zip([0,1,2],["red","green","blue"]):

4

# .flatten() converts 2D list to 1D

5

plt.hist(image[:,:,(index)].flatten(),50

6

,label=label,edgecolor="k",linewidth=1,

7

facecolor=label[0],alpha=0.75)

8

plt.xlabel("pixel intensity",size=16)

9

plt.xlim([0,255])

10

plt.ylabel("frequency",size=16)

11

plt.tight_layout()

12

plt.savefig("./../images/histogram_"+label+".png")

13

plt.close()

13 / 27

Red Green Blue

14 / 27

15 / 27

Image enhancement - histogram equalization

Histogram equalization (HE) Take a grayscale image

◮ attempt to distribute intensities more evenly along the range

• f possible values (0 to 1)

◮ pixels still rank the same

◮ a pixel with a higher value than another will still have a higher

value after the transform

◮ ...but the image as a whole becomes far more contrasted and

normalized We’ll use the submodule exposure to perform HE

◮ API

http://scikit-image.org/docs/dev/api/skimage. exposure.html

16 / 27

HE with Python’s skimage module

1

from skimage.exposure import equalize_hist

2 3

gray_image = rgb2gray(image)

4

print(gray_image[0,0])

5

# prints: 1.0

6

equalized_image = equalize_hist(gray_image)

7

print(equalized_image[0,0])

8

# prints: 1.0

9

io.imsave("./../images/monkey_HE.jpg",equalized_image) Based on what you have learned about HE

◮ why does the top-left most pixel’s value not change?

17 / 27

Grayscale Histogram equalization Why does the image become more contrasted?

◮ pixels that started with similar intensity values ◮ which were relatively hard to differentiate ◮ are now more distinctly separated

Let’s look at the histograms for both images

18 / 27

1

# plot hist of HE pixel intensities

2

plt.hist(equalized_image[:,:].flatten(),50,label="HE",

3

edgecolor="k",linewidth=1,facecolor="blue",

4

alpha=0.75)

5

# plot hist of grayscale pixel intensities

6

plt.hist(gray_image[:,:].flatten(),50,

7

label="grayscale",edgecolor="k",linewidth=1,

8

facecolor="red",alpha=0.75)

9

plt.xlim([0,1])

10

plt.xlabel("pixel intensity",size=16)

11

plt.ylabel("frequency",size=16)

12

plt.legend(loc="best")

13

plt.tight_layout()

14

plt.savefig("./../images/histogram_HE.png")

15

plt.close()

19 / 27

20 / 27

Image enhancement - binarizing and blurring

Sometimes, it helps to simplify an image even further

◮ instead of grayscale, binarize the image ◮ results in each pixel hold only one of two values ◮ more commonly recognized as a pure black and white image

The objective is to separate the foreground from the background

◮ to make feature generation even easier

A simple way of doing this is to just choose a threshold

◮ every pixel above that threshold is set to 1 ◮ every pixel below it to 0

21 / 27

Binarizing and blurring

In our case,

◮ we’ll select the mean of our grayscale image as the threshold ◮ every pixel above the mean is set to white (1.0) ◮ those below are set to black (0.0)

1

import numpy as np

2 3

gray_image = rgb2gray(image)

4

#print(gray_image[0,0])

5

# prints: 1.0

6

binary_image = np.where(gray_image > np.mean(gray_image)

7

,1.0,0.0)

8

io.imsave("./../images/monkey_binary.jpg",binary_image)

9

print(binary_image[0,0])

10

# prints: 1.0

22 / 27

23 / 27

Image enhancement - blurring/smoothing

Binary images may capture more detail than is helpful

◮ for example, the objective is to identify prominent features of

the image

◮ monkey’s hands and fur, foliage etc.

◮ the position for every piece of fur (or leaf) isn’t necessary ◮ blurring/smoothing the image is a reasonable alternative

Scikit-image’s Gaussian filter (filter submodule)

◮ takes a weighted average of surrounding pixels ◮ so individual pixels incorporate local intensities into their own ◮ this produces a pretty recognizable blur/smoothing effect

24 / 27

1

from skimage.filters import gaussian

2 3

equalized_image = equalize_hist(gray_image)

4

for sigma,name in zip([3,6],["blurred","really_blurred"]):

5

blurred_image = gaussian(equalized_image,sigma=sigma)

6

fig,ax = plt.subplots()

7

ax.imshow(blurred_image,vmin=0, vmax=1)

8

# remove ticks

9

ax.set_xticks([])

10

ax.set_yticks([])

11

# remove spines

12

for spine in ["top","bottom","right","left"]:

13

ax.spines[spine].set_visible(False)

14

plt.savefig("./../images/monkey_"+name+".jpg")

15

plt.close()

25 / 27

Sigma = 3 Sigma = 6 The equalized image (rather than grayscale) has been used

◮ maintains a high level of contrast through the filtering ◮ as sigma increases, so does the blurring

26 / 27

Next time in COMP 364

More digital image analyses!

◮ edge and corner detection ◮ maybe more?

27 / 27