Counting Sugar Crystals using Image Processing Techniques Bill - - PowerPoint PPT Presentation

Counting Sugar Crystals using Image Processing Techniques

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Lucky Daniel January 22, 2019

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 1 / 23

Contents

Introduction Objectives Data Methodology Results Conclusion

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 2 / 23

Background

Commercial sugar crystal analysis consists of: Crystal Habit: A description of the crystals shape. Includes Elongation Ratio, Coefficient Variation, Mean Aperture Sugar Counting: Reflects efficiency of sugar refinement process

Figure 1: A microscope image of unrefined sugar from Illovo Sugar in Swaziland

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 3 / 23

Objectives

The objectives of the project are:

1 to count the number of sugar crystals per unit of area 2 to determine the length-to-width ratio of each crystal (the Elongation

Ratio)

3 to comment on the “D” shape occurrence

We design and implement an image processing pipeline to address the first two objectives

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 4 / 23

Processing Pipeline

Our Processing Pipeline consists of the following steps: Pre-processing Segmentation Object Detection Object Classification We compare actual in-sample results with algorithmic results to determine

• ptimal parameters.

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 5 / 23

Data

Data consists of: 1300 Microscope images: may differ by resolution, and level of magnification

Unrefined Sugar: vary by company, location of the sugar mill, week of the year Refined Sugar: year, the run, the testing point, and duration of heating

Spreadsheets of image details: crystal count, crystal size distribution

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 6 / 23

Data: Some Examples

a: A Well-behaved Image b: Large Overlap c: Varying Magnification d: High transparency

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 7 / 23

Methodology: Implemented Pipeline

Our results are generated from the following pipeline:

1 Conversion to Gray Scale 2 Filtering (Gaussian Blurring) 3 Contrast Stretching 4 Image Dilation 5 Segmentation (Otsu’s Method) 6 Object Detection (Countours) 7 Object Classification Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 8 / 23

Methodology: Grayscale Conversion

Grayscale conversion changes image from a 3-D array to a 2-D array, simplifying subsequent manipulation.

Figure 2: A BW image represented as a 2D matrix. Numbers in the matrix are pixel intensity. An RGB image is represented by three 2D matrices.

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing TechniquesJanuary 22, 2019 9 / 23

Methodology: Spatial Filtering

The Filtering Transformation:

Figure 3: Spatial Filtering. A linear filter equally weights elements in the input mask

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 10 / 23

Methodology: Contrast Enhancement

We consider two methods of Contrast Enhancement: Histogram Equalisation Transforms the histogram of pixel intensities to have a uniform empirical Cumulative Distribution Function. Contrast Stretching Enhance contrast using the following function: b (x, y) =

• 2N − 1
• × a (x, y) − min (a)

max (a) − min (a) where a is the original image, b is the transformed image, and 2N − 1 is the dynamic range.

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 11 / 23

Methodology: Contrast Enhancement

Figure 4: An example of contrast enhancement applied to an image of sugar

• crystals. The increase in contrast is represented by the widening of the histogram.

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 12 / 23

Methodology: Dilation and Segmentation

Dilation is a morphologocial operation that expands objects and increases connectivity between them. Thresholding is a binary function that sets all pixels above the threshold to the max value (white), and all values below the threshold to min value (black)

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 13 / 23

Methodology: Dilation

Figure 5: Dilation has the effect of making the background (light areas) wider and dark areas smaller. The net result is improved noise reduction

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 14 / 23

Methodology: Segmentation

Figure 6: A series of automatic thresholding procedures to seperate foreground

• bjects from the background

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 15 / 23

Methodology: Summary

Figure 7: Preprocessing: Conversion to Grayscale, Blurring, Contrast Enhancement, Dilation, Segmentation

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 16 / 23

Methodology: Summary

Figure 8: Preprocessing: Conversion to Grayscale, Blurring, Contrast Enhancement, Dilation, Segmentation

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 17 / 23

Methodology: Crystal Detection

We want to identify the crystals in the binarised image: Gradients of pixel intensities are calculated to determine the edges of

• bjects. A boundary is drawn where there is a large change in

gradient. The boundary is stored in memory as an array of pixels. The length of the array proxies the size of the crystal.1. A threshold array length was chosen to separate crystals from noise, impurities, and other objects

1An equivalent micro-meter unit of measurement can be calculated Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 18 / 23

Methodology: Crystal Detection

Figure 9: Original and labelled images

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 19 / 23

Results: Crystal Counting

Table 1: Actual vs predicted number of crystals in out-of-sample images

Image Actual Predicted Error 1 15 20 5 2 21 20

• 1

3 16 19 3 4 13 15 2 5 17 20 3 6 13 21 8 7 16 26 8 16 14

• 2

9 24 24 10 15 18 3 11 16 21 5 12 17 17

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 20 / 23

Methodology: Width-to-Length

To calculate the length-to-width ratio: We applied the Hough transform on the segmented image The result is coordinates for a set of lines on the boundary of each

• bject

Using co-ordinates, we aimed to approximate a rectangle around each

• bject, but did not complete this step

Figure 10: Probabilistic Hough Transformation to provide co-ordinates of boundary lines

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 21 / 23

Challenges Faced

IMAGES: Improved quality of images (for instance, images with less blur) could lighten amount of work done on the image processing, since it proved difficult to detect objects that have soft or blurred edges. OVERLAPPING CRYSTALS: The crystals were immersed in a liquid of low density. This made the image segmentation an issue. To gain a depth perception of sorts, one could use different thresholding parameters for the binarisation of the images during the pre-processing phase. TOUCHING CRYSTALS: If two distinct crystals touch, the threshold used in the preprocessing phase was unable to distinguish the two as separate objects. To combat this issue, one could use image erosion on the binarised image to increase the sharpness of edges in the image. Fatigue

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 22 / 23

Conclusion

To recap: We developed an automated procedure to count sugar crystals, assuming they are well-separated, with strong out-of-sample performance We were unable to complete the calculation of the length-to-width ratio Future Work and Considerations: Much of the difficulty lies with correctly separating overlapping crystals and identifying transparent crystals Spreadsheets with data labels may allow for supervised learning procedures

Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Counting Sugar Crystals using Image Processing Techniques January 22, 2019 23 / 23