Industrial Image Analysis Group Challenges, Contributions, & - - PowerPoint PPT Presentation

▶
industrial image analysis group
SMART_READER_LITE
LIVE PREVIEW

Industrial Image Analysis Group Challenges, Contributions, & - - PowerPoint PPT Presentation

Oct 26, 2023 319 likes •853 views

Industrial Image Analysis Group Challenges, Contributions, & Roadmap Dr Matthew Thurley www.ltu.se/staff/m/mjt Challenges Which ones are worth doing? The Long News: stories that might still matter in 50, 100, or ten thousand years

slide-1
SLIDE 1

Industrial Image Analysis Group

Challenges, Contributions, & Roadmap

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-2
SLIDE 2

Challenges

  • Which ones are worth doing?
  • The Long News: stories that might still matter in 50, 100, or

ten thousand years from now (TED talk)

  • Can we think about our research this way?
  • Can we challenge accepted thinking and really innovate?
  • Ray Anderson – CEO Interface carpets

– Listen to Ray outline how he lead his company to reduce greenhouse gas emission by 82% in tonnage and double profit in 12 years

  • Amory Lovins – Chief Scientist, Rocky Mountain Institute

– Listen to Amory outline how capatilism and business for profit can eliminate America’s dependancy on foriegn oil

slide-3
SLIDE 3

Challenges

  • How will my work be viewed in 20, 50, 100 years?
  • How can I make a difference for tomorrow’s child?
  • These are hard questions to answer
  • One way to start. Be inspired by the work of others
  • TED.com – Riveting talks by remarkable people
  • If you only take one thing from my presentation

today, take TED

slide-4
SLIDE 4

The Challenge that Inspired me

  • Between 1993 and 1997 I worked for an Australian research organisation,

the CSIRO in their exploration and mining division

  • It was interesting enough but I was still trying to figure out my career
  • By 1997 it was time to for a change, it was time for a PhD.
  • Particle size distribution measurement of rock piles using machine vision
  • More efficient, greener mining - This could make a difference in a huge

industry, this was an inspiring challenge

slide-5
SLIDE 5

Automated Online Particle Size Measurement using 3D Range Data

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-6
SLIDE 6

The Challenge – Increased Energy Efficiency and Product Quality through Feedback and Control of Particle Processes

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-7
SLIDE 7

The objective is to facilitate this future

■ Facilitate this future by developing the necessary measurement systems to provide fast feedback ■ Fully automated online measurement of the particle size distribution ■ Particle size matters in many industries (cement, construction, steel, paper, agriculture, glass, chemical industry, laundry power, ...)

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-8
SLIDE 8

Pilot Installations & Demonstrators

■ Pilot installation for automated online measurement of green pellets on conveyor belt – LKAB 2007 ■ Pilot installation for automated online measurement of limestone particles on conveyor belt – Nordkalk 2009/10 ■ Demonstration project for automated off-line measurement of rocks in excavator buckets – LKAB 2008

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-9
SLIDE 9

3D data is collected of the surface

Advanced Algorithms

determine the particle size distribution from 3D surface data

  • Identifies invidual particles using

image segmentation

  • Identifies overlapped & non-
  • verlapped particles preventing

mis-sizing of overlapped particles as small particles

Robust Image Analysis

using 3D data overcomes limitations

  • f 2D photographic imaging.
  • Unaffected by variation in; particle

color, shadows, ambient lighting

  • Unaffected by scaling errors and

perspective distortion Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-10
SLIDE 10

Identifies non-overlapped particles

Advanced Algorithms

Automatic classification of

  • verlapped and non-overlapped

particles (Thurley & Ng, 2008) 82% classification accuracy using validation against a hold-out data set (Andersson & Thurley, 2008) Dr Matthew Thurley www.ltu.se/staff/m/mjt

  • M. J. Thurley and K. C. Ng. Identification and

sizing of the entirely visible rocks from a 3d surface data segmentation of laboratory rock

  • piles. Computer Vision and Image

Understanding, 111(2):170–178, Aug. 2008.

  • T. Andersson and M. J. Thurley. Visibility

classification of rocks in piles. Proceedings of the Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2008), pages 207 – 213, Dec. 2008.

slide-11
SLIDE 11

Automated sizing on conveyor – pellets

LKAB - Malmberget Industrial Prototype

Over two years of maintenance free

  • peration
  • Fully automated measurment
  • Continuously estimates the sieve

size distribution, once per minute

  • 9 size classes between 5mm and

16mm (down to 0.5mm spacing between classes)

Automatic Control

  • Sizing results facilitate fully

automatic control of the pelletizing process

Thurley, M.J., Anderson, T. An industrial 3D vision system for size measurement of iron ore green pellets using morphological image segmentation, Minerals Engineering Journal, Vol 28, 5, pp 405-415, 2008

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-12
SLIDE 12

Automated sizing on conveyor – pellets

LKAB - Malmberget Industrial Prototype

Two years of operation

  • Fully automated measurment
  • Continuously estimates the sieve

size distribution, once per minute

  • 9 size classes between 5mm and

16mm (down to 0.5mm spacing between classes)

Automatic Control

  • Sizing results allow fully automatic

control of the pelletizing process

Dr Matthew Thurley matthew.thurley@ltu.se

slide-13
SLIDE 13

Estimating the Sieve-Size Distribution – Sources of Error

Dr Matthew Thurley www.ltu.se/staff/m/mjt

  • Segregation error (brazil-nut-effect): vibration causes a separation effect

where large fragments move to the surface

  • Surface bias due to size: larger fragments are more likely to be visible due

to their size

  • These errors limit the range of sizes we can see (only the larger sizes)
  • Overlapped particle error: overlapped particles will look like smaller

particles

  • Profile error: Is the visible profile of a particle indicative of its size
  • Weight transformation: How to convert number of rocks recorded by

imaging, to a weight of rocks equivalent to sieving

slide-14
SLIDE 14

Automated sizing on conveyor – rocks

Nordkalk – Gottland Limestone Quarry

Requirements

  • Fully automated measurment
  • Quality control of material

size during ship loading.

  • Capability to report loading of

the wrong size class

  • Ability to report deviations in

desired material size during loading of the ship in order to detect mechanical failure in the screen decks.

Thurley, M.J. Automated online measurement of the particle size distribution using 3D range data, IFAC MMM Workshop, Vina del Mar, Chile, October 2009

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-15
SLIDE 15

Automated Segmentation

  • 1. Raw 3D surface data
  • 2. Fully automated segmentation
  • 3. Fully automated exclusion of
  • verlapped rocks

1 2 3

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-16
SLIDE 16
slide-17
SLIDE 17

■ By the end of 2009 we demonstrated that the online fully automated results trend in the right direction ■ Research is ongoing to improve accuracy of the absolute size results ■ The following three images were taken from the online measurement system during the 14th and 15th of December 2009

Automated sizing on conveyor – Preliminary Online Results

Nordkalk – Gottland Limestone Quarry: Measurement trends in the right direction

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-18
SLIDE 18

20-40mm being loaded 14.12.2009 17:23

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-19
SLIDE 19

10-90mm being loaded 14.12.2009 20:56

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-20
SLIDE 20

10-90mm being loaded 15.12.2009 9:02

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-21
SLIDE 21

■ The raw size measurement results trend in the right direction because they are based a physically observable property, the Best-Fit-Rectangle (BFR) area

  • f the non-overlapped rocks

■ 20-40mm product

■ Median value 882mm2

■ 40- 70mm product

■ Median value 1901mm2

Automated sizing on conveyor – rocks

Nordkalk – limestone quarry: Measurement trends in the right direction

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-22
SLIDE 22

■ Using this physically

  • bservable property, the

BFR area of the non-

  • verlapped rocks we can

identify the product being loaded ■ For each measurement data set (surface data for approx 1m length of the belt) we calculate the median and IQR of the sample

Identifying the Product

Nordkalk – limestone quarry

Dr Matthew Thurley www.ltu.se/staff/m/mjt

Anderson, T., Thurley M., Carlson, J.E. Online Product Identification during ship loading of limestone based on machine vision. Machine Vision & Applications 2010 (in submission)

slide-23
SLIDE 23

Identifying the Product

Nordkalk – limestone quarry

Dr Matthew Thurley www.ltu.se/staff/m/mjt

Andersson, T. and Thurley, M.J. and Carlson, J. Online Product Identification during Ship Loading of Limestone using Machine Vision, In submission, 2010

Product identification probabilities: 98.8 % accurate classification

slide-24
SLIDE 24

■ Collected a rock sample ■ Sieved into 3 size classes, and painted each rock to identify it by size class ■ Blended the rocks into mixed pile in a cylindrical bucket ■ Captured 3D surface data of the pile ■ Manually characterised the surface

■ Size of each visible rock ■ How visible is each rock?

■ Entirely visible = non-overlapped ■ Predominantly visible = only a minor corner or edge overlap ■ Overlapped = everything else

■ Develop algorithms

■ Segment the rocks ■ Identify overlapped vs non-overlapped

How was this overlapped/non-overlapped classification achieved

Collected good experimental data

slide-25
SLIDE 25

■ Detecting overlapped vs non-overlapped particles

Key Algorithm

slide-26
SLIDE 26

Automated sizing in buckets

Fast feedback to blasting, estimation of the amount of fines

Short Demonstration Project

  • Proof-of-concept*
  • Collect 3D data without

interuption of the LHD unit

  • Fully automated analysis
  • Estimate the sieve size of

the visible fragments only

  • Detection of visible fines

Constraints

  • No opportunity to collect

sieve data

Thurley, M.J. Fragmentation Size Measurement in LHD Buckets using 3D Surface Imaging, Fragblast 9, Granada, Spain, 2009

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-27
SLIDE 27

Automated sizing in buckets – Collect 3D data

Without interuption of the LHD Unit

Image courtesy of LKAB Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-28
SLIDE 28

Segmentation Exclusion of Fines

Details in the paper

  • 1. Raw 3D surface data
  • 2. Fully automated segmentation
  • 3. Fully automated exclusion of areas
  • f fines

1 2 3

Thurley, M.J. Fragmentation Size Measurement in LHD Buckets using 3D Surface Imaging, Fragblast 9, Granada, Spain, 2009

slide-29
SLIDE 29

Automated sizing in buckets Proportion of Visible Surface Classified as Fines over 3 Days

■ Proportion of the surface area of the bucket classified as fines varies between approximately 25 to 85%

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-30
SLIDE 30

Good 3D data provides the necessary foundation for smart algorithms that; ■ Achieve a sufficiently accurate fully automatic particle delineation ■ Distinguish between overlapped and non-overlapped particles ■ Identify areas of fine material ■ Facilitate fully automated industrial prototype systems ■ Provide a new range of automatic control opportunities ■ This research lets us break the problem down into three separate problems that can be handled in different ways. Non-overlapped rocks Overlapped rocks Areas of fines The Pile = + +

Key Research Benefits - Good 3D Data and Smart Algorithms

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-31
SLIDE 31

Dr Matthew Thurley www.ltu.se/staff/m/mjt

Demonstrators

■ Demonstrators are short-term projects, typically 3 man-months ■ Very applied research ■ Goal is to demonstrate proof-of-concept to industry ■ Build confidence in industry, ■ Encourage industry participation in a subsequent larger research project, ■ Reduces risk both from the industry point of view and the research planning ■ 2010 Demonstrators

■ Crack Detection in Steel Slabs ■ Optical microscopy - Automated detection of magnetite/hematite in iron ore pellets

slide-32
SLIDE 32

Crack Detection in Steel Slabs

  • Objectives:

– Measure surface defects on steel slabs (not red hot slabs) – Measure length wise cracks – Measure width wise cracks (can be found in oscillation marks) – Measure corner defects

  • Note: After over 1 year we currently only have examples of

length wise cracks and normal oscillation marks

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-33
SLIDE 33

Continuous casting

Image courtesy of SSAB

slide-34
SLIDE 34

Measured Steel Slabs

  • Measurement of three different slabs was

performed: two with scales and cracks, one scarfed

slide-35
SLIDE 35

Measurement Technologies

  • Laser triangulation

– Off the shelf technology, robust and reliable, SICK Ranger camera series – 0.1mm resolution in width and length – 5um resolution in depth measurement

  • Photometric stereo

– One camera, two sources of illumination, colored lights in separate wavelengths – Possibility to detect smaller cracks – Technology is not off-the-shelf – Ongoing area of measurement technology research

  • Industrial results using only 3D from laser triangulation
slide-36
SLIDE 36

Developing the measurement system module (KTUAS)

  • Both methods implemented to carriage
  • The first measurements using the carriage at SSAB during 22.3.2010
slide-37
SLIDE 37

Laser Triangulation - 3D Data with Cracks

slide-38
SLIDE 38

Laser Triangulation - 3D Data of Scales, Edge Marks

slide-39
SLIDE 39

3D data from laser triangulation, 0.1mm x,y resolution, 5.3um z resolution

slide-40
SLIDE 40

Laser Triangulation - Current Results

  • 3 slabs, 2 slabs with cracks
  • 708 images of the 3 slabs
  • 697 of 708 images do not have cracks
  • 697 of 697 images automatically detected as not having cracks
  • 11 of 708 images have cracks
  • 8 of 11 images automatically detected as having cracks
  • 2 slabs of 2 detected as having cracks
  • We need more examples of slabs with different cracks
  • We want examples of horizontal cracks, cracks in oscillation marks,

longitudinal cracks not in the trench

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-41
SLIDE 41

Crack detection Laser triangulation

slide-42
SLIDE 42

Oscillation mark detection Laser Triangulation

slide-43
SLIDE 43

Crack Detection Summary

  • Demonstrated proof-of-concept for detecting cracks as

needed by the Swedish and Finnish steel industry (currently represented by SSAB Luleå)

  • One more measurement campaign planned at SSAB
  • Test algorithms on new data
  • Present results to a workshop in Luleå to Swedish and Finnish

steel partners; SSAB Luleå, SSAB Borlänge, Outukumpu Avesta, Outukumpu Tornio, Ovako, Sandvik, Mefos

  • Evaluate potential for a consortium project to develop a

prototype (pilot installation)

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-44
SLIDE 44

Optical Microscopy - Pellets

  • LKAB wants to detect magnetite and hematite percentages in
  • ptical microscope images of iron ore pellets
  • Pellets are set in an epoxy disk, and then sliced through the

middle.

  • Analysis is performed on this 2D cross-section of the pellet
  • These proportions can only be observed in an optical

microscope

  • No automated analysis technique exists to perform this

identificaiton

  • Masters Thesis Project: Frida Nellros

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-45
SLIDE 45

Optical Microscopy - Pellets

  • Control the optical microscope to automatically

image an entire pellet (400 images at 20 times resolution)

  • Combine the images into an entire pellet
  • Automatically identify the boundary of the pellet and

exclude the external epoxy

  • Automatically identify magnetite, hematite, pores
  • Graph the proportions in each image based on

average distance to the pellet edge

slide-46
SLIDE 46

Optical Microscopy - Pellets

slide-47
SLIDE 47

Optical Microscopy - Pellets

slide-48
SLIDE 48

Optical Microscopy - Pellets

slide-49
SLIDE 49

Optical Microscopy - Summary

  • Automated control of the microscope achieved
  • Automated identificaiton of magnetite and hematite achieved
  • Automated analysis of an entire pellet achieved
  • Graphing proportions based on distance to the pellet edge

achieved, allowing some observation of the reduction degree in a given pellet

  • Applied for a spin-off doctoral thesis project to HLRC

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-50
SLIDE 50

■ Develop the particle size measurement research in new pilot installations ■ Build competence in automated image analysis of microscopy and tomography data for addressing both industry and research needs within LTU (chemistry, mineral processing, metallurgy, ore geology) ■ Build larger projects that provide time for answering core research questions ■ Build a dynamic group ■ Help build a dynamic subject

Roadmap for the Group

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-51
SLIDE 51

Build a consortium to support the development of two industry prototype permanent installations; ■ Feedback to blasting via automated online measurement of fragment size and bucket volume of LHD buckets. ■ Feedback for automatic crusher control via automated online measurement of the size distribution on conveyor Research Objectives ■ Development of fines detection algorithms to ensure robust validated operation. ■ Further development of the system to handle the broad range of material sizes typical in blasting and primary/secondary crushing. ■ Support complementary research that can use the technology, in automatic control, and feedback to blasting

Roadmap for Particle Size Measurement

Dr Matthew Thurley www.ltu.se/staff/m/mjt

slide-52
SLIDE 52

Industrial Image Analysis Group

Challenges, Contributions, & Roadmap

Dr Matthew Thurley www.ltu.se/staff/m/mjt