CS570 Introduction to Data Mining Department of Mathematics and - PowerPoint PPT Presentation

CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong

Data Exploration and Data Preprocessing � Data and attributes � Data exploration � Data pre!processing 2

What is Data? �� Collection of data objects and their � attributes �� An attribute is a property or �� characteristic of an object �� Examples: eye color of a � �� person, temperature, etc. person, temperature, etc. �� Attribute is also known as � �� variable, field, characteristic, or �� feature �� A collection of attributes describe � �� an object �� Object is also known as record, � �� point, case, sample, entity, or �� instance �� 3

Types of Attributes � Categorical (qualitative) Nominal � � Examples: ID numbers, eye color, zip codes Ordinal � � Examples: rankings (e.g., taste of potato chips on a scale from 1!10), grades, height in {tall, medium, short} 1!10), grades, height in {tall, medium, short} � Numeric (quantitative) Interval � � Examples: calendar dates, temperatures in Celsius or Fahrenheit. Ratio � � Examples: temperature in Kelvin, length, time, counts 4

Properties of Attribute Values � The type of an attribute depends on which of the following properties it possesses: � Distinctness: = ≠ � Order: < > � Addition: � Addition: + ! + ! � Multiplication: * / � Nominal attribute: distinctness � Ordinal attribute: distinctness & order � Interval attribute: distinctness, order & addition � Ratio attribute: all 4 properties 5

�� χ % �� !"�# �� $ �� ≠ � &�� # �� $�� )�� '��(� �� *�� +�� .�� /��0�� *�� !�� ,��- � 1�� *�� 4�� 2��3� ��5�� 6

Discrete and Continuous Attributes Discrete Attribute � � Has only a finite or countably infinite set of values � Examples: zip codes, counts, or the set of words in a collection of documents � Often represented as integer variables. � Note: binary attributes are a special case of discrete attributes Continuous Attribute Continuous Attribute � � Has real numbers as attribute values � Examples: temperature, height, or weight. � Continuous attributes are typically represented as floating!point variables. Typically, nominal and ordinal attributes are binary or discrete � attributes, while interval and ratio attributes are continuous Exception? � 7

Types of data sets � �� Graph World Wide Web � Molecular Structures � � Ordered � Ordered Spatial Data � Temporal Data � Sequential Data � Genetic Sequence Data � 8

Record Data Data that consists of a collection of records, each of which consists of � a fixed set of attributes Points in a multi!dimensional space, where each dimension � represents a distinct attribute Represented by an m by n matrix, where there are m rows, one for � each object, and n columns, one for each attribute �� 9

Document Data � Each document becomes a `term' vector, � each term is a component (attribute) of the vector, � the value of each component is the number of times the corresponding term occurs in the document. �� % �� "�� #�� !� $ � 10

Transaction Data � A special type of record data, where � each record (transaction) involves a set of items. � For example, the set of products purchased by a customer during one shopping trip constitute a transaction, while the individual products that were transaction, while the individual products that were purchased are the items. �� ! � �� 11

Data Quality Issues � Data in the real world is dirty � incomplete: lacking attribute values, lacking certain attributes of interest, or containing only aggregate data � e.g., occupation=“ ” � noisy: containing errors or outliers � noisy: containing errors or outliers � e.g., Salary=“!10” � inconsistent: containing discrepancies in codes or names � e.g., Age=“42” Birthday=“03/07/1997” � e.g., Was rating “1,2,3”, now rating “A, B, C” � e.g., discrepancy between duplicate records � duplicate: containing duplicate records Data Mining: Concepts and Techniques 12 12

CS570 Introduction to Data Mining Department of Mathematics and - PowerPoint PPT Presentation

CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Data Exploration and Data Preprocessing Data and attributes Data exploration Data pre!processing 2 What is Data?

CS570 Introduction to Data Mining Frequent Pattern Mining and Association Analysis Cengiz Gunay

CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Today

CS570 Introduction to Data Mining Classification and Prediction Partial slide credits: Han and

CS570 Data Mining Frequent Pattern Mining and Association Analysis 2 Cengiz Gunay Slide

CS570 Data Mining Classification: Ensemble Methods Cengiz Gnay Dept. Math & CS, Emory

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

Introduction What is data mining? to Data Mining: On what kind of data? Data Mining

LECTURE 1: INTRODUCTION TO DATA MINING Dr. Dhaval Patel CSE, IIT-Roorkee What is data mining?

Data Mining and Exploration Data Mining and Exploration: Introduction Course Introduction Amos

Introduction to Web Mining What is Web Mining? Discovering useful information from the

Introduction to Data Mining Methods and Tools by Michael Hahsler Agenda What is Data Mining?

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

Course Content Principles of Knowledge Introduction to Data Mining Discovery in Data

Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

P ( X | Y ) P ( Y ) P ( Y | X ) P ( X ) Introduction to Data Mining, 2 nd

Web Mining Web Mining Web mining is the use of data mining techniques to automatically

Data Visualization Principles: Color CSC444 Acknowledgments for todays lecture: Tamara

Digital Image Processing (CS/ECE 545) Lecture 9: Color Images (Part 2) & Introduction to

Learning Classifiers for Target Domain with Limited or No Labels Pengkai Zhu, Hanxiao Wang,

Coding and computation by neural ensembles in the primate retina Liam Paninski Department of

Data Mining: Data Lecture Notes for Chapter 2 Slides by Tan, Steinbach, Kumar adapted by Michael

Review for Final Exam 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom Unit 1: Probability 1.

What is colour? Eliminativism : The view that either (a) colours as we perceive them do not exist

Mathematical Induction n The inductive proof will sometimes point out an algorithmic solution to a

CS570 Introduction to Data Mining Department of Mathematics and - PowerPoint PPT Presentation

CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Data Exploration and Data Preprocessing Data and attributes Data exploration Data pre!processing 2 What is Data?

CS570 Introduction to Data Mining Frequent Pattern Mining and Association Analysis Cengiz Gunay

CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Today

CS570 Introduction to Data Mining Classification and Prediction Partial slide credits: Han and

CS570 Data Mining Frequent Pattern Mining and Association Analysis 2 Cengiz Gunay Slide

CS570 Data Mining Classification: Ensemble Methods Cengiz Gnay Dept. Math &amp; CS, Emory

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

Introduction What is data mining? to Data Mining: On what kind of data? Data Mining

LECTURE 1: INTRODUCTION TO DATA MINING Dr. Dhaval Patel CSE, IIT-Roorkee What is data mining?

Data Mining and Exploration Data Mining and Exploration: Introduction Course Introduction Amos

Introduction to Web Mining What is Web Mining? Discovering useful information from the

Introduction to Data Mining Methods and Tools by Michael Hahsler Agenda What is Data Mining?

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

Course Content Principles of Knowledge Introduction to Data Mining Discovery in Data

Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

P ( X | Y ) P ( Y ) P ( Y | X ) P ( X ) Introduction to Data Mining, 2 nd

Web Mining Web Mining Web mining is the use of data mining techniques to automatically

Data Visualization Principles: Color CSC444 Acknowledgments for todays lecture: Tamara

Digital Image Processing (CS/ECE 545) Lecture 9: Color Images (Part 2) &amp; Introduction to

Learning Classifiers for Target Domain with Limited or No Labels Pengkai Zhu, Hanxiao Wang,

Coding and computation by neural ensembles in the primate retina Liam Paninski Department of

Data Mining: Data Lecture Notes for Chapter 2 Slides by Tan, Steinbach, Kumar adapted by Michael

Review for Final Exam 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom Unit 1: Probability 1.

What is colour? Eliminativism : The view that either (a) colours as we perceive them do not exist

Mathematical Induction n The inductive proof will sometimes point out an algorithmic solution to a

CS570 Data Mining Classification: Ensemble Methods Cengiz Gnay Dept. Math & CS, Emory

Digital Image Processing (CS/ECE 545) Lecture 9: Color Images (Part 2) & Introduction to