[PPT] - Subgroup Discovery Exploratory Data Analysis Exploratory Data PowerPoint Presentation

SLIDE 1

Subgroup Discovery

SLIDE 2

Exploratory Data Analysis

SLIDE 3

Exploratory Data Analysis

§ Classification: model the dependence of the target on the remaining attributes.

§ problem: sometimes uses only some of the available dependencies, or classifier is a black-box. § for example: in decision trees, some attributes may not appear because of overshadowing.

SLIDE 4

Exploratory Data Analysis

§ Classification: model the dependence of the target on the remaining attributes.

§ problem: sometimes uses only some of the available dependencies, or classifier is a black-box. § for example: in decision trees, some attributes may not appear because of overshadowing.

§ Exploratory Data Analysis: understanding the effects of all attributes on the target.

SLIDE 5

Exploratory Data Analysis

§ Classification: model the dependence of the target on the remaining attributes.

§ problem: sometimes uses only some of the available dependencies, or classifier is a black-box. § for example: in decision trees, some attributes may not appear because of overshadowing.

§ Exploratory Data Analysis: understanding the effects of all attributes on the target. Q: How can we use ideas from C4.5 to approach this task?

SLIDE 6

Exploratory Data Analysis

§ Classification: model the dependence of the target on the remaining attributes.

§ problem: sometimes uses only some of the available dependencies, or classifier is a black-box. § for example: in decision trees, some attributes may not appear because of overshadowing.

§ Exploratory Data Analysis: understanding the effects of all attributes on the target. Q: How can we use ideas from C4.5 to approach this task? A: Why not list the info gain of all attributes, and rank according to this?

SLIDE 7

Interactions between Attributes

§ Single-attribute effects are not enough § XOR problem is extreme example: 2 attributes with no info gain form a good model § Apart from

A=a, B=b, C=c, …

§ consider also

A=a∧B=b, A=a∧C=c, …, B=b∧C=c, … A=a∧B=b∧C=c, … …

SLIDE 8

Subgroup Discovery Task

“Find all subgroups within the inductive constraints that show a significant deviation in the distribution

f the target attribute”

§ Inductive constraints:

§ Minimum support § (Maximum support) § Minimum quality (Information gain, X2, WRAcc) § Maximum complexity § …

SLIDE 9

Confusion Matrix

§ A confusion matrix (or contingency table) describes the frequency of the four combinations of subgroup and target:

§ within subgroup, positive § within subgroup, negative § outside subgroup, positive

T F T .42 .13 .55 F .12 .33 .54 1.0 subgroup target

SLIDE 10

Confusion Matrix

§ High numbers along the TT-FF diagonal means a positive correlation between subgroup and target § High numbers along the TF-FT diagonal means a negative correlation between subgroup and target § Target distribution on DB is fixed

T F T .42 .13 .55 F .12 .33 .45 .54 .46 1.0 subgroup target

SLIDE 11

Confusion Matrix

§ High numbers along the TT-FF diagonal means a positive correlation between subgroup and target § High numbers along the TF-FT diagonal means a negative correlation between subgroup and target § Target distribution on DB is fixed

T F T .42 .13 .55 F .12 .33 .45 .54 .46 1.0 subgroup target

SLIDE 12

Confusion Matrix

§ High numbers along the TT-FF diagonal means a positive correlation between subgroup and target § High numbers along the TF-FT diagonal means a negative correlation between subgroup and target § Target distribution on DB is fixed

T F T .42 .13 .55 F .12 .33 .45 .54 .46 1.0 subgroup target

SLIDE 13

Confusion Matrix

§ High numbers along the TT-FF diagonal means a positive correlation between subgroup and target § High numbers along the TF-FT diagonal means a negative correlation between subgroup and target § Target distribution on DB is fixed

T F T .42 .13 .55 F .12 .33 .45 .54 .46 1.0 subgroup target

SLIDE 14

Confusion Matrix

§ High numbers along the TT-FF diagonal means a positive correlation between subgroup and target § High numbers along the TF-FT diagonal means a negative correlation between subgroup and target § Target distribution on DB is fixed

T F T .42 .13 .55 F .12 .33 .45 .54 .46 1.0 subgroup target

SLIDE 15

Quality Measures

A quality measure for subgroups summarizes the interestingness

f its confusion matrix into a single number

WRAcc, weighted relative accuracy

§ WRAcc(S,T) = p(ST) – p(S)⋅p(T) § between −.25 and .25, 0 means uninteresting § Balance between coverage and unexpectedness

T F T .42 .13 .55 F .12 .33 .54 1.0

WRAcc(S,T) = p(ST)−p(S)⋅p(T) = .42 − .297 = .123

subgroup target

SLIDE 16

Quality Measures

§ WRAcc: Weighted Relative Accuracy § Information gain § X2 § Correlation Coefficient § Laplace § Jaccard § Specificity

SLIDE 17

Subgroup Discovery as Search

T

SLIDE 18

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 …

SLIDE 19

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 …

T F T .42 .13 .55 F .12 .33 .54 1.0

SLIDE 20

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 …

SLIDE 21

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 … A=a2∧B=b1 A=a1∧B=b1 A=a1∧B=b2 … …

SLIDE 22

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 … A=a2∧B=b1 A=a1∧B=b1 A=a1∧B=b2 … … … A=a1∧B=b1∧C=c1

SLIDE 23

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 … A=a2∧B=b1 A=a1∧B=b1 A=a1∧B=b2 … … … A=a1∧B=b1∧C=c1 minimum support level reached

SLIDE 24

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 … A=a2∧B=b1 A=a1∧B=b1 A=a1∧B=b2 … … … A=a1∧B=b1∧C=c1

SLIDE 25

Subgroup Discovery as Search

T A=a1 A=a2 B=b1 B=b2 C=c1 … A=a2∧B=b1 A=a1∧B=b1 A=a1∧B=b2 … … … A=a1∧B=b1∧C=c1

SLIDE 26

Refinements are (anti-)monotonic

target concept entire database

SLIDE 27

Refinements are (anti-)monotonic

target concept subgroup S1 entire database

SLIDE 28

Refinements are (anti-)monotonic

target concept subgroup S1 S2 refinement of S1 entire database

SLIDE 29

Refinements are (anti-)monotonic

target concept subgroup S1 S2 refinement of S1 S3 refinement of S2 entire database

SLIDE 30

Refinements are (anti-)monotonic

target concept subgroup S1 S2 refinement of S1 S3 refinement of S2

Refinements are (anti-) monotonic in their support…

entire database

SLIDE 31

Refinements are (anti-)monotonic

target concept subgroup S1 S2 refinement of S1 S3 refinement of S2

Refinements are (anti-) monotonic in their support… …but not in interestingness. This may go up or down.

entire database

SLIDE 32

Subgroup Discovery and ROC space

SLIDE 33

ROC Space

Each subgroup forms a point in ROC space, in terms of its False Positive Rate, and True Positive Rate. TPR = TP/Pos = TP/TP+FN (fraction of positive cases in the subgroup) FPR = FP/Neg = FP/FP+TN (fraction of negative cases in the subgroup) ROC = Receiver Operating Characteristics

SLIDE 34

ROC Space Properties

SLIDE 35

ROC Space Properties

‘ROC heaven’ perfect subgroup

SLIDE 36

ROC Space Properties

‘ROC heaven’ perfect subgroup ‘ROC hell’ random subgroup

SLIDE 37

ROC Space Properties

‘ROC heaven’ perfect subgroup ‘ROC hell’ random subgroup perfect negative subgroup

SLIDE 38

ROC Space Properties

‘ROC heaven’ perfect subgroup ‘ROC hell’ random subgroup entire database empty subgroup perfect negative subgroup

SLIDE 39

ROC Space Properties

‘ROC heaven’ perfect subgroup ‘ROC hell’ random subgroup entire database empty subgroup minimum support threshold perfect negative subgroup

SLIDE 40

Measures in ROC Space

WRAcc Information Gain positive negative

source: Flach & Fürnkranz

SLIDE 41

Measures in ROC Space

WRAcc Information Gain positive negative

source: Flach & Fürnkranz

isometric

SLIDE 42

Other Measures

Precision Gini index Correlation coefficient Foil gain

SLIDE 43

Refinements in ROC Space

SLIDE 44

Refinements in ROC Space

Refinements of S will reduce the FPR and TPR, so will appear to the left and below S.

SLIDE 45

Refinements in ROC Space

Refinements of S will reduce the FPR and TPR, so will appear to the left and below S.

SLIDE 46

Refinements in ROC Space

Refinements of S will reduce the FPR and TPR, so will appear to the left and below S. Blue polygon represents possible refinements of S. With a convex measure, f is bounded by measure of corners.

SLIDE 47

Refinements in ROC Space

Refinements of S will reduce the FPR and TPR, so will appear to the left and below S. Blue polygon represents possible refinements of S. With a convex measure, f is bounded by measure of corners.

.. . . .

If corners are not above minimum quality or current best (top k?), prune search space below S.

SLIDE 48

Combining Two Subgroups

SLIDE 49

Combining Two Subgroups

SLIDE 50

Combining Two Subgroups

SLIDE 51

Combining Two Subgroups

SLIDE 52

Combining Two Subgroups

SLIDE 53

Multi-class problems

§ Generalising to problems with more than 2 classes is fairly staightforward:

C1 C2 C3 T .27 .06 .22 .55 F .03 .19 .23 .45 .3 .25 .45 1.0

combine values to quality measure

subgroup target

SLIDE 54

Multi-class problems

§ Generalising to problems with more than 2 classes is fairly staightforward:

X2 Information gain

C1 C2 C3 T .27 .06 .22 .55 F .03 .19 .23 .45 .3 .25 .45 1.0

combine values to quality measure

subgroup target

SLIDE 55

Numeric Subgroup Discovery

SLIDE 56

Numeric Subgroup Discovery

h = 2200 h = 3100

SLIDE 57

Numeric Subgroup Discovery

§ Target is numeric: find subgroups with significantly higher or lower average value

h = 2200 h = 3100

SLIDE 58

Numeric Subgroup Discovery

§ Target is numeric: find subgroups with significantly higher or lower average value § Trade-off between size of subgroup and average target value

h = 2200 h = 3100 h = 3600

SLIDE 59

Quiz 1

SLIDE 60

Quiz 1

Q: Assume you have found a subgroup with a positive WRAcc (or infoGain). Can any refinement of this subgroup be negative?

SLIDE 61

Quiz 1

Q: Assume you have found a subgroup with a positive WRAcc (or infoGain). Can any refinement of this subgroup be negative? A: Yes.

SLIDE 62

Quiz 1

Q: Assume you have found a subgroup with a positive WRAcc (or infoGain). Can any refinement of this subgroup be negative? A: Yes.

SLIDE 63

Quiz 2

.

SLIDE 64

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup?

.

SLIDE 65

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes.

.

SLIDE 66

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes. Think of the XOR problem. A ∧ B is either completely positive or negative.

.

SLIDE 67

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes. Think of the XOR problem. A ∧ B is either completely positive or negative.

.

SLIDE 68

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes. Think of the XOR problem. A ∧ B is either completely positive or negative.

.

SLIDE 69

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes. Think of the XOR problem. A ∧ B is either completely positive or negative.

.

SLIDE 70

Quiz 2

Q: Assume both A and B are uninteresting subgroups. Can the subgroup A ∧ B be an interesting subgroup? A: Yes. Think of the XOR problem. A ∧ B is either completely positive or negative.

. . .

SLIDE 71

Quiz 3

SLIDE 72