MapReduce and Streaming Algorithms for Diversity Maximization in - - PowerPoint PPT Presentation

MapReduce and Streaming Algorithms for Diversity Maximization in Metric Spaces of Bounded Doubling Dimension

Andrea Pietracaprina

• Dip. Ingegneria dell’Informazione, Universit`

a di Padova

Joint work with:

• M. Ceccarello, G. Pucci (U. Padova), and E. Upfal (Brown U.)

VLDB’17

Outline

◮ Problem definition and applications ◮ Background ◮ Summary of results ◮ Our approach:

◮ Core-set construction ◮ Streaming implementation ◮ MapReduce implementation

◮ Experiments ◮ Diversity maximization under matroid costraints ◮ Summary and future work

Problem definition and applications

Diversity maximization Objective:

For a given dataset

Diversity maximization Objective:

Determine the most diverse subset of given (small) size k

Applications

Applications Aggregator websites (e.g., Google News)

◮ Documents (possibly clustered into categories) ◮ Diversified set of representative docs (from various clusters)

Applications Aggregator websites (e.g., Google News)

◮ Documents (possibly clustered into categories) ◮ Diversified set of representative docs (from various clusters)

E-commerce

◮ Consideration set: products returned by shopping portal in

reply to user query

◮ Diversity of returned products (w.r.t. unspecified attributes)

correlates to user satisfaction

Applications Aggregator websites (e.g., Google News)

◮ Documents (possibly clustered into categories) ◮ Diversified set of representative docs (from various clusters)

E-commerce

◮ Consideration set: products returned by shopping portal in

reply to user query

◮ Diversity of returned products (w.r.t. unspecified attributes)

correlates to user satisfaction

Facility location

◮ Franchise location (noncompetition) ◮ Strategic facilities (dispersion against simultaneous attacks)

Diversity maximization: formal definition

Diversity maximization: formal definition Given:

• 1. Set S of n points in a metric space ∆
• 2. Distance function d : ∆ × ∆ → R+ ∪ {0}
• 3. Integer k > 1
• 4. (Distance-based) diversity function div : 2∆ → R+ ∪ {0}

Diversity maximization: formal definition Given:

• 1. Set S of n points in a metric space ∆
• 2. Distance function d : ∆ × ∆ → R+ ∪ {0}
• 3. Integer k > 1
• 4. (Distance-based) diversity function div : 2∆ → R+ ∪ {0}

Return

S∗ ⊂ S, |S∗| = k s.t. S∗ = argmaxS′⊆S,|S′|=k div(S′)

Diversity maximization: formal definition Given:

• 1. Set S of n points in a metric space ∆
• 2. Distance function d : ∆ × ∆ → R+ ∪ {0}
• 3. Integer k > 1
• 4. (Distance-based) diversity function div : 2∆ → R+ ∪ {0}

Return

S∗ ⊂ S, |S∗| = k s.t. S∗ = argmaxS′⊆S,|S′|=k div(S′) The k-diversity of S is divk(S) = div(S∗)

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

◮ MSTAR(S′): min-weight star in G(S′) (≡ complete graph over S′)

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

◮ MSTAR(S′): min-weight star in G(S′) (≡ complete graph over S′) ◮ MBIP(S′): min-weight balanced bipartition of S′ in G(S′)

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

◮ MSTAR(S′): min-weight star in G(S′) (≡ complete graph over S′) ◮ MBIP(S′): min-weight balanced bipartition of S′ in G(S′) ◮ MST(S′): min-weight spanning tree of G(S′)

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

◮ MSTAR(S′): min-weight star in G(S′) (≡ complete graph over S′) ◮ MBIP(S′): min-weight balanced bipartition of S′ in G(S′) ◮ MST(S′): min-weight spanning tree of G(S′) ◮ TSP(S′): min-weight tour of S′ in G(S′)

Diversity functions studied in this work

Problem div(S′) Remote-edge minp,q∈S′ d(p, q) Remote-clique

• p,q∈S′ d(p, q)

Remote-star w(MSTAR(S′)) Remote-bipartition w(MBIP(S′)) Remote-tree w(MST(S′)) Remote-cycle w(TSP(S′))

◮ MSTAR(S′): min-weight star in G(S′) (≡ complete graph over S′) ◮ MBIP(S′): min-weight balanced bipartition of S′ in G(S′) ◮ MST(S′): min-weight spanning tree of G(S′) ◮ TSP(S′): min-weight tour of S′ in G(S′)

Except for remote-clique, all problems are max-min optimizations.

Background

Previous work Sequential approximation and hardness results

Problem

• Seq. Approx.

LB Remote-edge 2 ≥ 2 Remote-clique 2 ≥ 2 − ǫ Remote-star 2 – Remote-bipartition 3 – Remote-tree 4 ≥ 2 Remote-cycle 3 ≥ 2 Specialized results (hardness and better approx. ratios) for remote clique and remote edge under Euclidean distances

Previous work β-core-set [Agarwal et al.’04]

Previous work β-core-set [Agarwal et al.’04]

◮ (Small) subset T ⊆ S such that divk(T) ≥ (1/β) divk(S)

Previous work β-core-set [Agarwal et al.’04]

◮ (Small) subset T ⊆ S such that divk(T) ≥ (1/β) divk(S)

Previous work β-core-set [Agarwal et al.’04]

◮ (Small) subset T ⊆ S such that divk(T) ≥ (1/β) divk(S) ◮ T filters out redundancy

Previous work β-core-set [Agarwal et al.’04]

◮ (Small) subset T ⊆ S such that divk(T) ≥ (1/β) divk(S) ◮ T filters out redundancy ◮ Approximate solution can be computed on T.

Previous work β-composable core-set [Indyk et al.’14]

Previous work β-composable core-set [Indyk et al.’14]

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ

Previous work β-composable core-set [Indyk et al.’14]

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ β-composable core-sets Ti ⊂ Si ⇒ ∪Ti is a β-core-set

Previous work β-composable core-set [Indyk et al.’14]

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ β-composable core-sets Ti ⊂ Si ⇒ ∪Ti is a β-core-set

Previous work β-composable core-set [Indyk et al.’14]

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ β-composable core-sets Ti ⊂ Si ⇒ ∪Ti is a β-core-set ◮ Application to MapReduce and Streaming frameworks

Previous work Known β-composable core-sets for k-diversity maximization

([Indyk et al.’14,Aghamolaei et al.’15]) β αseq β · αseq Remote-edge 3 2 6 Remote-clique 6 + ǫ 2 12 + ǫ Remote-star 12 2 24 Remote-bipartition 18 3 54 Remote-tree 4 4 16 Remote-cycle 3 3 9

Previous work Known β-composable core-sets for k-diversity maximization

([Indyk et al.’14,Aghamolaei et al.’15]) β αseq β · αseq Remote-edge 3 2 6 Remote-clique 6 + ǫ 2 12 + ǫ Remote-star 12 2 24 Remote-bipartition 18 3 54 Remote-tree 4 4 16 Remote-cycle 3 3 9

◮ αseq = best sequential approximation ratio

Previous work Known β-composable core-sets for k-diversity maximization

([Indyk et al.’14,Aghamolaei et al.’15]) β αseq β · αseq Remote-edge 3 2 6 Remote-clique 6 + ǫ 2 12 + ǫ Remote-star 12 2 24 Remote-bipartition 18 3 54 Remote-tree 4 4 16 Remote-cycle 3 3 9

◮ αseq = best sequential approximation ratio ◮ General metric spaces

Previous work Known β-composable core-sets for k-diversity maximization

([Indyk et al.’14,Aghamolaei et al.’15]) β αseq β · αseq Remote-edge 3 2 6 Remote-clique 6 + ǫ 2 12 + ǫ Remote-star 12 2 24 Remote-bipartition 18 3 54 Remote-tree 4 4 16 Remote-cycle 3 3 9

◮ αseq = best sequential approximation ratio ◮ General metric spaces ◮ Core-set size: k

Computational Frameworks for Massive Data Analysis

Computational Frameworks for Massive Data Analysis MapReduce

◮ Targets distributed cluster-based architectures ◮ Computation: sequence of rounds where data are partitioned into

(small) subsets, processed in parallel

◮ Goals: few rounds, sublinear local space, linear overall space.

Computational Frameworks for Massive Data Analysis MapReduce

◮ Targets distributed cluster-based architectures ◮ Computation: sequence of rounds where data are partitioned into

(small) subsets, processed in parallel

◮ Goals: few rounds, sublinear local space, linear overall space.

Streaming

◮ Data provided as a continuous stream and processed using limited

local space (much smaller than input size)

◮ Multiple passes over data may be allowed ◮ Goals: few passes, sublinear local space

Computational Frameworks for Massive Data Analysis MapReduce

◮ Targets distributed cluster-based architectures ◮ Computation: sequence of rounds where data are partitioned into

(small) subsets, processed in parallel

◮ Goals: few rounds, sublinear local space, linear overall space.

Streaming

◮ Data provided as a continuous stream and processed using limited

local space (much smaller than input size)

◮ Multiple passes over data may be allowed ◮ Goals: few passes, sublinear local space

Fact: Known composable core-sets for k-diversity maximization yield 1-pass/2-round Streaming/MapReduce algorithms with O √ kn

• space.

Doubling space Doubling space

Metric space such that any ball of radius r is covered by ≤ 2D balls of radius r/2 (⇒1/ǫD balls of radius ǫr), for some D = O(1). E.g.: Euclidean spaces of constant dimension (under ℓ1 or ℓ2 norm); Shortest-path distances of mildly expanding topologies

Summary of Results

Summary of results For doubling spaces of dim D, all div’s, and ǫ > 0:

Summary of results For doubling spaces of dim D, all div’s, and ǫ > 0:

◮ (1 + ǫ)-(composable) core-sets

Summary of results For doubling spaces of dim D, all div’s, and ǫ > 0:

◮ (1 + ǫ)-(composable) core-sets ◮ 1-pass Streaming and 2-round MapReduce (MR) algorithms

– approximation ratio: αseq + ǫ – local space requirements:

Summary of results For doubling spaces of dim D, all div’s, and ǫ > 0:

◮ (1 + ǫ)-(composable) core-sets ◮ 1-pass Streaming and 2-round MapReduce (MR) algorithms

– approximation ratio: αseq + ǫ – local space requirements:

Streaming MR (det.) MR (rand.) r-edge/cycle O(k/ǫD) O

• kn/ǫD
• –
• ther div’s

O(k2/ǫD) O

• k
• n/ǫD
• O
• kn log n/ǫD

Summary of results For doubling spaces of dim D, all div’s, and ǫ > 0:

◮ (1 + ǫ)-(composable) core-sets ◮ 1-pass Streaming and 2-round MapReduce (MR) algorithms

– approximation ratio: αseq + ǫ – local space requirements:

Streaming MR (det.) MR (rand.) r-edge/cycle O(k/ǫD) O

• kn/ǫD
• –
• ther div’s

O(k2/ǫD) O

• k
• n/ǫD
• O
• kn log n/ǫD
• ◮ One extra pass/round brings deterministic space bounds for
• ther div′ s down to those for r-edge/cycle

Our approach

Core-set construction: algorithm

Input dataset: S Optimal solution OPT ⊂ S, with |OPT| = k

Core-set construction: algorithm

Input dataset: S Optimal solution OPT ⊂ S, with |OPT| = k MAIN IDEA: Compute core-set T such that each o ∈ OPT has a (distinct) proxy p(o) ∈ T with “small” d(o, p(o))

Core-set construction: algorithm

Input dataset: S Optimal solution OPT ⊂ S, with |OPT| = k MAIN IDEA: Compute core-set T such that each o ∈ OPT has a (distinct) proxy p(o) ∈ T with “small” d(o, p(o))

• 1. Partition S into k′ > k clusters of small radius

Core-set construction: algorithm

Input dataset: S Optimal solution OPT ⊂ S, with |OPT| = k MAIN IDEA: Compute core-set T such that each o ∈ OPT has a (distinct) proxy p(o) ∈ T with “small” d(o, p(o))

• 1. Partition S into k′ > k clusters of small radius
• 2. T = {cluster centers}

Core-set construction: algorithm

Input dataset: S Optimal solution OPT ⊂ S, with |OPT| = k MAIN IDEA: Compute core-set T such that each o ∈ OPT has a (distinct) proxy p(o) ∈ T with “small” d(o, p(o))

• 1. Partition S into k′ > k clusters of small radius
• 2. T = {cluster centers}
• 3. If injectivity of p(·) required:

T = {cluster centers} ∪ {f (k) ≤ k − 1 delegates per cluster}.

Core-set construction: algorithm

◮ k = 3, k′ = 8

Core-set construction: algorithm

◮ Compute k′-center clustering

Core-set construction: algorithm

◮ No injectivity required: T = {cluster centers} (|T| = k′)

Core-set construction: algorithm

◮ Injectivity required: T = {1 center + 2 delegates per cluster}

Core-set construction: analysis

Setting for the analysis:

Core-set construction: analysis

Setting for the analysis:

◮ S from doubling space of dimension D ◮ Focus on remote-clique (similiar for other div’s) ◮ Fix ǫ ∈ (0, 1/2).

Core-set construction: analysis

Setting for the analysis:

◮ S from doubling space of dimension D ◮ Focus on remote-clique (similiar for other div’s) ◮ Fix ǫ ∈ (0, 1/2).

Theorem

If k′ = (16/ǫ)Dk and k − 1 delegates per clusters are taken, T is a (1 + ǫ)-core-set for S, of size O

• k2(1/ǫ)D

Core-set construction: analysis Proof.

Core-set construction: analysis Proof.

◮ Let ρ = div(OPT)/

k

2

• .

Claim: The radius of an optimal k′-clustering of S is rk′ ≤ (ǫ/8)ρ

Core-set construction: analysis Proof.

◮ Let ρ = div(OPT)/

k

2

• .

Claim: The radius of an optimal k′-clustering of S is rk′ ≤ (ǫ/8)ρ

◮ ∃ an injective p(·) such that for each o ∈ OPT, p(o) ∈ T and

d(o, p(o)) ≤ 2rk′ ≤ (ǫ/4)ρ

Core-set construction: analysis Proof.

◮ Let ρ = div(OPT)/

k

2

• .

Claim: The radius of an optimal k′-clustering of S is rk′ ≤ (ǫ/8)ρ

◮ ∃ an injective p(·) such that for each o ∈ OPT, p(o) ∈ T and

d(o, p(o)) ≤ 2rk′ ≤ (ǫ/4)ρ

◮ Hence:

divk(T) ≥

• 1,o2∈OPT

d(p(o1), p(o2)) (injectivity!) ≥ divk(S)/(1 + ǫ) (claim+triangle ineq.)

Core-set construction: composability

Core-set construction: composability Composability

Core-set construction: composability Composability

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Extract a core-set Ti ⊂ Si as before ◮ ∪Ti is a (1 + ǫ)-core-set of size O

• ℓk2(1/ǫ)D

.

Core-set construction: composability Composability

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Extract a core-set Ti ⊂ Si as before ◮ ∪Ti is a (1 + ǫ)-core-set of size O

• ℓk2(1/ǫ)D

. ⇒ the Ti’s are (1 + ǫ)-composable core-sets

Core-set construction: composability Composability

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Extract a core-set Ti ⊂ Si as before ◮ ∪Ti is a (1 + ǫ)-core-set of size O

• ℓk2(1/ǫ)D

. ⇒ the Ti’s are (1 + ǫ)-composable core-sets

Saving space

Core-set construction: composability Composability

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Extract a core-set Ti ⊂ Si as before ◮ ∪Ti is a (1 + ǫ)-core-set of size O

• ℓk2(1/ǫ)D

. ⇒ the Ti’s are (1 + ǫ)-composable core-sets

Saving space

◮ Initial random partition ◮ Build each Ti picking Θ (max{k/ℓ, log n}) delegates per

cluster (rather than k − 1)

◮ ∪Ti is a (1 + ǫ)-core-set of size O

• k(k + ℓ log n)(1/ǫ)D

, w.h.p.

Streaming implementation

Streaming implementation

◮ One-pass

◮ Fix k′ = O

• k/ǫD

◮ Run adaptation of 8-approximate k′-center algorithm of

[Charikar et al.’04] to compute a (1 + ǫ)-core-set of size O (k′ · k) = O

• k2/ǫD

◮ Run sequential approximation on the core-set

Streaming implementation

◮ One-pass

◮ Fix k′ = O

• k/ǫD

◮ Run adaptation of 8-approximate k′-center algorithm of

[Charikar et al.’04] to compute a (1 + ǫ)-core-set of size O (k′ · k) = O

• k2/ǫD

◮ Run sequential approximation on the core-set

◮ If D is unknown, it can be guessed in O (1) passes

MapReduce implementation

MapReduce implementation

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ

MapReduce implementation

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Two rounds

◮ Round 1: run 2-approximation k′-center algorithm in each Si

to find Ti (e.g., Gonzalez’s)

◮ Round 2: run best sequential algorithm for k-diversity on ∪Ti

MapReduce implementation

◮ Input partition S = S1 ∪ S2 ∪ · · · ∪ Sℓ ◮ Two rounds

◮ Round 1: run 2-approximation k′-center algorithm in each Si

to find Ti (e.g., Gonzalez’s)

◮ Round 2: run best sequential algorithm for k-diversity on ∪Ti

◮ Need not know D! Stop when radius of k′-clustering is a

factor ǫ of radius of k-clustering

Experiments

Experiments: setup

Experiments: setup Datasets:

◮ Synthetic data: [4M ÷ 1.6G] points in R3 ◮ Real data: musiXmatch dataset

◮ ≈ 250K songs. ◮ bag-of-words model, cosine distance

Experiments: setup Datasets:

◮ Synthetic data: [4M ÷ 1.6G] points in R3 ◮ Real data: musiXmatch dataset

◮ ≈ 250K songs. ◮ bag-of-words model, cosine distance

Platform:

◮ 16-node cluster (Intel i7) ◮ 18GB-RAM/256GB-SSD per node ◮ 10Gbps ethernet ◮ MapReduce: Apache Spark ◮ Streaming: simulation

Experiments: setup Datasets:

◮ Synthetic data: [4M ÷ 1.6G] points in R3 ◮ Real data: musiXmatch dataset

◮ ≈ 250K songs. ◮ bag-of-words model, cosine distance

Platform:

◮ 16-node cluster (Intel i7) ◮ 18GB-RAM/256GB-SSD per node ◮ 10Gbps ethernet ◮ MapReduce: Apache Spark ◮ Streaming: simulation

Experiments: effectiveness of approach Streaming algorithm on real dataset

◮ X-axis: K ◮ Y-axis: approx-ratio (w.r.t. empirical best solution)

Experiments: comparison with state-of-art Our algorithm (CPPU) vs [Aghamolei et al.’15] (AFZ)

◮ MapReduce using 16 machines ◮ 4M points in R3 (max feasible for AFZ) ◮ Our algorithm: k′ = 128

approximation time (s) k AFZ CPPU AFZ CPPU 4 1.023 1.012 807.79 1.19 6 1.052 1.018 1,052.39 1.29 8 1.029 1.028 4,625.46 1.12

Experiments: scalability Scalability in MapReduce wrt input size

◮ Synthetic data

Experiments: scalability Scalability in MapReduce wrt number of processors

◮ Synthetic data: 100M points ◮ 1 processor ≡ streaming algorithm

Diversity maximization under matroid costraints

Diversity maximization under matroid costraints

Diversity maximization under matroid costraints

◮ Consider diversity maximization problems where solutions are

required to satisfy some given matroid constraint

Diversity maximization under matroid costraints

◮ Consider diversity maximization problems where solutions are

required to satisfy some given matroid constraint

◮ [Abassi et al. KDD’13]: sequential (2 + ǫ)-approximation

(based on local-search) for remote-clique diversity under matroid constraint

Diversity maximization under matroid costraints

◮ Consider diversity maximization problems where solutions are

required to satisfy some given matroid constraint

◮ [Abassi et al. KDD’13]: sequential (2 + ǫ)-approximation

(based on local-search) for remote-clique diversity under matroid constraint

◮ Our approach can be generalized to provide

(1 + ǫ)-composable core-sets for all div’s under partition/transversal matroid constraint in doubling spaces. Main adaptations required:

Diversity maximization under matroid costraints

◮ Consider diversity maximization problems where solutions are

required to satisfy some given matroid constraint

◮ [Abassi et al. KDD’13]: sequential (2 + ǫ)-approximation

(based on local-search) for remote-clique diversity under matroid constraint

◮ Our approach can be generalized to provide

(1 + ǫ)-composable core-sets for all div’s under partition/transversal matroid constraint in doubling spaces. Main adaptations required:

◮ Selection of k′

Diversity maximization under matroid costraints

◮ Consider diversity maximization problems where solutions are

required to satisfy some given matroid constraint

◮ [Abassi et al. KDD’13]: sequential (2 + ǫ)-approximation

(based on local-search) for remote-clique diversity under matroid constraint

◮ Our approach can be generalized to provide

(1 + ǫ)-composable core-sets for all div’s under partition/transversal matroid constraint in doubling spaces. Main adaptations required:

◮ Selection of k′ ◮ Selection of delegates in each cluster to include in core-set

Summary and future work

Summary

Summary

◮ (1 + ǫ)-(composable) core-sets for diversity maximization in

doubling spaces (under partition/transversal matroid costraint)

◮ 1-pass/2-round Streaming/MapReduce algorithms ◮ Experiments on real and synthetic data reveal effectiveness,

efficiency and scalability of our approach

Future Work

Future Work

◮ Further exploit randomization to obtain lower space

constraints in MapReduce

◮ Extend to broader classes of metric spaces ◮ Deal with general matroid constraints ◮ Incorporate other constraints (e.g., density)

Future Work

◮ Further exploit randomization to obtain lower space

constraints in MapReduce

◮ Extend to broader classes of metric spaces ◮ Deal with general matroid constraints ◮ Incorporate other constraints (e.g., density)