Reddicommend Content recommendations for Reddit Motivation Reddit - - PowerPoint PPT Presentation

▶
reddicommend
SMART_READER_LITE
LIVE PREVIEW

Reddicommend Content recommendations for Reddit Motivation Reddit - - PowerPoint PPT Presentation

Sep 12, 2022 226 likes •380 views

Reddicommend Content recommendations for Reddit Motivation Reddit is the fourth-most visited site in the US (8th globally) The ability of users to find content relevant to their interests is important to its utility as a

slide-1
SLIDE 1

Reddicommend

Content recommendations for Reddit

slide-2
SLIDE 2

Motivation

  • Reddit is the fourth-most visited site in the US (8th

globally)

  • The ability of users to find content relevant to their

interests is important to its utility as a content-aggregating site

  • Reddicommend is an engine for personalized subreddit

recommendations

slide-3
SLIDE 3

Pipeline

slide-4
SLIDE 4

Implementation

Collaborative filtering using similarity measure based on the Pearson correlation:

subreddit 1 subreddit 2 subreddit 3 u s e r 1 u s e r 2 u s e r 3 u s e r 4 u s e r 5 subreddit 1 subreddit 2 subreddit 3 user 1 user 2 user 3 user 4 user 5 Subreddit correlation matrix

slide-5
SLIDE 5

Scalable matrix multiplication in Spark

  • Managing time complexity

○ Dense-matrix multiplication is O(n3) (for

square matrices) ■ > 1 million users per batch ■ > 100,000 subreddits

slide-6
SLIDE 6

Scalable matrix multiplication in Spark

0. 1. 0. 1. 0. 1. 3. 2. 0. 1. 0. 0. 0. 5. 0. 1.

= x

M N P M: [MatrixEntry(0, 0, 3), MatrixEntry(0, 1, 2), MatrixEntry(1, 0, 1)] N: [MatrixEntry(0, 1, 1), MatrixEntry(1, 1, 1), MatrixEntry(2, 1, 1)]

slide-7
SLIDE 7

0. 1. 0. 1. 0. 1. 1. 2. 0. 0. 0. 0. 0. 3. 0. 0.

= x

M N P M: [MatrixEntry(0, 0, 1), MatrixEntry(0, 1, 2)] N: [MatrixEntry(0, 1, 1), MatrixEntry(1, 1, 1), MatrixEntry(2, 1, 1)] [(0, (0, 1)), (1, (0, 2)] [(0, (1, 1)), (1, (1, 1)), (2, (1, 1))]

Scalable matrix multiplication in Spark

slide-8
SLIDE 8

0. 1. 0. 1. 0. 1. 1. 2. 0. 0. 0. 0. 0. 3. 0. 0.

= x

M N P [(0, (0, 1)), (1, (0, 2)] [(0, (1, 1)), (1, (1, 1)), (2, (1, 1))] [(0, ((0, 1), (1, 1))), (1, ((0, 2), (1, 1)))]

Scalable matrix multiplication in Spark

slide-9
SLIDE 9

0. 1. 0. 1. 0. 1. 1. 2. 0. 0. 0. 0. 0. 3. 0. 0.

= x

M N P [(0, ((0, 1), (1, 1))), (1, ((0, 2), (1, 1)))] [((0, 1), 1), ((0, 1), 2)]

Scalable matrix multiplication in Spark

slide-10
SLIDE 10

0. 1. 0. 1. 0. 1. 1. 2. 0. 0. 0. 0. 0. 3. 0. 0.

= x

M N P [(0, ((0, 1), (1, 1))), (1, ((0, 2), (1, 1)))] [((0, 1), 1), ((0, 1), 2)] [((0, 1), 3)]

Scalable matrix multiplication in Spark

slide-11
SLIDE 11

0. 1. 0. 1. 0. 1. 1. 2. 0. 0. 0. 0. 0. 3. 0. 0.

= x

M N P [(0, ((0, 1), (1, 1))), (1, ((0, 2), (1, 1)))] [((0, 1), 1), ((0, 1), 2)] [((0, 1), 3)] [MatrixEntry(0, 1, 3)]

Scalable matrix multiplication in Spark

slide-12
SLIDE 12

Demo

reddicommend.ddns.net

slide-13
SLIDE 13

Oliver Hoidn

  • B.S. in Physics, Harvey Mudd

College

  • PhD in Physics, University of

Washington Past and current interests:

  • Hiking
  • Violin
  • Giant pumpkin cultivation
  • Lisp and functional

programming