SLIDE 1
Apache PredictionIO End-to-End Machine Learning Server with Apache - - PowerPoint PPT Presentation
Apache PredictionIO End-to-End Machine Learning Server with Apache - - PowerPoint PPT Presentation
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated
SLIDE 2
SLIDE 3
What is Machine Learning?
Examples of machine learning applications:
- Facial recognition software
- E-mail spam detectors
- Automated personalized recommendations
SLIDE 4
What is Apache PredictionIO?
SLIDE 5
What is Apache PredictionIO?
Video Streaming E-mail Service Image Data Video Rec. Engine Spam Detection Engine Facial Recognition Engine
Event Server
Data Query via REST Predicted Result
SLIDE 6
An Engine Template is the basic skeleton of an engine.
Recommendation Engine Template Text Classification Engine Template
Item Recommendation Engine User Recommendation Engine Semantic Analysis Engine Spam Detector Engine
Engine user usage: quick, ready, and customizable ML solutions
SLIDE 7
- DASE - the “MVC” for Machine Learning
- Data: Data Source and Data Preparator
- Algorithm(s)
- Serving
- Evaluator
Engine Building an Engine with
Separation of Concerns (SoC)
SLIDE 8
- A. Train deployable predictive model(s)
- B. Respond to dynamic query
- C. Evaluation
Engine Functions of an Engine
SLIDE 9
Engine A. Train predictive model(s)
class DataSource(…) extends PDataSource
def readTraining(sc: SparkContext) ==> trainingData
class Preparator(…) extends PPreparator
def prepare(sc: SparkContext, trainingData: TrainingData) ==> preparedData
class Algorithm1(…) extends PAlgorithm
def train(prepareData: PreparedData) ==> Model
$ pio train
SLIDE 10
Engine A. Train predictive model(s)
Event Server
Algorithm 1 Algorithm 3 Algorithm 2 PreparedDate Engine Data Preparator Data Source TrainingDate
Model 3 Model 1 Model 2
SLIDE 11
class Algorithm1(…) extends PAlgorithm
def predict(model: ALSModel, query: Query) ==> predictedResult
class Serving extends LServing
def serve(query: Query, predictedResults: Seq[PredictedResult]) ==> predictedResult
- B. Respond to dynamic query
Engine
Query via REST
SLIDE 12
- B. Respond to dynamic query
Engine
Algorithm 1 Model 1 Serving
Mobile App
Algorithm 3 Model 3 Algorithm 2 Model 2 Predicted Results Query (input) Predicted Result (output) Engine
SLIDE 13
Engine DASE Factory
- bject RecEngine extends IEngineFactory {
def apply() = { new Engine( classOf[DataSource], classOf[Preparator], Map("algo1" -> classOf[Algorithm1]), classOf[Serving]) } }
SLIDE 14
Running on Production
- Install PredictionIO
$ bash -c "$(curl -s http://install.prediction.io/install.sh)"
- Start the Event Server
$ pio eventserver
- Deploy an Engine
$ pio build; pio train; pio deploy
- Update Engine Model with New Data
$ pio train; pio deploy
SLIDE 15
Production Features
- Model Updating and Versioning
- Offline and Online Evaluation
- Multiple Engine Variants
- Query and Prediction Tracking
SLIDE 16
Universal Recommender
https://templates.prediction.io
SLIDE 17