2016/11/08 - - PDF document

08/11/16 1

第五章 漫谈服务计算新技术

授课：孙海龙 北京航空航天大学 2016/11/08

提纲

• 无处不在的“服务”
• Mashup：一种新型服务组合
• 服务推荐：更加有效的服务发现
• 服务互联网：服务的大规模整合

08/11/16 2

Web 2.0和云计算催生了服务的大 发展: XaaS 云计算的分层体系

08/11/16 3

其他应用服务

• 博客服务
• 微博服务
• 社交服务
• 位置服务
• 多媒体服务
• 网络存储服务
• 搜索服务
• 地图服务
• 邮件服务
• 文档共享服务

5

“服务”的本质

• 让用户以尽可能少的代价实现对其问题的处理

– 不泌需安装特定的软件，使⽤甩浏览器塀完成所有的操作 – 不泌必关⼼忄软件的升级和维护 – 使⽤甩价格低廉，最好免费 – 服务的质量勵要有保障

• 响应速度
• 可⽤甩性
• 可信性

6

08/11/16 4

服务计算路在何方？

• 服务计算解决的问题是什么？

7

网络软件运行环境

服务 服务 服务 服务 服务

服务 服务

无序 有序

服务

无序 有序

自 底 向 上 动 态 组 合 面向服务的软件开发

服务

提高效率：重用+组合 确保质量：动态演化

支付服务 订单服务 配送服务 商品查找服务

• 服务的泛在化：IaaS, PaaS, SaaS

– 软件服务化：软件资源、开发过程、运⾏行降平台 – 数量勵多、规模⼤夨：Web service, Web API

服务化软件面临的问题（1）

问题一：如何管理大规模服务资源， 以支持服务的按需组合？

08/11/16 5

• 以用户为中心

– ⽤甩户量勵快速增⻓門，例禮如Facebook有超过5亿⽤甩户 – ⽤甩户需求的多样性

• 功能性
• ⾮靟功能性

– ⽤甩户需求动态多变

服务化软件面临的问题（2） 问题二：如何动态按需组合服务， 以适应用户需求变化多样的特征？

• 传统软件开发平台

– ⼯左具、中间件维护 – 技术⻔闩槛⾼髙 – 开发效率低

• 平台的开放化趋势

– 编程模型 – 托管平台

服务化软件面临的挑战（3）

测 试 运行环境

开发工具

问题三：如何为服务化软件提供按需、 可信、透明的部署和运行支持？

08/11/16 6

提纲

• 无处不在的“服务”
• Mashup：一种新型服务组合
• 服务推荐：更加有效的服务发现
• 服务互联网：服务的大规模整合

Mashup/Web Mashup

• 中文：混搭、糅合
• A mashup, in web development, is a web

page, or web application, that uses and combines data, presentation or functionality from two or more sources to create new services.-----From Wikipedia.

• Web2.0的典型应用之一，一种将不同来源的数

据和功能无缝组合，形成全新、集成式服务的 模式

• 主要的技术特征

– combination, visualization, aggregation – Easy and fast integration

08/11/16 7

Web Mashup的代表性应用

• 地图 mashups
• Google Maps, Yahoo Maps,

Microsoft Virtual Earth

• 视频与图片mashups
• Flickr, Youtube
• 搜索与购物mashups
• eBay, Amazon
• 新闻 mashups
• Diggdot.us= Digg.com +

Slashdot.org + Del.icio.us Example: http://www.mapdango.com/

新闻Mashup实例

• News+Map

08/11/16 8

国内外研究现状

• 开发平台发展历史

2005年6月29日， Google公开 Google Map API， 引发了地图 Mashup的热潮

国内外研究现状

• 开发平台发展历史

2005年6月29 日，Google Map Yahoo于2007 年2月推出 Yahoo! pipes， 使用户很容易 的创建 Mashup

08/11/16 9

国内外研究现状

• 开发平台发展历史

2005年6月29日， Google推出 Google Map 2007年2月， Yahoo推出 Yahoo! pipes 2007年5月，微软 推出Web Mashup 应用开发工具 Popfly

国内外研究现状

• 开发平台发展历史

2005年6月29日 Google推出 Google Map 2007年2月 Yahoo推出 Yahoo! pipes 2007年5月 微软推出 Popfly 2007年5月31日， Google发布了 Google Mashup Editor测试版，支 持在线的Mashup 创建、调试和测试

08/11/16 10

国内外研究现状

• 开发平台发展历史

2005年6月29日 Google推出 Google Map 2007年2月 Yahoo推出 Yahoo! pipes 2007年5月 微软推出 Popfly 2007年5月31日 Google发布 GME 2008年4月22日， Intel也发布了一个 Mash Maker的网页 插件，可以把网页变 为Widget并混合在一 起构建新的应用

国内外研究现状

• 开发平台发展历史

2005年6月29日 Google推出 Google Map 2007年2月 Yahoo推出 Yahoo! pipes 2007年5月 微软推出 Popfly 2007年5月31日 Google发布 GME 2008年4月22日 Intel发布 Mash Maker 2008年7月1日， IBM发布企业 Mashup平台 IBM Mashup Center

08/11/16 11

国内外研究现状

• 各大厂商推出的Mashup构建平台比较

Yahoo！ Pipes GME Microsoft Popfly Intel Mash Maker IBM Mashup Center 开发界面 拖拽方式 文本编辑 拖拽、文本 鼠标操作 拖拽方式 目标人群 高级Web用 户 程序员 普通web用 户 普通web用户 企业用户 系统需求 普通浏览器 普通浏览器 SilverLight Mash Maker插 件 WebSphere Application Server 内容类型 数据、逻辑 数据、逻辑、 界面 数据、逻辑、 界面 数据 数据、界面 其他 重数据显示 轻数据操作 目标提升用户 浏览网页体验

Yahoo! Pipes : Example

08/11/16 12

Yahoo! Pipes : Example 国内外研究现状

• IBM Mashup Center

08/11/16 13

Mashup体系结构

• Source
• API/content providers
• Web Protocols: REST, Web Services, RSS/ATOM
• Screen Scraping
• The mashup site
• This is where the mashup logic resides, it is not

necessarily where it is executed

• Server-side: Dynamic content aggregation
• Client-side: Client side scripting
• The client's Web browser
• This is where the application is rendered graphically and

where user interaction takes place

08/11/16 14

客户端Mashup

Advantages Disadvantages

• 1. Can be easy to implement.
• 2. Don't need to provide a server-side

component

• 3. Better Performance
• 4. Reduces processing load on the server
• 1. Allows maximum two or three

XMLHttpRequests to execute concurrently, can limit performance.

• 2. Provides no buffer to shield the client

from problems in the other web site.

• 3. Needs to handle data of any size that

the other web site returns.

• 4. Needs to handle data in any format that

the other web site returns. 5. You cannot make concurrent and asynchronous calls to many data sources at the same time from the client.

服务器端Mashup

Advantages Disadvantages

• 1. The Java EE and Java SE platforms provide many

libraries that make it easy to access other web sites from the server.

• 2. The proxy used in a server-side mashup can

serve as a buffer between the client and the

• ther web site.
• 3. Allows you to transform the data returned by

a service into a different format and in smaller chunks. 4. You can handle security requirements more easily on the server. 5. You can make concurrent and asynchronous calls to many data sources at the same time from the server. 1. You need to provide a server-side proxy.

• 2. A service request and response needs

to go from the browser to the server- side proxy, and then from the server-side proxy to the mashup server. This can result in a significant delay in receiving the response. 3. You need to protect the server-side proxy from unauthorized use. For example, suppose you mash up with a service that requires the user to pay a

• fee. A malicious user might be able to use

the server-side proxy to access the service and in this way avoid paying the fee.

08/11/16 15

所面临的技术挑战

• 数据描述与语义：不同的数据schema
• 数据质量：incomplete and noisy data
• 服务的可用性与可靠性问题
• 个性化的Mashup
• 自动化的Mashup构建

Mashup相关论文

• MashRank: Towards Uncertainty-Aware and Rank-Aware Mashups.

ICDE 2009.

• Entity Search Strategies for Mashup Applications. ICDE 2012.
• Damia: Data Mashups for Intranet Applications. SIGMOD 2008.
• Data Integration in Mashups. SIGMOD 2008.
• Autocompletion for Mashups. VLDB 2009.
• Building Ranked Mashups of Unstructured Sources with Uncertain
• Information. VLDB 2010.
• Navigating in Complex Mash-Up Applications. VLDB 2010.
• Remash! Blueprints for RESTful Situational Web Applications. WWW

2009.

• Towards the Web of Things: Web Mashups for Embedded Devices.

WWW 2009.

• Mashroom: End-User Mashup Programming Using Nested Tables.

WWW 2009.

• Refactoring Pipe-like Web Mashups for End-User Programmers. ICSE

2011.

30

08/11/16 16

提纲

• 无处不在的“服务”
• Mashup：一种新型服务组合
• 服务推荐：更加有效的服务发现
• 服务互联网：服务的大规模整合

服务推荐

• 服务发现的挑战

– 服务的数量勵多 – 描述模型不泌统⼀丁 – ⽤甩户的需求难以清晰表达

• 提高服务发现的效率: 推荐

– Enhance user experience

• Assist users in finding information
• Reduce search and navigation time

– Increase productivity – Increase credibility – Mutually beneficial proposition

32

推荐：Show Me What I Want

08/11/16 17

33

Recommender Systems

• A recommender system (RS) helps people

that have not sufficient personal experience

• r competence to evaluate the, potentially
• verwhelming, number of alternatives
• ffered by a Web site.

– In their simplest form RSs recommend to their users personalized and ranked lists of items – Provide consumers with information to help them decide which items to purchase

Bracha Shapira. Recommender Systems And Search Engines-Two Sides of the Same Coin?

Example applications

08/11/16 18

35

WHAT BOOK SHOULD I BUY?

36

What movie should I watch?

• The Internet Movie Database (IMDb)

provides information about actors, films, television shows, television stars, video games and production crew personnel.

• Owned by Amazon.com since 1998
• 796,328 titles and 2,127,371 people
• More than 50M users per month.

08/11/16 19

08/11/16

§ In October 2006, Netflix announced it would give a $1 million to whoever created a movie-recommending algorithm 10% better than its own. § Within two weeks, the DVD rental company had received 169 submissions, including three that were slightly superior to Cinematch, Netflix's recommendation software § After a month, more than a thousand programs had been entered, and the top scorers were almost halfway to the goal § But what started out looking simple suddenly got hard. The rate of improvement began to slow. The same three or four teams clogged the top of the leader-board. § Progress was almost imperceptible, and people began to say a 10 percent improvement might not be possible. § Three years later, on 21st of September 2009, Netflix announced the winner.

abcd The Nextflix prize story

37 38

WHAT NEWS SHOULD I READ?

08/11/16 20

39

Where should I spend my vacation?

Tripadvisor.com

I would like to escape from this ugly an tedious work life and relax for two weeks in a sunny place. I am fed up with these crowded and noisy places … just the sand and the sea … and some “adventure”.

I would like to bring my wife and my children on a

holiday … it should not be to expensive. I prefer mountainous places… not too far from home. Children parks, easy paths and good cuisine are a must. I want to experience the contact with a completely different

• culture. I would like to be fascinated by the people and

learn to look at my life in a totally different way.

08/11/16

Usage in the market/products Recommendation State-of-the-art solutions

Method Commonness Examined Solutions Jinni Taste Kid Nanocrowd Clerkdogs Criticker IMDb Flixster Movielens Netflix Shazam Pandora LastFM YooChoose Think Analytics Itunes Amazon Collaborative Filtering v v v v v v v v v v v v Content-Based Techniques v v v v v v v v v v v Knowledge-Based Techniques v v v v v v v Stereotype-Based Recommender Systems v v v v v v v Ontologies and Semantic Web Technologies for Recommender Systems v v v Hybrid Techniques v v v v v v v Ensemble Techniques for Improving Recommendation v future Context Dependent Recommender Systems v v v v v v Conversational/Critiquing Recommender Systems v v Community Based Recommender Systems and Recommender Systems 2.0 v v v v v

40

08/11/16 21

协同过滤：Collaborative Filtering

41

08/11/16

§ kNN - Nearest Neighbor § SVD – Matrix Factorization § Similarity Weights Optimization (SWO)

§ The method of making automatic predictions (filtering) about the interests of a user by collecting taste information from many users (collaborating). The underlying assumption of CF approach is that those who agreed in the past tend to agree again in the future. Collaborative Filtering

1

Description Selected Techniques

Collaborative Filtering

42

08/11/16 22

08/11/16

COLLABORATIVE FILTERING

§ Trying to predict the opinion the user will have on the different items and be able to recommend the “best” items to each user based on: the user’s previous likings and the opinions of other like minded users abcd The Idea

? Positive Rating Negative Rating

43

08/11/16

abcd How it works

How collaborative filtering works? “People who liked this also liked…”

44

Item to Item User to User abcd User-to-User

§ Recommendations are made by finding users with similar tastes. Jane and Tim both liked Item 2 and disliked Item 3; it seems they might have similar taste, which suggests that in general Jane agrees with Tim. This makes Item 1 a good recommendation for Tim. This approach does not scale well for millions of users.

Item-to-Item

§ Recommendations are made by finding items that have similar appeal to many users. Tom and Sandra are two users who liked both Item 1 and Item 4. That suggests that, in general, people who liked Item 4 will also like item 1, so Item 1 will be recommended to Tim. This approach is scalable to millions of users and millions of items.

08/11/16 23

08/11/16

Hamming distance

5 6 6 5 4 8

0 Dislike 1 Like ? Unknown 1 ? 1 1 1 1 1 1 1 1

Current User Users Items User Model = interaction history

1

1st item rate 14th item rate

KNN - NEAREST NEIGHBOR

§ Nearest Neighbor

abcd

45

§ This user did not rate the item. We will try to predict a rating according to his neighbors.

abcd Unknown Rating

§ There are other users who rated the same item. We are interested in the Nearest Neighbors.

abcd Other Users

§ We are looking for the Nearest

• Neighbor. The
• ne with the

lowest Hamming distance.

abcd Nearest Neighbors

§ The prediction was made based

• n the nearest

neighbor.

abcd Prediction

§ The Hamming distance is named after Richard Hamming. § In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different.

abcd Hamming Distance

IMPORTANT ISSUES

• Cold Start
• Implicit/Explicit Rating
• Sparsity

– Long Tail problem - many items in the Long Tail have only few ratings

• Portfolio Effect: Non Diversity Problem

– It is not useful to recommend all movies by Antonio Banderas to a user who liked one of them in the past

• Beyond Popularity

– Gray sheep problem – Black sheep problem

• Inormation Security

– Misuse – Privacy

08/11/16 24

基于内容的推荐系统 Content-based Recommender

47 48

CONTENT-BASED RECOMMENDATION

• In content-based recommendations the

system tries to recommend items that matches the User Profile.

• The Profile is based on items user has liked in

the past or explicit interests that he defines.

• A content-based recommender system

matches the profile of the item to the user profile to decide on its relevancy to the user.

08/11/16 25

49

SIMPLE EXAMPLE

Read update User Profile New books User Profile Recommender Systems Match recommendation

基于上下文：Context-based Recommenders

50

08/11/16 26

08/11/16

§ The recommender system uses additional data about the context of an item consumption. § For example, in the case of a restaurant the time or the location may be used to improve the recommendation compared to what could be performed without this additional source of information. § A restaurant recommendation for a Saturday evening when you go with your spouse should be different than a restaurant recommendation on a workday afternoon when you go with co-workers abcd Overview

Context-Based Recommender Systems

51

08/11/16

§ Recommend a vacation § Winter vs. summer § Recommend a purchase (e-retailer) § Gift vs. for yourself § Recommend a movie § To a student who wants to watch it on Saturday night with his girlfriend in a movie theater. Motivating Examples

Context-Based Recommender Systems

52

08/11/16 27

08/11/16

§ Recommend music § The music that we like to hear is greatly affected by a context, such that can be thought of a mixture of our feelings (mood) and the situation or location (the theme) we associate it with. § Listen to Bruce Springteen "Born in USA" while driving along the 101. § Listening to Mozart's Magic Flute while walking in Salzburg. Motivating Examples

Context-Based Recommender Systems

53

08/11/16

abcd Musicovery.com

§ An Interactive personalized WebRadio § A mood matrix propose a relationship between music and mood. § Ethnographic studies have shown that people choose music peaces according to their mood or mood change expectation.

abcd Details

Information Discovery: Example

“Tell me the music that I want to listen NOW"

54

08/11/16 28

08/11/16

Context-Based Recommender Systems

§ What is the user when asking for a recommendation? § Where (and when) the user is ? § What does the user (e.g., improve his knowledge or really buy a product)? § Is the user

• r with other

? § Are there products to choose or only ? What simple recommendation techniques ignore?

55

08/11/16

§ What is the user when asking for a recommendation? § Where (and when) the user is ? § What does the user (e.g., improve his knowledge or really buy a product)? § Is the user

• r with other

? § Are there products to choose or only ? Plain recommendation technologies forget to take into account the user context.

Context-Based Recommender Systems

What simple recommendation techniques ignore?

56

08/11/16 29

08/11/16

§ Obtain sufficient and reliable data describing the user context § Selecting the right information, i.e., relevant in a particular personalization task § Understand the impact of contextual dimensions on the personalization process § Computational model the contextual dimension in a more classical recommendation technology § For instance: how to extend Collaborative Filtering to include contextual dimensions? abcd Major obstacle for contextual computing

Context-Based Recommender Systems

57

08/11/16

§ Each item in the data base ( ) is a candidate for splitting § Context defines ( ) all possible splits of an item ratings vector § We test all the possible splits – we do not have many contextual features § We choose one split (using a single contextual feature) that maximizes an impurity measure and whose impurity is higher than a threshold abcd Item Split - Intuition and Approach

Context-Based Recommender Systems

58

08/11/16 30

社会推荐：Social Recommenders

59

08/11/16

§ Intuition – Users tend to receive advice from people they trust, i.e., from their friends. § Trusted friends can be defined explicitly by the users or inferred from social networks they are registered to. . abcd Overview

Social Based (Trust based) Recommender Systems

60

08/11/16 31

? 3

Active user

Rating prediction

Trust- based Collaborative Filtering

Active users’ trusted friends

Trust metrics

• Global metrics: computes a single

global trust value for every single user (reputation on the network)

• Pros:

– Based on the whole community opinion

• Cons:

– Trust is subjective (controversial users)

a b d c 1 3 3 2 3

08/11/16 32

Trust metrics (cont.)

• Local metrics: predicts (different) trust

scores that are personalized from the point of view of every single user

• Pros:

– More accurate – Attack resistance

• Cons:

– Ignoring the “wisdom of the crowd”

a b d c 1 5 3 2 ?

推荐系统面临的主要挑战

• 数据稀疏性问题
• 冷启动问题
• 大数据处理与增量问题
• 多样性与精确性的取舍
• 推荐系统的脆弱性
• 用户行为模式的挖掘与利用
• 推荐系统的效果评估
• 用户界面与用户体验
• 多维数据的交叉利用
• 社会推荐

64

个性化推荐的十大挑战. 计算机学会通讯. 2011.

08/11/16 33

推 荐 系 统 的 常 见 评 价 指 标

65

提纲

• 无处不在的“服务”
• Mashup：一种新型服务组合
• 服务推荐：更加有效的服务发现
• 服务互联网：服务的大规模整合

08/11/16 34

67

2015 1990

Specialised networks Internet

Open Servicenet

Ambient Intelligence Ubiquitous Semantic Trusted Platforms

1960 2007

WEB 2.0

Computer at the Centre USERS at the Edge

Hardware Network Devices Abstraction Middleware Grid Virtualization Services Process Semantic

Services For Users Citizens and Business USERS at the Centre Computers at the Edge

«Internet of Services »

欧洲的视角：服务的互联网

－>摘自EU NESSI

System Platform

Consumer Adaptation Service consumers Infrastructure Service Composition BPM view Service Platform

Monitoring

Service Communication Service Discovery Mediation SLA Negotiation Service Coordination Integration Services Reasoning Service Execution Lifecycle Management Service Registration Infrastructure and Data Abstraction Data Management Resources Management Business Process Execution SVN Lifecycle Management Information Services Interaction Services End user Interface Context Handling Mapping users perspectives to business/Integration Native Services External Services Knowledge Modelling Service Modelling SVN Modeling Infrastructure Modelling SBS/SBA Modelling Context Modelling Business Process Modeling

Engineering tools

Requirements Capture

• ffered as

services

08/11/16 35 System Platform

Consumer Adaptation Service consumers Infrastructure Service Composition BPM view Service Platform

Monitoring

Service Communication Service Discovery Mediation SLA Negotiation Service Coordination Integration Services Reasoning Service Execution Lifecycle Management Service Registration Infrastructure and Data Abstraction Data Management Resources Management Business Process Execution SVN Lifecycle Management Information Services Interaction Services End user Interface Context Handling Mapping users perspectives to business/Integration Native Services External Services Knowledge Modelling Service Modelling SVN Modeling Infrastructure Modelling SBS/SBA Modelling Context Modelling Business Process Modeling

Engineering tools

SLA@SOI SOA4ALL MASTER EzWeb RESERVOIR

70

服务的统一描述

Towards a Unified Service Description Language for the Internet of Services: Requirements and First Developments. SCC 2010

08/11/16 36

71

Service Engineering for IoS CHOReOS: A EU FP7 Project

72

08/11/16 37

73 74

08/11/16 38

75

Distributed service choreography

76

A Distributed Framework for Reliable and Efficient Service Choreographies. WWW 2011

08/11/16 39

从互联网到务联网

77

务联网的概念结构

78

08/11/16 40

务联网的科学理论基础

79

Q&A