[PPT] - Bayesian Trust Models and Information Security Mozhgan Tavakolifard PowerPoint Presentation

SLIDE 1

www.q2s.ntnu.no

Bayesian Trust Models and Information Security

Mozhgan Tavakolifard Trial Lecture 30 August 2012 Centre for Quantifiable Quality of Service in Communication Systems Centre of Excellence NTNU, Norway

SLIDE 2

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Introduction

(The need for trust)

Bayesian Trust Models and Information Security

SLIDE 3

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

The Need

Local Computing

– Entities exist in a single administrative domain – Interactions governed by common rules

Common understanding of “correct” behavior

– Single authority defines and enforces the rules

National laws, company policies,

social/religious codes – Common infrastructure (software, services, …)

Global Computing

– Entities may roam between multiple domains – No common rules – No authority is able to enforce “correct” behavior – No common infrastructure can be assumed

Bayesian Trust Models and Information Security

1/42 - Introduction

SLIDE 4

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Example 1: Difficulties in adopting security evaluation

standards and importance of quantifications

– Common Criteria Drawbacks

The result is biased as the evaluation is done by one or a few evaluators
Result is not a security level statement, but rather is an assurance level  hard

to rely on for decision making

Evaluations are time and resource demanding

– E.g., we need 1.5 million NOK (about $250,000) and 2-3 working days for EAL 4/4+ in Norway

Required documentation and tests may not be suitable for a particular system
r deployment environment
Example 2: Access control as one of important security services

– Existing models are suited for centralized and relatively static environments

Bayesian Trust Models and Information Security

2/42 - Introduction

SLIDE 5

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

New Security Properties

Large number of (previously unknown) entities

– No permanent and global connectivity can be assumed

No centralized/unique/legitimate authority

– Infrastructure administered by multiple authorities

Increased uncertainty arising from lack of control
No knowledgeable system administrator can be assumed

– Not economically viable

All of the above properties implies increased risk

Bayesian Trust Models and Information Security Bayesian Trust Models and Information Security

3/42 - Introduction

SLIDE 6

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Mitigating Risks

Take out insurance

– Requires risks to be well understood

Not currently the case in global computing
Establish legal contract

– Resolve conflicts according to local rules – Single authority to enforce rules

Establish common authority to mediate interactions

– Trusted third parties (e.g., PKI, Kerberos) – Poor scalability, effectively equivalent to local computing

Restrict interactions to a few “local” domains to avoid risks
Develop trust to allow risk to be assessed and accepted

– Allows interactions with unknown peers – Takes advantage of new services

Bayesian Trust Models and Information Security Bayesian Trust Models and Information Security

4/42 - Introduction

SLIDE 7

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Bayesian Trust Models and Information Security

One of the proposed approaches is to use a notion of computational trust, resembling the concept of trust among human beings

5/42 - Introduction

SLIDE 8

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Bayesian Trust Models and Information Security

3/ - Introduction 6/42 - Introduction

SLIDE 9

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Computational Trust Models Reputation-based Trust Models Probabilistic Trust Models Bayesian Trust Models

SLIDE 10

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Need for formal models of trust

Bayesian Trust Models and Information Security

7/42 - Introduction

SLIDE 11

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Definition Examples

Gambetta: Is Trust Only About Predictability?

– ‘Trust is the subjective probability by which an individual, A, expects that another individual, B, performs a given action on which its welfare depends’[Gambetta,1988]

Focusing only on the mental aspect
Evaluation trust
Mayer, Davis, & Schoorman: Is Trust Only Willingness, for Any

Kind of Vulnerability?

– ‘The willingness of a party to be vulnerable to the actions of another party based on the expectation that the other party will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party’ [Mayer, 1995]

Focusing only on the action aspect

– ‘I trust John but not enough’.

Decision trust

Bayesian Trust Models and Information Security

8/42 - Introduction

Evaluation trust

Decision trust

SLIDE 12

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

McKnight: The Black Boxes of Trust

– Only correlations and mutual influences, precise nature and ‘mechanics’ of the process need to be defined

(McKnight and Chervany, 2001)

Trust as Based on Reciprocity

– ‘the willingness to take some risk in relation to other individuals on the expectation that the others will reciprocate’ (Omstrom and Walker, 2003)

What about those cases that are not based at all on some exchange or cooperation?
Then trust with Technology is meaningless

Bayesian Trust Models and Information Security

9/42

SLIDE 13

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Giddens: Is Trust Based on Norms?

– Trust prediction is based on inference based on some “rules”

However not all expectations are rule-based!
Normality and regularity are not sufficient or necessary for trust
Luhmann: risk, vulnerability and dependability

– “Trust begins where knowledge ends: trust provides a basis dealing with uncertain, complex, and threatening images of the future.” (Luhmann,1979)

O’Hara: Degrees of trust

– Implies that trust is not uniform, but can be described in terms of degree (O’Hara, 2004)

Bayesian Trust Models and Information Security

10/42

SLIDE 14

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Properties

Trust can be a disposition, but also an 'evaluation', and also a

'prediction' or better an 'expectation';

It is a 'decision' and an 'action', and 'counting on' (relying) and

'depending on' somebody;

and which is the link with uncertainty and risk taking (fear and

hope);

It creates social relationships;
It is a dynamic phenomenon with loop-effects;
It derives from several sources.

Bayesian Trust Models and Information Security

11/42 - Introduction

SLIDE 15

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Sources of Trust

Previous Direct Experience
Inference from a class or category
Analogy
Pseudo-transitivity
Reputation
Norms & Policies
Generalized Trust; Trust atmosphere; Trust by Default
…

Bayesian Trust Models and Information Security

12/42 - Introduction

SLIDE 16

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Computational Trust Models Reputation-based Trust Models Probabilistic Trust Models Bayesian Trust Models

SLIDE 17

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Reputation

Behavioral

– “The estimation of the consistency over time of an attribute or entity” [Herbig et al.] – Perception that an agent creates through past actions about its intentions and norms of behavior

Social

– “Information that individuals receive about the behavior of their partners from third parties and that they use to decide how to behave themselves” [Buskens, Coleman...] – Calculated on the basis of observations made by others

An agent’s reputation may affect the evaluation trust that others

have toward it

Bayesian Trust Models and Information Security

13/42 - Introduction

SLIDE 18

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

A good trust model should be [Fullam et al, 05]:
Accurate

– provide good previsions

Adaptive

– evolve according to behaviour of others

Multi-dimensional

– Consider different agent characteristics

Efficient

– Compute in reasonable time and cost 14/42 - Introduction

What is a good trust model?

SLIDE 19

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Reputation-based Trust Models

Bayesian Trust Models and Information Security

Rank ordering
Simple summation or average of ratings
Probabilistic models
Fuzzy models
Flow models
Game theoretical models
Stochastic models
Belief models
Semantic web and ontologies
Spread activation networks
Social network measures
Custom-desinged models

15/42 - Introduction

SLIDE 20

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Computational Trust Models Reputation-based Trust Models Probabilistic Trust Models Bayesian Trust Models

SLIDE 21

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Probabilistic Trust Models

(A short overview)

Bayesian Trust Models and Information Security

SLIDE 22

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

The Mental Aspect (Evaluation Trust)

Bayesian Trust Models and Information Security

SLIDE 23

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Assumptions

– A probabilistic model is assumed for the behavior of the principal – Strong correlation between present and future behavior

Goal

– Predicting the behavior of the principal in the future given the model and the behavior of principal in the past

Trust computation algorithm

– Input: a sequence of observations about the principal behavior – Output: a probability distribution

Uncertainty

– Stochastic: resulting from the randomness of a system itself

Due to the trustee’s behavior
Represented by the probability distribution

– Epistemic: resulting from the observer’s lack of knowledge about the system

Related to amount of information that has been collected about the trustee ↓
Uncertainty about the modeling of probability for stochastic uncertainty

Bayesian Trust Models and Information Security

16/42 – Overview of Models

SLIDE 24

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Frequentist Model

Bayesian Trust Models and Information Security

The simplest probabilistic model
Each principal behaves in each interaction according to a fixed and

independent probability p of positive outcome (and therefore 1-p of negative

utcome)
p is unknown
Input: r observed positive outcome and s observed negative outcome
Output: trust, the probability of positive outcome in the next interaction
According to frequentist statistics, the best (maximum likelihood) estimate for

p is

Main problem: not capturing the epistemic uncertainty

r r  s

17/42 – Overview of Models

SLIDE 25

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Maximum Likelihood Estimation method (MLE)

Fitting a statistical model to data

Sample set, random variable, probability distribution

Estimates the parameter values in a way the likelihood of the sample set be maximized

Exact value of the parameters

On Some Challenges for Online Trust and Reputation Systems

18/42 – Overview of Models

SLIDE 26

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Maximum Likelihood Model (Despotovic and

Aberer)

Bayesian Trust Models and Information Security

18/42 – Overview of Models

SLIDE 27

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Computational Trust Models Reputation-based Trust Models Probabilistic Trust Models Bayesian Trust Models

SLIDE 28

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Bayesian Models

Bayesian Trust Models and Information Security

Prior probability Posterior probability

19/42 – Overview of Models

SLIDE 29

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Beta Model (Mui et al)

Beta(  r,  s) reputation  E(Beta(  r,  s))     

20/42 – Overview of Models

SLIDE 30

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

21/42 – Overview of Models

SLIDE 31

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Epistemic uncertainty as entropy An information theoretic measure that quantifies the amount of available information

22/42 – Overview of Models

SLIDE 32

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

TRAVOS (Teacy et al.)

Similar formulation for reputation as Bayesian models
Confidence metric: how much of the probability density falls within

some distance of the reputation value

Level of trust  stochastic uncertainty
Confidence  epistemic uncertainty

Bayesian Trust Models and Information Security

23/42 – Overview of Models

SLIDE 33

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Frequentist vs. Bayesian

Criticisms to the Frequentist approaches

– Limited applicability – Misleading observations for small samples

Criticisms to the Bayesian approaches

– The need to assume a priori distribution

A simple comparison

– the Frequentist (F) approach can be worse than the Bayesian (B) approach even when the trials give a “good” result!!!! – E.g.,

p=1/2
n=1, O: {r}, F: {0,1}, B: {0.33,0.66}

– Average difference from true distribution, F: 0.5, B: 0.16

n=2, O: {r, r}, F: {0,1/2,1}, B: {1/4,1/2,3/4}

– Average difference from true distribution, F:0.25, B: 0.12

Bayesian Trust Models and Information Security

24/42 – Overview of Models

SLIDE 34

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Dirichlet Model (Jøsang et al.)

Bayesian Trust Models and Information Security

25/42 – Overview of Models

SLIDE 35

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

BLADE (Regan et al.)

A Bayesian Network

– Nodes: set of random variables – Directed edges: conditional relationships nodes

Bayesian Trust Models and Information Security

26/42 – Overview of Models

Truster Trustee 1 Trustee 2 Recommender 1 Recommender 2 Recommender 3

SLIDE 36

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no Bayesian Trust Models and Information Security

Main drawback: assumption of a fixed distribution to represent principals

27/42 – Overview of Models

BRS (Jøsang et al.)

SLIDE 37

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Hidden Markov Model (ElSalamouny et al.)

Static behavior  as a probability distribution in Bayesian models
Dynamic behavior  Hidden Markov Models (HMM)

– The probability distribution over outcomes changes over time

(Q, π, A, O, B)

– Q: set of states – π: initial distribution on Q – A: Q×Q [0,1]

state transition matrix, probability of changing from one state to another

– O: set of observations – B: Q×O [0,1], observation probability matrix, probability of a particular

bservation in a particular state
Reputation estimation

– Estimation of the probability of each possible outcome in the next interaction (the predictive probability distribution) given a particular sequence of observations – E.g., Forward-Backward algorithm

Bayesian Trust Models and Information Security

28/42 – Overview of Models

SLIDE 38

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

History: 10 successes & 2 failures
A counting algorithm would then assign high probability to a success
ccurring next
But he last two failures suggest a state change might have occurred,
which would in reality make that probability very low

Bayesian Trust Models and Information Security

HMM vs. Frequentist

29/42 – Overview of Models

SLIDE 39

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

System stability

– the expected probability of the HMM remaining in the same state

Bayesian Trust Models and Information Security

HMM vs. BRS

30/42 – Overview of Models

SLIDE 40

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Unstable System

Bayesian Trust Models and Information Security

31/42 – Overview of Models

SLIDE 41

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Stable System

Bayesian Trust Models and Information Security

32/42 – Overview of Models

SLIDE 42

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Very Stable System

Bayesian Trust Models and Information Security

33/42 – Overview of Models

SLIDE 43

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

HMM vs. BRS

Traditional Beta trust models are unable of coping with

dynamic behavior systems

Using a decay scheme enhances Beta estimation in cases

where the system is very stable

Beta estimation error is subject to choosing the optimal

value of decay which depends on the system parameters

Bayesian Trust Models and Information Security

34/42 – Overview of Models

SLIDE 44

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Way of thinking

(binary or fuzzy)

Bayesian Trust Models and Information Security

SLIDE 45

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Evidence Theory (Yu & Singh)

Dempster-Shafer theory: a generalization of probability theory

– The sum of probabilities on all pairwise exclusive possibilities does not need to add up to one – Explicit expression of the epistemic uncertainty – Trust opinion = (belief, disbelief, uncertainty)

Basic Probability Assignment (bpa)

– belief : proportion of the observation that were above the satisfactory threshold – disbelief: proportion of the observation that were below the unsatisfactory threshold – uncertainty : proportion of the observations between two thresholds (epistmeic uncertainty)

Main drawback  Epistemic uncertainty

– This model: a single scalar value – Bayesian models: a distribution over each possible value of the probability modeling stochastic uncertainty (a richer presentation)

Bayesian Trust Models and Information Security

35/42 – Overview of Models

SLIDE 46

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

The Strength of Two

(Probabilistic Calculus + Belief Theory)

Bayesian Trust Models and Information Security

SLIDE 47

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Subjective Logic (Jøsang)

Bayesian Trust Models and Information Security

36/42 – Overview of Models

SLIDE 48

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

The Action Aspect

(Decision Trust)

Bayesian Trust Models and Information Security

SLIDE 49

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Markov Decision Processes

Missing part  the action part of trust (decision to trust)

– How reputation information is collected? – How trust decisions are eventually made?

A natural way to model the decision making process given uncertainty

about possible utility is a Markov Decision Process

MDP: (S, A, T, R)

– S: states (representing stochastic uncertainty)

Each state hold a reputation for each trustee

– A: actions

E.g., asking for recommendations or performing an interaction with others
Cause a move to another state probabilistically as determined by T

– T: state transition function – R: reward function

state follows transaction  value of transaction
Other states  small negative reward

Bayesian Trust Models and Information Security

37/42 – Overview of Models

SLIDE 50

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Information about a subset of a trustee’s past interactions 
nly partial knowledge of the underlying stochastic process
This epistemic uncertainty can be modeled by extending MDP

to POMDP

– By placing a belief distribution over the possible states and using

bservations to adjust this belief
POMDP

– Observations

ask: recommendation
transaction: outcome

– Observation function: probability distributions over possible

bservations for each action and its resulting state
We need to find the best course of action that maximizes expected

rewards – Policy is a mapping from belief (probability distribution over state) to action – E.g., by using dynamic programming or policy search techniques

Bayesian Trust Models and Information Security

Partially Observable Markov Decision Processes

38/42 – Overview of Models

SLIDE 51

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Comparison

Bayesian Trust Models and Information Security

SLIDE 52

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Expected Cross entropy

– Cross entropy: an information-theoretic measure for comparing distributions

The average amount of information discriminating two distributions

Bayesian Trust Models and Information Security

Theoretical Comparison (ElSalamouny et al.)

39/42 – Comparison

SLIDE 53

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Example: BRS vs. MLE

Comparison of BRS model with the Frequentist model

– A1: Frequentist model

Probability of good behavior and probability of negative behavior

– A2: BRS model

Probability of good behavior and probability of negative behavior

– A3: Beta model

Comparison result

– p=0 or p=1

CrossEntropy(A1,A3)=0 < CrossEntropy(A2,A3)
A1 computes the exact distribution, whereas A2 does not

– 0<p<1

CrossEntropy(A1,A3)=∞ => A2 is always better!

Bayesian Trust Models and Information Security

r r  s s r  s r 1 r  s  2 s 1 r  s  2

40/42 – Comparison

SLIDE 54

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Conclusion

Bayesian Trust Models and Information Security

SLIDE 55

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Features of Ubiquitous Computing like scalability, mobility, and incomplete

information deeply affect security requirements

Trust-based security decision making
Two main sources of information

– Previous direct experiences – Reputation

An overview of some probabilistic trust models and their challenges

– Trust definition – Trust evaluation vs. trust decision – Binary or fuzzy way of thinking – Representation of uncertainty – Dynamic and static behavior

Bayesian Trust Models and Information Security

41/42 - Conclusion

SLIDE 56

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Advantages of probabilistic trust models

– Incomplete information normally results in probabilistic decision making – Sound and rigorous basis – Accurate explanation of hypothesis and guarantees – Efficient trust computations – Possibility of probabilistic reasoning about principal’s behavior – Common objective and structure

Assume a particular probabilistic model for principal behavior
Put forward algorithms for approximating the behavior of principals

– More generality and flexibility – Considerable amount of research is on probabilistic trust management

Disadvantages

– Complexity, not suitable for human decision making – Lack of scalability – Does not consider rationality of agents (sanctioning-fostering honest behavior)

Bayesian Trust Models and Information Security

42/42 – Summary

SLIDE 57

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

Thank you for your attention

Bayesian Trust Models and Information Security

SLIDE 58

www.q2s.ntnu.no www.q2s.ntnu.no www.q2s.ntnu.no

References

Karl Sentz and Scott Ferson. Combination of evidence in Dempster-Shafer theory. Technical report, Sandia National

Laboratories, 2003.

ElSalamouny, E., Sassone, V., Nielsen, M.: HMM-based trust model. In: Degano, P., Guttman, J.D. (eds.) Formal

Aspects in Security and Trust. LNCS, vol. 5983, pp. 21–35. Springer, Heidelberg (2010)

A. Jøsang and R. Ismail. The Beta Reputation System. Proceedings of the 15th Bled Conference on Electronic

Commerce, Bled, Slovenia, 17-19 June 2002.

Audun Jøsang and Jochen Haller. Dirichlet Reputation Systems. Proceedings of the Second International Conference
n Availability, Reliability and Security (ARES 2007), Vienna, April 2007
B. Yu and M. P. Singh. An evidential model of distributed reputation management. In Proceedings of First

International Joint Conference on Autonomous Agents and Multiagent Systems, pages 294–301, 2002.

K. Regan, P. Poupart, and R. Cohen. Bayesian Reputation Modeling in E-Marketplaces Sensitive to Subjectivity,

Deception and Change. In Proc. of AAAI-06, 2006.

Z. Despotovic, K. Aberer, Maximum Likelihood Estimation of Peers’ Performances in P2P Networks, in: 2nd

Workshop on the Economics of Peer-to-Peer Systems, Cambridge, MA, USA, 2004.

W. T. Teacy , Jigar Patel , Nicholas R. Jennings , Michael Luck, TRAVOS: Trust and Reputation in the

Context of Inaccurate Information Sources, Autonomous Agents and Multi-Agent Systems, v.12 n.2, p.183- 198, March 2006

Bayesian Trust Models and Information Security