Governance for Artificial Intelligence/ Machine Learning Akbar - - PDF document

10/7/2019 1

Governance for Artificial Intelligence/ Machine Learning

Akbar Siddiqui Technical Director Civil Liberties, Privacy, and Transparency Office National Security Agency

1 2

10/7/2019 2

The N SA Mission

Signals Intelligence

The National Security Agency is responsible for:

Providing our nation’s policy makers and military commands with foreign intelligence to gain a decisive advantage.

U.S. Cybersecurity

Protecting and defending sensitive information systems and networks critical to national security and infrastructure.

What is AI/ML?

Unsupervised Learning

MACHINE LEARNING

Reinforcement Learning Supervised Learning Game AI Skills Acquisition Learning Tasks Robot Navigation Real-time Decisions Estimating Life Expectancy Population Growth Prediction Market Forecasting Weather Forecasting Advertising Popularity Prediction

Classification Regression

Diagnostics Customer Retention Image Classification Identity Fraud Detection Meaningful Compression

Clustering

Big Data Visualization Structure Discovery Feature Elicitation Recommender Systems Targeted Marketing Customer Segmentation

Dimensional Reduction

Raw Data Labeled Training Data Experimental Parameters Models Output

3 4

10/7/2019 3

Governance in the Process

Purpose Collect Process Evaluate Retain

Disseminate Authority

Training Guidance Compliance Controls Technical Safeguards

Governance in the Process

Purpose Collect Process Evaluate Retain

Disseminate Authority

Training Guidance Compliance Controls Technical Safeguards

5 6

10/7/2019 4

Governance in the Process

Purpose Collect Process Evaluate Retain

Disseminate

Purpose Training Data Develop Model Apply to Data Use Outputs

Feedback

Machine Learning Process Safeguards

Purpose

• Authorities, Ethical

Use

• Explainable

Purpose and Methods

Training Data

• Collect or generate

training data and test data

• Documentation:

Datasheets for Datasets

Develop Model

• Explainability
• Testing/Validation
• Confidence Levels
• Documentation:

Model Cards

Apply to Data

• Use Limitation
• User Interpretation

and Validation

Use Outputs

• Accountability
• Explainability and

Redress

• Human control
• Confidence level

follows outputs

Feedback

• Build user into workflow
• Check for adversarial

techniques

7 8

10/7/2019 5

Defin fined Pur urpose an and Use se

• Governance Bodies
• Check for Authorities
• Check for Ethical Use (Principles)
• Explainable Purpose and Methods

Purpose

Trai aining Data

• Collect or generate training data and test data
• Data Selection, Feature Engineering, Labeling
• Issue: Collecting and maintaining “negative” examples
• Documentation: Datasheets for Datasets
• Identify and document features, purpose, limitations, and

known issues

• biases (explicit and implicit)

Training Data

Purpose

9 10

10/7/2019 6

Mod

• del

l Devel elopment

• Explainability
• Testing and Validation
• Check for bias in weights/methodology
• ID situations where model performs poorly/unreliably or is vulnerable to

adversarial techniques

• Confidence Level
• Documentation: Model Cards

Develop Model Training Data Purpose

Stakeholders

Model Mission Review Peer Review Senior Operations Data Officer (SODO), Senior Operations Analytics Officer (SOAO) Sharing Equities (Equities Review Board) Mission Risk Acceptance (Mission Element Owners) Security Business Intelligence Metrics (ROI) Labeled Data Civil Liberties, Privacy, and Transparency (CLPT) Compliance for Dissemination (LPOC) Legal Deployment Compliance (AVG) Chief Data Officer (CDO)

11 12

10/7/2019 7

App pply lying Mod

• dels to
• Data
• Use Limitation
• User Interpretation and Validation

Develop Model Training Data

Purpose

Apply to Data

Usin sing Out utputs

• Accountability
• Explainability and Redress
• Human control
• Confidence level follows outputs

Develop Model Training Data Purpose Apply to Data Use Outputs 13 14

10/7/2019 8

Fee eedback

• Build user corrections into workflow
• Drift
• Biased weights
• Check for adversarial techniques

Develop Model Training Data Purpose Apply to Data Use Outputs

Feedback

Q & A

15 16

10/7/2019 9

18

Mitigating Adversarial Machine Learning

*Popular Science: Fooling The Machine, The

Byzantine science of deceiving artificial intelligence; Dave Gershgorn (March 30, 2016)

Original

• utcome →

Tricked

• utcome →

Machine learning (ML) can be a solution to scalable defensive and offensive measures for

• cybersecurity. These can range from semi-

automated decision support to fully-automated

• capabilities. However, ML models can be exploited

in at least four ways. Adversaries can: (a) poison training data used to train ML algorithms to degrade prediction quality,

• r redirect predictions, altogether;

(b) evade by manipulating runtime data to ensure ML models misclassify malicious behavior as benign; (c) infer records into the training data; and (d) reconstruct the ML model for further analysis and exploitation. When ML models of varying qualities are integrated into an ensemble, an adversary can exploit weaknesses in individual models to coordinate a malicious effect in the overall system.

17 18