2020 LAS Collaborators Week Dr. Alyson Wilson Dr. Matt Schmidt - - PowerPoint PPT Presentation

▶
2020 las collaborators week
SMART_READER_LITE
LIVE PREVIEW

2020 LAS Collaborators Week Dr. Alyson Wilson Dr. Matt Schmidt - - PowerPoint PPT Presentation

Feb 12, 2024 260 likes •604 views

2020 LAS Collaborators Week Dr. Alyson Wilson Dr. Matt Schmidt Jamie Roseborough LAS Principal Investigator LAS Director of Programs LAS Director of Outreach and Engagement Dr. Christine Brugh Dr. Jascha Swisher Lori Wachter LAS

slide-1
SLIDE 1

1

2020 LAS Collaborators’ Week

  • Dr. Alyson Wilson

LAS Principal Investigator

  • Dr. Matt Schmidt

LAS Director of Programs

Jamie Roseborough

LAS Director of Outreach and Engagement

  • Dr. Christine Brugh

LAS Technical Program Manager

  • Dr. Jascha Swisher

LAS Technical Program Manager

Lori Wachter

LAS Technical Program Manager

June 15-18, 2020

slide-2
SLIDE 2

Contact Info

  • General Inquiries: lasoutreach@ncsu.edu
  • Specific Inquiries:

○

Alyson Wilson, agwilso2@ncsu.edu

○

Matt Schmidt, mcschmid@ncsu.edu

○

Jamie Roseborough, jvrosebo@ncsu.edu

  • LAS Collaborators Week Website:

https://ncsu-las.org/2020-las-collaborators-day/

2

slide-3
SLIDE 3

3

LAS Collaborators Week Schedule

  • Monday, June 15: Plenary Session
  • Tuesday, June 16: “How to Work with LAS” Sessions
  • Wednesday, June 17: Technical “Office Hour” Sessions

○

Analytic Rigor and Performance

○

Data Triage

○

Influence Campaigns

  • Thursday, June 18: Technical “Office Hour” Sessions

○

Machine Learning Integrity

○

Human Machine Collaboration

○

Selected Cybersecurity Challenges

○

Additional Use Cases

slide-4
SLIDE 4

Plenary Session

Monday, June 15

  • Overview of LAS and how we work
  • Overview of 2021 LAS interest areas
  • Overview of the white paper submission process

4

slide-5
SLIDE 5

“How to Work with LAS” Sessions

Tuesday, June 16

  • Purpose

○

Answer questions about logistics of working with LAS

○

Provide general suggestions about how your research interests might align with the different LAS interest areas.

  • Individual sessions conducted via Zoom

○

Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/

○

Two available blocks of time slots

■

09:00a – 11:00a (EDT)

■

01:00p – 03:00p (EDT)

5

slide-6
SLIDE 6

Technical “Office Hour” Sessions

Wednesday, June 17 & Thursday, June 18

  • Purpose

○

Provide an opportunity to speak with LAS staff who have related interests about potential project ideas and collaborations.

  • Individual sessions conducted via Zoom

○

Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/

○

Wednesday, June 17

■

09:00a – 11:30a (EDT) : Analytic Rigor and Performance (CFWP Section 3.1)

■

12:00p – 02:30p (EDT) : Influence Campaigns (CFWP Section 4.1)

■

02:30p – 05:00p (EDT) : Data Triage (CFWP Section 3.4) ○

Thursday, June 18

■

09:00a – 11:30a (EDT) : Machine Learning Integrity (CFWP Section 3.2)

■

12:00p – 02:30p (EDT) : Selected Cyber Security Challenges (CFWP Section 4.2)

■

12:00p – 02:30p (EDT) : Additional Use Cases (CFWP Section 4.3)

■

02:30p – 05:00p (EDT) : Human Machine Collaboration (CFWP Section 3.3)

6

slide-7
SLIDE 7

Questions

  • If you would like to ask a question please use the Q&A feature in

Zoom

  • We have multiple places in the talk where we will pause to answer

questions from the Q&A

  • If you are unable to ask your question through the Q&A feature

today, please email lasoutreach@ncsu.edu with your question, and we will get back to you.

7

slide-8
SLIDE 8

What is the Laboratory for Analytic Sciences?

LAS is a mission-oriented academic-industry-government research collaboration that works at the intersection of technology and tradecraft.

8

https://ncsu-las.org/

slide-9
SLIDE 9

9

Advance the tradecraft of intelligence analysis while leveraging novel and recent advances in research and technology

  • Investigate technical approaches with the potential to

address analysis challenges

  • Develop analytic tradecraft that leverages research

and technology to address mission needs

  • Transition technology and tradecraft to partners who

can operationalize and scale solutions

slide-10
SLIDE 10

10

How do we work at LAS?

  • Mission-relevant projects
  • ~90% of our work is unclassified
  • Integrated, team-based approach
  • Guidance is intentionally open-ended, as we are expecting you to

help shape the direction of the projects

slide-11
SLIDE 11
  • 24 faculty (and ≈ 35 students) at 9 unique universities
  • 7 industry partners and 1 national lab
  • ≈ 50 government staff/IC partners
  • 14 NCSU staff

11

Who is participating with LAS in 2020?

slide-12
SLIDE 12

12

What are we looking for in 2021 Collaborators?

  • Immersive

○

Iterative approaches to solutions

○

Opportunistic approaches to solutions

  • Interdisciplinary

○

Researchers, developers, and practitioners

○

STEM, humanities, and social sciences

  • Relevant Expertise

○

Relevant to their own activities

○

Potentially relevant to other activities

slide-13
SLIDE 13

13

What are we looking for in 2021 Projects?

  • Impact

Will a successful outcome have a positive impact for intelligence analysts?

  • Innovation

Is a new approach proposed, or does it utilize new capabilities?

  • Engagement

Are LAS stakeholders interested in collaborating on the project?

slide-14
SLIDE 14
  • New Understanding

○

Experimental data

○

Research Papers

  • New Tradecraft

○

Storyboards

○

Documented Workflows

  • New Capabilities

○

Proofs-of-concept (e.g. Jupyter Notebooks)

○

Prototypes

14

What are we looking for in 2021 Outcomes?

slide-15
SLIDE 15

Questions

  • If you would like to ask a question please use the Q&A feature in

Zoom

15

slide-16
SLIDE 16

16

Human-Machine Collaboration Machine Learning Integrity Influence, Cybersecurity, and Other Use Cases Analytic Rigor and Performance Triage

What are our areas of interest for 2021?

slide-17
SLIDE 17

17

Analytic Rigor and Performance

Analytic production and journalism Evaluating rigor in analytic workflows Augmenting analytic performance Applying rigor to language analysis

Defining and Evaluating rigor and its components Identifying the “Fundamental Five” of analyst performance

slide-18
SLIDE 18

from Amershi et al (2019)

Machine Learning Integrity

18

ML in Production

Define and support best practices for machine learning operations

People and ML

Improve interactions between humans and algorithms

End Users

Encourage appropriate trust in automated predictions ML human factors

Data Scientists

Accelerate development of reliable models ML explainability

End Users as Data Scientists

Empower individual end users to address their own challenges through ML User-centric document classification Label, build, deploy, monitor R&D not finished products

slide-19
SLIDE 19

Human Machine Collaboration

19

Recognizing Intent

Understand what an analyst is trying to do

Modeling intent in open-world environments

Useful interventions

Effectively support analysts in achieving their goals

Comparative studies

Microsoft Office Assistant, used with permission from Microsoft. From Wikipedia User:Norm from Crouser et al (2020) from Farrell and Ware (2020) from Hong and Watson (to appear) from Guo et al (2020)

slide-20
SLIDE 20

20

Data Triage

Data Tagging

Data Retention

Information Retrieval Data Prioritization Data Exploration & Survey

Data Triage concerns the Classic Challenges of Big Data

slide-21
SLIDE 21

Influence Campaigns

Indicators,

  • rigins, &

provenance Message content Impact & effectiveness Countering malign influence 21 “The collection of tactical information about an adversary as well as the dissemination of propaganda in pursuit of a competitive advantage

  • ver an opponent” (RAND)

Influence can be:

  • Online or offline
  • Authentic or

inauthentic activity

  • Targeted or broad
  • “Innocuous” or malign
slide-22
SLIDE 22

Selected Cybersecurity Challenges

22

  • Vulnerability Detection

○

Symbolic Execution without Source Code

  • Malware Evolution and Triage

○

Polymorphic vs Metamorphic Obfuscation & Detection Techniques

  • Endpoint Detection and Response

○

Machine Learning Research for EDR

  • Cybersecurity Policy

○

Effectiveness Assessment

slide-23
SLIDE 23

Prioritization of Voice Data User-Centric Document Categorization Processing Uniquely Structured Forms Handwriting Recognition in Scanned Docs

Additional Use Cases

23

slide-24
SLIDE 24

Questions

  • If you would like to ask a question please use the Q&A feature in

Zoom

24

slide-25
SLIDE 25

25

LAS White Paper and Proposal Timeline

  • May 28, 2020

Call for White Papers

  • June 15-18, 2020

LAS Collaborators Week

  • July 17, 2020

White Papers Due

  • Sept 15, 2020

Preliminary Notifications

  • Nov 1, 2020

Final Notifications

  • Jan 1, 2021

Begin Period of Performance

  • Dec 31, 2021

End Period of Performance

slide-26
SLIDE 26

White Paper Submissions

  • In order to propose work, you must submit a white paper
  • More than one submission is fine. You should submit one white

paper for each project idea you have.

  • You may submit team white papers with more than one performer.

White Papers Due July 17, 2020

26

slide-27
SLIDE 27

White Paper Submissions

  • All white papers must be submitted through web-based tool

○

Link: https://whitepapers.ncsu-las.net

  • Each white paper submission must include:

○

Title

○

All Funded PIs and Main POC

○

Abstract

○

Budget Request

○

Technical Description

We ask that your abstract and white paper NOT contain classified, proprietary, or sensitive information of any kind.

27

NOTE: These will be entered separately in the submission tool and do not have to be repeated in the Technical Description

slide-28
SLIDE 28

White Paper Content Guidelines

  • Detailed guidelines for what to include in a white paper are given in

Section 6.2 of the Call for White Papers

○

Link: https://ncsu-las.org/2021-call-for-white-papers/

  • Generally, the most important parts of your white paper will be the

descriptions of:

○

The proposed effort (what question are you answering or problem are you solving)

○

The proposed approach (how will you address the question/problem)

○

How the work aligns with LAS areas of interest

○

The specific deliverables you expect from your work

28

slide-29
SLIDE 29

White Paper Structure Guidelines

  • Each whitepaper should be no more than 2 pages
  • Optional additional page to discuss possible extensions to 2022
  • Optional additional page to describe team capabilities, although a

link to a website is preferred

29

slide-30
SLIDE 30

White Paper Budget Guidelines

Academic Partners

  • Standard award is equivalent to:

○

One month of summer salary support or academic release, plus

○

One 12-month graduate student, plus

○

$3,600 in other direct costs

  • Award can be used for post-docs, undergrads, etc., but must stay

within total budget

  • You may submit up to three additional scope options at the level of
  • ne additional graduate student each

30

slide-31
SLIDE 31

White Paper Budget Guidelines

Industry Partners

  • Standard award is $250k or less, which includes all direct and

indirect costs

  • You may submit up to three additional scope options at the level of

an additional $100k each

  • If these levels of effort do not seem appropriate to the work you

would like to propose, please contact Dr. Matt Schmidt, LAS Director of Programs, mcschmid@ncsu.edu, to discuss other

  • ptions.

31

slide-32
SLIDE 32

Questions

  • If you would like to ask a question please use the Q&A feature in

Zoom

32

slide-33
SLIDE 33

33

LAS Collaborators Week Schedule

  • Monday, June 15: Plenary Session
  • Tuesday, June 16: “How to Work with LAS” Sessions
  • Wednesday, June 17: Technical “Office Hour” Sessions

○

Analytic Rigor and Performance

○

Data Triage

○

Influence Campaigns

  • Thursday, June 18: Technical “Office Hour” Sessions

○

Machine Learning Integrity

○

Human Machine Collaboration

○

Selected Cybersecurity Challenges

○

Additional Use Cases