Department of Defense Budget Level Analysis of Science and - - PowerPoint PPT Presentation

1

Department of Defense Budget Level Analysis of Science and Technology (DoD BLAST)

SYST/OR 699 Master’s Project

Team

Craig Jayson (SE) Josh Singh (SE) Phillip Sutton (OR)

Sponsor Systems Planning and Analysis, Inc. (SPA) is a professional services contractor located in Alexandria, VA. Project This project is a continuation of a capstone project selected last year that supported categorizing and analyzing past, present, and future budget submissions. Purpose of Brief Provide an overview of the DoD BLAST team’s work on the Budget Exhibit Analysis Tool (BEAT). The team will cover where we started, what we did, and recommendations for future work.

Agenda

2

Problem Definition Scope Architecture Data & Visualization Way Forward

• Use Cases
• Structured Analysis
• BEAT Methodology
• Key Definitions
• Background
• Problem Statement
• Stakeholders
• BEAT Database

Update

• Program Analysis

Dashboard

• Project Analysis

Dashboard

• Reporting Functions
• Scope
• Requirements
• Deliverables
• Third Offset

Example

• Recommendations
• Conclusion

Key Defenitions

3

Problem Definition Scope Architecture Data/Visualization Way Forward

Presidential Budget

The president submits a comprehensive budget request to Congress in early February which outlines the administration’s policy and funding priorities and economic

• utlook for coming year.

DoD Budget

The annual appropriation of discretionary spending that goes to any military related expenditures.

Source: OMB

RDT&E

RDT&E appropriations finance research, development, test and evaluation efforts performed by contractors and government installations to develop equipment, material, or computer application software.

Budget Activity A category within each appropriation and fund account that identify the purposes, projects, or types

• f activities financed by the

appropriation or fund Program Element The primary data element in the RDT&E budget and the smallest aggregation of resources used by the Office

• f the Secretary of Defense

(OSD) for analysis.

Background (1 of 3)

In support of the National Military Strategy (NMS), the Department of Defense (DoD) budget places emphasis on a number of items to include the need to invest in future capabilities to ensure the U.S. remains a modernized, technologically superior gloabal force that is able to project power across the globe. This funding is found in RDT&E appropriation.

4

Supplemental Funding

Figure: DoD Budget from 1950 – 2025 in FY16 dollar (future years estimated) Figure: FY 2016 DoD and OCO budget (source: DoD.Gov)

Research, Development, Testing & Evaluation (RDT&E) makes up 11.8% of the total DoD budget in FY 2016.

Problem Definition Scope Architecture Data/Visualization Way Forward

Background (2 of 3)

Last Year: System Planning and Analysis teamed with students in the Master’s of Systems Engineering and Operations Research to initiate Phase 1 of a multi-year effort to develop a Decision Management System that assesses trends in Science & Technology (S&T) investing within DoD.

5

DOD Programs (Found in DoD RDT&E) 01 Strategic Forces 02 General Purpose Force 03 Intelligence and Communication 04 Mobility Forces 05 Guard and Reserve Forces 06 Research and Development 07 Central Supply and Maintenance 08 Training, Medical, and Other 09 Administrative and Associated Activities 10 Support of Other Nations 11 Special Operations Forces Budget Activity BA-1 Basic Research BA-2 General Purpose Force BA-3 Advance Technology Development BA-4 Advance Component and Prototypes BA-5 System Development and Demonstration BA-6 RDT&E Management Support BA-7 Operational System Development PE Number PE Name 063286E Advanced Aerospace Systems 0603211F Aerospace Technology Dev/Demo

Problem Definition Scope Architecture Data/Visualization Way Forward PE Number Key 06 – DoD Program 3 – Budget Activity 28 – Serial Number Assigned 6 – Equipment / Activity Type E- Code for Service / Agency

PE / BLI Program Element / Budget Line Item (BLI) Title Include in TOA FY 2015 (Base & OCO) FY 2016 Base Enacted FY 2016 OCO Enacted FY 2016 Total Enacted FY 2017 Base FY 2017 OCO FY 2017 Total Classification 0603286E Advanced Aerospace Systems Y 123,292 173,631 173,631 182,327 182,327 U 0603287E Space Programs and Technology Y 172,504 126,692 126,692 175,240 175,240 U 0603739E Advanced Electronics Technologies Y 81,119 76,021 76,021 49,807 49,807 U 0603760E Command, Control and Communications Systems Y 229,945 201,335 201,335 155,081 155,081 U 0603766E Network-Centric Warfare Technology Y 350,323 425,861 425,861 428,894 428,894 U 0603767E Sensor Technology Y 283,905 240,127 240,127 241,288 241,288 U

PE: 0603286E – Advanced Aerospace Systems – FY17 Total $182,327,000

Background (3 of 3)

6

BEA BEAT T Ve Versio ion 1.0

Budget Exhibit Analysis Tool (BEAT) is a Decision Support Tool that that allows users to visualize RDT&E data based on selective queries in order to conduct follow-on exploratory analysis.

6

Problem Definition Scope Architecture Data/Visualization Way Forward

Problem Statement

There is no known method or available tool to digest and depict multiple budget cycles (POM) to facilitate the analysis and characterization of past, current, and future budget information. The Fall 2016 Capstone team was successful in aggregating budget data into a database and creating an Excel tool that performed such analysis at the PE level.

Understanding that there is important information available at the more granular project level, the BEAT has the capacity to provide further information and insights, but currently that capability has not been realized.

7

• DoD Budget data is disbursed into many separate

documents/locations

• Manual formatting of data standard of practice
• No efficient way exists to isolate and visualize data

to meet user-specified criteria

Problem Definition Scope Architecture Data/Visualization Way Forward

Stakeholders

8

Figure: Stakeholders include publishers, resourcing, and recipients (source: gmuseorspa.wordpress.com)

Industry: Private entities whose interests and level of involvement with the DoD vary significantly FFRDCs/UARC Research partnerships in which the DoD is the sponsor of a research facility which is under the administration of either a corporate entity (FFRDCs) or a University. Community of Interest: A network of technical groups that serve as a lasting structure to integrate with experience and technical expertise. DoD: This includes the Services and the S&T

• community. They

receive the tech. and support and collaborate with multiple stakeholders. OSD: Primary defense policy advisor to the President and responsible for formulating and executing policy through the DoD. President/Congress: The DoD submits an annual budget request to the President for review. The President then submits a budget request to Congress. This request is reviewed and subjected to a process of revisions before it is returned to the President, at which point it is either signed into law or vetoed.

Problem Definition Scope Architecture Data/Visualization Way Forward

Scope/Requirements

The project team for Spring 2017 Capstone team will focus on taking the PEs and continuing to add to the database by including all project level data (excludes classified data).

• Time Frame of Study (2011 – 2020) this covers FY’16 PB and POM.

9

Highest Level Requirements MR1: The BEAT system shall be able to ingest project level information and data from DoD documents that summarize PEs. MR2: The BEAT system shall ingest project level description information for each project. MR3: The BEAT system shall ingest five future years of expected project level data. MR4: The BEAT system shall include accomplishment level data for specific projects. MR5: The BEAT system shall visualize RDT&E, specifically project level data, over a pre-determined period. MR6: The BEAT system shall be able to display Program Budget (PB) new program/project/accomplishments have started anew or ended.

Link to Requirements Diagram

Problem Definition Scope Architecture Data/Visualization Way Forward

10

Structural Analysis

A Process Oriented Approach of BEAT Use Cases (Two Primary) Activity Modeling

• IDEF0 (A-0, A0, A5 diagrams)

Data Modeling

• Entity Relationship Diagram

Rule Modeling

• Decision Table

Dynamic Modeling

• Sequence Diagrams

Source: SYST 520 course at https://mymasonportal.gmu.edu

Use Cases

Problem Definition Scope Architecture Data/Visualization Way Forward

BEAT Version 1.0 (Fall 2016 Capstone Project)

R2 Output

Problem Definition Scope Architecture Data/Visualization Way Forward

R1 Data

R1 Data: The R-1 document contains budget information for all RDT&E programs undertaken by the branches of the DoD. For each FY, the document lists the budget for the previous two FY, as well as the budget for the current FY. R2 Data: The R-2 document is specific to each program, identified by a PE number in the R-1. The R-2 serves as the item justification for each program, as the proposed budget for the next four FYs.

12

Database Update

12

• Presidential Budget XML file parsing was migrated from Java to VBA and the database

was moved from a MySQL database to MS Access

• The BEAT has been linked to the Access Database and two new worksheets have been

created to query the data

• The current data set goes back to 2011 and contains 4366 Program Element Records

across the DoD comprising of 9388 Project Records

Problem Definition Scope Architecture Data/Visualization Way Forward

BEAT Version 2.0 Methodology

13

http://www.dtic.mil/dtic/

Download PB data from DTIC (Manual Process) Open BEAT and Select “Upload new R2 files”

BEAT (Excel) Opens Access and runs the XML parsing Code

Excel Sheets Update Existing Queries and Connections

Visualization

Visualization Tool within BEAT that allows user to graphically display data based on inputs

Input Cells for scoping data

User conducts exploratory analysis and informs management

Descriptive Information Problem Definition Scope Architecture Data/Visualization Way Forward

14

• BEAT Program and Project Analysis Dashboards allows the User to analyze

a specific program or projects funding trends and relate those variations to the notes from the budgets all in a single environment

Program/Project Analysis Dashboard

Problem Definition Scope Architecture Data/Visualization Way Forward

15

Program Analysis Dashboard (1 of 6)

• The User can narrow the organizations and programs by selecting a (or all) Budget

Activity of Interest

• From there, the User selects among the organizations and agencies that have

programs under the selected Budget Activity

• The “Select Program” options are then updated and the user can select a Program

belonging to the organization or agency by name of Program Number

Problem Definition Scope Architecture Data/Visualization Way Forward

16

Program Analysis Dashboard (2 of 6)

• Additionally, the User can search for a program using the “Program Search” function
• The User can filter all available Programs by Service/Agency, Budget Activity, or

Keyword Searches

• Once a Program has been identified, the User can select that Program and the

Program Analysis Dashboard will update to the selected Program

Problem Definition Scope Architecture Data/Visualization Way Forward

17

Program Analysis Dashboard (3 of 6)

• Information about the Program selected is populated from the database:
• Program Description
• Earliest/Latest PB Years Available
• Program Notes Per PB Year
• Additionally there is a button below the Program Description to Select the

current Program for analysis of its Projects, described later

Problem Definition Scope Architecture Data/Visualization Way Forward

18

Program Analysis Dashboard (4 of 6)

• Chart plots funding across the various Presidential Budget years providing

the User a visual representation of the changes in funding across the budgets

• Includes the data in a table below the chart to provide values for data

Problem Definition Scope Architecture Data/Visualization Way Forward

19

Program Analysis Dashboard (5 of 6)

• Summary statistics are generated to provide metrics describing the behavior
• f each Fiscal Years budget projections
• Statistics include Mean, Standard Deviation, Variance, Min, Median, Max,

and Range

• A stable budget will have minimal Standard Deviations and Variances

between budget submissions

Problem Definition Scope Architecture Data/Visualization Way Forward

20

Program Analysis Dashboard (6 of 6)

• A heat map highlights the variance in the deltas between Presidential

Budgets (PB)

• The N/A is used to indicate that there is no delta to calculate:

(In PB14 we have a prior year data point in FY 2013, however in PB15, there is no value for FY13, resulting in an #N/A) Note: The PB covers current FY and next FY and then 4 years out (tied to the POM)

Problem Definition Scope Architecture Data/Visualization Way Forward

21

Project Analysis Dashboard (1 of 4)

• The Project Analysis allows the User to investigate the different projects

that belong to each program

• The down selecting process follows the same as the Program Analysis

Dashboard, but adds a Project Selection by Project Name or Project Number

Problem Definition Scope Architecture Data/Visualization Way Forward

22

Project Analysis Dashboard (2 of 4)

• Similar to the Program Analysis Dashboard, the User can filter all available Projects

by Service/Agency, Budget Activity, or Program and Project Keyword Searches

• Once a Project has been identified, the User can select that Project and the Project

Analysis Dashboard will update to the selected Project

Problem Definition Scope Architecture Data/Visualization Way Forward

23

Project Analysis Dashboard (3 of 4)

• The Project Mission, (Earliest and Latest PB Data Years), and Notes are all

imported for the select project, the same as the Program Analysis Dashboard

• Additionally, the tool provides the funding chart and heat map to highlight

the deltas in funding

Problem Definition Scope Architecture Data/Visualization Way Forward

24

Project Analysis Dashboard (4 of 4)

• Removed from the Project Analysis Dashboard are the summary statistics
• In their stead is a pie chart that describes the all the Projects under the

governing Programs control

• This pie chart provides insight as to how the Project selected compares to

the other Projects in that purview

Problem Definition Scope Architecture Data/Visualization Way Forward

25

Reporting Functions: Delta Analysis

• The BEAT also has an analytical feature that will analyze all Programs and Projects

to identify Programs/Projects that have user defined budget variations as a percentage of the budget

• Those Programs and Projects that exceed the delta threshold are logged in a report

that can be reviewed and then explored further within the Program and Project Analysis Dashboards

Problem Definition Scope Architecture Data/Visualization Way Forward

26

Reporting Functions: ID New or Recently Ended Programs

• The BEAT will also parse through both Program and Project Level data to identify

programs that are new to this years PB data or have ended in the last 3 years

Problem Definition Scope Architecture Data/Visualization Way Forward

BEAT v1.0

• JAVA XML Parsing
• MySQL Database
• Database not directly linked to BEAT
• Manual data extraction, transformation, and

loading from MySQL into BEAT

• Visualizes R-1 and R-2 data at the Program

Element level

• Requires the purchase of Smart Charts for

Excel by DEVSCOPE for R-1 Visualizations

• Compares multiple R-2 Programs in a single

Presidential Budget year

BEAT v2.0

• VBA XML Parsing
• MS Access Database
• Access database directly linked to queries

within BEAT (MS Excel)

• Queries automatically refresh after new data

is uploaded into Access

• Program Analysis Dashboard allows for in-

depth analysis of a specific program

• Project Analysis Dashboard allows for in-

depth analysis of a specific project

• Reporting Functions
• Delta Analysis analyzes all data to identify

programs and project with significant variation

• New/Ended Program Identification

BEAT Comparison

Scope Architecture Data/Visualization Way Forward Problem Definition

The updates to the database and XML parsing resulted from sponsor request during initial meetings

27

Deliverables

28

 BEAT continued refinement and project level data uploaded  BEAT user interface improvement with ability to visualize new information  BEAT user manual updated  BEAT test case for final presentation (example of BEAT – the JLTV)  Written report and briefing summarizing results of the analysis  Website with all products (https://stbudgetgmuspring17.wixsite.com/gmuproject)  Listing of all sources  Visualization Tool that identifies when projects begin and end  Identification of future budget trends of projects that are nested with Office of Secretary of Defense’s offset strategy (Next Slides will Provide Example)

Problem Definition Scope Architecture Data/Visualization Way Forward

Third Offset Strategy

Technological superiority has been a staple of the DoD strategy since post WWII. This philosophy and follow-on investment has allowed U.S. to out pace its peers and remain at the forefront of battlefield innovation. However, since the current budget drawdown and the military mission shift to counterinsurgency (COIN), it is no longer safe to assume technological superiority.

29

Problem Definition Scope Architecture Data/Visualization Way Forward

TOTAL RDT&E BUDGET AUTHORITY BY SERVICE (FY 2001–FY 2017)

Counterinsurgency

Drawdown

Source: http://csbaonline.org/uploads/documents/CSBA6196-2017-Budget-Analysis

Opportunity

DoD investing $3.6 billion in FY 2017 and $18 billion over the FYDP (in current year dollars) in capabilities related to the Third Offset Strategy.

Third Offset Strategy (if maintained) provides significant opportunities for future RDT&E investment- should observe trends of increased investments in these capabilities.

Third Offset Strategy

(PE and Project Examples of Tool)

The President’s FY2017 budget resulted in DoD investing in anti-access technology, undersea warfare capabilities, human-machine collaboration, precision munitions, and asymmetric capabilities (EW, Cyber, Space, Laser).

PE and projects of interest (should see an uptick in funding) I) U.S. Army – High Energy Laser (PE: Advanced Weapons Technology) II) U.S. Airforce- Cyber Technology Pattern Recognition and Autonomous Learning (PE: Aerospace Sensors)

30

Problem Definition Scope Architecture Data/Visualization Way Forward

Existing Project with Uptick in future spending New Project

Recommendations

31

Problem Definition Scope Architecture Data/Visualization Way Forward

Retrieve and house accomplishment data in the database Improve the database by creating more tables and linking them appropriately Explore and incorporate predictive analytics tools such as Linear Regressions, Support Vector Machines, K-Nearest Neighbors, etc. to better analyze data (may require more advanced analytical software packages/programming languages Julia, Python, R, SAS) Explore and incorporate textual analytics on the program/project descriptions and notes to investigate correlations between keywords/phrases and budget projections (may require more advanced analytical software packages/programming languages (Julia, Python, R, SAS) Continue to link policy (National Military Strategy, Defense Innovation Initiative) to budgetary trend analysis using BEAT (Significant opportunity for future RDT&E investment if current administration and DoD continues to invest in a Third Offset Strategy) Upgrades to BEAT:

• Heat Chart with conditional formatting based on relative changes
• Link each program to its page in the corresponding budget file

Program/Project/ Accomplishment Data

• Projects, Accomplishment, and Plans

data fields are verbose

• E.g. NLP, Support Vector Machine, K-

Nearest Neighbors

Conclusion

Decision Management System

• Begin with the Decision in Mind
• Transparent & Agile
• Predictive
• Test, learn, continuous improvement

FYDP Data

Lends itself to data analytics because:

• Abundance of training data
• Structured
• Released annually

PB17

Mile 1.0

Mile 5.0

Mile 20

Mile 10

Mile 2.5

Program/Project Data

• Create repeatable process for

importing future data

• Run surface level analytics to

identify trends, significant changes,

• r beginning/ending of funding
• BEAT can present data in minutes

32

Next Step

Problem Definition Scope Architecture Data/Visualization Way Forward

Acknowledgements

• System Planning and Analysis, Inc. – Andy Dichter, Slider

Griese, Chris Anderson, and team

• SEOR Faculty – Prof. Laskey
• OR /SYST 699 Classmates

33

Questions

34

Backup

35

Project Acronyms

36

Acronym Name Definition

BA Budget Activity A Budget Activity (BA) is a category within each appropriation and fund account that identify the purposes, projects, or types of activities financed by the appropriation or fund BEAT Budget Exhibit Analysis Tool Decision Management Support Tool that allows users to visualize RDT&E data based on selective queries in order to conduct follow-on exploratory analysis DoD Department of Defense The executive branch department of the federal government of the United States charged with coordinating and supervising all agencies and functions of the government concerned directly with national security and the United States Armed Forces FFRDC Federally Funded Research and Development Centers Federally Funded Research and Development Centers (FFRDCs) are public-private partnerships which conduct research for the United States Government FY Fiscal Year Budgetary cycle for the DoD that starts in October and ends in September. FYDP Future Years Defense Program Summarizes resources (equipment, manpower, forces) associated by fiscal year, with DoD programs, as approved by the Secretary of Defense. PB President’s Budget The president submits a comprehensive budget request to Congress in early February which

• utlines the administration’s policy and funding priorities and economic outlook for coming year

PE Program Element The primary data element in the FYDP and normally the smallest aggregation of resources used by the Office of the Secretary of Defense (OSD) for analysis. PEs are designed and quantified to be comprehensive and mutually exclusive. POM Program Objective Memorandum

The 5-year FYDP and presents the Services and Defense Agencies proposal on how they will balance their allocation of available resources. The POM includes an analysis of missions, objectives, alternative methods to accomplish objectives, and allocation of resources.

R1/R2 R1/R2 Data The R-1 document contains budget information for all RDT&E programs undertaken by the branches of the DoD. R-2 document is specific to each program, identified by a PE number in the R-1. RDT&E Research, Development, Test & Evaluation RDT&E appropriations finance research, development, test and evaluation efforts performed by contractors and government installations to develop equipment, material, or computer application software

Refrences

37

American Association for the Advancement of Science 2016 [3] - Govini. No Date. DoD Financial Management Regulation Volume 2B, Chapter 5 Department of Defense, Future Years Defense Program (FYDP) Structure, April 2014 at http://www.prim.osd.mil/Documents/DoD_704507_future_years_defense.pdf Department of Defense (DoD) Releases Fiscal Year 2017 President’s Budget Proposal Defense Technical Information Center (DTIC) – www.dtic.mil GMUSEORSPA at https://gmuseorspa.wordpress.com/deliverables/ Military budget of the United States, Wikipedia U.S. House of Representatives, Committee on the Budget at http://budget.house.gov/budgetprocess/ "US Department of Defense Third Offset Standard Market Taxonomy." Analyst Report, Washington, D.C.

Need Statement

There is a need to advance the BEAT system to analyze project level budget data in

• rder to project future trends for better decision making.

The Budget Exhibit Analysis Tool (BEAT) will provide a mechanism for categorizing and analyzing past, present, and future Department of Defense (DoD), Research, Development, Testing & Evaluation (RDT&E) Presidential Budget (PB) submissions.

38

Mission Statement

BEAT SYSTEM User

Problem Definition Scope Architecture Data/Visualization Way Forward

Requirements Diagram

39

Return to Scope/Requirements

Use Case 1: Load R-2 PE and Project Level Data into BEAT

40

System Boundary

Schedule: Main Use Case; must be tested before providing to sponsor Priority: Top, must be executed Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20 Frequency: Every time a download is required due to release of new Presidential Budget for corresponding FY Return to BEAT Functional Description

Use Case 2: Perform Visualization of Project Level Data in BEAT

41

Schedule: Main Use Case; must be tested before providing to sponsor. Priority: Top, must be executed Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20 Frequency: Every time a query and filter for specific project level data is executed

Query data in order to determine what to visualize

Return to BEAT Functional Description

IDEF0 for BEAT (A-0 Diagram)

42

Return to Structural Architecture

IDEF0 for BEAT (A0 Diagram)

43

Return to Structural Architecture

IDEF0 for BEAT (A5 Diagram)

44

Return to Structural Architecture

Entity Relationship Diagram for BEAT

45

Return to Structural Architecture

BEAT Decision Tables (1 of 2)

(Based on IDEF0)

46

Return to Structural Architecture

BEAT Decision Tables (2 of 2)

(Based on IDEF0)

47

Return to Structural Architecture

48

Sequence Diagrams for BEAT

(Based on Use Case 1)

Return to Structural Architecture

49

Sequence Diagrams for BEAT

(Based on Use Case 2)

Return to Structural Architecture

50

Types of Equipment / Activity

51

Types of Service /Agency