A Research Framework for Empirical Software Engineering - PowerPoint PPT Presentation

Software Engineering Center Information-technology Promotion Agency, Japan A Research Framework for Empirical Software Engineering Collaboration and Its Application in a Software Development Project Yoshiki Mitani*, Nahomi Kikuchi**, Tomoko Matsumura***, Satoshi Iwamura****, Mike Barker***, Ken-ichi Matsumoto*** *IPA/SEC,NAIST(EASE),**IPA/SEC,***NAIST(EASE),****NTT Software(EASE) Demonstrate and verify empirical software engineering methods A new research framework In response to government policy Field data sharing between Industry & Academia Concept of a “Macro-measurement tools” Target project: A Governmental Multi-Vendor Consortium type Software Project IWFST2005 1 IPA: Information Technology Promotion Agency, Japan EASE: Empirical Approach to Software Engineering project SEC: Software Engineering Center NAIST: Nara Institute of Science and Technology

In-process Project Measurement and Feedback Structure Feedback to in-process Software Development Project EPM Basic Analyzer RDB Development XML Standard Data Format EPM Environment Extended Configuration Analyzer Program Management Development System Operation (CVS) Rile A A Code Clone Source Code B B Bug Analyzer Bug Tracking (GNATS) Bug Report 40 Management Items System Management Operation Rule, Mailing-list Input Web Template C C Management Mail e-Mail System Operation Rule Review 20 Management Items Review Report Sheet Operation Rule, Input Template 類似度の分布 度数 Collaborating 1 0 0 9 0 8 0 7 0 6 0 5 0 Filtering 4 0 3 0 400 Management Items 2 0 1 0 Benchmark Data Report 0 tool 0 0 0 0 0 0 0 0 0 0 . 9 . 8 . 7 6 . . 5 4 . 3 . 2 . 1 . 0 . ～1 ～0 ～0 ～0 ～0 ～0 ～0 ～0 ～0 ～0 . 0 . 9 8 . 7 . . 6 5 . 4 . . 3 . 2 . 1 類似度の値域 統合 スコ 統合 スコ 400 SEC Benchmark Database 5 5 課題管理 課題管理 ープ ープ 4 4 モチベーショ モチベーショ ン ン タ タ イ イ ム ム Items 3 3 (over 1,000projects) 2 2 Q&A Interview to PM 基本動作 基本動作 コ コ スト スト 1 1 Interview 0 0 組織 組織 品質 品質 Check-list 技術 技術 人的資源 人的資源 Q&A Check-list 顧客 顧客 コ コ ミ ミ ュ ュ ニケーショ ニケーショ ン ン Analyzer 調達 調達 リ リ スク スク IWFST2005 2

Table1 Measurement Data Items Proceeding Data item Review record, Bug report(40 items), Specification Change report, Project Inter company mail with basic document, Basic Design Document, CVS repository with source code Data Tool CVS, GNATS, Mailman, EPM Data input form & template: Review report form, GNATS web template form Operation rule CVS, GNATS, Mail, EPM Collected data are kept safely in SEC secret data room. Data collecting Individual IT vender development process: Weekly collect data by media from CVS and GNATS. Method Integration test process: Use single GNATS system and collect data from it. CVS data are same as individual development process. Ended Data item SEC defined 400 bench mark data items, IT skill data based on Project IPA/ITSS Data/ Data collecting SEC defined bench mark data input form( EXCEL template) Bench mark Data Method Data collection timing; end of basic design and end of development Project Monitoring Observe whole development process, Join to project meeting. Context project EPM: Empirical Project Monitor IWFST2005 3 ITSS: Information Technology Skill Standard

Table 2 Data Analysis Example Basic Analysis Source line of code transition, Check-in opportunity, Check-out opportunity & frequency, Number & opportunity of mail, Number & opportunity of bur report, Pending bug transition, Residual bug transition, Extended Analysis Review quality analysis from IEEE 982.1(1988):Fault Density, Defect Density, Measurement from PSP (Personal Software Process) ODC (Orthogonal Defect Classification): Review quality File renewal analysis Code Clone Analysis Source Code Clone Analysis Bench mark Retrieve similar project from 1,000 past project benchmark data . Analysis Skill data Analysis Analyze human factor from IT skill data of development persons IWFST2005 4

Development Measurement Target ASDP: Advanced Software Development Project Project Project Management & Operation Software Tools Software Tool CVS, GNATS, Mailman Management & Operation Measurement Measurement Platform EPM,400itemBenchmarkData,ITSS Macro Platform Measurement Management & Operation Tools A&I Collaboration Industry & Academia EASE project, SEC Management & Operation Collaboration Framework MEXT, METI Trigger Ministry Office Policy Budget and National Policy Policy Prime Minister Leadership Government Policy Fig.1 “Macro Measurement Tools” and its Management & Operation for Software Project Measurement EPM: Empirical Project Monitor MEXT: Ministry of Education, Culture, Sports, Science and Technology ITSS: Information Technology Skill Standard METI: Ministry of Economy, Trade and Industry EASE: Empirical Approach to Software Engineering SEC: Software Engineering Center IWFST2005 5

The major factors to realize measurement and data-sharing • The fundamental I&A collaboration research framework • Concept and equipment of measurement platform • Thoughtful measurement plan • Total framework coordination Acknowledgment This work is supported by IPA/SEC, METI and MEXT of Japan as part of the Comprehensive Development of e-Society Foundation Software program. We thank the researchers in SEC and the EASE project who kindly support Software our project. Engineering IWFST2005 6 Center Information-technology Promotion Agency, Japan