Failure Prediction for decision makers in Data Centers using Data - - PowerPoint PPT Presentation

▶
failure prediction for decision makers in data centers
SMART_READER_LITE
LIVE PREVIEW

Failure Prediction for decision makers in Data Centers using Data - - PowerPoint PPT Presentation

Sep 28, 2023 536 likes •1.02k views

Failure Prediction for decision makers in Data Centers using Data Mining. Group ID- 39WDIT Team Members IT 11 6002 44 D.G.S.M. Wijayarathne IT 11 6049 90 W.K.S.D Fernando IT 11 6005 58 A.S.M.S Sharfaan IT 11 6073 42 J.S.D

slide-1
SLIDE 1

Failure Prediction for decision makers in Data Centers using Data Mining.

Group ID- 39WDIT

slide-2
SLIDE 2

Team Members

 IT 11 6002 44 D.G.S.M. Wijayarathne  IT 11 6049 90 W.K.S.D Fernando  IT 11 6005 58 A.S.M.S Sharfaan  IT 11 6073 42 J.S.D Fernando  IT 11 6104 58 M.P.L Mendis Internal Supervisor: Mr. Dilhan Manawadu

IT 11 6049 90

slide-3
SLIDE 3

Topics to be covered….

  • 1. Introduction
  • 2. Overall Descriptions
  • 3. Specific Requirements
  • 4. References

IT 11 6049 90

  • 5. Appendices
slide-4
SLIDE 4

Introduction

slide-5
SLIDE 5

Overview

Develop team will implement a system called “WinSeer” to predict Data Center failures. What is the need of predicting Data Center failures? System targets for decision makers.

IT 11 6049 90

slide-6
SLIDE 6

Objectives

IT 11 6049 90

Select best data mining algorithm. Develop a data mining model. Predict data center failures. Acknowledge decision makers about failures.

slide-7
SLIDE 7

Software Architecture Diagram

IT 11 6049 90

slide-8
SLIDE 8

Product perspective.

 Existing Researches.

  • Research 1: Prediction of Hard Drive Failures

via Rule Discovery from Auto Support Data by V. Agrawal, C. Bhattacharyya, T. Niranjan, S. Susarla in Sep 2009 .

  • Research 2: Effective Failure Prediction in

Hadoop Clusters by R. Dudko, A. Sharma, and J. Tedesco .

  • Research 3: A Failure Detection and Prediction

Mechanism for Enhancing Dependability of Data Centers by Q. Guan, Z. Zhang, and S. Fu in October 2012 .

  • Research 4: Host Load Prediction in a Google

Compute Cloud with a Bayesian Model by S. Di1,

  • D. Kondo, W. Cirne in 2012 .
slide-9
SLIDE 9

Product perspective

IT 11 6002 44

“WinSeer” Project Research 1 Research 2 Research 3 Research 4 Prediction factor Data center server failures. Hard Drive Failures. Failure prediction in Hadoop Clusters. Enhancing Dependability of Data Centers. Host Load Prediction. Target Audients Decision makers in

  • rganizations.

Data center administrators. Operators and managers of the cluster. Data center administrators. Google users. Business Goal To increase the data centers’ availability. To avoid loss of data and performance degradation. Data management and monitoring large clusters. Provide high accuracy to the Data Centers. To increase the availability of the search engine. Model Type Open source data mining models. Rule learning algorithms. Novel approach. Bayesian and decision trees models. Based on Bayes model. User Interface Features. Web interface. Net Application. Monitoring systems. Monitoring systems. Web interface.

slide-10
SLIDE 10

Product functions

IT 11 6002 44

slide-11
SLIDE 11

Login View Predictions Product functions

IT 11 6002 44

slide-12
SLIDE 12

Login View Predictions Update Profile Add a new user Product functions

IT 11 6002 44

slide-13
SLIDE 13

Two classes of users.

  • Organizational decision maker
  • System administrator.

Ability to read and understand English. Familiarity with the operation of the basic Graphical User Interface (GUI) of a web browser. Should have an e-mail account to get email alerts.

User characteristics

IT 11 6002 44

slide-14
SLIDE 14

The server monitoring function will be done by the currently available system and the pattern recognition and the failure predictions generating only will be done by the proposed system.

Assumptions and dependencies

IT 11 6002 44

slide-15
SLIDE 15

 Prototype – Processing data set and Integrating tools with ASP.net.  Mid Review - WinSeer data mining model.  Finals - Complete project, focus on predicting the failures at least before 2 weeks’ time.

  • Processing XML data set – Shamini, Premeshini
  • Integrate the open source data mining tools with

ASP.net. – Samith, Sameera

  • Research the mining model algorithms – Saumy,

Sameera, Samith

  • Generate reports - Saumy

Distribution of requirements

IT 11 6002 44

slide-16
SLIDE 16

Specific Requirements

slide-17
SLIDE 17

Detailed user interfaces

  • Login Interface

External interface requirements

IT 11 6005 58

slide-18
SLIDE 18

 Home Page

External interface requirements

IT 11 6005 58

slide-19
SLIDE 19

 Register New User.

Detailed user interfaces

IT 11 6005 58

slide-20
SLIDE 20

 Edit Profile.

Detailed user interfaces

IT 11 6005 58

slide-21
SLIDE 21

 Update and view user details.

Detailed user interfaces

IT 11 6005 58

slide-22
SLIDE 22

Software interface integrations

  • Weka 3.6
  • Knime 2.6
  • RapidMiner 5.3

Communication interface integrations

  • Internet connection is required to feed the

web pages and to access the web interface by the user.

Detailed user interfaces

IT 11 6005 58

slide-23
SLIDE 23

Classes/Objects

IT 11 6005 58

slide-24
SLIDE 24

Performance Requirements

Response Time How fast the system handle individual requests. Should not render resident computer useless for other purposes.

IT 11 6073 42

slide-25
SLIDE 25

Performance Requirement

Throughput How many requests the system can handle. “Winseer” prediction handles datasets of up to 20 GB in size

IT 11 6073 42

slide-26
SLIDE 26

Design Constraints

Easy to access the system. Develop the mining model by using open source tools. Software Interfaces used in “WinSeer”.

IT 11 6073 42

slide-27
SLIDE 27

Software System Attributes.

Reliability

IT 11 6073 42

Availability.

slide-28
SLIDE 28

Software System Attributes.

Security.

IT 11 6073 42

Maintainability.

slide-29
SLIDE 29

Supporting Information

slide-30
SLIDE 30

 [1] HowStuffWorks.com Contributors, "Are data mining and data warehousing related?", 20 April 2011. HowStuffWorks.com, [Online]. Available: http://www.howstuffworks.com/are-data-mining-and-data- warehousing-related.htm. [Accessed: March. 23, 2013].  [2] “Database Fundamentals,” 2008. [Online]. Available: http://www.personal.psu.edu/glh10/ist110/topic/topic07/topic07_09.htm

  • l. [Accessed: Mar. 23, 2013].

 [3] B. Sudeshna, Georgia, "DATA MINING," 1997. [Online]. Available: http://www.siggraph.org/education/materials/HyperVis/applicat/data_mi ning/data_mining.html [Accessed: Mar.23, 2013].  [4] M. Bruno, "4 open source data mining tools (with GUI)," April 21

  • 2009. [Online]. Available:

http://www.analyticbridge.com/profiles/blogs/4-open-source-data- mining.  [5] “Data Mining: What is Data Mining?,” [Online]. Available: http://www.anderson.ucla.edu/faculty/jason.frand/teacher/technologies/ palace/datamining.htm. [Accessed: Mar.24, 2013].

References

IT 11 6104 58

slide-31
SLIDE 31

 [6] C.G Carrier, and O Povel, "Characterizing Data Mining software," Intelligent Data Analysis 7, pp. 181-185, August 2003.  [7] V. Agrawal, C. Bhattacharyya, T. Niranjan and S. Susarla, “Prediction of Hard Drive Failures via Rule Discovery from Auto Support Data” pp.782-786, 2009 International Conference on Machine Learning and Applications, Dec. 2009.  [8] R. Dudko, A. Sharma and J. Tedesco, “Effective Failure Prediction in Hadoop Clusters,” Available:https://wiki.engr.illinois.edu/download/attachments/19576688 7/JAR-2nd.pdf? version=3&modificationDate=1333424381000 [Accessed Mar 28, 2013].  [9] Q. Guan, Z. Zhang, and S. Fu, A Failure Detection and Prediction Mechanism for Enhancing Dependability of Data Centers, Vol. 4, No. 5, International Journal of Computer Theory and Engineering, 2012.  [10] S. Di, D. Kondo and W. Cirne, “Host Load Prediction in a Google Compute Cloud with a Bayesian Model,” In Proceedings of IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, Nov. 2012.

References

IT 11 6104 58

slide-32
SLIDE 32

 [11] “Performance requirements documentation,” [Online]. Available: http://pic.dhe.ibm.com/infocenter/aix/v7r1/index.jsp?topic=%2Fcom.ib m.aix.prftungd%2Fdoc%2Fprftungd%2Fdoc_perf_reqs.htm. [Accessed: March. 23, 2013].  [12] “How to write Performance Requirements with Example,” [Online]. Available: http://www.1202performance.com/atricles/how-to-write- performance-requirements-with-example/. [Accessed: March. 23, 2013].  [13]M. Bruno, "4 open source data mining tools (with GUI)," April 21

  • 2009. [Online]. Available:

http://www.analyticbridge.com/profiles/blogs/4-open-source-data-

  • mining. [Accessed: Mar.23, 2013].

 [14]Z. Li, "using data mining techniques to improve software reliable," 2006.  [15]A. Alzghoul, M. Löfstrand,” Increasing availability of industrial systems through data stream mining”, Computers &Industrial Engineering, 2010. [Accessed: Mar.23, 2013].

References

IT 11 6104 58

slide-33
SLIDE 33

 [16]"Non Functional Requirements," 2, Aug 26 2010. [Online]. Available: http://c2.com/cgi/wiki?NonFunctionalRequirements.[Accessed: March. 22, 2013].

References

IT 11 6104 58

slide-34
SLIDE 34

 How you define the scale of your company? (Scale

  • f the servers.)

 What kind of data do your servers handle? (How Critical)  Have you faced any server failures in your company?  How often failures are happening?  How do you get to know when a failure occurred in your company?  How failures affect to your company?  After a failure happens, what are your next action steps?

Interview Questions

IT 11 6104 58

slide-35
SLIDE 35

 How long does it take to recover from a failure?  Do you have any server failure prediction mechanism?

  • If yes;
  • What kind of mechanism do you have to predict failures

in your data centers?

  • Is it cost effective?
  • How early can you get about the failure?
  • Are you satisfied with your system?
  • If no;
  • If you have a failure prediction mechanism, will it be

helpful to your decisions and your company?

  • What is your idea about a failure prediction mechanism?

Interview Questions Cont..

IT 11 6104 58

slide-36
SLIDE 36

National Water Supply & Drainage Board – Ratmalana. Engineer IT 10 years of experience Decentralized method Interviews

IT 11 6104 58

slide-37
SLIDE 37

Billing data, financial data, ERP, Access Directory and Exchange. 3 major failures. Depend on the backup Not any Server monitoring system. Prediction system will be a big help Interviews

IT 11 6104 58

slide-38
SLIDE 38

System Manager - 8 years of experience. Senior Systems Engineer. Data Center and Server rooms -internal purposes. Interviews

IT 11 6104 58

slide-39
SLIDE 39

SOS control systems, Communication systems, Active Directory services, 3 years back – a server failure.

  • 2-3 hours to recover.

Currently - monitoring systems “Yes. It will be a very helpful system” Interviews

IT 11 6104 58

slide-40
SLIDE 40

Nestle Lanka PLC IS Manager - 22 years of experience IT Manager (Infrastructure & Communication) – 10 years of experience Main servers – Sydney, Australia Interviews

IT 11 6104 58

slide-41
SLIDE 41

Sri Lanka- File Servers & Backup Servers. 5 years back major failure – 2 days to recover. Only a monitoring system. Interviews

IT 11 6104 58

slide-42
SLIDE 42

Lanka Communication Services (Private) Limited Manager Pre Sales – 4 years Information Security Engineer – 3 years One & only ‘data only operator’ Data Scale- depends on customer Interviews

IT 11 6104 58

slide-43
SLIDE 43

Monitoring system only No Prediction system It will be a help. Interviews

IT 11 6104 58

slide-44
SLIDE 44

 View Reports

Sequence diagrams

IT 11 6104 58

slide-45
SLIDE 45

 Add User

Sequence diagrams

IT 11 6104 58

slide-46
SLIDE 46

“WinSeer” Failure Prediction for Decision Makers in Data Centers Using Data Mining