SOEN6461 - Software Design Methodologies Software Process and - - PowerPoint PPT Presentation

SOEN6461 - Software Design Methodologies Software Process and Agile Modeling

Winter 2019

Fabio Petrillo Invited Lecturer

This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 3.0 Unported License

Département d'informatique et mathématique

Software Process

Why study software process?

“The main reason for using software processes and software process models as part of IT governance is that they can help to increase the predictability of processes and their results, and in particular the transparency of the work done.” [Kneuper, 2018]

Therac-25 Accidents

• Radiation therapy machine
• Companies

○ Atomic Energy of Canada Limited ○ CGR (France)

• Failure

○ Beta radiation overdose - 100 times the normal dose ○ 1985 - 1887

• Impact

○ the death of at least three patients ○ several injuries

Therac-25 Accidents - Causes

• Software complex race condition

○ https://devopedia.org/race-condition-software ○ https://stackoverflow.com/questions/34510/what-is-a-race-condition

• AECL did not have the software code independently reviewed
• AECL had never tested the combination of software and hardware until it

was assembled at the hospital

• Developed by a single person using PDP11 assembly
• Several years of development!
• Lack of documentation and software test plan
• The investigators could not reproduce the fault condition!

Leveson, Nancy G.; Turner, Clark S. (July 1993). "An Investigation of the Therac-25 Accidents". IEEE Computer. 26 (7): 18–41.

HealthCare.gov Application Crash (“ObamaCare”)

• USA Federal Health Insurance
• Company

○ CGI Group Inc. (Montreal/Canada) ○ CGI was a key contractor

• Failure

○ problematic launch - application poor performance ○ 2013

• Impact

○ The program does not achieve its objective of facilitating health insurance services for millions of Americans ○ American election results??? Venkatesh, V., Hoehle, H., Aljafari, R. (2014). "A usability evaluation of the Obamacare website". Government Information Quarterly. Volume 31, Issue 4, 2014, Pages 669-680

“ObamaCare” Application Crash - Causes

• integration/infrastructure issues
• Lots of subcontractors (47!!!)

○ who was ultimately responsible???

• ~60k user sim -> 250k (naïve)
• Large requirements
• “Big Bang” approach
• Huge pression
• Lack of integration testing

“We didn’t see full end-to-end testing until a couple of days leading up to the launch”

http://adage.com/article/digitalnext/lessons-user-experience-healthcare-gov/244933 http://www.nytimes.com/2013/10/25/us/politics/bipartisan-dismay-over-health-plan-woes-at-house-hearing.html

Can we “manufacture” a software?

Can we “manufacture” a software?

No (not yet)!

Software is developed or engineered, it is not manufactured in the classical sense.

Is there a “Software Factory”?

Is there a “Software Factory”?

• Source code, as high-level languages (as COBOL, C, Rust, or Python), is a

design artefact!

• The compiler is a true “software factory”!
• Compiler -> automation
• When you write code, we are doing a design task.
• Programming is design, not production.

What is a Death March project?

Death March Defined

A Death March is a project is destined to fail In most software death march projects, this usually means one or more of the following constraints has been imposed:

• The schedule has been compressed to less than half the amount of time

estimated by a rational estimating process

• The budget and associated resources have been cut in half
• The staff has been reduced to less than half the number of people that

would normally be assigned to a project of this size and scope

• The functionality, features, performance requirements, or other technical

aspects of the project are twice what they would be under normal circumstances and/or discussed previously (change of requirements)

Why it happens?

Politics, politics, politics.

Death March - Why it happens (more)? (1997)

• Naive promises made by marketing, senior executives, naive project

managers, etc.

• Naive optimism of youth: "We can do it over the weekend!"
• The "start-up" mentality of fledgling, entrepreneurial companies.
• The "Marine Corps" mentality: Real programmers don't need sleep
• Intense competition caused by globalization of markets.
• Intense competition caused by the appearance of new technologies.
• Intense pressure caused by unexpected government regulations.
• Unexpected and/or unplanned crises — e.g., your hardware/software vendor

just went bankrupt, or your three best programmers just died of Bubonic Plague.

So, how to address that problems? What could we do?

There is no way to rescue Death March Projects! Quit as quick as possible, if it is possible.

Software Process

• A structured set of activities required to develop a software system.
• A software process model is an abstract representation of a process. It

presents a description of a process from some particular perspective.

• Process -> we usually talk about the activities or tasks

Main (traditional) software process activities

• Software specification, where customers and engineers define the

software that is to be produced and the constraints on its operation.

• Software development, where the software is designed and

programmed.

• Software validation, where the software is checked to ensure that it is

what the customer requires.

• Software evolution, where the software is modified to reflect changing

customer and market requirements.

Software Process

• Process descriptions may also include:

○ Products, which are the outcomes of a process activity; ○ Roles, which reflect the responsibilities of the people involved in the process; ○ Pre- and post-conditions, which are statements that are true before and after a process activity has been enacted or a product produced.

• Plan-driven or prescriptive process. All of the process activities are

planned in advance and progress is measured against this plan.

• Agile processes. Planning is incremental and it embrace an evolutive

approach to reflect changing customer requirements.

• In practice, most practical processes include elements of both plan-driven and

agile approaches.

• There are no right or wrong software processes.

Software process models

• Waterfall or Sequential model

○ Plan-drive model. Separate and distinct phases of specification, development, testing, validation, delivery.

• Incremental or Evolutionary model

○ Specification, development and validation are interleaved. May be plan-driven or agile.

• Integration and configuration model

○ The system is assembled from existing configurable components.

• In practice, most large systems are developed using a process that

incorporates elements from all of these models.

The waterfall model

The waterfall model (stage gates)

http://solovatsoft.com/waterfall-model-software-development.html

Results from Bianchi at. all (2018) show that the adoption of Stage-Gate principles is negatively associated with speed and cost performance.

Mattia Bianchi, Giacomo Marzi, Massimiliano Guerini, Agile, Stage-Gate and their combination: Exploring how they relate to performance in software development, Journal of Business Research, 2018, http://www.sciencedirect.com/science/article/pii/S0148296318302273

V Model

https://www.360logica.com/blog/basic-characteristic-of-lifecycle-process-models/

Advantages in Waterfall Development

Petersen K., Wohlin C., Baca D. (2009) The Waterfall Model in Large-Scale Development. In: Bomarius F., Oivo M., Jaring P., Abrahamsson

• P. (eds) Product-Focused Software Process Improvement. PROFES 2009. Lecture Notes in Business Information Processing, vol 32. Springer,

Berlin, Heidelberg

• Easy to understand
• Predictable
• Pays attention to planning the architecture and structure of the software system

Issues in Waterfall Development (State of the Art)

Petersen K., Wohlin C., Baca D. (2009) The Waterfall Model in Large-Scale Development. In: Bomarius F., Oivo M., Jaring P., Abrahamsson

• P. (eds) Product-Focused Software Process Improvement. PROFES 2009. Lecture Notes in Business Information Processing, vol 32. Springer,

Berlin, Heidelberg

• High effort and costs for writing and approving documents for each development phase.
• Extremely hard to respond to changes.
• When iterating a phase the iteration takes considerable effort for rework.
• When the system is put to use the customer discovers problems of early phases very late and

system does not reflect current requirements.

• Problems of finished phases are left for later phases to solve.
• Management of a large scope of requirements that have to be baselined to continue with

development.

• Big-bang integration and test of the whole system in the end of the project can lead to unexpected

quality problems, high costs, and schedule overrun.

• Lack of opportunity for customer to provide feedback on the system.
• The waterfall model increases lead-time due to that large chunks of software artifacts have to be

approved at each gate.

What is the main drawback

• n the Waterfall model?

What is the main drawback

• n the Waterfall model?

The main drawback of the waterfall model is the

difficulty of accommodating changes after the

process is underway. In principle, a phase has to be complete before moving onto the next phase.

Incremental development

Incremental development

Spiral Model

Advantages of Incremental development

• The cost of accommodating changing customer requirements is reduced.

○ The amount of analysis and documentation that has to be redone is much less than is required with the waterfall model.

• It is easier to get customer feedback on the development work that has been done.

○ Customers can comment on demonstrations of the software and see how much has been implemented.

• More rapid delivery and deployment of useful software to the customer is possible.

○ Customers are able to use and gain value from the software earlier than is possible with a waterfall process.

Incremental development problems (classical vision)

• The process is not visible.
• System structure tends to degrade as new increments are added.

○ Unless time and money is spent on refactoring to improve the software, regular change tends to corrupt its structure. Incorporating further software changes becomes increasingly difficult and costly.

• It is a classical vision -> answer: agile process!

Rational Unified Process (RUP)

• Rational Software Corporation
• Rational Objectory Process (ROP) in 1996
• Use-case driven, architecture-centric, roles, tasks
• Iterative and incremental
• Unified Modeling Language (UML)
• Six “engineering disciplines”

○

Business modelling

○

Requirements

○

Analysis and design

○

Implementation

○

Test

○

Deployment

What is the main issue on plan-based models?

To use the same approach for any kind of software project!!!

What is the main issue on plan-based models?

No Silver Bullet!

(Brooks, 1975)

Software is a wicked problem

“A wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. ...not all hard-to-solve problems are wicked, only those with an indeterminate scope and scale. ”

Complex interdependencies!

https://www.wickedproblems.com/1_wicked_problems.php

Agile Process

Agile Process

• The aim of agile methods is to reduce overheads in the software process

and to be able to respond quickly to changing requirements without excessive rework.

• Deliver value to customers/users frequently

○ Satisfy the customer through early and continuous delivery of valuable software ○ Working software is the primary measure of progress

• Continuously sustainable pace
• Constant feedback
• Welcoming changes

○ Uncertain world ○ Liquid modernity - Zygmunt Bauman ○ The End of Certainty - Ilya Prigogine

Agile (or Agility) is an evolutionary approach!

Agile Process

• Is driven by customer descriptions of what is

required (scenarios or user stories)

• Recognizes that plans are short-lived
• Develops software iteratively with a heavy

emphasis on construction activities

• Delivers multiple ‘software increments’
• Adapts as changes occur

Agile Process

• the process molds to the needs of the people

and team, not the other way around

• people on an agile team must be

○ Competence ○ Common focus and collaboration ○ Decision-making ability ○ Fuzzy problem-solving ability ○ Mutual trust and respect ○ Self-organization

Agile is evolving what fits together

Lean software development

• 1. Eliminate waste
• 2. Amplify learning
• 3. Decide as late as possible
• 4. Deliver as fast as possible
• 5. Empower the team
• 6. Build integrity in
• 7. See the whole

Lean Kanban and Work-In-Progress

Is Quality a variable in Agile context?

Is Quality a variable? Quality is not a variable in Agile context! Scope, Time and Resources are variables.

Problems with agile methods

• It can be difficult to keep the interest of customers who

are involved in the process.

• Team members may be unsuited to the intense

involvement that characterises agile methods.

• Prioritising changes can be difficult where there are

multiple stakeholders (politics!!!).

• Maintaining simplicity requires extra work.
• Contracts may be a problem as with other approaches to

iterative development.

Is Agile better than Incremental Process?

• Following results from Tarhan and Yilmaz [2014]
• Comparison demonstrated that the Agile Process had performed better than

the Incremental Process in terms of

○ productivity (79%) ○ defect density (57%) ○ defect resolution effort ratio (26%) ○ test Execution V&V effectiveness (21%), and ○ effort prediction capability (4%).

• These results indicated that development performance and product quality

achieved by following the Agile Process was superior to those achieved by following the Incremental Process (to short term projects [research scope])

Ayca, Seda Gunes Yilmaz, Systematic analyses and comparison of development performance and product quality

• f Incremental Process and Agile Process, Information and Software Technology, Volume 56, Issue 5, 2014,

Pages 477-494, http://www.sciencedirect.com/science/article/pii/S0950584913002310

Large-Scale Agile Development

• Agile methods were originally developed for small
• rganizations
• Current agile approaches do not provide good blueprints

for what a scaled agile organization

• Results show that the most significant reasons for

bottlenecks are related to requirements engineering. (Our next course…)

Maria Paasivaara, Benjamin Behm, Casper Lassenius,Minna Hallikainen, Large-scale agile transformation at Ericsson: a case study, Empir Software Eng (2018) https://doi.org/10.1007/s10664-017-9555-8 Kai Petersen, Mahvish Khurum, Lefteris Angelis, Reasons for bottlenecks in very large-scale system of systems development, Information and Software Technology,Volume 56, Issue 10, 2014, Pages 1403-1420, http://www.sciencedirect.com/science/article/pii/S0950584914001074

Trends -> DevOps

67 https://www.enterpriseirregulars.com/116202/race-pipeline-atlassian-aint-playin-introducing-devops-marketplace/

Trends -> Scaling Agile the Spotify

https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/ https://blog.crisp.se/wp-content/uploads/2012/11/SpotifyScaling.pdf http://usir.salford.ac.uk/48199/1/camera%20ready%20submitted.pdf

The software development methodologies used

Survey results indicate that although agile methodologies are more prevalent than 10 years ago, traditional methodologies are still popular.

• L. R. Vijayasarathy and C. W. Butler, "Choice of Software Development Methodologies: Do Organizational, Project, and Team

Characteristics Matter?," in IEEE Software, vol. 33, no. 5, pp. 86-94, Sept.-Oct. 2016. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7006383&isnumber=7548893

So, what process should I use?

Collaboration

Strictly plan-driven Agile/open source Low High

The software development process in reality

72

http://agilemodeling.com/

Agile Modeling

• Agile principles and practices applying for modeling
• Following Ambler (2002):

○ Agile Modeling (AM) is a chaordic, practice-based methodology for effective modeling and documentation of software-based systems.

Agile Modeling Values and Principles

• Communication
• Simplicity (Keep it simple)

○ Don’t “What if” Yourself to Death ○ Avoid applying complex patterns too soon ○ Avoid over-architecting your system to support potential future requirements ○ Avoid to develop a complex infrastructure ○ To develop a complex infrastructure

• Feedback

○ Develop the model as a team ○ Review the model with your target audience ○ Implement the model

Agile Modeling Values and Principles

• Software Is Your Primary Goal: we’re not documentation developers, or

even model developers; we’re software developers.

• Travel Light: create just enough models and documentation to get by.
• Model with a Purpose
• Maximize Stakeholder Investment: System Documentation is a Business

Decision, Not a Technical One

• Content is more important than representation

○ You don’t need to jump into using a complicated CASE tool right away

Agile Modeling Values and Principles

• Multiple Models
• Each artifact has an objective/semantic/strengths/weaknesses

○ Use the right tool for the job ○ No single model is sufficient for your modeling needs. ○ Start by learning a subset of artifacts ○ The UML is not complete ○ Understandability Is More Important than Following Standards

• Create Several Models in Parallel
• Iterate to Another Artifact
• Use the simplest tools
• Model with Others

○ Active Stakeholder Participation during modeling sessions ○ Display Models Publicly

• Collective Ownership -> Display Models Publicly

• Modeling or documenting?

Agile modeling

77

Agile modeling - modeling

• Modeling or documenting?
• Modeling = to make models to understand a problem
• Comprehension
• Several levels of abstraction
• Sketch
• “Perishable”
• Take a picture and throw in the trash

78

• Modeling or documenting?
• Documenting = to make artefacts for communication
• Describe and store/registry decisions
• For “perinity” or “posterity”
• More details
• More structured documents (templates)
• “Formal” modeling notation -> UML, SysML, Archimate, …
• Documents aren’t always models

○ Ex.: User manual is not a model.

• Maximize stakeholder investment
• Travel light!

Agile modeling - documenting

79

Agile modeling session

80

Agile Modeling Session

81

Agile Modeling

82

• Make essential modeling
• Whiteboard
• Process -> Requirements -> Architecture
• UML notation as reference -> simple diagrams
• Make several models simultaneously
• Diagrams work together (views)
• Decide -> modeling or documenting?

Agility beyond the hype Experiences and lessons learned from the trenches

Agile is not new...

~ 20 years!

Agile is trending...

Agile is buzzword...

Agile is a business...

But finally, what is agility in in practice?

I started working at very young age ...

Software Architect Developer System Administrator Agile Coach Manager Agile team leader DevOps Engineers Data Scientist Researcher Professor Software Engineer R&D Engineers

So, I’m here for sharing

some stories and ...

Lesson #1 Is impossible possible?

When is impossible possible?

• Integrated Project Systems
• Team experience: 2 years to do one system.
• Goal 5 months to do 3 more!
• Deadline Jan 2th, 2002.
• Thousands of specifications (use cases was

not introduced yet…)

• I was a newbie (fresh meat in the company)

Yes, it is impossible, but on what paradigm???

We changed the paradigm!

Delphi OO Visual Form Inheritance!

Lesson #1 - change the paradigm to do “impossible” things!

sds

Lesson #2 Do a job well, ever!

Lesson #2 - if you do something wrong, but it “works”, it will stay forever!

Do a job well, ever!

Lesson #3 Detached from your babies!

• Integrated Project Systems
• Deadline Jan 2th, 2002.
• Thousands of specifications (use cases was

not introduced yet…)

• Thousands of validated screen

prototypes!

Delphi OO Visual Form Inheritance!

How the analysts saw my proposal?

We applied the change and..

… what the clients said about the new screens?

NOTHING!!!

They forgot the prototype! But why?

Because the new screen are more consistent and easy to use!!!

Lesson #3 - be personally attached from a system or practice is a challenge for improvements!

The system is not your baby!

Lesson #4 Good results is not equal to success!

VS

VS

Despite all, we did a better system system than Globo with maybe 100x less resources!!! So, great success? Not exactly...

Lesson #04 - Sometimes great

results are a fail… Do your best, but understand that decisions are political driven!

Politics is the most important problem in SE!

Lesson #5 Do stuff simple; push fast...

Complex systems need complex architectures?

In general, yes, but why not simplify it?

I did and It worked!

Lesson #05 - do something simple and useful as soon as possible, and push to production! If the project is important new resources will be allocate on!

Lesson #6 Decide your team by technology or your technology by the team...

Lesson #7 Eliminate the blockers

• r…

Lesson #8 Time to stop changing...

Launching monday. We closed a version

• Friday. Do we have to

improve the software?

The team decide work on during the weekend… What happened?

Lesson #9 You will not stay the forever...

• https://youtu.be/T2Ny45d0cPo
• http://www.procergs.rs.gov.br/1-modulo-do-sgo-e-entregue-atraves-de-metodos-ageis
• http://www.procergs.rs.gov.br/procergs-recebe-visita-do-tribunal-de-justica

Coach a team for continuing your missions… Create a perennity environment for improvements….

Main lessons

Perseverance:

The time is the master

• f truth…

Main lessons Agility works because it is evolutionary … Keep trying!

Main lessons

It is not your fault:

Politics is a driver of decisions...

References

• Software Engineering: A Practitioner’s Approach​, R. Pressman, B. Maxim,

McGraw-Hill, 8th Edition, 2015.

• Software Processes and Life Cycle Models, Kneuper, 2018

○ https://link.springer.com/book/10.1007%2F978-3-319-98845-0

• Software Engineering 10th Edition, Ian Sommerville

○ https://iansommerville.com/software-engineering-book/

• Foundations of Software Engineering

○ http://www.cs.cmu.edu/~ckaestne/15313/2018/index.html

• Nathan Marz -- The Epistemology of Software Engineering

○ https://www.youtube.com/watch?v=_7Lp66By6-o

• An Investigation of the Therac-25 Accidents

○ https://web.stanford.edu/class/cs240/old/sp2014/readings/therac-25.pdf