[PPT] - Software Evolution Massimo Felici Massimo Felici Software PowerPoint Presentation

SLIDE 1

Software Evolution

Massimo Felici

Massimo Felici Software Evolution c 2011

SLIDE 2

Title Slide

Acknowledgements: James Bednar, Massimo Felici, Conrad Hughes, Perdita Stevens

SLIDE 3

1

Change is inevitable

Even brand-new systems will often or typically need to:

Interface

with existing components that cannot be replaced (usually undocumented, sometimes without source code)

Reproduce functionality of old, badly designed systems in current use, without

breaking anything

Go through many revisions during development

Massimo Felici Software Evolution c 2011

SLIDE 4

2

What is software evolution?

Change over time

all kinds of maintenance: corrective, adaptive, perfective, preventive
reengineering
end-of-life, system withdrawal

Various experts have asserted that most of the cost of software ownership arise after delivery, i.e. during maintenance — e.g. > 90%! (Erlikh, 2000) However, evolution is not as maintenance (Lutz, Mikulski, 2003)

Massimo Felici Software Evolution c 2011

SLIDE 5

Slide 2: What is software evolution?

The process by which programs change over time, e.g. for:

Maintenance:

Fixing bugs, adapting when external components change, adding features

Reverse engineering: Analysing an existing system, e.g. to create accurate

documentation

Refactoring: Cleaning up code without modifying functionality
Reengineering: Creating a maintainable system out of an unmaintainable one

(at a larger scale than refactoring)

Len Erlikh.

2000. Leveraging Legacy System Dollars for E-Business. IT Professional 2, 3 (May 2000), 17-23. http://dx.doi.org/10.1109/6294. 846201

Robyn R. Lutz and Ines Carmen Mikulski. 2003. Operational anomalies as a

cause of safety-critical requirements evolution. J. Syst. Softw. 65, 2 (February 2003), 155-161. http://dx.doi.org/10.1016/S0164-1212(02)00057-2

SLIDE 6

3

Why does software need to change?

because it wasn’t built right
because something has changed: business environment, software environment,

hardware environment...

Massimo Felici Software Evolution c 2011

SLIDE 7

4

Kinds of maintenance

More precisely, even though software doesn’t wear out in the same was as physical artifacts, it still needs to be:

fixed (corrective maintenance),
adapted to changing needs (adaptive maintenance),
improved in performance or maintainability (perfective maintenance)
improved by fixing bugs before they activate (preventive maintenance)

Massimo Felici Software Evolution c 2011

SLIDE 8

Slide 4: Kinds of maintenance

ISO/IEC 14764, Software Engineering – Software Life Cycle Processes –

Maintenance following Swanson’s dimensions

E. Burton Swanson. 1976. The dimensions of maintenance. In Proceedings
f the 2nd international conference on Software engineering (ICSE ’76). IEEE

Computer Society Press, Los Alamitos, CA, USA, 492-497.

SLIDE 9

5

What should we think of this?

Success: tells of flexible systems that needn’t be thrown away?
Failure: tells of systems that aren’t correct or flexible as built?
Whatever...

figures like these do tell us that how maintenance is done is important: doing it better may save money. (And doing it less may too, of course.)

Massimo Felici Software Evolution c 2011

SLIDE 10

6

Lehman’s laws

Manny Lehman, the “Father of Software Evolution”, wrote many papers from the mid 70s onwards, proposing “Laws of Software Evolution” for “E-type systems”. Systems classified into:

S-type: formally specified and verified; static by definition
E-type: real-world system

Massimo Felici Software Evolution c 2011

SLIDE 11

7

Lehman’s laws (adapted from 2001 talk by MML)

I Continuing Change An E-type system must be continually adapted else it becomes progressively less satisfactory in use II Increasing Complexity As an E-type system is evolved its complexity increases unless work is done to maintain or reduce it III Self regulation Global E-type system evolution processes are self-regulating IV Conservation

f

Organisational Stability Average activity rate in an E-type process tends to remain constant

ver system lifetime or segments of that lifetime

V Conservation

f

Familiarity In general, the average incremental growth (growth rate trend) of E-type systems tends to decline VI Continuing Growth The functional capability of E-type systems must be continually enhanced to maintain user satisfaction over system lifetime VII Declining Quality Unless rigorously adapted to take into account changes in the

perational environment, the quality of an E-type system will appear

to be declining as it is evolved VIII Feedback System E-type evolution processes are multi-level, multi-loop, multi-agent feedback systems Massimo Felici Software Evolution c 2011

SLIDE 12

8

Criticism of Lehman’s laws

“Laws”?
Based on data?
Contentful?

Massimo Felici Software Evolution c 2011

SLIDE 13

9

Two kinds of scenario

Let us consider separately:

Business-as-usual software evolution: how things should work
Disaster recovery software evolution: coping when things don’t work properly

Massimo Felici Software Evolution c 2011

SLIDE 14

10

Managing change

Realising that software evolution is normal and necessary, have a process in place:

1. A bug report or change request is submitted, e.g. via an issue tracking tool;
2. Show-stopping problems may have to be dealt with immediately (maybe even

by a binary patch);

3. But as a general rule, issues and bugs are classified and prioritised;
4. Subsequent software releases incorporate fixes for a selection of reported issues.

There is always a trade-off between fixing old issues and introducing new

functionality. Some problems are cheap to live with.

Massimo Felici Software Evolution c 2011

SLIDE 15

11

Why evolution doesn’t always go smoothly

Maintenance is seen as uninteresting and low-status: hard to get the best

developers to spend long on a maintenance team.

There is always more work than can be done, so corners tend to be cut.
Even if resource isn’t an issue, the intention behind the original design is easily

lost.

So the software gradually loses its integrity: architectural degradation.

Massimo Felici Software Evolution c 2011

SLIDE 16

12

Legacy Systems

A system which still has value, but which significantly resists modification and evolution. Stereotypically old – but that can mean 5 years. Problems include:

architectural degradation
reliance on unmaintained software or hardware
loss of expertise
not designed for evolution

Massimo Felici Software Evolution c 2011

SLIDE 17

Slide 12: Legacy Systems

A legacy system is one that is difficult to evolve.

Cynical definition 1: legacy = any system actually being used. Systems are
bsolete as soon as they are shipped — technology evolves, requirements

change, etc., yet, it is always difficult to make changes without disrupting existing users.

Cynical definition 2:

legacy = any system without a complete test suite. Without good tests, making changes is scary, so people become conservative rather than doing large-scale maintenance like refactoring. Without constant vigilance (continuous refactoring and reengineering), this tradeoff results in a series of small, scattered patches that eventually destroy system integrity and make systems unmaintainable.

SLIDE 18

13

Bad situations

There is a system: it is in use, but nobody deeply understands how it works.

It needs to be changed, but there’s a problem.

Maybe its source code is a mess.
Maybe it can no longer be compiled.
Maybe its source code has even been lost! (Perhaps only partially.)

Massimo Felici Software Evolution c 2011

SLIDE 19

14

So what to do?

Basically three options:

Soldier on
Reengineer
Scrap

The attempt to understand the system is an essential part of the decision process.

Massimo Felici Software Evolution c 2011

SLIDE 20

15

Techniques for understanding legacy systems

Program comprehension tools can help, well, comprehend the program. Many

techniques can be used, providing different information useful in different

circumstances. For example, a slicing tool can be used to identify which lines
f a program affect the value of a specific variable at a point of interest.
Reverse engineering tools construct a high level view of a system from a low

level one. This may be source code from object code (useful if source code has been lost), a call graph in which nodes representing modules or functions are connected if one calls the other, or perhaps a UML class diagram.

Massimo Felici Software Evolution c 2011

SLIDE 21

16

And then what?

If only small modifications are needed, may now be able to modify and rebuild the system, even if much of it is still undocumented. Or it may be more cost-effective to replace or reengineer. E.g.

Reengineer:

renovate the system by restoring a suitable architecture and updating as needed.

Replace in a fairly mechanical way (with or without automation): e.g., where

a system is written in an obsolete language, have it translated function-by- function into a modern language.

Replace, starting from gathering current requirements. Consider using COTS

(commercial off-the-shelf) software.

Massimo Felici Software Evolution c 2011

SLIDE 22

17

Reengineering Legacy Code

Assume that we have want to add features to a legacy system that has a complicated, suboptimal design. Refactoring + Testing approach:

1. Set up tests to capture current behaviour (time-consuming)
2. Gradually refactor as code is understood (also slow)
3. Once the design is relatively clean and appropriate for the types of changes

now expected, start adding features (now easy) Benefit: it’s usually obvious what to do next

Massimo Felici Software Evolution c 2011

SLIDE 23

18

Refactoring

Assume that we have either a partially implemented system or a legacy system to which we would like to add a feature.

Ideal: Just find where the new feature would go, and write the appropriate

bit of code.

Actual: Extensive changes are often needed to the existing design before the

new feature can be added. Refactoring: improving the existing design without adding functionality. Refactoring helps avoid doing scattered patches, to keep the overall structure clean.

Massimo Felici Software Evolution c 2011

SLIDE 24

Slide 18: Refactoring Approach

Whenever the current design makes it unwieldy to implement a desired function

r feature:
1. Step back and re-design the existing code so that it will make the feature easy

to add

2. Make sure that the code meets the same tests as before, i.e., provides the

same functionality.

3. Integrate the changes with the rest of your team.
4. Make the change, pass the tests, and integrate again.

SLIDE 25

19

Regression Testing

Crucial for maintenance: Build up an automated library of tests that are run

regularly to uncover newly introduced bugs.

For a legacy system, often one slowly adds unit tests to the regression test

suite as one understands bits of the code.

Simple way to blindly generate regression tests: collect output from numerous

runs of the current software, assuming whatever it does has been good enough so far, then only investigate when the output changes.

Massimo Felici Software Evolution c 2011

SLIDE 26

20

Refactoring and Testing

Refactoring is much easier with good regression tests.
If none exist, first add tests for all the functionality you are planning to modify,

and make sure that they are fully automated.

The tests can verify that a refactoring does not change the program behaviour.
Typically: run tests as-is before and after refactoring, then modify the code

and the tests, and again verify the tests run ok.

Massimo Felici Software Evolution c 2011

SLIDE 27

Slide 20: Revision Control and Refactoring

Adding a feature using SVN, testing, and refactoring:

1. svn commit .

(Commit all outstanding edits)

2. emacs

(Refactor, not changing behavior at all)

3. make tests

(Regression test)

4. svn diff

(Will have many changes)

5. svn commit -m ‘‘No visible changes’’.
6. edit files

(Add new feature)

7. make tests

(Regression test)

8. svn diff

(Short list: only the new code)

9. svn commit -m ‘‘Added feature Y’’.

Goal: validate nearly all changes against existing tests, then debug new user- visible change/feature by itself.

SLIDE 28

21

A complementary technique

Sometimes you want to save, but not continue to modify, a legacy system or

component. E.g., it’s written in an obsolete language, and/or it is incomprehensible (but apparently correct!)

You can use the Adapter design pattern – wrap it in a well-defined interface

(using a foreign function interface if necessary) usable from a modern language. All future code interacts with the legacy only through the adapter.

Pro: Old code doesn’t ‘infect’ new code
Con: Limited to cases where you can isolate the valuable legacy.

Massimo Felici Software Evolution c 2011

SLIDE 29

Slide 21: Adapter Pattern

Often, it’s better just to draw a fence around legacy code, and not attempt to

improve it. E.g. if there is an obscure, undocumented component that is known to be extremely reliable, refactoring would be difficult.

Instead, wrap up the old FORTRAN or COBOL component as a nice-looking
bject in the new development language, use the adapter in the future, and

never touch the old code again.

Benefit: Keeps old bad code from tainting new good code.

SLIDE 30

22

Keeping the System Running

New projects are often designed to replace an existing body of lousy code in

current use, but adding features.

Many such projects fail by being overly ambitious. Often, the developers go far
ut on a limb adding fun new features and improvements, before the system

has been put into real use. The longer this goes on, the lower the likelihood

f the new system ever being used.
The alternative of making small changes that keep the old system working

continuously is very hard, but it’s a good way to minimise the chance of total failure.

Massimo Felici Software Evolution c 2011

SLIDE 31

Slide 22: Keeping the System Running

Andreas Zeller. 1999. Yesterday, my program worked. Today, it does not. Why?. In Proceedings of the 7th European software engineering conference held jointly with the 7th ACM SIGSOFT international symposium on Foundations

f software engineering (ESEC/FSE-7). Springer-Verlag, London, UK, 253-267.

http://dx.doi.org/10.1145/318773.318946

SLIDE 32

23

Summary

Change is inevitable — prepare for it.
Refactoring helps keep you (and your design!) sane when making changes.
Refactoring, testing, and revision control together make it much more practical

to reengineer legacy code.

Massimo Felici Software Evolution c 2011

SLIDE 33

24