Introduction FreeBSD: Research results: incremental development - PowerPoint PPT Presentation

Introduction FreeBSD: Research results: incremental development � democracy can be very efficient (delegation of authority) and democratic organization � "see bug, fix bug, see bug fixed" is highly motivating Why FreeBSD ? � there are other projects than Linux + Apache ! Niels Jørgensen � Danish members of "core team" (2000) Roskilde University Center � not because of FreeBSD democracy (I didn’t know) nielsj@ruc.dk Open source / democracy Report: www.ruc.dk/~nielsj/research/freebsd/freebsd.pdf � source available, license allows use,copy, modify Data: www.ruc.dk/~nielsj/research/freebsd/freebsd.html � limited financial resources, "alternative" organization Slides: www.ruc.dk/~nielsj/research/freebsd/slides−dm−10−01−02.pdf � but not necessarily "flat" organization What is FreeBSD ? Outline 1. Intro Unix−type operating system � very similar to Linux 2. Incremental development � runs 15% of Internet−servers worldwide (Yahoo, Hotmail, ..) 3. FreeBSD background Berkely Software Distribution 1977− approx. 90: � key role in early Unix, Internet, and ”Open Source” history 4. FreeBSD’s "Life cycle for changes" � good starting point in 90 (when given free by Berkeley U.) 5. Case study of development of major new feature FreeBSD / Linux non−technical differences � elitist image vs. anarchy, tolerance, young nerds, .. 6. Conclusion � lawsuit vs. early 90s jump start � copy centrism vs. copy leftism Note: � FreeBSD license allows companies to modify � generic sw development perspective and re−distribute (not "infecting" as GNU license) � nothing about DM’s IT−network initiative

FreeBSD / Linux technical differences FreeBSD / Linux merge ? Full distribution vs. pure kernel (Not that it is a real issue, but it is tempting to speculate) � 4000+ pieces of third party software � 50+ Java−related ports (compilers, JVMs, JIT−compilers) Pros: � stronger alternative to MS Windows (46%+ of Internet servers) Internet server vs. desktop/laptop � manpower to smoothen installation/configuration � project focus on security (eg. security auditing of ports) � fewer drivers Cons: � alternative ideas can be tried out in practice I don’t know whether FreeBSD’s − � waste of resources to fight over project decisions � virtual memory system is more suited for servers ? � software is shared today (eg. LinuxThreads on FreeBSD) � kernel is more stable ? � not a constant pool of programmes � double manpower not same as double output Otherwise: � similar basic principles ("Unix−type") � shares software for graphical interface, shells, threads, .. Definition 2. Incremental development Piece−by−piece 1 piece = 1 week of programming/testing, for example Definition Preserve software in sound/working state Related life cycle models Main problem: Why study incremental development ? −quality preservation difficult to achieve because the software is constantly changing

Related life cycle models Why study incremental development? Incremental development ~ Incremental development appears suitable for open source projects ~ iterative development Incremental development is key property of Bazaar Model: � Release early, release often => ”parallel debugging” � but: in Boehm’s Spiral model, analysis and design � community of user/developers (or just advanced users) is required in each iteration Incremental development is related to ”flat” organization ~ evolutionary prototyping � long−term planning difficult without sales revenues � but: purpose of e.p. is to obtain user feedback on design � write and read permissions (check−in, check out) � when to integrate: hundreds of decision makers � document−based progress assessment impossible ~ software development as software maintenance (aka evolutionary system development) Incremental development challenges classical software engineering � bugfixes and new featueres � design then code vs. code−n−fix are merely different kind of changes � systematic testing vs. ad hoc debugging The FreeBSD product & service 3. More FreeBSD background Software is Operating system kernel + ports � kernel: state of the art (eg. networking, virtual memory) The FreeBSD product & service. � 4000+ pieces of third party software � 50+ Java−related ports (compilers, JVMs, JIT−compilers) The FreeBSD release tree Release 4.2 contains 75 diverse items listed in release notes The FreeBSD organization � adaptive changes: new drivers, ports updates, .. � corrective changes: 10 security fixes, .. � perfective changes: improved TCP/IP performance, .. The required design effort Several security advisories each month � victim program, problem, impact, workaround, fix � five versions of Bash shell fix distributed (3 Intel x86, 2 Compaq Alpha)

The FreeBSD release tree The FreeBSD organization 3.0 Core team (approx. 10), committers (200+), contributors (1200+) Major release Minor Absolutely necessary to have well−defined and well−understood 4.0 4.1 Snapshot processes because development branch is ”melting pot” 4.2 Core team elected by ”committers” for 2 years � ”being able to work together is our greatest asset” � assigns managers of documentation, PR, web, mailing lists, .. � manages access to repository � development decisions Major releases along main branch every 18 months (approx.) 4−5 minor releases each year as improvements on latest major release Basic organizational ”unit” is maintainer � sources are maintained by formal or ”de facto” maintainer Distribution � maintainers are responsible for all changes not just corrective � major/minor releases on CD � committers do corrective, perfective, etc. work interchangeably � snapshots and major/minor releases always downloadable 4. FreeBSD’s "Life cycle for changes" The required design effort "Code, then fix" works for Linux (or FreeBSD) because: code � "By the time Linux came around, requirements and architecture review pre−commit development test parallel defects had already been flushed out during the development of release production debugging release many previousgenerations of Unix" (Steve McConnell, IEEE Software Magazine) I postulate a six phase model Argument may apply even more to FreeBSD � applies to each individual change (bugfix, small feature, part of large feat.) � FreeBSD goes back 20+ years � consistent with but not immediately apparent in Handbook, guidelines, etc. � Software that has been widely studied / taught � model consists of phases + key points for each phase I believe model captures "essense" of FreeBSD’s development process New requirements: Internet standards and processor technology � why it works � some requirements are very demanding � model is explaining, captures mechanism � symmetric multi−processing � not just descriptive � but most work on FreeBSD is maintenaince−type of work

(i) Code Method code review pre−commit development parallel test release production debugging release Method Code guidelines exist � webbased survey involving 72 project participants Visibility of code is important: � quantitative + qualitative � 85% encouraged to improve skills because of visibility (Q 26) Questions designed to verify/falsify (partially) � Eric S. Raymond’s Bazaar model � "Embarrassment is a powerful thing" � Classical life cycle model � Hypotheses about learning (ii) Review (iii) Pre−commit test code code pre−commit pre−commit review review development development test parallel test parallel release production release production debugging debugging release release The implied responsibility of the committer to Reviews strongly suggested but not absolutely required integrate changes is a learning incentive Committers frequently seek and receive code review � 82% had improved their technical skills � 85% distributed code for review last 3 months by debugging build failures Main obstacle to more frequent reviews on design is lack of feedback � Pre−commit test is really integration test ! � "Getting design feedback is like pulling teeth"