Outline Operating System Security, Buffer overflows Continued - PDF document

Outline Operating System Security, • Buffer overflows Continued • Designing secure operating systems CS 239 • Assuring OS security Computer Security • Logging and auditing February 27, 2006 Lecture 12 Lecture 12 Page 1 Page 2 CS 239, Winter 2006 CS 239, Winter 2006 Buffer Overflows What Is a Buffer Overflow? • One of the most common causes for • A program requests input from a user compromises of operating systems • It allocates a temporary buffer to hold • Due to a flaw in how operating the input data systems handle process inputs • It then reads all the data the user –Or a flaw in programming languages provides into the buffer, but . . . –Or a flaw in programmer training • It doesn’t check how much was provided –Depending on how you look at it Lecture 12 Lecture 12 Page 3 Page 4 CS 239, Winter 2006 CS 239, Winter 2006 For Example, Well, What If the User Does? int main(){ • The code continues reading data into char name[31]; memory printf(“Please type your name: “); gets(name); –That’s how gets() works printf(“Hello, %s”, name); • The first 32 bytes go into name return (0); } • Where do the remaining bytes go? • What if the user enters more than 32 characters? • Onto the stack Lecture 12 Lecture 12 Page 5 Page 6 CS 239, Winter 2006 CS 239, Winter 2006 1

Using Buffer Overflows to Munging the Stack Compromise Security • The temporary variable name is allocated • Carefully choose what gets written into on the stack the instruction pointer – Close to the record of the function • So that the program jumps to currently being run something you want to do • The overflow will spill into whatever’s next –Under the identity of the program on the stack that’s running • Commonly, that’s effectively going to • Such as, execute a command shell overwrite the instruction pointer Lecture 12 Lecture 12 Page 7 Page 8 CS 239, Winter 2006 CS 239, Winter 2006 Effects of Buffer Overflows Are Buffer Overflows Common? • Remote or unprivileged local user gets • You bet! to run a program with greater • Weekly occurrences in major privileges systems/applications • If buffer overflow is in a root program, • Probably one of the most common gets all privileges, essentially security bugs • Common mechanism to allow attackers to break into machines Lecture 12 Lecture 12 Page 9 Page 10 CS 239, Winter 2006 CS 239, Winter 2006 Some Recent Buffer Overflows Fixing Buffer Overflows • Windows Media Player Plug-In • Check the length of the input • Use programming languages that prevent them • Microsoft Windows Web Client • Put in OS controls that prevent overwriting the • LibPNG Graphics Library stack • Put things in different places on the stack, making • Metamail message processing it hard to find the return pointer • Blackberry Enterprise Server • Why aren’t these things commonly done? • And two others, just in last week’s • Presumably because programmers and designers SANS vulnerability report neither know nor care about security Lecture 12 Lecture 12 Page 11 Page 12 CS 239, Winter 2006 CS 239, Winter 2006 2

Desired Security Features of a Extra Features for a Trusted OS Normal OS • Authentication of users • Mandatory and discretionary access • Memory protection control • File and I/O access control • Object reuse protection • General object access control • Enforcement of sharing • Complete mediation • Fairness guarantees • Audit capabilities • Secure IPC and synchronization • Intruder detection capabilities • Security of OS protection mechanisms Lecture 12 Lecture 12 Page 13 Page 14 CS 239, Winter 2006 CS 239, Winter 2006 How To Achieve OS Security Advantages of Kernelization • Kernelized design • Smaller amount of trusted code • Separation and isolation mechanisms • Easier to check every access • Virtualization • Separation from other complex pieces of the system • Layered design • Easier to maintain and modify security features Lecture 12 Lecture 12 Page 15 Page 16 CS 239, Winter 2006 CS 239, Winter 2006 Assurance of Trusted Operating Reference Monitors Systems • An important security concept for OS • How do I know that I should trust design someone’s operating system? • A reference monitor is a subsystem • What methods can I use to achieve the that controls access to objects level of trust I require? –It provides (potentially) complete mediation • Very important to get this part right Lecture 12 Lecture 12 Page 17 Page 18 CS 239, Winter 2006 CS 239, Winter 2006 3

Secure Operating System Assurance Methods Standards • Testing • If I want to buy a secure operating system, how do I compare options? • Formal verification • Use established standards for OS • Validation security • Several standards exist Lecture 12 Lecture 12 Page 19 Page 20 CS 239, Winter 2006 CS 239, Winter 2006 Some Security Standards The U.S. Orange Book • U.S. Orange Book • The earliest evaluation standard for trusted operating systems • European ITSEC • Defined by the Department of Defense • U.S. Combined Federal Criteria in the late 1970s • Common Criteria for Information • Now largely a historical artifact Technology Security Evaluation Lecture 12 Lecture 12 Page 21 Page 22 CS 239, Winter 2006 CS 239, Winter 2006 Purpose of the Orange Book Orange Book Security Divisions • To set standards by which OS security • A, B, C, and D could be evaluated – In decreasing order of degree of security • Fairly strong definitions of what features • Important subdivisions within some of the and capabilities an OS had to have to divisions achieve certain levels • Requires formal certification from the • Allowing “head-to-head” evaluation of government (NCSC) security of systems – And specification of requirements – Except for the D level Lecture 12 Lecture 12 Page 23 Page 24 CS 239, Winter 2006 CS 239, Winter 2006 4

Some Important Orange Book The C2 Security Class Divisions and Subdivisions • C2 - Controlled Access Protection • Discretionary access –At fairly low granularity • B1 - Labeled Security Protection • Requires auditing of accesses • B2 - Structured Protection • And password authentication and protection of reused objects • Windows NT has been certified to this class Lecture 12 Lecture 12 Page 25 Page 26 CS 239, Winter 2006 CS 239, Winter 2006 The B1 Security Class The B3 Security Class • Requires careful security design • Includes mandatory access control –With some level of verification –Using Bell-La Padula model • And extensive testing –Each subject and object is assigned a • Doesn’t require formal verification security level –But does require “a convincing • Requires both hierarchical and non- argument” hierarchical access controls • Trusted Mach is in this class Lecture 12 Lecture 12 Page 27 Page 28 CS 239, Winter 2006 CS 239, Winter 2006 Basics of Common Criteria The Common Criteria Approach • Modern international standards for • Something of an alphabet soup – computer systems security • The CC documents describe • Covers more than just operating systems –The Evaluation Assurance Levels • Design based on lessons learned from (EAL) earlier security standards • The Common Evaluation Methodology • Lengthy documents describe the Common (CEM) details guidelines for Criteria evaluating systems Lecture 12 Lecture 12 Page 29 Page 30 CS 239, Winter 2006 CS 239, Winter 2006 5

Another Bowl of Common What’s This All Mean? Criteria Alphabet Soup • TOE – Target of Evaluation • Highly detailed methodology for • TSP – TOE Security Policy specifying : – Security policy of system being evaluated 1. What security goals a system has • TSF – TOE Security Functions 2. What environment it operates in – HW, SW used to enforce TSP • PP – Protection Profile 3. What mechanisms it uses to achieve its security goals – Implementation-dependent set of security requirements 4. Why anyone should believe it does so • ST – Security Target – Predefined sets of security requirements Lecture 12 Lecture 12 Page 31 Page 32 CS 239, Winter 2006 CS 239, Winter 2006 Logging and Auditing Logging • No security system will stop all attacks • An important part of a complete security solution –Logging what has happened is vital to dealing with the holes • Practical security depends on knowing • Logging also tells you when someone what is happening in your system is trying to break in • Logging and auditing is required for –Perhaps giving you a chance to close that purpose possible holes Lecture 12 Lecture 12 Page 33 Page 34 CS 239, Winter 2006 CS 239, Winter 2006 Access Logs Other Typical Logging Actions • Logging failed login attempts • One example of what might be logged for security purposes – Can help detect intrusions or password crackers • Listing of which users accessed which • Logging changes in program permissions objects – Often done by intruders –And when and for how long • Logging scans of ports known to be • Especially important to log failures dangerous Lecture 12 Lecture 12 Page 35 Page 36 CS 239, Winter 2006 CS 239, Winter 2006 6