Outline Operating System Security, Buffer overflows Continued - PDF document

Outline Operating System Security, • Buffer overflows Continued • Designing secure operating systems CS 239 • Assuring OS security Computer Security • Logging and auditing February 23, 2005 Lecture 11 Lecture 11 Page 1 Page 2 CS 239, Winter 2005 CS 239, Winter 2005 Buffer Overflows What Is a Buffer Overflow? • One of the most common causes for • A program requests input from a user compromises of operating systems • It allocates a temporary buffer to hold • Due to a flaw in how operating the input data systems handle process inputs • It then reads all the data the user –Or a flaw in programming languages provides into the buffer, but . . . –Or a flaw in programmer training • It doesn’t check how much was provided –Depending on how you look at it Lecture 11 Lecture 11 Page 3 Page 4 CS 239, Winter 2005 CS 239, Winter 2005 For Example, Well, What If the User Does? int main(){ • The code continues reading data into char name[31]; memory printf(“Please type your name: “); gets(name); –That’s how gets() works printf(“Hello, %s”, name); • The first 32 bytes go into name return (0); } • Where do the remaining bytes go? • Onto the stack • What if the user enters more than 32 characters? Lecture 11 Lecture 11 Page 5 Page 6 CS 239, Winter 2005 CS 239, Winter 2005 1

Using Buffer Overflows to Munging the Stack Compromise Security • The temporary variable name is allocated • Carefully choose what gets written into on the stack the instruction pointer – Close to the record of the function • So that the program jumps to currently being run something you want to do • The overflow will spill into whatever’s next –Under the identity of the program on the stack that’s running • Commonly, that’s effectively going to • Such as, execute a command shell overwrite the instruction pointer Lecture 11 Lecture 11 Page 7 Page 8 CS 239, Winter 2005 CS 239, Winter 2005 Effects of Buffer Overflows Are Buffer Overflows Common? • Remote or unprivileged local user gets • You bet! to run a program with greater • Weekly occurrences in major privileges systems/applications • If buffer overflow is in a root program, • Probably one of the most common gets all privileges, essentially security bugs • Common mechanism to allow attackers to break into machines Lecture 11 Lecture 11 Page 9 Page 10 CS 239, Winter 2005 CS 239, Winter 2005 Desired Security Features of a Fixing Buffer Overflows Normal OS • Check the length of the input • Authentication of users • Use programming languages that prevent • Memory protection them • File and I/O access control • Put in OS controls that prevent overwriting • General object access control the stack • Enforcement of sharing • Why aren’t these things commonly done? • Fairness guarantees • Presumably because programmers and • Secure IPC and synchronization designers neither know nor care about • Security of OS protection mechanisms security Lecture 11 Lecture 11 Page 11 Page 12 CS 239, Winter 2005 CS 239, Winter 2005 2

Extra Features for a Trusted OS How To Achieve OS Security • Mandatory and discretionary access • Kernelized design control • Separation and isolation mechanisms • Object reuse protection • Virtualization • Complete mediation • Layered design • Audit capabilities • Intruder detection capabilities Lecture 11 Lecture 11 Page 13 Page 14 CS 239, Winter 2005 CS 239, Winter 2005 Advantages of Kernelization Reference Monitors • An important security concept for OS • Smaller amount of trusted code design • Easier to check every access • A reference monitor is a subsystem • Separation from other complex pieces that controls access to objects of the system –It provides (potentially) complete • Easier to maintain and modify security mediation features • Very important to get this part right Lecture 11 Lecture 11 Page 15 Page 16 CS 239, Winter 2005 CS 239, Winter 2005 Assurance of Trusted Operating Assurance Methods Systems • How do I know that I should trust • Testing someone’s operating system? • Formal verification • What methods can I use to achieve the • Validation level of trust I require? Lecture 11 Lecture 11 Page 17 Page 18 CS 239, Winter 2005 CS 239, Winter 2005 3

Secure Operating System Some Security Standards Standards • If I want to buy a secure operating • U.S. Orange Book system, how do I compare options? • European ITSEC • Use established standards for OS • U.S. Combined Federal Criteria security • Common Criteria for Information • Several standards exist Technology Security Evaluation Lecture 11 Lecture 11 Page 19 Page 20 CS 239, Winter 2005 CS 239, Winter 2005 The U.S. Orange Book Purpose of the Orange Book • To set standards by which OS security • The earliest evaluation standard for could be evaluated trusted operating systems • Fairly strong definitions of what features • Defined by the Department of Defense and capabilities an OS had to have to in the late 1970s achieve certain levels • Allowing “head-to-head” evaluation of • Now largely a historical artifact security of systems – And specification of requirements Lecture 11 Lecture 11 Page 21 Page 22 CS 239, Winter 2005 CS 239, Winter 2005 Some Important Orange Book Orange Book Security Divisions Divisions and Subdivisions • A, B, C, and D • C2 - Controlled Access Protection – In decreasing order of degree of security • B1 - Labeled Security Protection • Important subdivisions within some of the • B2 - Structured Protection divisions • Requires formal certification from the government (NCSC) – Except for the D level Lecture 11 Lecture 11 Page 23 Page 24 CS 239, Winter 2005 CS 239, Winter 2005 4

The C2 Security Class The B1 Security Class • Discretionary access • Includes mandatory access control –At fairly low granularity –Using Bell-La Padula model • Requires auditing of accesses –Each subject and object is assigned a • And password authentication and security level protection of reused objects • Requires both hierarchical and non- • Windows NT has been certified to this hierarchical access controls class Lecture 11 Lecture 11 Page 25 Page 26 CS 239, Winter 2005 CS 239, Winter 2005 The B3 Security Class The Common Criteria • Requires careful security design • Modern international standards for computer systems security –With some level of verification • Covers more than just operating systems • And extensive testing • Design based on lessons learned from • Doesn’t require formal verification earlier security standards –But does require “a convincing • Lengthy documents describe the Common argument” Criteria • Trusted Mach is in this class Lecture 11 Lecture 11 Page 27 Page 28 CS 239, Winter 2005 CS 239, Winter 2005 Basics of Common Criteria Another Bowl of Common Approach Criteria Alphabet Soup • TOE – Target of Evaluation • Something of an alphabet soup – • TSP – TOE Security Policy • The CC documents describe – Security policy of system being evaluated –The Evaluation Assurance Levels • TSF – TOE Security Functions (EAL) – HW, SW used to enforce TSP • PP – Protection Profile • The Common Evaluation Methodology – Implementation-dependent set of security (CEM) details guidelines for requirements evaluating systems • ST – Security Target – Predefined sets of security requirements Lecture 11 Lecture 11 Page 29 Page 30 CS 239, Winter 2005 CS 239, Winter 2005 5

What’s This All Mean? Logging and Auditing • Highly detailed methodology for • An important part of a complete specifying : security solution 1. What security goals a system has • Practical security depends on knowing 2. What environment it operates in what is happening in your system 3. What mechanisms it uses to achieve its • Logging and auditing is required for security goals that purpose 4. Why anyone should believe it does so Lecture 11 Lecture 11 Page 31 Page 32 CS 239, Winter 2005 CS 239, Winter 2005 Logging Access Logs • No security system will stop all attacks • One example of what might be logged –Logging what has happened is vital for security purposes to dealing with the holes • Listing of which users accessed which • Logging also tells you when someone objects is trying to break in –And when and for how long –Perhaps giving you a chance to close • Especially important to log failures possible holes Lecture 11 Lecture 11 Page 33 Page 34 CS 239, Winter 2005 CS 239, Winter 2005 Other Typical Logging Actions Problems With Logging • Logging failed login attempts • Dealing with large volumes of data – Can help detect intrusions or password • Separating the wheat from the chaff crackers –Unless the log is very short, auditing • Logging changes in program permissions it can be laborious – Often done by intruders • System overheads and costs • Logging scans of ports known to be dangerous Lecture 11 Lecture 11 Page 35 Page 36 CS 239, Winter 2005 CS 239, Winter 2005 6