Bypass Testing of Web Applications Jeff Offutt George Mason - - PowerPoint PPT Presentation

▶
bypass testing of web applications
SMART_READER_LITE
LIVE PREVIEW

Bypass Testing of Web Applications Jeff Offutt George Mason - - PowerPoint PPT Presentation

Nov 05, 2022 271 likes •587 views

Bypass Testing of Web Applications Jeff Offutt George Mason University Professor of Software Engineering www.cs.gmu.edu/~offutt My bypass testing playmates Ye Wu Vasileios Papadimitriou Joann J. Ordille Xiaochen Du Qingxian Wang Upsorn

slide-1
SLIDE 1

Bypass Testing

  • f

Web Applications

Jeff Offutt George Mason University

Professor of Software Engineering

www.cs.gmu.edu/~offutt

slide-2
SLIDE 2
  • f 30

My bypass testing playmates

November 2020 Jeff Offutt 2

Xiaochen Du Qingxian Wang Upsorn Praphamontripong Hong Huang Joann J. Ordille Ye Wu Vasileios Papadimitriou

slide-3
SLIDE 3
  • f 30

Jeff Offutt 3

Deploying software

  • Bundled : Pre-installed on computer
  • Shrink-wrap : Bought and installed by end-users
  • Contract : Purchaser pays developer to develop and install,

usually for a fixed price

  • Embedded : Installed on a hardware device, usually with no

direct communication with user

  • Web : Executed across the Internet through HTTP

November 2020

slide-4
SLIDE 4
  • f 30

Problem parameters

  • HTTP is a stateless protocol

– Each request is independent of previous requests

  • Servers have little information about where a request

comes from

  • Web site software is extremely loosely coupled

– Coupled through the Internet – separated by space – Coupled to diverse hardware devices – Written in diverse software languages

November 2020 Jeff Offutt 4

slide-5
SLIDE 5
  • f 30

Web application input validation

Sensitive Data

Bad Data

  • Corrupts data base
  • Crashes server
  • Security violations

Check data Check data

Malicious Data

Can “bypass” data checking

Client

Server

November 2020 Jeff Offutt 5

slide-6
SLIDE 6
  • f 30

Jeff Offutt 6

Bypass testing

  • Users can easily “bypass” client-side constraint

enforcement

  • Bypass testing constructs tests to intentionally violate

constraints :

– Eases test automation – Validates input validation – Checks robustness – Evaluates security

November 2020

J Offutt, Y Wu, X Du, and H Huang. Bypass Testing of Web Applications. IEEE International Symposium on Software Reliability Engineering, November 2004

slide-7
SLIDE 7
  • f 30

Simple example web page

November 2020 Jeff Offutt 7

User Name: Small $150 Version to purchase: Age: Large $500 Medium $250

slide-8
SLIDE 8
  • f 30

Proper behavior

November 2020 Jeff Offutt 8

User Name: Small $150 Version to purchase: Age: Large $500 Medium $250

Username should be plain text only. Age should be between 18 and 150. Invalid data, please correct …

Alan<Turing 500

slide-9
SLIDE 9
  • f 30

Jeff Offutt 9

Abbreviated HTML

<form> <input type=“text” name=“username” size=20> <input type=“text” name=“age” size=3 maxlength=3> <p> Version to purchase: … <input type=“radio” name=“version” value=“150” Checked> <input type=“radio” name=“version” value=“250”> <input type=“radio” name=“version” value=“500”> <input type="submit" onClick="return checkInfo(this.form)"> <input type=“hidden” isLoggedIn=“no”> </form>

November 2020

slide-10
SLIDE 10
  • f 30

Bypass behavior

November 2020 Jeff Offutt 10

Extremely loose coupling … combined with the stateless protocol … allows users to easily bypass client-side checking

Users can save and modify the HTML

slide-11
SLIDE 11
  • f 30

Jeff Offutt 11

Saved & modified HTML

<form> <input type=“text” name=“username” size=20> <input type=“text” name=“age” size=3 maxlength=3> <p> Version to purchase: … <input type=“radio” name=“version” value=“150” Checked> <input type=“radio” name=“version” value=“250”> <input type=“radio” name=“version” value=“500”> <input type="submit" onClick="return checkInfo(this.form)"> <input type=“hidden” isLoggedIn=“no”> </form>

November 2020

Allows an input with arbitrary age, no checking, cost=$25 … ‘<‘ can crash an XML parser Text fields can have SQL statements 25 yes

slide-12
SLIDE 12
  • f 30

In-class exercise 1

November 2020 Jeff Offutt 12

Saved and modified HTML Open the HTML file: https://cs.gmu.edu/~offutt/calculate.html

  • 1. Save it to your computer:

i. <ctrl>-u in your browser ii. Copy and paste to your local computer

  • 2. Open the HTML file in your computer
  • 3. Identify elements that could be “bypassed”
slide-13
SLIDE 13
  • f 30

Applying bypass testing

  • Analyze HTML to extract form elements
  • Model constraints imposed by HTML and JavaScript
  • Rules for data generation :

– From client-side constraints – Typical security violations – Common input mistakes

November 2020 Jeff Offutt 13

Validating input data on the client is like asking your opponent to hold your shield in a sword fight

slide-14
SLIDE 14
  • f 30

Client-side constraint rules

November 2020 Jeff Offutt 14

Violate size restrictions on strings Introduce values not included in static choices

  • Radio boxes
  • Select (drop-down) lists

Violate hard-coded values Use values that JavaScripts flag as errors Change “transfer mode” (get, post, …) Change destination URLs

slide-15
SLIDE 15
  • f 30

Server-side constraint rules

November 2020 Jeff Offutt 15

Data type conversion Data format validation Inter-field constraint validation Inter-request data fields (cookies, hidden)

slide-16
SLIDE 16
  • f 30

Example security violation rules

November 2020 Jeff Offutt 16

Potential Illegal Character Symbol Empty String Commas , Single and double quotes ’ or ” Tag symbols Tag symbols < and > Directory paths .. ../ Strings starting with forward slash / Strings starting with a period . Ampersands & Control character NIL, newline Characters with high bit set 254 and 255 Script symbols <javascript> or <vbscript>

slide-17
SLIDE 17
  • f 30

In-class exercise 2

November 2020 Jeff Offutt 17

What can happen? Run this program: https://cs.gmu.edu:8443/offutt/jsp/gradecalculate.jsp Look at the source (<ctrl>-u) and identify client-side validation rules

slide-18
SLIDE 18
  • f 30

Research validation of bypass testing

  • Autobypass : A web application that accepts a URL and

generates input data for the HTML form fields

– Also accepts any needed login data – MS thesis by Vasileios Papadimitriou

  • Built on top of HttpUnit (which extends JUnit)

– Parses HMTL pages – Identifies formsand their fields – Creates bypass test cases – Submitstest cases to the application’s server

November 2020 Jeff Offutt 18

Parse interface Set default values Generate test vases & run tests Review results

slide-19
SLIDE 19
  • f 30

Types of client input validation

  • Client side input validation is performed by HTML form

controls, their attributes, and client side scripts

  • Validation types are categorized as HTML and scripting

– HTML supports syntactic validation – Client scripting can perform both syntactic and semantic validation

November 2020 Jeff Offutt 19

HTML constraints Scripting constraints

  • Length (max input characters)
  • Value (preset values)
  • Transfer Mode (GET or POST)
  • Field Element (preset fields)
  • Target URL (links with values)
  • Data Type (e.g. integer check)
  • Data Format (e.g. ZIP code format)
  • Data Value (e.g. age value range)
  • Inter-Value (e.g. credit # + exp. date)
  • Invalid Characters (e.g. <,../,&)
slide-20
SLIDE 20
  • f 30

Example interface: yahoo registration form

November 2020 Jeff Offutt 20

Limited Length (HTML) Preset Values (HTML) Preset Transfer Mode in form definition (HTML) Preset number of Fields (HTML) URL with preset Values (HTML) Data Value, Type, & Format validation (script) Inter Value validation (script)

slide-21
SLIDE 21
  • f 30

Real-world examples

November 2020 Jeff Offutt 21

atutor.ca

Atalker

demo.joomla.or

Poll, Users

phpMyAdmin

Main page, Set Theme, SQL Query, DB Stats

brainbench.com

Submit Request Info, New user

myspace.com

Events & Music Search

bankofamerica.com

ATM locator, Site search

comcast.com

Service availability

ecost.com

Detail submit, Shopping cart control

google.com

Froogle, Language tools

pageflakes.com

Registration

wellsfargolife.com

Quote search

nytimes.com

Us-markets

mutex.gmu.edu

Login form

yahoo.com

Notepad, Composer, Search reminder, Weather Search

barnesandnoble.com

Cart manager, Book search/results

amazon.com

Item dispatch, Handle buy

Pure black-box testing means no source (or permission) needed

slide-22
SLIDE 22
  • f 30

Results

November 2020 Jeff Offutt 22

v

J Offutt, V Papadimitriou, and U Praphamontripong. A Case Study on Bypass Testing

  • f Web Applications. Empirical Software Engineering, 19(1):69-104, February 2014
slide-23
SLIDE 23
  • f 30

In-class exercise 3

November 2020 Jeff Offutt 23

Construct bypass tests for gradeCalculate Encode your tests using URL rewriting

https://cs.gmu.edu:8443/offutt/jsp/gradecalculate.jsp?courseCount= 2&courseName1=xx&courseGrade1=A&courseCredit1=3&courseNam e2=yy&courseGrade2=B&courseCredit2=3

Example valid test, 2 courses courseCount=2 courseName1=C1 courseName2=C2 courseGrade1=A courseGrade2=B courseCredit1=3 courseCredit2=3 Expected output: GPA=3.5

slide-24
SLIDE 24
  • f 30

Industrial case study

  • Inventionsfrom scientists are slow to move into industrial

practice

  • We investigated whether the obstacles are :

– Technical difficulties of applying to industrial use – Social barriers – Businessconstraints

  • Applied bypass testing through the research arm of a

software company

November 2020 Jeff Offutt 24

slide-25
SLIDE 25
  • f 30

Avaya’s NPP technology

  • NPP : Notification Preference Portal

– Used to notify users of events – NPP is a highly user interactive web application

  • Javascript is used on the client to :

– Validate inputs – Dynamically modify screen by manipulating DOM – Encode input data into XML before sending to the server

  • This study was part of system testing
  • Encoded values into XML, fed to NPP with HtmlUnit

November 2020 Jeff Offutt 25

slide-26
SLIDE 26
  • f 30

NPP testing results

  • We tested six NPP screens
  • Tests are invalid inputs
  • Did not check effects on back-end

– Failure analysis based on response screens only

November 2020 Jeff Offutt 26

Web screen Tests Failing tests Unique failures Points of contact 42 23 12 Time profile 53 23 23 Notification profile 34 12 6 Notification filter 26 16 7 Change PIN 5 1 1 Create account 24 17 14

TOTAL 184 92 63

slide-27
SLIDE 27
  • f 30

Types of faults

  • Invalid data saved into the database

– These lead to more visible failures later – Example : An invalid password was accepted, but the account could not subsequently be used – Example : Messages could not be sent to invalid contacts

  • Exposure errors (security issue)

– Internal exception message sent in the response screen

  • No response at all

– A software component probably failed – Database or web server sometimes crashed

November 2020 Jeff Offutt 27

slide-28
SLIDE 28
  • f 30

Summary of Avaya study

  • Bypass testing worked very well in an industrial context

– There is no technical obstacle to adoption

  • Even generating tests by hand was cheaper

– There is no valid business barrier

  • Most problems are unlikely with non-malicious users and a

correct implementation

– But client-side validation is notoriously error-prone

November 2020 Jeff Offutt 28

We concluded the primary obstacle is social

Offutt, Wang, and Ordille. An Industrial Case Study of Bypass Testing

  • n Web Applications. 1st International Conference on Software Testing, April 2008

The project lead was very defensive

slide-29
SLIDE 29
  • f 30

In-class exercise 4

November 2020 Jeff Offutt 29

Reflect on research to practice The research literature is full of ideas that have never been used in practice List at least 3 and no more than 10 reasons why research ideas may not be used Write one sentence for each reason

slide-30
SLIDE 30
  • f 30

Google’s motto

November 2020 Jeff Offutt 30

Debugging sucks Testing rocks

Professor Jeff Offutt cs.gmu.edu/~offutt

  • ffutt@gmu.edu