1、An Industrial Case Study of Bypass Testing on Web Applications,Joint research Dr. Ye Wu, Xiaochen Du, Hong Huang, Vasileios Papadimitriou, Qingxiang Wang and Joann J. Ordille of Avaya Labs Research Based on papers in ISSRE 2004 and ICST 2008,Jeff Offutt Software Engineering George Mason University F
2、airfax, VA USA www.cs.gmu.edu/offutt/ offuttgmu.edu,Expanded version of a talk given at the first International Conference on Software Testing, Verification and Validation,Outline of Talk,Motivation Applying bypass testing early results Automating bypass testing Real-world examples Industrial case s
3、tudy Conclusions and future work,ICST 2008, Jeff Offutt,2,Web Application Input Validation,Sensitive Data,Client,Server, Jeff Offutt, 2004,4,Deploying Software,Bundled : Pre-installed on computer Shrink-wrap : Bought and installed by end-users Contract : Purchaser pays developer to develop and insta
4、ll, usually for a fixed price Embedded : Installed on a hardware device, usually with no direct communication with user,Web : Executed across the Internet through HTTP, Jeff Offutt,5,Problem Parameters,HTTP is a stateless protocol Each request is independent of previous request Servers have little i
5、nformation about where a request comes from Web site software is extremely loosely coupled Coupled through the Internet separated by space Coupled to diverse hardware devices Written in diverse software languages,ICST 2008, Jeff Offutt,6,Bypass Testing,“bypass” client-side constraint enforcementBypa
6、ss testing constructs tests to intentionally violate constraints : Eases test automation Validates input validation Checks robustness Evaluates security,ICST 2008, Jeff Offutt,7,User Name:,Small $150,Version to purchase:,Age:,Large $500,Medium $250,Simple Example Web Page,ICST 2008, Jeff Offutt,8,Us
7、er Name:,Small $150,Version to purchase:,Age:,Large $500,Medium $250,Proper Behavior,ICST 2008, Jeff Offutt,9,Abbreviated HTML,Version to purchase:,ICST 2008, Jeff Offutt,10,Bypass Behavior,Extremely loose coupling combined with the stateless protocol allows users to easily bypass client-side checki
8、ng :,Users can save and modify the HTML,ICST 2008, Jeff Offutt,11,Saved & Modified HTML,Version to purchase:,Allows an input with arbitrary age, no checking, cost=$25 can crash an XML parser Text fields can have SQL statements,25,yes,ICST 2008, Jeff Offutt, 2004,12,SQL Injection, Jeff Offutt,13,Appl
9、ying Bypass Testing,Analyze HTML to extract each form element Model constraints imposed by HTML and JavaScript Rules for data generation : From client-side constraints Typical security violations Common input mistakes,ICST 2008,Validating input data on the client is like asking your opponent to hold
10、 your shield in a sword fight, Jeff Offutt, 2004,14,Example Client-Side Constraint Rules,Violate size restrictions on strings Introduce values not included in static choices Radio boxes Select (drop-down) lists Violate hard-coded values Use values that JavaScripts flag as errors Change “transfer mod
11、e” (get, post, ) Change destination URLs, Jeff Offutt, 2004,15,Example Server-Side Constraint Rules,Data type conversionData format validationInter-field constraint validationInter-request data fields (cookies, hidden), Jeff Offutt, 2004,16,Example Security Violation Rules, Jeff Offutt, 2004,17,Firs
12、t Example CyberChair,CyberChair : Web-based conference management www.cyberchair.org ICSE , ICST, ISSRE, ICSM, Bypass testing found 5 types of faults Submission without authentication Unsafe use of hidden form field Disclosing information (program crashes) Lack of validation of file type Allows pape
13、rs of negative length,Automating Bypass Testing,Autobypass : A web application that accepts a URL and generates input data for the HTML form fields Also accepts any needed login data MS thesis by Vasileios Papadimitriou Built on top of HttpUnit Parses HMTL pages Identifies forms and their fields Cre
14、ates bypass test cases Submits test cases to the applications server,ICST 2008, Jeff Offutt,18,Types of Client Input Validation,Client side input validation is performed by HTML form controls, their attributes, and client side scripts that access DOM Validation types are categorized as HTML and scri
15、pting HTML supports syntactic validation Client scripting can perform both syntactic and semantic validation,ICST 2008, Jeff Offutt,19,Example Interface: yahoo registration form,ICST 2008, Jeff Offutt,20,Limited Length (HTML),Preset Values (HTML),Preset Transfer Mode in form definition (HTML),Preset
16、 No of Fields (HTML),URL with preset Values (HTML),Data Value, Type, & Format validation (script),Inter Value validation (script),Test Value Selection,Challenge: How to automatically provide effective test values? Semantic Domain Problem (SDP) Values within the application domain are needed Enumerat
17、ion of all possible test values is inefficient Possible Solutions Random Values (ineffective lots of junk) Automatically generated values (very hard) Taking values from session log files (feasible but incomplete) Tester input (feasible) AutoBypass uses an input domain created by parsing the interfac
18、e and tester input,ICST 2008, Jeff Offutt,21,AutoBypass,AutoBypass steps (the big picture),ICST 2008, Jeff Offutt,22,Parse Interface,Set Default Values,Generate Test Cases & Run Tests,Review Results,All HTML violation rules are used to generate test cases First version of AutoBypass does NOT automat
19、ically violate scripting validation, but : AutoBypass behaves as a browser with scripts disabled Tester can provide test inputs that will bypass scripting validation.,Real-World Examples,ICST 2008, Jeff Offutt,23,atutor.caAtalkerdemo.joomla.orPoll, UsersphpMyAdmin Main page, Set Theme, SQL Query, DB
20、 SSubmit Request Info, New Events & Music Search,ATM locator, Site Service Detail submit, Shopping cart Froogle, Language R Quote search,Us-marketsmutex.gmu.eduLogin Notepad, Composer, Search reminder, Weather SCart manager, Book search/ Item dispatch,Handle buy,Pure black-box testing means no s
21、ource (or permission) needed !,Classifying Output Responses,(V) Valid Responses : invalid inputs are adequately processed by the server(F) Faults & Failures : invalid inputs that cause abnormal server behavior (typically caught by web server when application fails to handle the error) (E) Exposure :
22、 invalid input is not recognized by the server and abnormal software behavior is exposed to the users,ICST 2008, Jeff Offutt,24,Results,ICST 2008, Jeff Offutt,25,v,ICST 2008, Jeff Offutt,26,“Knowing is not enough, we must apply. Willing is not enough, we must do.” Goethe,Research to Practice,Theyre
23、teaching a new way of plowing over at the Grange tonight - you going?,Naw - I already dont plow as good as I know how.,Industrial Case Study,Inventions from scientists are slow to move into industrial practice We wanted to investigate whether the obstacles are : Technical difficulties of applying to
24、 industrial use Social barriers Business constraints Tried to technology transition bypass testing to the research arm of a software company,ICST 2008, Jeff Offutt,27,Avayas NPP Technology,Avaya Labs Research creates research prototypes of software systems, then turns successful prototypes over to p
25、roduct groups NPP : Notification Preference Portal Users specify how and when they should be contacted Types include phone, email and SMS Contacts an be made in parallel or sequentially Used to notify users of events This study was part of system testing NPP is now in production,ICST 2008, Jeff Offu
26、tt,28,NPP Design and Implementation,NPP is a highly user interactive web application Uses many screens Javascript is used on the client to : Validate inputs Dynamically modify screen by manipulating DOM Encode input data into XML before sending to the server The second two uses necessitated changes
27、to how bypass testing was applied,ICST 2008, Jeff Offutt,29,NPP Bypass Tests,Bypass testing analyzes HTML (statically) and generates inputs that violate input constraints The extensive modification of the HTML DOM meant HTML could not be analyzed statically Instead, the input requirements of the ser
28、ver software were identified, and tests encoded in XML A special-purpose tool was written to convert XML tests into HtmlUnit tests,ICST 2008, Jeff Offutt,30,NPP Testing Results,Six NPP screens were tested Tests are invalid inputs exceptions are expected Effects on back-end were not checked Failure a
29、nalysis just based on response screens,ICST 2008, Jeff Offutt,31,Types of Faults,Invalid data saved into the database These lead to more visible failures later Example : An invalid password was accepted, but the account could not subsequently be used Example : Subsequent messages could not be sent t
30、o invalid contacts No response at all Probably a software component failed Database or web server sometimes crashed Exposure errors Internal exception message sent in the response screen,ICST 2008, Jeff Offutt,32,Conclusions,Bypass testing worked very well in an industrial context There is no techni
31、cal obstacle to adoption Source is not needed Even hand generation of tests was quite cheap in comparison with other methods There is no valid business barrier Most problems are unlikely with non-malicious users and a correct implementation But client-side validation is notoriously error-prone,ICST
32、2008, Jeff Offutt,33,We conclude the primary obstacle is social,Future Work,A major observability problem with web application testing is detecting invalid database values A comprehensive valid data model could allow database auditors to be developed Javascript needs to be fully parsed and analyzed
33、Implement scripting violation rules Widen the scope of testing from a form/ to a site Test sequence of events Application level Input Domain Explore possibilities for automated response evaluation Ajax allows client-server messages to be sent asynchronously through message passing This introduces more controllability and observability problems,ICST 2008, Jeff Offutt,34,
