1、 Reference numberISO/IEC/IEEE 29119-1:2013(E)ISO/IEC 2013IEEE 2013INTERNATIONAL STANDARD ISO/IEC/IEEE29119-1First edition2013-09-01Software and systems engineering Software testing Part 1: Concepts and definitions Ingnierie du logiciel et des systmes Essais du logiciel Partie 1: Concepts et dfinitio
2、ns ISO/IEC/IEEE 29119-1:2013(E) COPYRIGHT PROTECTED DOCUMENT ISO/IEC 2013 IEEE 2013 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permissi
3、on in writing from ISO, IEC or IEEE at the respective address below. ISO copyright office IEC Central Office Institute of Electrical and Electronics Engineers, Inc. Case postale 56 3, rue de Varemb 3 Park Avenue, New York CH-1211 Geneva 20 CH-1211 Geneva 20 NY 10016-5997, USA Tel. + 41 22 749 01 11
4、Switzerland E-mail stds.iprieee.org Fax + 41 22 749 09 47 E-mail inmailiec.ch Web www.ieee.org E-mail copyrightiso.org Web www.iec.ch Web www.iso.org Published in Switzerland ii ISO/IEC 2013 All rights reserved IEEE 2013 All rights reservedISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 All rights reserve
5、d IEEE 2013 All rights reserved iiiContents Page Foreword . vg3Introduction vig31 Scope 1g32 Conformance . 1g33 Normative references 1g34 Terms and definitions . 1g35 Software Testing Concepts 12g35.1 Introduction to Software Testing . 12g35.1.1 The Role of Testing in Verification and Validation 14g
6、35.1.2 Exhaustive Testing 14g35.1.3 Testing as a Heuristic . 14g35.2 Software Testing in an Organizational and Project Context . 14g35.2.1 The Test Process . 17g35.3 Generic Testing Processes in the Software Life cycle 19g35.3.1 Development Project Sub-processes and their Results . 20g35.3.2 On-goin
7、g Maintenance and its Results . 21g35.3.3 Support Processes for the Software Development Life Cycle . 22g35.4 Risk-based Testing 24g35.4.1 Using Risk-Based Testing in the Organizational Test Process 25g35.4.2 Using Risk-Based Testing in the Test Management processes . 25g35.4.3 Using Risk-Based Test
8、ing in the Dynamic Testing processes 25g35.5 Test Sub-process 26g35.5.1 Test Objectives 26g35.5.2 Test Item . 27g35.5.3 Testing of Quality Characteristics . 27g35.5.4 Test Basis . 28g35.5.5 Retesting and Regression Testing 29g35.5.6 Test Design Techniques . 29g35.6 Test Practices 30g35.6.1 Introduct
9、ion 30g35.6.2 Requirements-Based Testing . 31g35.6.3 Model-Based Testing 31g35.6.4 Mathematical-Based Testing 32g35.6.5 Experience-Based Testing . 32g35.6.6 Scripted and Unscripted Testing . 33g35.7 Automation in Testing . 34g35.8 Defect Management . 34g3Annex A (informative) The Role of Testing in
10、Verification and Validation 35g3Annex B (informative) Metrics and Measures 36g3B.1 Metrics and Measures . 36g3Annex C (informative) Testing in Different Life Cycle Models . 37g3C.1 Overview . 37g3C.2 Agile Development and Testing . 37g3C.2.1 Agile Development Principles 37g3C.2.2 Test Management in
11、Agile Development 38g3C.2.3 Test Sub-processes in Agile Development. 39g3C.3 Sequential Development and Testing . 40g3C.3.1 Sequential Development Principles 40g3ISO/IEC/IEEE 29119-1:2013(E) iv ISO/IEC 2013 All rights reserved IEEE 2013 All rights reservedC.3.2 Test Management in Sequential Developm
12、ent .40g3C.3.3 Test Sub-processes in Sequential Development .41g3C.4 Evolutionary Development and Testing 41g3C.4.1 Evolutionary Development Principles .41g3C.4.2 Test Management in Evolutionary Development 42g3C.4.3 Test Sub-processes in Evolutionary Development 42g3Annex D (informative) Detailed T
13、est Sub-process Examples .44g3D.1 Overview .44g3D.2 Acceptance Test Sub-process .45g3D.3 Detailed Design Test Sub-process .45g3D.4 Integration Test Sub-process .46g3D.5 Performance Test Sub-process 48g3D.6 Regression Test Sub-process 49g3D.7 Retest Test Sub-process .51g3D.8 Story Set Test Sub-proces
14、s 51g3D.9 Story Test Sub-process 51g3D.10 System Test Sub-process .52g3D.11 Component Test Sub-process 53g3Annex E (informative) Roles and Responsibilities in Testing 54g3E.1 Testing Roles .54g3E.2 Communication in Testing 54g3E.3 Independence in Testing .54g3Bibliography 56g3ISO/IEC/IEEE 29119-1:20
15、13(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved vForeword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC pa
16、rticipate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governme
17、ntal and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating Committ
18、ees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its standards through a consensus development process, approved by the American National Standards Institute, which brings together volunteers representing varied viewpoints and interests to achieve the final product.
19、 Volunteers are not necessarily members of the Institute and serve without compensation. While the IEEE administers the process and establishes rules to promote fairness in the consensus development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of the informa
20、tion contained in its standards. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of ISO/IEC JTC 1 is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to na
21、tional bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is called to the possibility that implementation of this standard may require the use of subject matter covered by patent rights. By publication of t
22、his standard, no position is taken with respect to the existence or validity of any patent rights in connection therewith. ISO/IEEE is not responsible for identifying essential patents or patent claims for which a license may be required, for conducting inquiries into the legal validity or scope of
23、patents or patent claims or determining whether any licensing terms or conditions provided in connection with submission of a Letter of Assurance or a Patent Statement and Licensing Declaration Form, if any, or in any licensing agreements are reasonable or non-discriminatory. Users of this standard
24、are expressly advised that determination of the validity of any patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Further information may be obtained from ISO or the IEEE Standards Association. ISO/IEC/IEEE 29119-1 was prepared by Joint Technical Commi
25、ttee ISO/IEC JTC 1, Information technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Software as well as how automation can be used to support testing. The involvement of testing in defect management is also discussed. Annex A describes the role of testing within
26、the larger scope of verification and validation. Annex B provides a brief introduction to metrics used to monitor and control testing. Annex C contains a set of examples showing how to apply the standard in different life cycle models. Annex D provides examples on detailed test sub-processes. Annex
27、E provides additional information on the roles and responsibilities typically encountered in test groups and tester independence. Finally, the Bibliography is at the end of the document. Note that Title Case is used throughout this part of ISO/IEC/IEEE 29119 to denote processes and documents that ar
28、e specified in ISO/IEC/IEEE 29119-2 and ISO/IEC/IEEE 29119-3 (e.g. Test Planning Process, Test Plan), whereas lowercase letters are used for documents that form parts of other documents (e.g. the project test strategy is an element of the Project Test Plan). The test process model that the ISO/IEC/I
29、EEE 29119 series of software testing standards are based on is defined in detail in ISO/IEC/IEEE 29119-2 Test Processes. ISO/IEC/IEEE 29119-2 covers the software testing processes at the organizational level, test management level and for dynamic test levels. Testing is the primary approach to risk
30、treatment in software development. This standard defines a risk-based approach to testing. Risk-based testing is a recommended approach to strategizing and managing testing that allows testing to be prioritized and focused. Templates and examples of test documentation that are produced during the te
31、sting process are defined in ISO/IEC/IEEE 29119-3 Test Documentation. Software testing techniques that can be used during testing are defined in ISO/IEC/IEEE 29119-4 Test Techniques. Together, this series of international standards aims to provide stakeholders with the ability to manage and perform
32、software testing in any organization. INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved 1Software and systems engineering Software testing Part 1: Concepts and definitions 1 Scope This part of ISO/IEC/IEEE 29119 specifies definitions a
33、nd concepts in software testing. It provides definitions of testing terms and discussion of concepts key to the understanding of the ISO/IEC/IEEE 29119 series of software testing international standards. 2 Conformance ISO/IEC/IEEE 29119-1 is informative and no conformance with it is required. The IS
34、O/IEC/IEEE 29119 software testing series of standards contain three standards where conformance may be claimed: g127 test processes; g127 test documentation; g127 test techniques. Conformance is addressed in ISO/IEC/IEEE 29119-2, ISO/IEC/IEEE 29119-3 and ISO/IEC/IEEE 29119-4. 3 Normative references
35、This document does not require the use of any normative references. Standards useful for the implementation and interpretation of this part of ISO/IEC/IEEE 29119 are listed in the Bibliography. 4 Terms and definitions For the purposes of this document, the terms and definitions given in ISO/IEC/IEEE
36、 24765 and the following apply. NOTE The following terms and definitions are provided to assist with the understanding and readability of parts 1, 2, 3 and 4 of the ISO/IEC/IEEE 29119 Software Testing standards so some terms defined in ISO/IEC/IEEE 29119-1 will not be used in ISO/IEC/IEEE 29119-1 an
37、d will only be used in another standard in the ISO/IEC/IEEE 29119 series. Only terms critical to the understanding of these standards are included; this clause is not intended to provide a complete list of testing terms. The systems and software engineering Vocabulary ISO/IEC/IEEE 24765 should be re
38、ferenced for terms not defined in this clause. This source is available at the following web site: http:/puter.org/sevocab. ISO/IEC/IEEE 29119-1:2013(E) 2 ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved4.1 accessibility testing type of usability testing used to measure the degree to w
39、hich a test item can be operated by users with the widest possible range of characteristics and capabilities 4.2 actual results set of behaviours or conditions of a test item, or set of conditions of associated data or the test environment, observed as a result of test execution EXAMPLE Outputs to h
40、ardware, changes to data, reports, and communication messages sent. 4.3 backup and recovery testing type of reliability testing that measures the degree to which system state can be restored from backup within specified parameters of time, cost, completeness, and accuracy in the event of failure 4.4
41、 black-box testing see specification-based testing (4.39) 4.5 capacity testing type of performance efficiency testing conducted to evaluate the level at which increasing load (of users, transactions, data storage, etc.) compromises a test items ability to sustain required performance 4.6 compatibili
42、ty testing type of testing that measures the degree to which a test item can function satisfactorily alongside other independent products in a shared environment (co-existence), and where necessary, exchanges information with other systems or components (interoperability) 4.7 coverage item see test
43、coverage item (4.54) 4.8 decision types of statements in which a choice between two or more possible outcomes controls which set of actions will result Note 1 to entry: Typical decisions are simple selections (e.g. if-then-else), to decide when to exit loops (e.g. while-loop), and in case (switch) s
44、tatements (e.g. case-1-2-3-N). 4.9 dynamic testing testing that requires the execution of the test item 4.10 endurance testing type of performance efficiency testing conducted to evaluate whether a test item can sustain a required load continuously for a specified period of time ISO/IEC/IEEE 29119-1
45、:2013(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved 34.11 equivalence partition subset of the range of values of a variable, or set of variables, within a test item or at its interfaces such that all the values in the partition can reasonably be expected to be treated similarly b
46、y the test item (i.e. they may be considered “equivalent“) by the test item 4.12 equivalence partition coverage proportion of identified equivalence partitions of a test item that are covered by a test set Note 1 to entry: In many cases, the identification of equivalence partitions is subjective (es
47、pecially in the sub-partitioning of “invalid“ partitions), so a definitive count of the number of equivalence partitions in a test item could be impossible. 4.13 equivalence partitioning test design technique in which test cases are designed to exercise equivalence partitions by using one or more re
48、presentative members of each partition 4.14 error guessing test design technique in which test cases are derived on the basis of the testers knowledge of past failures, or general knowledge of failure modes Note 1 to entry: The relevant knowledge could be gained from personal experience, or might be
49、 encapsulated in, for example, a defects database or a “bug taxonomy“. 4.15 expected results observable predicted behaviour of the test item under specified conditions based on its specification or another source 4.16 exploratory testing experience-based testing in which the tester spontaneously designs and executes tests based on the testers existing relevant knowledge, prior exploration of the test item (including the results of previous tests), and heuristic “rules of thumb“ regarding common software
