1、INTERNATIONAL TELECOMMUNICATION UNION ITU-T TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU DATA NETWORKS AND OPEN SYSTEM COMMUNICATIONS OPEN SYSTEMS INTERCONNECTION - CONFORMANCE TESTING X.294 (04195) OS1 CONFORMANCE TESTING METHODOLOGY AND FRAMEWORK FOR PROTOCOL RECOMMENDATIONS ON TEST LABORATORIE
2、S AND CLIENTS FOR THE CONFORMANCE ASSESSMENT PROCESS FOR ITU-T APPLICATIONS - REQUIREMENTS ITU-T Recommendation X.294 (Previously “CCITT Recommendation”) 4862591 ObOb770 580 FOREWORD The ITU-T (Telecommunication Standardization Sector) is a permanent organ of the International Telecommunication Unio
3、n (). The ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommen- dations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Conference (WTSC), which meets every four years, establishes t
4、he topics for study by the ITU-T Study Groups which, in their turn, produce Recommendations on these topics. The approval of Recommendations by the Members of the ITU-T is covered by the procedure laid down in WTSC Resolution No. 1 (Helsinki, March 1-12, 1993). 1 IT-T Recommendation X.294 was revise
5、d by ITU-T Study Group 7 (1993-1996) and was approved under the WTSC Resolution No. 1 procedure on the 10th of April 1995. NOTE In this Recommendation, the expression “Administration” is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. O ITU
6、 1996 All rights reserved. No part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from the ITU. ITU-T X-SERIES RECOMMENDATIONS DATA NETWORKS AND OPEN SYSTEM COMMUNICATIONS (F
7、ebruary 1994) ORGANIZATION OF X-SERIES RECOMMENDATIONS Subject area PUBLIC DATA NETWORKS Services and Facilities Recommendation Series X. 1 -X. 19 Interfaces Transmission, Signalling and Switching X.20-X.49 X.50-X. 89 Network Aspects Maintenance Administrative Arrangements I Service Definitions I X.
8、210-X.2 19 I X.90-X. 149 X.150-X. 179 X. 180-X. 199 OPEN SYSTEMS INTERCONNECTION Model and Notation X.200-X.209 I Security Protocols I X.270-X.279 I Connection-mode Protocol Specifications Connectionless-mode Protocol Specifications X.220-X.229 X.230-X.239 PICS Proformas Protocol Identification X.24
9、0-X.259 X.260-X.269 Layer Managed Objects Conformance Testing INTERWORKING BETWEEN NETWORKS X.280-X.289 X.290-X.299 General Mobile Data Transmission Systems I OS1 MANAGEMENT I X.700-X.799 I X300-X.349 X.350-X.369 Management MESSAGE HANDLING SYSTEMS I Commitment, Concurrency and Recovery I X.850-X.85
10、9 I X.370-X. 399 X.400-X.499 DIRECTORY OS1 NETWORKING AND SYSTEM ASPECTS X.500-X.599 Networking X.600-X.649 Naming, Addressing and Registration X.650-X.679 SECURTTY OS1 APPLICATIONS X.800-X.849 Transaction Processing Remote Ooerations X.860-X.879 X.880-X. 899 I OPEN DISTRIBUTED PROCESSING X.900-X.99
11、9 m LiBb2571 0606772 353 m r .- “11 ., . ._ - , . ,. F D 4862593 0606773 2T = CONTENTS Scope References Definitions Abbreviations . Overview of the conformance assessment process 5.2 Preparation for testing . 5.1 Introduction 5.3 Test operations 5.4 Test report production . Preparation for testing 6
12、.1 Introduction 6.2 6.3 Communication between the test laboratory and the client 6.4 Documentation for conformance assessment Requirements for testability of the System under Test . Test operations . 7.1 Introduction 7.2 Static conformance review 7.3 Test selection 7.4 Test parameterization 7.5 Veri
13、fication of Test Coordination Procedures 7.6 Test campaign . 7.7 Negotiated exits during the test campaign Test report production 8.1 Conformance test reports 8.2 8.3 System Conformance Test Report (SCTR) . Protocol Conformance Test Report (PCTR) . Compliance 9.1 Test laboratory role . 9.2 Client ro
14、le . Annex A . System Conformance Test Report (SCTR) template A . 1 Introduction A.2 Conformance . A.3 SCTR proforma Annex B - Protocol Conformance Test Report (PCTR) template . B . 1 Introduction B.2 Conformance . B.3 PCTR proforma Appendix I - Skeleton EUT template . 1.1 Introduction 1.2 WT profor
15、ma Recommendation X.294 (04/95) . Page 1 2 3 3 4 4 4 4 5 5 5 5 8 10 12 12 14 15 17 17 18 20 21 21 22 22 23 23 24 25 25 25 25 29 29 29 29 33 33 33 1 = 4862591 0606774 126 SUMMARY This Recommendation specifies requirements on both the test laboratory and client, for the conduct of the conformance asse
16、ssment process. The requirements are those necessary to achieve comparability of results of tests on similar implementations performed by different test laboratories. This Recommendation also provides some guidance on the conformance assessment process. The text was developed jointly with ISO/EC JTC
17、1 and the main intent of this revision is to reflect the changes as a result of the work on Protocol Profile Testing Methodology (PPTM) and on Multi-Party Testing Methodology (MFyTM). ii Recommendation X.294 (04195) INTRODUCTION Conformance testing requires mutual understanding and agreement between
18、 the test laboratory and the client. This Recommendation addresses the roles of both the test laboratory and the client during the conformance assessment process, the need to reach mutual agreements between them, and the requirements on each of them. The conformance assessment process is the most vi
19、sible process of conformance testing, where the results of test suite standardization are put to real use. This is also the stage at which there is potentially the most scope for variations to occur. As this Recommendation is concerned with the relatively formal process of testing implementations, i
20、t is important that the number and nature of such variations be very limited. One of the major objectives of standardizing the conformance testing process is to achieve an acceptable and useful degree of comparability of results of conformance assessments of similar implementations. If this is to be
21、 achieved, not only should the same source of tests be used (Le. as specified in appropriate standards), but also the methods of selecting and parameterizing these tests, and presenting their results, should be, to a large extent, the same. This Recommendation addresses the issues which should be ta
22、ken into account, by both the test laboratory and the client, if the necessary consistency of conformance assessment is to be achieved. The target audiences for this Recommendation are the test laboratories and their clients. The test laboratory is responsible for conducting the conformance assessme
23、nt of an OS1 implementation at the request of a client. Typically, test laboratories are: organizations developing or supplying OS1 implementations (first-party test laboratories); organizations willing to verify OS1 implementations themselves before using them (second-party test laboratories); orga
24、nizations, independent of suppliers or users of OS1 implementations, whose business is the testing of such implementations (third-party test laboratories). 1) 2) 3) Clients may be implementors or suppliers of real open systems or other OS1 systems, who are applying for their own implementations to b
25、e tested. Alternatively, they may be procurers of those implementations, or any other interested party. The applicability of this Recommendation is independent of the relationship between the client and the implementation. During the conformance assessment process, the client is responsible for the
26、conformance statements accompanying the System under Test (SUT) and for the configuration of the SUT. Secondary, but related, audiences to whom this Recommendation could also be of interest a) are test realizers; b) c) organizations responsible for the accreditation of first- , second- or third-part
27、y test laboratories; organizations responsible for the issue of test certificates which are based upon the conformance test reports issued by test laboratories; readers of conformance test reports. d) Within this Recommendation, the conformance assessment process relating to both the test laboratory
28、 and the client is divided into three phases: i) preparation for testing; ii) test operation; iii) production of test reports. An overview of these three phases is given in clause 5. Clauses 6 to 9 state requirements on how to conduct these three phases. For the purposes of this Recommendation, it i
29、s assumed that a test laboratory is available and is already organized to provide a conformance assessment service. The test laboratory is assumed to have acquired from a test realizer (whether or not the latter belongs to the same organization) means of Testing ILJTs, for one or more OS1 protocols
30、and according to one or more Abstract Test Methods (ATMs). This Recommendation specifies requirements on the test laboratory with respect to the conduct of the conformance assessment process for a particular client. . Recommendation X.294 (04/95) 111 m 4862593 0606776 TT m Similarly, it is assumed t
31、hat a client is ready to apply for conformance assessment of an OS1 implementation. The client is assumed to be familiar with the appropriate standards, with the concepts of conformance testing and Abstract Test Methods, and to be ready to cooperate with the test laboratory. This Recommendation spec
32、ifies requirements on the client with respect to both the testability of the proposed SUT and the conduct of the conformance assessment process. It is recommended that this Recommendation be read in conjunction with Recommendation X.290. This Recommendation is also to published as ISOhEC 9646-5: 199
33、4. iv Recommendation X.294 (04195) = L)Bb2591 0606777 935 Recommendation X.294 OS1 CONFORMANCE TESTING METHODOLOGY AND FRAMEWORK REQUIREMENTS ON TEST LABORATORIES AND CLIENTS FOR THE CONFORMANCE ASSESSMENT PROCESSI) FOR PROTOCOL RECOMMENDATIONS FOR ITU-T APPLICATIONS - (Geneva, 1992; revised in 1995
34、) 1 Scope 1.1 This Recommendation specifies requirements on both the test laboratory and the client, for the conduct of the conformance assessment process. The requirements are those necessary to achieve comparability of results of tests on similar implementations performed by different test laborat
35、ories. Implementations could support one base specification only, multiple base specifications or one or more profile(s) based on one or more base specifications. This Recommendation also provides some guidance on the conformance assessment process. 1.2 The requirements include: a) b) c) requirement
36、s for the testability of the implementation with respect to Abstract Test Methods; general requirements on the test laboratory and the client applicable to any conformance assessment process; exchange of technical and administrative information, including a System Conformance Statement, an Implement
37、ation Conformance Statement for each relevant base or profile specification, and Implementation extra Information for Testing for each Abstract Test Suite (ATS) to be used for testing each base specification, combination of base specifications or component of each profile, as appropriate, plus for p
38、rofile testing the Profile Test Specification Summary for each profile; cooperation between the test laboratory and the client to reach an agreement on the definition of the Implementation under Test, on the Abstract Test Methods and ATSs to be used and on the conditions under which testing will be
39、performed; e) requirements for the structure and content of the conformance test reports that document the results of the conformance assessment process. This Recommendation is applicable equally to those test laboratories which are affiliated to suppliers or This Recommendation is applicable to con
40、formance assessment of implementations of - d) 1.3 procurers, and those which are independent. 1.4 OS1 and Integrated Services Digital Network (ISDN) base specifications that comply with the relevant requirements for testability in Recommendation X.29 1 ; andor OS1 and ISDN profile specifications th
41、at comply with the relevant requirements for testability in Recommendation X.295, based on: conformance testing specifications specified in compliance with Recommendation X.291; and if appropriate, based on Profile Test Specification Summaries and Profile Specific Test Specifications specified in co
42、mpliance with Recommendation X.295; and using Means of Testing (MOT) in compliance with Recommendation X.293. - - - - I) Recommendation X.294 and ISO/IEC 9646-5, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 5: Requirements on test labor
43、atories and clients for the conformance assessment process, are technically aligned. Recommendation X.294 (04/95) 1 4862591 0606778 871 1.5 The following are outside the scope of this Recommendation: a) b) c) accreditation of test laboratories; d) the production of diagnostic trace information, addi
44、tional to that in the conformance log, resulting from testing performed by the test laboratory, and the supply to the client; aspects of test laboratory operations which are not specific to conformance testing implementations of OS1 base specifications and profiles; certification of implementations
45、of OS1 protocols. 2 References The following Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subjec
46、t to revision: all users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of currently valid ITU-T Recommendations is regularly published. ITU-T Recommendation X.200 (19
47、94) I ISO/IEC 7498-1: 1994, Information technology - Open Systems Interconnection - Basic Reference Model: The Basic Model. ITU-T Recommendation X.210 (1993) I ISO/IEC 10731: 1994, Information technology - Open Systems Interconnection - Conventions for the definition of OSI services. ITU-T Recommend
48、ation X.290 (1993, OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications - General concepts. ISOIIEC 9646-1 : 1994, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part I: General concepts. ITU-T
49、Recommendation X.291 (1993, OSI conformance testing methodology and framework for protocol Recommendations for ITU-T Applications -Abstract test suite specijkation. ISO/IEC 9646-2: 1994, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 2: Abstract test suite specification. CCITT Recommendation X.292 (1992), OSI conformance testing methodology and framework for protocol Recommendations for CCIT applications - The tree and tabular Combined Notation (7TCN). ISO/IEC 9646-3: 1992, Information technology
