1、 Reference number ISO/IEC TR 19795-3:2007(E) ISO/IEC 2007TECHNICAL REPORT ISO/IEC TR 19795-3 First edition 2007-12-15Information technology Biometric performance testing and reporting Part 3: Modality-specific testing Technologies de linformation Essais et rapports de performances biomtriques Partie
2、 3: Essais de modalit spcifique ISO/IEC TR 19795-3:2007(E) PDF disclaimer This PDF file may contain embedded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the
3、computer performing the editing. In downloading this file, parties accept therein the responsibility of not infringing Adobes licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to
4、create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform
5、 the Central Secretariat at the address given below. COPYRIGHT PROTECTED DOCUMENT ISO/IEC 2007 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, witho
6、ut permission in writing from either ISO at the address below or ISOs member body in the country of the requester. ISO copyright office Case postale 56 CH-1211 Geneva 20 Tel. + 41 22 749 01 11 Fax + 41 22 749 09 47 E-mail copyrightiso.org Web www.iso.org Published in Switzerland ii ISO/IEC 2007 All
7、rights reservedISO/IEC TR 19795-3:2007(E) ISO/IEC 2007 All rights reserved iii Contents Page Foreword iv Introduction v 1 Scope . 1 2 Normative references . 1 3 Terms and definitions. 1 4 Symbols and abbreviated terms . 2 5 Modality-specific test design. 2 5.1 Development flow . 2 5.2 Modality-speci
8、fic factors that may impact performance (Step 1) 3 5.3 Modality-specific policies pertaining to Test Subjects (Step 2) 3 5.4 Modality-specific policies pertaining to data collection (Step 3). 5 5.5 Modality-specific policies in impostor transactions (Step 4). 7 5.6 Modality-specific reporting polici
9、es (Step 5) . 9 6 Evaluation of modality-specific influential factors 10 6.1 Robustness tests 10 6.2 Base example: face. 11 6.3 Other examples: fingerprint, iris, vein, and voice . 13 7 Test design principles on “new” modalities 18 Bibliography . 19 ISO/IEC TR 19795-3:2007(E) iv ISO/IEC 2007 All rig
10、hts reservedForeword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards th
11、rough technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, al
12、so take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to pr
13、epare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. In exceptional circumstances, the joi
14、nt technical committee may propose the publication of a Technical Report of one of the following types: type 1, when the required support cannot be obtained for the publication of an International Standard, despite repeated efforts; type 2, when the subject is still under technical development or wh
15、ere for any other reason there is the future but not immediate possibility of an agreement on an International Standard; type 3, when the joint technical committee has collected data of a different kind from that which is normally published as an International Standard (“state of the art”, for examp
16、le). Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to be reviewed until the data they provide are considered to be no longer valid
17、 or useful. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. ISO/IEC TR 19795-3, which is a Technical Report of type 1/2/3, was prepared by Jo
18、int Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics. ISO/IEC TR 19795 consists of the following parts, under the general title Information technology Biometric performance testing and reporting: Part 1: Principles and framework Part 2: Testing methodologies
19、for technology and scenario evaluation Part 3: Modality-specific testing Technical Report Performance and interoperability testing of data interchange formats will form the subject of a future Part 4. ISO/IEC TR 19795-3:2007(E) ISO/IEC 2007 All rights reserved v Introduction In biometric performance
20、 testing and reporting, careful consideration should be given to the characteristic differences of each modality (fingerprint, face, iris, etc.). These differences naturally require variations within the general methodology defined in ISO/IEC 19795-1. Since there are many modality-dependent variatio
21、ns, it is desirable that each modality be specified with specific testing and reporting methods. This ensures that not only specialists of biometrics but also non- specialists can carry out reasonably accurate testing. The characteristics that affect evaluation methodology can at least be discussed
22、from the following viewpoints: characteristics of users, including the definition of impostors; restrictions that come from practical situations in which that biometric modality is used in applications. Characteristics of impostors: There are two factors to consider regarding the definition of impos
23、tors: (1) multiple biometric data from one person, and (2) impostor attempts for behaviour-based modalities, such as voice or signature. For modalities in which multiple biometric data can be collected from one person, e.g. finger (ten fingerprints from one person) and iris (two iris-images from one
24、 person), a rule for permitting or prohibiting use of these data as impostor attempts needs to be clearly defined. In the case of behaviour-based modalities, testing results regarding impostor attempts false match rate (FMR) or false acceptance rate (FAR) can be influenced depending on whether (or h
25、ow much) an impostor tries to imitate an authorized users behaviour or not. For instance, the case in which an impostor physically traces an authorized users signature that the impostor obtained differs significantly in FMR or FAR from the case where the impostor only looks at the signature and imit
26、ates it. For these modalities, a criterion regarding impostor attempts needs to be defined. Characteristics of modality specific to applications In general, almost all modalities of biometrics are used for user authentication. However, some modalities are expected to be used in different classes of
27、applications; for example, face-based identification is widely used in surveillance applications. While we can expect a users cooperation in the former, we cannot expect it in the latter case. Thus variation of testing methodologies needs to be considered depending on the way the modality is used in
28、 real applications. These restrictions can be divided into two classifications: factors relating to users, such as facial expressions that affect the countenance of the face, wearing eye- glasses or contact lenses for the iris; factors relating to external environments that are uncontrollable by the
29、 algorithm or system, such as illumination change for face or background noise for voice. These factors naturally affect the performance, and the types and number of factors are different in each modality. These modality-dependent variations need to be considered in testing and reporting. In additio
30、n, a concept of robustness testing needs to be introduced to evaluate the sensitivity or robustness of the technology toward environmental factors, in case the variation of the factors strongly influences the observed performance. ISO/IEC TR 19795-3:2007(E) vi ISO/IEC 2007 All rights reservedThis pa
31、rt of ISO/IEC 19795 is intended to describe the variations in methodologies relating to these modality- dependent characteristics. It presents and defines methods for determining, given a specific biometric modality, how to develop a technical performance test. This part of ISO/IEC 19795 has been de
32、veloped based on six technical reports 1-6, which are the outputs of standardization activities in Japan for biometric testing and reporting, and have been published by the Japanese Standards Association as JIS-TRs (Japanese Industrial Standards Technical Reports). They have been prepared through di
33、scussions by experts of respective modalities, and have extensive considerations particular to different procedures specific to each modality. They are intended to define detailed procedures for testing, including specifications of a test database and how to collect data. These documents can be used
34、 as reference when designing specific evaluation procedures. TECHNICAL REPORT ISO/IEC TR 19795-3:2007(E) ISO/IEC 2007 All rights reserved 1 Information technology Biometric performance testing and reporting Part 3: Modality-specific testing 1 Scope In biometric performance testing and reporting, car
35、eful consideration needs to be given to the characteristic differences of each modality (fingerprint, face, iris, etc.). These differences naturally require variations within the general methodology defined in ISO/IEC 19795-1. This part of ISO/IEC 19795 describes the methodologies relating to these
36、modality-dependent variations. It presents and defines methods for determining, given a specific biometric modality, how to develop a technical performance test. 2 Normative references The following referenced documents are indispensable for the application of this document. For dated references, on
37、ly the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO/IEC 19795-1:2006, Information technology Biometric performance testing and reporting Part 1: Principles and framework 3 Terms and definitions For the purposes o
38、f this document, the terms and definitions given in ISO/IEC 19795-1:2006 and the following apply. 3.1 influencing factors factors that have influence on performance 3.2 robustness test test to evaluate how much a certain influential factor affects biometric performance 3.3 active forgery attempt imp
39、ostor attempt in which an individual tries to match the stored template of a different individual by presenting a simulated or reproduced biometric sample, or by intentionally modifying his/her own biometric characteristics 3.4 forgery type type of method of impostor forgery attempts ISO/IEC TR 1979
40、5-3:2007(E) 2 ISO/IEC 2007 All rights reserved3.5 forgery level difficulty level of impostor forgery attempts 4 Symbols and abbreviated terms ROC receiver operating characteristic CMR cumulative matching rate CMC cumulative match characteristic FNMR false non-match rate FMR false match rate 5 Modali
41、ty-specific test design 5.1 Development flow When designing performance evaluation tests, the following test planning sequence should be used to ensure that the primary modality-dependent factors are accounted for (see Figure 1): Step 1: Identify and analyze “influencing factors“ that may impact per
42、formance Step 2: Consider and establish policy on test subjects Step 3: Consider and establish policy on data collection Step 4: Consider and establish policy on impostor transactions Step 5: Consider and establish policy on performance reporting Figure 1 Development flow This sequence assumes that
43、the modality and test type (i.e. technology / scenario / operational) have been defined. Identify and analyze “influencing factors“ Consider and establish policy test subjects data collection impostor transactions performance reporting ISO/IEC TR 19795-3:2007(E) ISO/IEC 2007 All rights reserved 3 5.
44、2 Modality-specific factors that may impact performance (Step 1) Factors that influence performance (“influencing factors”) should be identified and analyzed in Step 1 of the test design development flow because biometric performance can be greatly affected by a wide variety of influencing factors.
45、The same biometric device may generate different test results if these influencing factors differ. Controlling, recording, and reporting influencing factors is indispensable to executing repeatable performance tests and to predicting operational performance. When developing test procedures, the test
46、 designer should identify and analyze those influencing factors known to impact performance for a given modality. The designer may also include modality-specific factors whose impact on performance is postulated but not known. In identifying influencing factors, the designer should consider, at a mi
47、nimum, the following (see also Annex C of ISO/IEC 19795-1:2006: A) Biometric sensor quality and characteristics B) Biological or behavioural characteristics of the subject relevant to data collection (essential historical or demographic data): Invariable: Gender, ethnic origin, occupation Variable:
48、Biological: age, body dimensions / anthropometric data (height, weight, etc), musculoskeletal disorders Habitual / Social factors: smoking preference, hairstyle, makeup, eyewear (glasses, contacts, etc), clothing C) Environmental factors applicable to the biometric device, sensor, or application suc
49、h as: Temperature Humidity Illumination Type (standard incandescent, fluorescent, tungsten halogen, reflector lamps, light emitting diodes (LEDs), sunlight, etc) Noise Position of sensor with regards to the user D) Temporal change of the biometric features E) Impact of active forgery attempts on false acceptance, particularly in behavioural modalities F) Differences between the data capture and signal processing subsystems used in the enrolment phase and th
