SAE J 2830-2016 Process for Comprehension Testing of In-Vehicle Symbols.pdf

《SAE J 2830-2016 Process for Comprehension Testing of In-Vehicle Symbols.pdf》由会员分享，可在线阅读，更多相关《SAE J 2830-2016 Process for Comprehension Testing of In-Vehicle Symbols.pdf（20页珍藏版）》请在麦多课文档分享上搜索。

1、 _ SAE Technical Standards Board Rules provide that: “This report is published by SAE to advance the state of technical and engineering sciences. The use of this report is entirely voluntary, and its applicability and suitability for any particular use, including any patent infringement arising ther

2、efrom, is the sole responsibility of the user.” SAE reviews each technical report at least every five years at which time it may be revised, reaffirmed, stabilized, or cancelled. SAE invites your written comments and suggestions. Copyright 2016 SAE International All rights reserved. No part of this

3、publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE. TO PLACE A DOCUMENT ORDER: Tel: 877-606-7323 (inside USA and Canada) Tel: +1 724-776-49

4、70 (outside USA) Fax: 724-776-0790 Email: CustomerServicesae.org SAE WEB ADDRESS: http:/www.sae.org SAE values your input. To provide feedback on this Technical Report, please visit http:/www.sae.org/technical/standards/J2830_201606 SURFACE VEHICLE RECOMMENDED PRACTICE J2830 JUN2016 Issued 2008-07 R

5、evised 2016-06Superseding J2830 JUL2008 Process for Comprehension Testing of In-Vehicle Symbols RATIONALE Symbols also referred to as icons - can be used to communicate information to the driver in a manner that is not dependent on language and can save valuable space on in-vehicle displays. Incompr

6、ehensible symbols, however, have the potential to negatively affect safety (e.g., if the driver does not understand the symbol and either does not respond in a timely manner, or responds in a manner that makes things worse). Despite the ubiquity of symbols within the in-vehicle environment, few guid

7、elines exist for testing of symbols. Key shortcomings of existing symbol testing procedures include: (1) a lack of contextual information provided to experimental participants, (2) over-reliance on evaluator judgment regarding how well an experimental participant comprehended a symbols meaning, and

8、(3) a lack of prescriptive information - feedback to symbol designers regarding how individual symbols could be improved based on the results of the comprehension testing. This update to the 2008 version of J2830 includes (1) references to ISO symbols testing guidelines, (2) more recent research in

9、this area (e.g., Chi Huang, Shieh, these are within the scope of this recommended practice. The process described in this recommended practice includes criteria that are used to identify how well the perceived meaning matches the intended meaning for a representative sample of drivers. The data from

10、 this process are analyzed to determine the drivers comprehension of the symbol. These data provide guidance as to the symbols that can be improved and used. Although the process described in this recommended practice emphasizes a paper-and-pencil approach to administer the test, a computer could be

11、 used instead. NOTE: This process was initially developed specifically for testing active safety symbols (e.g., collision avoidance functions), or other symbols that reflect some in-vehicle message (e.g., navigation, vehicle status, or infotainment functions). For that reason, many of the examples p

12、rovided concern those systems. 2. REFERENCES 2.1 Applicable Documents The following publications form a part of this specification to the extent specified herein. Unless otherwise indicated, the latest issue of SAE publications shall apply. 2.1.1 SAE Publications Available from SAE International, 40

13、0 Commonwealth Drive, Warrendale, PA 15096-0001 USA, phone: 877-606-7323 (inside USA and Canada) or +1 724-776-4970 (outside USA), www.sae.org. Campbell, J. L., Kludt, K., FMVSS 101) Washington DC: National Highway Traffic Safety Administration Green, P. (1993). Design and evaluation of symbols for

14、automobile controls and displays. In B. Peacock Docket No. NHTSA-2010-0053). Retrieved from http:/www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+DOT+Releases+Guidelines+to+Minimize+In-Vehicle+Distractions. Richman, J. B., Campbell, J. L., their validity and use. Proceedings of the Human Factors and E

15、rgonomics Society Annual Meeting , Santa Monica, CA: Human Factors and Ergonomics Society, 979-983). SAE INTERNATIONAL J2830 JUN2016 Page 6 of 20 Zwaga, H. J., The role of design and human factors. Proceedings of the Human Factors and Ergonomics Society Annual Meeting.Santa Monica, CA: Human Factors

16、 and Ergonomics Society, 782-785). 3. DEFINITIONS 3.1 COMPREHENSION The degree to which a user understands the intended meaning of a symbol. 3.2 COUNTERBALANCING Using all of the possible orders of conditions, or randomly determining the order for each participant to control for order effects (Cozby

17、, 2009). 3.3 CRITICAL CONFUSION A response to the symbol undergoing comprehension testing indicating that the test participant perceived the meaning to convey a potentially unsafe action/meaning or one actually prohibited by the symbol. 3.4 “MAJOR” MESSAGE ELEMENTS Those words or phrases within the

18、definition or message associated with a symbol that are critical to adequate comprehension. For example, for a “Tire pressure warning-low” symbol, the “tire” and “warning” parts of the message are “major” elements. See 4.1.2.1. 3.5 MESSAGE The idea, status, or action that the symbol is intended to r

19、epresent. NOTE: ISO 9186 and other documents use the term referent. 3.6 “MINOR” MESSAGE ELEMENTS Those words or phrases within the definition or message associated with a symbol that are less critical to adequate comprehension. For example, for a “Tire pressure warning-low” symbol, the “pressure” an

20、d the “low” parts of the message are “minor” elements. See 4.1.2.1. 3.7 SAFETY CRITICAL SYMBOLS Symbols that are intended to communicate a likely or immediate hazard, status, or required action. 3.8 SYMBOL A visually perceptible figure used to transmit information, usually independently of language,

21、 produced by drawing, printing or other means (see also ISO 2575) NOTE: In this recommended practice, the term “symbols” does not refer to mathematical symbols (e.g., +, =, or ) or words/letters (e.g., CRUISE or BRAKE). 4. PROCESS STEPS 4.1 Step 1: Develop and Review Candidate Symbols for Subsequent

22、 Comprehension Testing 4.1.1 Obtain candidate symbols for testing Symbols should be provided by individuals and organizations interested in having the comprehensibility of candidate symbols assessed. All symbols should be submitted as electronic graphic files (e.g., .jpg or.gif). Alternative version

23、s of the same symbol should be as similar as possible, in terms of size, appropriate color, resolution, etc., to the intended future SAE INTERNATIONAL J2830 JUN2016 Page 7 of 20 application and real-world use of that symbol. Symbols may also be obtained from other SAE and ISO standards (e.g., ISO 25

24、75, SAE J1362, ISO 3767-1, FMVSS 101, ISO 7639) and from existing products. NOTE: One should start out with a wide variety of candidates because the testing that follows is expensive and time consuming (see also Campbell, Hoffmeister, Kiefer, Selke, Green, Eberhard and Green, 1989; Ng, Siu, and Chan

25、, 2013; Ng, Siu, and Chan, 2013; Ng and Chan, 2015). The overall goal of the production test is to generate ideas for several candidate symbols. Participants are asked to draw symbols that they think represent a particular message. Participants shall be provided with a description of what the symbol

26、 is to represent, not just a name (e.g., choke) and a box in which they can draw a symbol. These boxes shall be small (e.g., 1 inch x 1 inch) and participants shall use a fat point marker (e.g., a “Sharpie”) so they cannot draw symbols with too much detail, as most symbols are presented in a small s

27、pace and details will not be legible. Participants shall be told not to include letters as labels as symbols should be text free. NOTE: The production test relies on the participants ability to conceptualize the message and draw a symbol that includes the attributes of the message needed for a compr

28、ehensible symbol. With very complex or novel in-vehicle concepts, this may not be an efficient and effective process for symbol development. As noted above, the participants must have a basic understanding of the message being conveyed by the symbol in order to generate useful candidate symbols for

29、the message. Also, as the purpose of this test is to develop ideas for graphics, and participants often do not have professional graphics skills, the symbols developed based on production tests are often modifications of what participants draw, which may include a combination of concepts. 4.1.3.1.1

30、Participants Representative drivers, not engineers, should be used as test participants; engineers, particularly automotive engineers, tend to produce very unique and non-representative symbols during production tests. Engineers tend to draw internal, mechanistic representations for systems - for ex

31、ample a gear for transmission - while others tend to draw an external view (e.g., a shift lever). There should be 30-40 participants per each language region tested (participants can be tested in groups; see also section 4.2.2 below for a discussion about sample sizes). 4.1.3.1.2 Output The output o

32、f the production test is the variety of graphic representations of a message that participants propose. However, the production test will not result in a final symbol selection; just because a particular image is drawn more often than others does not mean that that image is the best candidate for a

33、symbol. NOTE: Some participants responses during the production test may reflect population stereotypes for symbol elements, icons, or even colors. Examples of population stereotypes might include the use of the color “red” to denote danger, a large “X” to denote a prohibited action, or a schematic

34、of a flame to indicate a fire. To the extent that these population stereotypes accurately reflect the true meaning of the in-vehicle message, they can be a powerful aid to the development an appropriate in-vehicle symbol. However, some stereotypes can reflect an incorrect or outdated perception of r

35、eality (e.g., a depiction of an old rotary style phone to represent “telephone”), or one believed by only a specific subpopulation within the larger driving public. 4.1.3.2 Conduct appropriateness ranking test The purpose of the appropriateness ranking test is to screen the candidate symbols generat

36、ed during the production test to a more manageable number of high-potential candidates for further testing (e.g., Green and Pew, 1978). This screening of candidate symbols for a message is necessary to make further testing of the symbols more feasible and cost-effective. Participants are shown a mes

37、sage and several symbols that could be used to represent that message, typically 4-8. Participants rank order the symbols from best to worst in terms of communicating the desired message (though Likert-scale ratings and magnitude estimation approaches can be used as well). Participants are encourage

38、d not to include ties (e.g., these are both best), as ties occur more often for symbols that are less understandable. SAE INTERNATIONAL J2830 JUN2016 Page 9 of 20 4.1.3.2.1 Participants Representative drivers should be the test participants. There should be 30-40 participants per each language regio

39、n tested (participants can be tested in groups; see also section 4.2.2 below for a discussion about sample sizes), which is sufficient to capture the range of participant views. 4.1.3.2.2 Output After these ranking data have been gathered, the candidate symbols with the highest ranking perhaps the t

40、hree with the highest mean ranking shall be selected for further testing. 4.1.3.3 Obtain or generate a message for each symbol All symbols shall have a message, as well as a general in-vehicle function (i.e., navigation, collision avoidance, vehicle status, etc.) assigned to them. The message assign

41、ed to each symbol should be carefully selected by the individual or organization submitting the symbol(s), as this message will be compared to participants responses when determining overall comprehension scores. Also the individual or organization submitting the symbol(s) shall, if applicable, iden

42、tify any “major” vs. “minor” elements of the message to support the data analyses conducted in Step 4. The message (or definition) identified for each symbol shall be clear and complete to support accurate and efficient scoring of the participants responses in Step 4 of this procedure. Example - If

43、the intended actual message (or definition) of a candidate symbol is “Tire pressure warning-low”, using “Tire pressure” as the intended message (or definition) would likely lead to higher comprehension scores than are warranted. Even small changes to the intended message can lead to large difference

44、s in the final comprehension scores for individual symbol. If appropriate, comprehension responses from the participants can be scored against more than one definition. 4.2 Step 2: Prepare for Comprehension Testing 4.2.1 Prepare test materials. Assemble test materials as follows: a. Candidate symbol

45、s shall be prepared so that they are as similar as possible in terms of size, appropriate color, luminance contrast, resolution, etc., to the intended future application and real-world use of the symbol. b. Candidate symbols shall be placed on separate sheets of paper, slides, or computer screens, d

46、epending on the presentation method planned for the study. A single presentation medium (i.e., paper, slides, or computer screens) shall be used for a particular experiment so as not to confound results with perceptions altered by varying presentation media. NOTE: One presentation approach that can

47、work well when testing a symbol that has already been introduced into a vehicle fleet is to use a picture as the test stimuli, showing the symbol-to-be-tested in its real-world context (Figure 1). Although using pictures can make the test preparation more difficult, this approach improves a high lev

48、el of realism associated with the comprehension testing and increases the validity of the test. The pictures should show the symbol co-located with functionally similar symbols, as well as any associated controls. SAE INTERNATIONAL J2830 JUN2016 Page 10 of 20 Figure 1 - Example of using a photograph

49、 as test stimuli, showing the symbol-to-be-tested, as well as surrounding areas of the vehicle interior c. In addition to the candidate symbols tested, approximately 10-15% of the symbols in the test set shall be existing, reasonably well-understood symbols (e.g., seat belt, fuel, and engine temperature) to provide comparison data (see Campbell et al, 2004). Given the small sample sizes used, the absolute measures of unders