1、Best Practices Entry: Best Practice Info:a71 Committee Approval Date: 2000-03-07a71 Center Point of Contact: JPLa71 Submitted by: Wil HarkinsSubject: Common Review Methods for Engineering Products Practice: Practice: Conduct technical reviews to validate engineering designs using a common, consisten
2、t approach which has been proven to lead to reliable and quality products. A technical review is an evaluation of the engineering status of products and processes by an independent group of knowledgeable people. Although major technical reviews for a project differ in their content and timing, there
3、 are practices common to most reviews which may be defined to assure review success. These practices provide a common framework for planning, conducting, documenting, and evaluating the review process.Abstract: Preferred Practice for Design from NASA Technical Memorandum 4322A, NASA Reliability Pref
4、erred Practices for Design and Test.Benefits:Standards established for common review methods are presently supporting reliability assurance by emphasizing early detection and correction of deficiencies through the increased use of working level, peer reviews (detailed technical reviews) in preparati
5、on for major design reviews. The standards also assure that reviews are scaled in accordance with criticality, complexity, and risk, and that the review process is optimized to produce results of value to the mission.Implementation Method:The definition of common review methods essential to successf
6、ul reviews reflects a need for effective planning and management of all technical reviews. Figure 1 provides an overview of the review process for a project responsible for an engineering product. The oval-shaped processes in Figure 1 are expanded in Figures 2 and 3.Review Planning:Review planning h
7、elps to ensure that the sponsoring organization is well prepared and that the anticipated results justify the significant resources to be expended in the review. The project (or task) manager prepares a review plan identifying the reviews to be held at the project, system and subsystem levels; their
8、 relationships to each other; when the reviews are to be held; and their major objectives. A carefully prepared, detailed statement of the objectives of the review and the success criteria for the product being reviewed provides the essential focus for the review process. Although each review is a m
9、ilestone visible to high-level management, a review should be rescheduled if the work is not sufficiently mature or the prerequisites are not met.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-refer to D descriptionD The project manager scales the s
10、tructure and formality of the review to the significance, complexity, and risk of the product. For smaller projects, reviews may be combined or eliminated to achieve the benefits of review at minimum expense. Factors considered in the review plan include:a71 Total project duration. The number and ti
11、ming of reviews should conform to the project schedule, but can be modified to match changes in project duration. Reviews should be considered whenever the project enters a new phase (viz, preliminary design, detail design, system test) or following a major change in requirements or resources.a71 a7
12、1 Level of inheritance. Projects that use inherited designs, hardware, or software should subject the items to inheritance reviews. This review may sometimes be substituted for, or included within, a preliminary design review (PDR) or critical design review (CDR).a71 a71 Sponsor-imposed reviews. Spe
13、cific reviews required by the sponsor should be incorporated into the internal review plan and not duplicated. External reviews, such as independent readiness reviews (IRRs) and independent annual reviews (IARs), should be identified in the review plan and consolidated with equivalent internal revie
14、ws.a71 a71 Technical Complexity. Complex tasks involving critical technology may require multiple subsystem reviews as well as system-level reviews, and high risk components may require a focus on risk assessment and mitigation.a71 Provided by IHSNot for ResaleNo reproduction or networking permitted
15、 without license from IHS-,-,-a71 Management Complexity. Activities with complex interfaces external to the project or task, such as contractor oversight or inter-organizational agreements, may require additional (or more rigorous and structured) reviews. Fully contracted tasks should defer to the c
16、ontractors in-house review practices to the maximum extent possible.Experience has demonstrated that perfunctory preparation is a precursor to poor performance.The selection and commitment of the review board is crucial to the success of the review. The board membership and review scope are defined
17、in a review board charter. Board members are selected for their ability to objectively evaluate the products and processes to be reviewed. The scope should include all relevant technical concerns and related cost considerations, but exclude management issues such as implementation plans, schedules,
18、and work breakdown structures.The essential focus for the review process is provided by a carefully prepared, detailed statement of the review objectives and success criteria for the product under review. Review objectives are stated in terms of specific product features- for example, “does the desi
19、gn satisfy the requirement for the 20-Gbit/s imaging data stream to have a 1.5-ns response time?“ Success criteria define measures of product readiness required to proceed to the next development step; if these criteria are not met, the review must be repeated. Essential prerequisites to the review-
20、 timely distribution of review material to participants and completion of analyses and tests- are also defined in advance. Figure 2 illustrates the review preparation process.refer to D descriptionD As an aid to review planning, Table 1 provides a standard review checklist, with items listed in the
21、order they are performed.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-REVIEW ACTIVITY LEAD PERSONGenerate project review plan. Project ManagerEstablish and document charter. Convening authorityEstablish and document scope, objectives, success crit
22、eria, and prerequisites.Responsible individualSelect review board.Convening authority, responsible individualAnnounce schedule and agenda. Responsible individualPrepare for review: a71 Schedule conference rooma71 Arrange for audiovisual equipment and support, refreshmentsa71 Identify presentation te
23、ama71 Develop presentation guidelinesa71 Hold presenters meetinga71 Assemble material to be reviewed, and distribute material to barda71 Generate presentation and backup materiala71 Dry run or story board presentationa71 Update, produce, and print presentation material, and distribute it to the boar
24、da71 Prepare slides and transparencies, and distribute them to the presenters.Responsible individualStudy material prior to review. Board membersConduct review. Board chairConduct post-review meeting: a71 Identify key findings and recommendationsa71 Develop board consensusa71 Draft board report.Boar
25、d chairConsolidate and filter recommendations for action (RFAs)Board chair, responsible individualProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Accept RFAs as action items, advisories, or rejected items a71 Identify critical action items.Responsibl
26、e individualComplete and issue final board report. Board chairSubmit metrics to the Office of Engineering and Mission Assurance.Responsible individualPrepare and issue RFA disposition plan. Responsible individualApprove disposition plan. Convening authorityApprove action item closures. Responsible i
27、ndividualReview action item closures; provide feedback to responsible individual and convening authority.Board chair, selected board membersTable 1. Standard Review Checklist Review Procedures:Reviews begin with the board chairs discussion of review objectives and instructions on desired audience pa
28、rticipation, including who may generate a recommendation for action (RFA), and guidelines for keeping the review on schedule. The chair remains responsible for keeping the review within the scope, on the agenda, and on schedule. The goal of each presentation is to identify issues in the review to be
29、 solved outside of the review. Questions directed by the board members to the presenter should be phrased in an objective, neutral, and non-inquisitory manner addressing the product under review rather than the merits of the presenter. Once an issue has been sufficiently clarified, the chair summari
30、zes the issue and assigns someone responsibility for writing an RFA. Presentations conclude with a summary of current work status, open issues and concerns, and future challenges.Effective resolution of technical issues by the review board requires prior analysis of design performance, including rel
31、iability, safety, security, and other product requirements. As a precursor to the design review, detailed technical reviews (DTRs) are recommended for important products to facilitate early detection and correction of design deficiencies. DTRs are informal, working-level, peer reviews intended to as
32、sess interface compatibility and to prevent propagation of deficiencies to later products. Like the formal design review to follow, organization of a DTR follows the steps of planning, scoping, team selection, materials distribution, conducting the meeting, and documentation of results. The primary
33、differences are the greater technical depth provided by the DTR, the corresponding higher specialization of the small (commonly 2 to 3-person) review team, and the greater simplicity of the supporting documentation and reports.Review Follow-Up:Provided by IHSNot for ResaleNo reproduction or networki
34、ng permitted without license from IHS-,-,-A post-review board meeting is held to achieve consensus on key issues, findings, and recommendations. The board chair then consolidates all RFAs into a set of action items, which are filtered and prioritized to form the basis of the boards recommendations.
35、A review board report is prepared listing the key findings and recommendations, any dissenting views, summary recommendations for action, and any advisories, actions items, and critical actions identified by the assigned responsible individual. The recommendations of the review board are advisory on
36、ly, even though some projects may use the review results as the basis of their decision to proceed to the next step in product development. Figure 3 portrays the flow of review follow-up activities.Responsibility for tracking, resolution, and close out of action items rests with the responsible indi
37、vidual. After categorization of the boards recommendations into accepted action items, advisories, or rejected items, and an assessment of their criticality, this person submits a board recommendation disposition plan to the board chair for comment and to the convening authority for approval. Follow
38、ing each review, the responsible individual and the board chair provide metrics to the JPL Office of Engineering and Mission Assurance. These data on the cost, results, and identified benefits of the review are used to evaluate the annual trends of the JPL-wide review process as one aid to continuou
39、s improvement.refer to D descriptionD The review process concludes with the action item assignees completing their assigned actions and documenting the recommendations for action item closure. The responsible individual approves the Provided by IHSNot for ResaleNo reproduction or networking permitte
40、d without license from IHS-,-,-written closures and distributes them to the convening authority and the review board members.Rationale:Experience has shown that the process of preparing for a review is highly valuable to a project- even if the review is never held. Reviews provide the flight project
41、 manager with a knowledgeable and independent evaluation of products or processes, permitting a timely assessment of status and implementation of corrective action. However, reviews require planning, preparation, and follow-up. Definition of clear standards which apply to all reviews aids in trainin
42、g and guiding participants in the activities necessary to assure reliable engineering design.Related Practices:1. Preliminary Design Review, Reliability Preferred Practice No. PD-ED-1215.12. Hardware Review/Certification Requirement, Reliability Preferred Practice No. PD-ED-1215.23. Critical Design
43、Review for Unmanned Missions, Reliability Preferred Practice No. PD-ED-1215.34. Subsystem Inheritance Review, Reliability Preferred Practice No. PD-ED-1262References:1. JPL Standard for Reviews, Jet Propulsion Laboratory document D-10401 (Initial Issue), May 5, 1995.2. Reviews, Laboratory Policy 4-1
44、6.3. Reviews, JPL Standard Practice Instruction SPI 4-16-1.4. Special Reviews, Laboratory Policy 4-18.5. Accident, Incident and Mission Failure Reviews, JPL Standard Practice Instruction SPI 4-18-1.6. Management of Major System Programs and Projects, NASA Management Instruction NMI 7120.4A,B (and NH
45、B 7120.5).7. Classification of NASA Payloads, NASA Management Instruction NMI 8010.1A, Appendix C.8. Technical Reviews, Department of Defense, Military Standard MIL-STD-1521B.Impact of Non-Practice: Impact of Non-Practice: Lacking clear standards for project and task review, and failing to conduct d
46、etailed technical reviews (DTRs) beforehand, design reviews have a tendency to emphasize formality and “packaging“ at the expense of substantive assessment of content and depth of analysis.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Related Practices: N/AAdditional Info: Approval Info: a71 Approval Date: 2000-03-07a71 Approval Name: Eric Raynora71 Approval Organization: QSa71 Approval Phone Number: 202-358-4738Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-
