1、Lessons Learned Entry: 1780Lesson Info:a71 Lesson Number: 1780a71 Lesson Date: 2007-04-10a71 Submitting Organization: JPLa71 Submitted by: Kenneth Atkinsa71 POC Name: Kenneth Atkins, Rick Grammiera71 POC Email: Kenneth.L.Atkinsjpl.nasa.gov, Richard.S.Grammierjpl.nasa.gova71 POC Phone: 818.354.4480 (
2、Atkins), 818.354.0596 (Grammier)Subject: How to Plan and Manage Project Reserves Abstract: Lessons from JPL spaceflight projects initiated in the 1990s demonstrate that the flight project manager (PM) must maintain a buffer of reserves that will be sufficient to overcome the problems endemic to the
3、latter stages of complex, high risk, system development. The Stardust project may be unique among these cost-capped missions in that it managed to complete development with the proposed reserve. Stardust project reserves management practices may serve as a model for the project manager in determinin
4、g whether project reserves are sufficient, when to release the reserves, and how they should be spent.Description of Driving Event: The JPL Stardust project proved very successful in accomplishing its objective of returning a sample of cometary material. This was due in part to its innovative use of
5、 programmatic resource management techniques, and to its firm commitment to managing the work against cost-to-complete (in contrast to merely managing actual costs against planned). NASA budgetary constraints, and a mandate to accomplish more with fewer resources, led in the mid-1990s to a faster, b
6、etter, cheaper (FBC) paradigm and competitive mission selections. Where NASA-funded spaceflight projects were previously requirement-driven and open-scope, successive JPL projects became severely cost and/or schedule-constrained. With NASA demonstrating a resolve to cancel projects that fail to demo
7、nstrate budget and schedule discipline, managers of essentially fixed-price flight projects have been pressed to exercise tight budget and schedule control. Although there were some significant successes under this new paradigm, problems with balancing project resources and risk led to a number of m
8、ission failures (e.g., Mars Climate Orbiter, Mars Polar Lander, Wide-Field Infrared Explorer, Contour). Achieving this culture change for planetary exploration and space science required processes and procedures for implementing “design-to-cost“ and “manage-to-budget.“ Lessons from JPL flight projec
9、ts initiated in the 1990s demonstrate that the flight project manager (PM) must maintain a buffer of reserves that will be sufficient to overcome the problems endemic to the latter stages of complex, high risk, system development. Consistent and rigorous reserve management is a preventative practice
10、 that assures that the project remains on track. In contrast, it is likely that a PM who spends too much too soon, or delays until no amount of reserves can salvage the project, will be unable to recover from adverse developments. Stardust may have been unique among these flight projects in completi
11、ng development (Phase D) with the proposed reserve. Stardust project management practices and results may serve as a model for PMs who wish to understand (1) whether project reserves established by a proposal team competing under very tight cost constraints are sufficient, (2) how to develop a plan
12、and criteria for release of reserve, and (3) how to integrate the releases with an earned value management (EVM) system (Reference (1). Specifically, the Stardust experience provides guidance on how the PM may: Provided by IHSNot for ResaleNo reproduction or networking permitted without license from
13、 IHS-,-,-1. Determine the adequate reserves buffer, 2. Develop a plan for expending the reserve in the context of risk, 3. Integrate the reserve plan with EVM and scope, and 4. Control reserve on “cost-to-go“ (i.e., cost to complete the baseline plan). The Stardust PM implemented a reserves manageme
14、nt plan early in Phase B to (1) derive a set of affordable flight system capabilities that matched the prioritized requirements, (2) produce a risk-adjusted scope of work, and (3) generate a work plan based on earning value (Reference (2). This established a planned cash flow inside the time boundar
15、ies to defeat known threats to mission success. Metrics were identified and criteria were pre-determined that would trigger the timely application of additional reserves to mitigating major risks (Figure 1). Planning the allocation of the reserve pool recognized that the risk scenario was only an es
16、timate: some expected problems did not occur, while some expected and unexpected problems did. Implementing the plan in 10 key steps, Stardust: Figure 1 is a set of two color graphs titled ?Performance Assessment Metric for Total Project? that plot the accomplishment of monthly earning events (basel
17、ine vs. actual) such as Stardust task initiation and completion. The topmost graph measures baseline versus actual monthly earning events in terms of percentages (y-axis) over a span of 24 months (x-axis). A solid green line representing the baseline remains constant at 100 percent across the graph.
18、 A solid blue line representing the actual starts at 100 percent in the first month and continues in a downward slope reaching a minimum after 6 months. Thereafter, the graph continues in an upward slope until it reaches the current state after 10 months. A red arrow directs our attention to the cur
19、rent state and displays the value at that point: 86% of the total number of project earning tasks are complete. The bottom graph plots the baseline versus actual monthly earning events in terms of events per month (y-axis) over a span of 24 months (x-axis). The graph contains three bars: green bar (
20、baseline), blue bar (actual), white bar (current schedule). The graph exhibits a trend of baseline exceeding the actual until the eighth month. From the eight to 10th month (current state), baseline and actual earning events are roughly the same. Thereafter, the projected current schedule (white bar
21、) exceeds the baseline plan (green bar) for the ensuing months.Figure 1. This Stardust performance assessment metric (PAM) plots the accomplishment of monthly earning events (baseline vs. actual) on the left ordinate as both raw counts and a ratio of actual to baseline. The curves (bottom plot) prov
22、ide the cumulative picture of baseline (green /long dashes) and critical path events (red/alternate dash-dot), with the actual (blue/solid bottom curve) accomplishment falling between. The middle (black/dashed) curve projects moving the actual progress off the critical path and recapturing schedule
23、slack. If the actual cannot be returned to the baseline within the target schedule completion, the project will fail the schedule and cost (time=$) success criteria. (That is, the schedule must slip, the baseline job cannot be completed as agreed, and cost will certainly increase, resulting in loss
24、of control over the scope.) To avoid this and achieve the baseline (top curve) job, Stardust planned to have adequate schedule and budget reserve. Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-1. Documented in an agreement (project plan or Mission
25、Definition and Requirements Agreement) co-signed by the customer a clear and unequivocal definition of mission success. The definition included measurable success criteria so that the PM could later prioritize the commitment of reserves. Stardust established a “performance buffer“ by defining a thre
26、e-level project scope hierarchyprimary, secondary, and tertiary. The primary category represents the minimal success statements that make the project worthwhile. The secondary category houses baseline-scope criteria where the achievement of most of them is acceptable. The tertiary category, also in
27、the baseline, contains those criteria that are important but would be the first to be de-scoped in response to unforeseen events. 2. Reviewed the design capabilities against the requirements to determine whether modifications to the existing design (or design of new elements) may be needed to captur
28、e the full baseline, and whether such changes can be made within the targeted cost and schedule. A trade study was performed to assess reserves needed to accommodate the uncertainty and to force any necessary de-scope decisions. This resulted in a Stardust review called a Capability vs. Requirements
29、 Review (CRR) that assessed the ability of the project to design-to-cost. The initial Significant Risk List (SRL) was generated from the threats and probabilities identified at the CRR. 3. Populated the Work Breakdown Structure (WBS), assessing and adjusting the constituent, low level task plans for
30、 risk and the estimated reserve required for each WBS item. The negotiated margins were weighted to reflect the probabilities identified at the CRR, and then summed to produce a risk-adjusted, justifiable, reserve pool. 4. Loaded the integrated schedule network software to determine the critical pat
31、h and any slack in the project. Added the schedule reserve at the end, protecting the “schedule“ success criterion. Fully aligned and checked links, paths to deliverables, key events, etc. 5. Next, Stardust loaded the financial software to apply current manpower cost rates, overheads, etc., to produ
32、ce the time-phased baseline budget (costing plan) and the baseline Earned Value (EV) structure. They performed a “Goldilocks“ validation to ensure they had a “just-right“ number and spacing of earning events. (Not so few events that the project wouldnt know when it was in trouble, and not so many as
33、 to micromanage the activity.) The EVM system was set up early in the preliminary design phase (Phase B) and used to clarify and capture the baseline work plan and validate its consistency with the constrained cost and schedule. This preparation provided the control infrastructure for success in Pha
34、se C/D. 6. Using the results of Step 3, the SRL was updated to include the latest information on probabilities and mitigation options, and to assess the cost-to-mitigate for major risks. Rough-order-of-magnitude (ROM) estimates were prepared for incorporation as “soft liens“ into the Budget Change L
35、og used in Step 7. 7. A Budget Change Log (Figure 2) was implemented as the control discipline for release of reserves, ensuring that task plans, contracts, networks, and the EV system were updated expeditiously and were consistent. For JPL subcontracts, the incentive-fee structure and the events th
36、at were to earn fee (e.g., risk reduction, change in scope) were clearly designated. Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-In Figure 2, a budget change log is divided into three main columns. The first column is titled ?Budget Change Log,?
37、the second ?Estimated Liens,? and the third ?Definitized Liens.? The first few rows summarize the JPL and contractor project reserves, respectively, against estimated and definitized liens. The remaining rows list the specific liens. For example, the first lien listed is ?EPS Test,? it is identified
38、 as a risk reduction type lien, with a date of August-97 and an apportionment to the ?Contractor? (not JPL-internal) account. The FY98 lien amount for the lien is -150? both estimated and definitized? and the Status is ?in December MPM Plan.?Figure 2. Stardust used a Budget Change Log to translate t
39、he project reserve into hard liens (authorized to debit reserves) and from soft liens (not yet authorized to debit reserves). Legend: R/R = risk reduction; C/S = change in scope; O/R = overrun 8. The reserve strategy was implemented, getting timely (days, not a week) information from the EVM system
40、on the effectiveness of reserve expenditures in mitigating threats. 9. Swept the project regularly for threats and opportunities to avoid trouble (i.e., to use reserves effectively on risk reductions, changes in scope, or overruns). Updated the SRL and Budget Change Log accordingly. 10. Managed the
41、unencumbered reserves against the cost-to-go. To maintain project control and assure that an acceptable product is delivered on time and within budget, it is essential to continuously monitor remaining costs to the conclusion of the project (cost-to-go). Stardust released reserves both proactively a
42、nd reactively using the Percent Reserve on Cost-To-Go metric as a “governor.“ Figure 3 provides an example of this index. Note that a 10 percent floor was set as a minimum reserve level, not to be broken until it became clear (e.g., late in development) that the project could be completed within thi
43、s reserve for “unknowns.“ A metric like Percent Reserve on Cost-To-Go provides a current picture of the reserve pools strength against the cost to complete the baseline plan. Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-In Figure 3, a line graph p
44、lots the Stardust reserves against cost-to-go (y-axis) over a twelve month span (x-axis). A blue line representing the unencumbered reserves begins at seventeen percent in the first month and continues in an increasing slope and reaches a maximum of twenty percent at the fifth month. Thereafter, it
45、continues in a decreasing slope until it reaches a minimum of eleven percent at the seventh month. Afterwards, it continues in an increasing slope until it reaches the current state at the tenth month. A green arrow directs our attention to the current state and displays the value at this point: fou
46、rteen percent. A pink line representing the reserves on Cost to Go remains constant at ten percent from the first month to the current state.Figure 3. An example trend in percent Stardust reserves against cost-to-go. This example depicts an initial growth in the percent reserves on cost-to-go, perha
47、ps resulting from favorable early experience where some anticipated risk mitigations were not required. The February 1997 drop after reaching the 20 percent reserves level indicates the project either (1) decided to preemptively address some threats by releasing reserve to mitigate or retire them, o
48、r (2) discovered a set of risks that demanded action, such as realizing that a major task element was underbid in the baseline. This drop may prompt a recovery action, such as a de-scope, to increase the pool. The post-April 1997 growth in the index indicates that the project is not being aggressive
49、 enough in releasing reserve to mitigate threats, or that it is saving reserve to address an expected need. Absent this level of programmatic resource management discipline, it is not likely that Stardust could have completed development (Phase D) with the original project reserve. The CRR process demonstrated by the Stardust project has been adopted by the ongoing Juno flight project, and the 1996-1999 Stardust approach to managing the work against cost-to-c
