1、Systems Engineering Cost Estimation Curriculum Development Workshop Debrief,Ray Madachy, Naval Postgraduate School Ricardo Valerdi, University of Arizona,Attendees,Mauricio Pena (USC/Boeing) Anca Juliana-Stoica (Uppsala University) Andy Nolan (Rolls-Royce) Ed Colbert (USC) Jorge Aguilar (UPAEP) Mary
2、 Boghosian (Aerospace Corp.) Sue Koolmanojwong (USC) Monvorath Phonpaibul (IBM/Thammasat University),Hunter Deane (ICONIX) Doug Rosenberg (ICONIX) Nupul Kukreja (USC) Warren Reid (WSR Consulting) Ray Madachy (Naval Postgraduate School) Ricardo Valerdi (University of Arizona),2,Workshop Overview,Goal
3、: Engage stakeholders from the cost community interested in education and training to learn and share best practices related to curriculum development and delivery. Our discussion guided by Links between cost estimation curriculum and ongoing efforts to update the systems engineering body of knowled
4、ge (BKCASE) and software engineering reference curriculum (GSwERC) Sharing of cost estimation course syllabi from the Naval Postgraduate School, University of Arizona, and other universities Review of Systems Engineering Cost Estimation Workbook being developed by Madachy and Valerdi Obtained feedba
5、ck to move the cost estimation profession forward.,3,Participant Teaching Experiences Overview,Thammasat University in Thailand cost estimation is part of the Project Management class (undergraduate and graduate students in software engineering) Uppsala University in Sweden software and systems engi
6、neering classes (Masters and PhD students in IT Project Management), deals with both HW and SW Rolls-Royce three courses: 1) how to estimate, 2) how to develop tools, 3) how to be a good recipient of an estimate. Management of uncertainty is a significant part. University of Southern California grad
7、uate level classes for Software Engineering Management & Economics, and Software Engineering contain cost estimation topics,4,Curricula Recommendations,Students should learn that cost estimation does not live in isolation E.g. business concerns, stakeholder management Start with model development, s
8、tatistical analysis and data collection Include feedback loop for students to compare their estimates with actuals, adjust and improve estimation skills,5,Workbook Discussion,Include example systems, learning curve problems, life cycle cost problems Be problem-based vs. model-specific Add “problem d
9、omain” dimension to coverage matrix Include problems with size expansion ratios, DODAF, estimation risk/confidence Add bottom-up and top-down cost estimates Change reuse questions for students to figure out how much of the system is new/modified/adopted vs. handing the values Tradeoff exercises betw
10、een cost and performance by negotiating scope; scope creep exercises to test cost model sensitivity,6,Workbook Discussion (Cont.),Introduce workbook with everyday estimation problems Cover art and science of estimation Introduce case studies to include lessons learned Sync cost estimation course wit
11、h another project course developing software so that students can estimate that development; then compare estimates vs. actuals Have examples include multi-criteria decision making (cost, schedule, risk, performance, value, etc.) Add annotated bibliography and glossary,7,Backups from Workshop,8,GSWE
12、RC,A reference curriculum for a Masters Degree in Software Engineering based on Software Engineering Body of Knowledge SWEBoK Joint industry, academic, and government initiative Modified SWEBOK in several places, including adding Engineering Economics and Risk Management to SWE Management Knowledge
13、Area,9,GSWERC Outcomes and Knowledge Areas,5. Understand the relationship between software engineering and systems engineering and be able to apply systems engineering principles and practices in the engineering of software. I.3. Risk Management7. Be able to reconcile conflicting project objectives,
14、 finding acceptable compromises within limitations of cost, time, knowledge, risk, existing systems, and organizations. I.1. Software Project Planning I.2. Risk Management I.7. Engineering Economics,10,GRCSE,The Body of Knowledge and Curriculum to Advance Systems Engineering (BKCASE) project is defi
15、ning a Systems Engineering Body of Knowledge (SEBoK) and developing an advanced Graduate Reference Curriculum for Systems Engineering (GRCSE) based on the (SEBoK) Joint industry, academic, and government initiative See v.0.5 at bkcase.orgSEBoK Knowledge Areas involving cost estimation Project Planni
16、ng, Measurement, Risk Management, Systems Analysis, others,11,Our Courses,NPS SE3011: Engineering Economics and Cost Estimation Offered multiple times per year in Monterey, CA (and virtually) UA SIE 464/564: Cost Estimation Offered every Spring in Tucson, AZ (and virtually),12,NPS MSSE Outcome Perfo
17、rmance Measures,Demonstrates proficiency in techniques, skills, and tools of systems engineering. Demonstrate proficiency in the assessment of a system. Demonstrate proficiency in economic and life-cycle analysis.,13,NPS Course Learning Outcomes,Understand how cost is used in choosing between altern
18、atives Understand basics of cost behavior and cost allocation Understand how costs change over the life-cycle of a system Choose between alternatives using economic reasoning Understand cost issues related to program management Develop life-cycle cost models Choose between alternatives using net pre
19、sent value analysis Apply cost estimating techniques to develop cost estimates Incorporate uncertainty into cost estimates Quantify and evaluate cost risk Develop and analyze system cost,14,NPS MSSE Degree Program Outcome Coverage,Demonstrate the ability to identify, formulate, and solve operational
20、, technical, and engineering problems in systems engineering and related disciplines using the techniques, skills, and tools of modern practice, including modeling and simulation. These problems may include issues of research, design, development, procurement, operation, maintenance or disposal of s
21、ystems and processes for military applications. Demonstrate proficiency in the systems engineering process, including defining requirements, conducting functional analysis, designing and architecting a system, analyzing it against requirements, allocation of requirements to sub-systems, conducting t
22、rade-off studies, determining the cost of the system, integrating human factors into the system, designing logistical supportability, and planning for its testing and evaluation. Demonstrate proficiency in core skills of systems analysis, to include deterministic and stochastic modeling of systems,
23、optimization, decision analysis, risk analysis, economic models, and lifecycle supportability analysis. This includes familiarity with combat simulations and combat modeling. Demonstrate the ability to work as a team member or leader in a large systems engineering project, and to provide leadership
24、in the systems engineering management process. The graduate must be able to interact with personnel from other services, industry, laboratories and academic institutions. Demonstrate competence in the planning and management of large systems engineering projects. Demonstrate proficiency in written a
25、nd oral presentation of technical material.,15,Workbook Objectives/Targets,Provide a flexible, one-stop resource for companies and Universities who provide education/training in cost modeling Example University courses being taught by Young Kwak, George Washington University Peggy Brouse, George Mas
26、on University Dan Nussbaum, Naval Postgraduate School John Farr, West Point Sue Koolmanojwong, USC Etc.,16,Workbook Contents,Systems Engineering Software Engineering Hardware Engineering Human Systems Integration Systems of Systems Product Lines Cost model extensions (quality, multiple modules, incr
27、ements, etc.) Exercises in all areas Spreadsheets and web-based estimation tools Instructors version,17,Workbook Boundary,IN Equations for parametric models Cost driver rating scales and effort multipliers Tables to decompose estimates by phases and activities Equations for other decision metrics su
28、ch as ROI, PV, etc. Worked out examples and student exercises for all models Calibration equations for models (linear and logarithmic) Instructor version includes all answers plus master tool to compose and solve modified problems OUT Background on how the models were developed Details of the phenom
29、ena behind the cost driver values See COSYSMO book, COCOMO II book, SOSE dissertation, COSYSMO reuse dissertation, etc. for above,18,Example Exercise Mapping,19,Discussion Topics,Application domains for examples Appropriate level of detail, decomposition, hierarchy, lifecycle coverage, etc. containe
30、d in each exercise Decision scenarios (i.e., risk management, ROI, tradeoffs, etc.) Any missing topics?,20,Example Exercise,21,Coverage: Systems engineering reuse; equivalent size; tradeoff analysisThree system configurations are under consideration that employ systems engineering reuse in varying d
31、egrees. The reuse categories are from the COSYSMO reuse model.Option 1: All New DevelopmentOption 2: Adapt Legacy System50% Deleted from the legacy systemThe system will then be comprised of: 50% New 50% ModifiedOption 3: Product Line Approach25% Designed for Reuse (across product line) 25% Modified
32、 50% Managed,Example Exercise,22,Coverage: Software cost; software reuse; tradeoff analysis; military applicationA top Army project is for an application that enables soldiers to find the location of resources for Forward Operating Bases (FOBs). To achieve the Geo-Mapping capability, a geographic so
33、ftware vendor is offering a tailored version of Google Maps at an up-front license price of $500K. However, an alternate proposal is to use a dedicated team with expertise in geographic applications and Google Maps to build a better and cheaper map system. Prepare a COCOMO estimate of the custom dev
34、elopment cost using these inputs: Adapt the Google Map software of 80 KSLOC with the reuse parameters: % Design Modified = 35 % Code Modified = 20 % Integration Required = 60 Assessment and Assimilation = 0 Software Understanding = 10 Unfamiliarity = 0.2,Example Exercise (Cont.),23,Coverage: Softwar
35、e cost; software reuse; tradeoff analysis; military applicationCost drivers are all Nominal except for the following: Programmer Capability and Analyst Capability: 75th percentile Personnel Continuity: stable staff with 3% annual turnover Multi-site Development: entire team is co-located together Us
36、e of Software Tools: strong toolset, moderately integratedThe cost per person-month is $8K. Additionally, the manager believes that they would have no case if they delivered the software system too late. Thus they must have the software completed 6 weeks ahead of the deadline, which would suppress t
37、he schedule to 85% of the nominal schedule. Determine the estimated cost of adapting the Geo-Mapping capability for the application. Is it a better option from a cost standpoint than the vendors tailored product?,Example Exercise,Coverage: Software and hardware lifecycle costs; software reuse; time
38、value of moneyDevelop an integrated software and hardware lifecycle cost over time using the equipment procurement from #1, the mini-drone first unit cost from problem #2, and the following for software labor.Software Portion The newly developed software size is 250,000 SLOC and modified software si
39、ze is 90,000 SLOC. - the schedule compression ratio is 85% - the effect of a software failure is a large equipment loss - average language and tool experience is 3 years - there is expected to be a major change in the operating system once per year - use of a comprehensive toolset fully integrated a
40、cross all activities. For the modified software: - 40 out of 200 design modules were modified - 30 KSLOC out of the total 90 KSLOC were modified - the integration of the modified code is estimated to be 2/3 of the effort compared to if it was all new - only basic module search and documentation was
41、required for its assessment and assimilation - the team did not develop the software being modified and are unfamiliar with it. - to understand the software, it is described as:structure - high cohesion, low coupling application clarity - moderate correlation between program and application self-des
42、criptiveness - moderate level of comments, headers, documentation.,24,Example Exercise (cont.),Coverage: Software and hardware lifecycle costs; software reuse; time value of moneyCreate a monthly budget profile of costs for the full project including labor by software engineering phase, the first dr
43、one development costs and non-labor items for the testing equipment. The drone development will dovetail with the software completion. Assume its monthly costs are constant for the duration of software Inception through Construction (i.e. spread evenly over this time). It finishes when software Tran
44、sition starts. Assume a nominal annual interest rate of 5% and labor rate of $10K/Person-Month. Your final output should provide the costs for every month of the project with separate items for the individual software engineering phases, hardware development, and other procurement costs. Provide the
45、 lifecycle totals for software, hardware, procurement and grand total cost.,25,Example Exercise,26,Coverage: Systems engineering process improvement; ROIIn a systems engineering organization, automated tools for architectural development and verification have been developed along with other process
46、improvements. Their re-calibrated COSYSMO equation shows a reduced size exponent from 1.06 to 1.03:Effort (Person-Months) = .254* System Requirements1.03On average, the organization develops 500 Equivalent System Requirements per year at $10,000/Person-Month. What is their estimated cost savings per
47、 year compared to before the improvements? What is their ROI over time for 5 years if the improvements cost 2 person-years?,Example Exercise,27,Coverage: Systems and software lifecycle costs; equivalent size; time value of money; tradeoff analysis; military applicationThe Navy desires to develop inf
48、ormation dominance capabilities to satisfy national security strategy objectives for Maritime Domain Awareness (MDA). These MDA capabilities will be developed through the use of new and emerging commercial software technologies. By combining these technologies, different Courses of Action (COAs) for
49、 the capability of “Develop Awareness” were developed for the OPNAV Capability Based Assessment. COA 1 assumes that the technologies will be inserted within 13 existing systems. COA 2 creates a new architecture that combines the systems into one overall system, so the software to be developed is dra
50、matically reduced. These ground rules and assumptions for these options are listed below.Ground Rules and Assumptions for the Materiel Solutions1. The wage inflation rate is 3% per year for all years beyond FY12. 2. Annual maintenance costs are 25% of the total software cost and do not start until the software development is complete. 3. Use $10,000 cost per person month for systems and software engineers (constant year dollars). 4. COAs are costed independently. 5. Assume both COAs start at beginning of FY12.,
