DOE G 200.1-1 App D, Appendix D In-Stage Assessment Process Guide

Software Engineering Methodology Appendix D In-Stage Assessment Process Guide Table of Contents Section Page 1.0 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Who Conducts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Timing/Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-1 Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-2 In-Stage ISA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-2 End of Stage ISA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-2 Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-3 Change Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-4 Relationship to Other System Development Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-4 Process Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1-4 2.0 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-1 Customers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-1 Suppliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-1 Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-1 Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-2 Plan Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-3 Schedule Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-3 Receive Deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-3 Conduct Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-3 Review Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-3 Prepare Review Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-4 Process Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-4 Responsibility Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2-7

3.0 Review Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-1 3.1 Reviewer Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-2 3.2 SEM Deliverables Review Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-3 3.2.1 Project Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-7 3.2.2 Structured Walkthrough Meeting Record (documentation) (level 2) . . . D.3-7 3.2.3 Feasibility Statement (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-7 3.2.3a Software Quality Assurance Plan (level 3) . . . . . . . . . . . . . . . . . . . . . . . D.3-8 3.2.4 Requirements Specification (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-8 3.2.5 Configuration Management Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . D.3-8 3.2.6 Project Test Plan (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-9 Table of Contents (continued) Section Page 3.2.7 Continuity of Operations (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-9 3.2.7a Acceptance Test Plan (draft) (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . D.3-10 3.2.8 Logical Model (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-10 3.2.9 Data Dictionary (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-10 3.2.10 Traceability Matrix (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-11 3.2.11 Functional Design Document (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . D.3-11 3.2.12 Program Specifications (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-12 3.2.13 Physical Model (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-12 3.2.14 Integration Test Plan (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-13 3.2.15 Conversion Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-13 3.2.16 System Design Document (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-13 3.2.17 System Test Plan (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-14 3.2.18 Software Baseline (level 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-14 3.2.19 Acquisition Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-14 3.2.20 Section Removed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-14 3.2.21 Transition Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-15 3.2.22 User's Guide/Manual (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-15 3.2.23 Operating Documentation (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-16 3.2.24 Test Reports (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-16 3.2.25 Training Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-17 3.2.26 Preacceptance Checklist (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-17 3.2.26a Installation Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-17 3.2.27 Acceptance Test Plan (final) (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . D.3-18 3.2.28 Operational System (level 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-18 3.2.29 Acceptance Test Report (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-18 3.2.30 Maintenance Plan (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-19

3.3 Project Management Review Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-20 3.3.1 Project Objectives Summary (level 1) . . . . . . . . . . . . . . . . . . . . . . . . . D.3-22 3.3.2 Development Approach (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-22 3.3.3 Project Team (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-22 3.3.4 Roles/Responsibilities (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-23 3.3.5 Problem Escalation (level 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-23 3.3.6 Assumptions/Constraints/Dependencies (level 3) . . . . . . . . . . . . . . . . . D.3-24 3.3.7 Estimates (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-24 3.3.8 Stage/Project Schedule (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-25 3.3.9 Status Reporting (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-25 3.3.10 Resource Planning (level 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-26 3.3.11 Budget - Plan Versus Actual (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . D.3-26 3.3.12 Sign-Offs (prior stage exit) (level 1) . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-27 3.3.13 Configuration Management (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-27 Table of Contents (continued) Section Page 3.3.13.1 SCI Identification (level 2) . . . . . . . . . . . . . . . . . . . . . . . D.3-27 3.3.13.2 Change Initiation (level 3) . . . . . . . . . . . . . . . . . . . . . . . . D.3-28 3.3.13.3 Change Evaluation (level 3) . . . . . . . . . . . . . . . . . . . . . . D.3-28 3.3.13.4 Change Approval (level 3) . . . . . . . . . . . . . . . . . . . . . . . D.3-28 3.3.13.5 Auditing (level 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3-29 3.3.13.6 Change Control Log (level 2) . . . . . . . . . . . . . . . . . . . . . D.3-29 3.3.13.7 Re-baseline Requirements (level 2) . . . . . . . . . . . . . . . . . D.3-29 Change Control Page Revision date: 10/12/95 Author: Andy Bicocchi Section(s): Change Control Page Page number(s): D-iv Summary of change(s): Added this change control page. Revision date: 10/30/95 Author: Andy Bicocchi a. Section(s): Throughout document Page number(s): Throughout document Summary of change(s): Revise the process to align it with the Software Engineering Methodology (SEM), which is replacing the System Development Guide (SDG). b. Section(s): 1.0 Who Conducts Page number(s): D.1-1 Summary of change(s): Added that the Quality Assurance Analyst will conduct ISAs. c. Section(s): Entire document Page number(s): Most Summary of change(s): Revised document to ensure alignment to, and consistency with, the SEM and a Departmentwide focus (versus a Headquarters focus) d. Section(s): 1.0 Overview, End-of-Stage ISA Page number(s): D.1-2 Summary of change(s): Revised wording to reflect that copies of the report may be provided, as appropriate, to persons other than the project manager. e. Section(s): 2.0 ISA Report Page number(s): D.2-6 Summary of change(s): Added Corrective Action Recommendations section to the ISA report. Revision date: Author: DOE G 200.1-1 1.0 Overview 5-21-97 In-Stage Assessment Process The SEM is the standard for all DOE projects.1 Date: June 28, 1994 Appendix D Page Rev Date: October 30, 1995 D.1-1

Section: 1.0 Overview Introduction: In-Stage Assessments (ISAs) are independent reviews of DOE system development and maintenance projects. ISAs are conducted in all stages of the software development and maintenance lifecycle (SDLC) in accordance with the ISA schedule, which is documented in the project plan. This document defines the process for planning and conducting ISAs. Purpose: The purpose of ISAs is to assure, via an independent assessment, that the established system development and project management processes and procedures are being followed effectively, and that exposures and risks to the current project plan are identified and addressed. An ISA is a project review that is conducted by a reviewer independent of the project. The reviewer assesses a project's processes, work products, and deliverables to verify adherence to standards and that sound system development and project management practices are being followed. An ISA is a paper review and does not require meetings among the involved parties. Who Conducts: Within the current framework of deployment of this process, the Quality Assurance Analyst will conduct the ISAs. Applicability: This process is applicable to all DOE development and maintenance efforts that would follow the DOE Software Engineering Methodology (SEM) .1 Timing/ Frequency: An ISA can be conducted anytime during a stage whenever a deliverable is stable enough, or near the end of a stage to prepare for stage exit. DOE G 200.1-1 1.0 Overview 5-21-97 In-Stage Assessment Process Date: June 28, 1994 Appendix D Page Rev Date: October 30, 1995 D.1-2 Diagram: The following diagram shows the timing of ISAs relative to the SEM lifecycle. The break out of the ISAs shown in the planning stage also would apply to the other SDLC stages. ISA | | ISA | | | |Stage exit (this is a separate process) | |? T ? ?T T T T T T T * T T* T T* T T* T T* T T* T T* T T* /)))2)))23)))2)))23)))2)))23)))2)))23)))2)))23)))2)))23)))2)))21 *Planning*Reqrmts *Func Dsg*Sys Dsgn*Programg*Soft Int*Installa* R RDefinit R R R R& TestngR & AcptnR S))))))))))))))))))))))) S D L C )))))))))))))))))))))))))))))Q< In-Stage ISA: One or more in-stage ISAs can be scheduled for a given stage of development. The purpose of this review is to assess one or more deliverables when development (of that deliverable) is far enough along to allow for review, and early enough to allow for revisions prior to stage exit. The results of the review are contained in a report that is submitted directly to the project manager. End of Stage ISA: An end-of-stage ISA must be conducted near the end of each stage of development. The purpose of this review is to assess the readiness of a project to proceed to the next stage by reviewing all the deliverables for the current stage. The results of this review are contained in a report that is submitted to the project manager. Copies of the report may be provided to the next-level manager and the client or system owner, as appropriate. In order to exit the current stage of development, the project manager must develop an acceptable action plan to address any open issues or qualifications. DOE G 200.1-1 1.0 Overview 5-21-97 In-Stage Assessment Process Date: June 28, 1994 Appendix D Page Rev Date: October 30, 1995 D.1-3

Process Ownership: The following table identifies the In-Stage Assessment process sponsor and process owner. The responsibilities of the process sponsor include approving the initial process definition document and changes during process improvement. The responsibilities of the process owner include initial process definition, process implementation, and conducting ongoing process improvement. The process was originally developed with the support of a cross-functional process team. Some of the team members have since moved on to other positions, and have been deleted from the table. New team members are being recruited. The team developed the In-Stage Assessment process under the direction of the DOE Chief Information Officer. After initial implementation, some members of the team have met periodically to review process improvement recommendations. NAME RESPONSIBILITY BUSINESS FUNCTION E-MAIL PHONE Wayne Jones Process Sponsor Technical Monitor wayne.jones@ (301) 903-4655 hq.doe.gov Andy Bicocchi Process Owner Quality Assurance andrew.bicocchi@ (301) 903-0616 hq.doe.gov Lew Bachman Consultant Quality Assur. lew.bachman@ (301) 903-0880 hq.doe.gov Beth Willis Consultant Software Development elizabeth.willis@ (301) 903-0878 hq.doe.gov DOE G 200.1-1 1.0 Overview 5-21-97 In-Stage Assessment Process Date: June 28, 1994 Appendix D Page Rev Date: October 30, 1995 D.1-4 Change Control: The ISA process is a component of the SEM. Changes to this process will be instituted using the same change mechanism that has been implemented to process changes to the SEM. All requests for change should be directed to the Manager of the site Quality Team. Relationship to Other System Development Processes: The ISA process is a primary component of the DOE SEM. Together with other processes it serves to assure a consistent and predictable outcome in the resulting software products. The ISA process is complementary to other processes such as Stage Exits and Structured Walkthroughs. Process Measurements: Process measurements are required in order to understand, in a quantifiable manner, the effectiveness of a process at work. If measurements indicate the process is not working as designed, a causal analysis should be conducted to identify the root cause of the breakdown, and changes should be implemented via the process improvement team. The ISA process is considered to be effective (working as designed) if: C All issues that must be resolved in the current stage are identified. C Unmet project objectives can be attributed to issues documented in an ISA. C All issues without an acceptable action plan become qualifications to exit the current stage of development. C All issues are properly documented. Specific procedures for capturing the data for the above measurements will be defined during process improvement. DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-1 Section: 2.0 Process Scope: The In-Stage Assessment process begins with the scheduling of the review and ends with the delivery of the report produced after the review. It is the responsibility of the project manager to develop and implement solutions for the issues and qualifications documented during the review. The project manager must develop an appropriate action plan for each issue. Customers: The customers of the ISA process are those individuals or organizations that will use the output of the ISA process. The primary customers of the process are:

C Project manager C Project manager's manager C Quality Assurance (QA) Secondary customers of the process are: C System owner C User POC C Client representative Suppliers: The project team produces work products and deliverables which become input to the ISA process. Input: The following are the minimum inputs to the ISA process: C Software development lifecycle deliverable(s) C Project plan developed during the planning stage, which includes the work breakdown structure and timeline in addition to other components C Updated project plan revised during all subsequent stages C Structured walkthrough minutes/conference records DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-2 Diagram: The following diagram depicts the ISA process flow. +))))))))))))))))), Planning * * ISA target dates Stage * PLAN FOR * defined in the project * THE REVIEW * plan for each stage * * .))))))))0))))))))- ? +))))))))))))))))), All * * Set date Stages * SCHEDULE * Secure agreement T * THE REVIEW * * * * * .))))))))0))))))))- * ? * +))))))))))))))))), * * * Stage deliverables * * RECEIVE * Updated project plan * * DELIVERABLES * Walkthrough minutes * * * Other applicable items * .))))))))0))))))))- * ? * +))))))))))))))))), * * * Review deliverables * * CONDUCT * Review project plan * * THE REVIEW * Review other items * * * Formulate assessment * .))))))))0))))))))- * ? * +))))))))))))))))), * * * Assessment of risk * * PREPARE * to achieve plan ? * REVIEW FINDINGS * List of concerns * * Recommendations .))))))))0))))))))- ? To Project Manager DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-3 Plan ISA(s): In the planning stage, the target date for conducting the ISA(s) at each stage of development is documented in the project plan. In-stage ISAs can be conducted at any logical point in the stage. End-of-stage ISAs should be scheduled near the end of a stage (e.g., 2 or 3 weeks ahead of the stage exit milestone). In-stage ISAs should be scheduled for the next stage only, since reviews for all subsequent stages might be difficult to plan in advance. Schedule Review: In each stage, as soon as practical, the actual assessment point should be established and agreed-to, and the activity scheduled so that the project manager is aware and there are no surprises. Receive Deliverables: The reviewer should be provided with a copy of the deliverables to be reviewed, plus the current project plan if it is an end-of-stage ISA. The deliverables will vary according to the project's stage of development. The System Development Review Level table in section 3.2 provides detailed review guidance. Conduct Review: The reviewer should examine each lifecycle deliverable. The depth of the examination

will vary according to the deliverable and the project's stage of development. Guidance to assist the reviewer is provided in Section 3.0, Review Guide. Review Levels: The following are the levels of review that can be performed on the SEM deliverables. For each deliverable, specific guidance is provided in Section 3.0, Review Guide. Level Explanation 1 Verify the existence of the work product or deliverable. Review to assure the work product or deliverable exists and is complete. 2 Verify minimum content exists. Review to ensure the minimum level of information has been provided. Verify the existence of content by checking sections/headings. 3 Verify content is rational. Review to make judgements as to the quality and validity of the deliverable. DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-4 Prepare Review Findings: The reviewer should document the results of the ISA and produce the report described in the Process Output section. Process Output: A report must be produced when the ISA process is executed. Each ISA requires a report even if no issues were identified during the review. The report should be brief with the focus on providing a clear statement of the issues(s); solutions may be suggested, but are the project manager's responsibility. The report should include the following elements: C A written assessment of the current project plan in terms of the following: - Risk to schedule and budget - Risk for next stage - Risk for remainder of project Risk categories: Low - Potential or existing problems must be addressed to avoid an impact to the current project plan. This would also apply if no issues were identified. Medium - Problems exist that have a high probability of impacting the current project plan or other dependencies. High - Serious problems exist (without an acceptable plan to resolve) that have a high probability of impacting user acceptance, the current project plan, or other dependencies. C A list of issues/concerns if any were formed during the review. An issue is logged if there is a problem without a visible plan for resolution. Once a list of issues has been compiled, it should be reviewed with the project manager to see if any new or additional information might mitigate or eliminate any of them. Remaining issues must be addressed with an action plan from the project manager. Issues from an end-of-stage ISA might become "qualifications" to exiting the current stage of development. Refer to the stage exit process documentation for additional information. DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-5 Process Output continued: The following are some examples of issues. - No description of the estimating methodology used. - No definition of a change control mechanism. - Signoff (concurrence) from the prior stage is not visible. - Concern about the appropriateness of the process used to arrive at technical decisions. In this example, the reviewer may recommend an additional in-depth review by technical experts as an action item. If no issues were identified, the report only needs to contain the name of the project, date of the review, reviewer name, and a statement that no issues were identified.

C Additional reviewer comments, as appropriate. These include suggestions and recommendations that would benefit the project. The reviewer is encouraged to provide this feedback based on his/her experience. Reviewer comments are provided for the benefit of the project manager and should not be logged as issues requiring an action plan. In certain cases the reviewer may also recommend a more in-depth review by an individual highly skilled in a certain area, to help validate technical decisions and conclusions. For in-stage ISAs, the written report is distributed to the following: C Project manager For end-of-stage ISAs, the written report is distributed to the following: C Project manager, with copies to: - Project manager's manager - Quality Assurance manager DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-6 ISA REPORT Project Name Stage Date Reviewer Phone No. Issues/Concerns: Resolved No. Corrective Action Recommendations: Assessment of risk to schedule: Low Medium High G G G For the next stage G G G For the remainder of the project DOE G 200.1-1 2.0 Process 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.2-7 Responsibility Matrix: The following matrix defines the responsibilities for the various organizations involved in the ISA process. Pr oj ec t T ea m Q ua lit y A ss ur an ce IS A R ev ie w er ( 1) Prepare deliverables P Schedule ISA P R Conduct ISA S P Compile list of issues R P Prepare assessment P Ensure issue resolution P Monitor process effectiveness P Continuous process improvement S P Legend P = Perform R = Review S = Support (1) As the process is currently implemented, the ISA is conducted by Quality Assurance. However, it is possible for the ISA to be conducted by other parties, e.g. a peer project manager. Section 3.1, Reviewer Selection, provides a description of the skills required to conduct ISAs. DOE G 200.1-1 3.0 Review Guide 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-1 Section: 3.0 Review Guide Introduction: This section contains guidance on selecting an ISA reviewer, the appropriate level of review to be performed for each deliverable (or work product), and what to look for in each. While some deliverables are prepared once during the applicable stage of system development or maintenance, others are subsequently updated as the project progresses. These variances are highlighted in the review level tables. Guidance: The following guidance is provided in this section. 3.1 Reviewer Selection 3.2 Deliverables Review Guidance 3.3 Project Management Review Guidance DOE G 200.1-1 3.1 Reviewer Selection 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-2 Section: 3.1 Reviewer Selection The review must be conducted by a person who does not report to the same organization as the development or maintenance team. This allows for an independent view of problems and issues that might exist and serves as a cross-training tool. The experience and skills required include:

C Hands-on experience planning and managing technically complex software development projects. C Working knowledge of the DOE Software Engineering Methodology (SEM). C Ability to deal with people and communicate well. Individuals who typically have the technical background, experience, and skills required include team managers, area managers, project managers, project leaders, task leaders, quality assurance representatives. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-3 Section: 3.2 SEM Deliverables Review Guidance Description: This section provides guidance on what to look for when reviewing and assessing the various SDLC deliverables and components of the project plan. Work products (deliverables) are produced throughout the SDLC. They serve to document all project related data and form the basis of understanding between all parties involved in developing systems. These deliverables are required at various stages of the SDLC. For a detailed description of the SDLC deliverables refer to the SEM. Guidance: The review guidance for the following deliverables is provided in this section. 3.2.1 Project Plan 3.2.2 Structured Walkthrough Meeting Record 3.2.3 Feasibility Statement 3.2.3a Software Quality Assurance Plan 3.2.4 Requirements Specification 3.2.5 Configuration Management Plan 3.2.6 Project Test Plan 3.2.7 Continuity of Operations 3.2.7a Acceptance Test Plan (draft) 3.2.8 Logical Model 3.2.9 Data Dictionary 3.2.10 Cross Reference Matrix 3.2.11 Preliminary Design Document 3.2.12 Program Specifications 3.2.13 Physical Model 3.2.14 Integration Test Plan 3.2.15 Conversion Plan 3.2.16 System Design Document 3.2.17 System Test Plan 3.2.18 Software Baseline 3.2.19 Acquisition Plan 3.2.20 Section Removed 3.2.21 Transition Plan 3.2.22 User's Guide/Manual 3.2.23 Operating Documentation 3.2.24 Test Reports 3.2.25 Training Plan 3.2.26 Preacceptance Checklist 3.2.26a Installation Plan 3.2.27 Acceptance Test Plan (final) DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-4 3.2.28 Operational System 3.2.29 Acceptance Test Report 3.2.30 Maintenance Plan The table on the following page identifies the appropriate level of review to be performed depending on the deliverable and the stage of development. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-5 System Development Deliverables Review Level Table SEM STAGE TABLE LEGEND SEM DELIVERABLE Pl an ni ng R eq ui re m en ts D ef in iti on Fu nc tio na l D es ig n Sy st em D es ig n Pr og ra m m in g So ft w ar e In te gr at io n & T es t In st al la tio n & A cc ep ta nc e Review Levels: - 1 - Verify the existence of the deliverable - 2 - Verify minimum content exists - 3 - Verify content is rational (1) See Project Management Review Level table for details Project Plan (1) 3 3 3 3 3 3 3 Walkthrough Records 2 2 2 2 2 2 2 Quality Assurance Plan 3 Feasibility Statement 2 Requirements Specification 3 Configuration Mgmt. Plan 2 Project Test Plan 3 Acceptance Test Plan (draft) 2 Continuity of Ops. Plan 2 Logical Model 2 Traceability Matrix 2 Data Dictionary 2 Functional Design Document 2

Program Specifications 2 Physical Model 2 System Design Doc. 2 Conversion Plan (as req.) 3 System Test Plan 3 Integration Test Plan 3 Software Baseline 1 Acquisition Plan 2 Transition Plan 2 DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process SEM STAGE TABLE LEGEND SEM DELIVERABLE Pl an ni ng R eq ui re m en ts D ef in iti on Fu nc tio na l D es ig n Sy st em D es ig n Pr og ra m m in g So ft w ar e In te gr at io n & T es t In st al la tio n & A cc ep ta nc e Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-6 User's Guide 2 Integration Test Plan (final) 3 Installation Plan 3 System Test Plan (final) 3 Operating Documentation 2 Test Reports 2 Training Plan 2 Pre-Acceptance Checklist 2 Acceptance Test Plan (final) 3 Operating Documentat. (final) 2 Installation Plan (final) 3 Operational System 1 Acceptance Test Report 1 Maintenance Plan 2 DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-7 Section: 3.2.1 Project Plan Refer to Section 3.3, Project Management Review Guidance, for project plan guidance. Section: 3.2.2 Structured Walkthrough Meeting Record (documentation) (level 2) The structured walkthrough meeting record (i.e. the documented results of the walkthrough), including the Management Summary and any other associated available documentation, should contain a list of any defects identified during that particular walkthrough, and a clear disposition for each defect. The reviewer's objective is to verify the following: C One or more structured walkthroughs were conducted depending on the stage of development in accordance with SEM guidelines. C All defects have been addressed and closed; no action item has been left open- ended. C If a followup structured walkthrough was required, the walkthrough was conducted and all action items were addressed. Refer to Appendix C, Conducting Structured Walkthroughs, for additional information about structured walkthroughs. Section: 3.2.3 Feasibility Statement (level 2) The feasibility statement is reviewed to determine if the following elements have been identified. C Project objectives C System automation alternatives C Potential technical solutions C Benefits and costs C Recommendations The project file should be reviewed to verify that a structured walkthrough was conducted and any action items resulting from the walkthrough were addressed or resolved. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-8 Section: 3.2.3a Software Quality Assurance Plan (level 3) The Software Quality Assurance Plan should clearly define the project's quality assurance policies and procedures. The plan should address the following: C When In-Stage assessments will be conducted and by whom. C Applicability of published standards and procedures. C Monitoring for application of applicable standards and procedures. C Assurance of resolution of discrepancies. C Assessment of project progress. C Assuring the integrity of the software product. Section: 3.2.4 Requirements Specification (level 3) The requirements document is reviewed (use random sampling as appropriate) to assure that it includes the following elements. C Project objectives are consistent with the objectives identified in the management plan.

C Requirements exhibit good attributes including clarity (no ambiguity), statement of a business problem or need (not the solution), and sufficient detail to allow for testing. C All system requirements are addressed including data, functional, operational, security, telecommunications, and implementation requirements. Section: 3.2.5 Configuration Management Plan (level 2) Refer to Section 3.3, Project Management Review Guidance, for configuration management plan guidance. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-9 Section: 3.2.6 Project Test Plan (level 3) This document is reviewed to assure that it includes the following elements. (System Test plan details should be provided in the System Design Stage) C Approach to testing (e.g. "...system testing will be conducted by an independent group...") C Purpose and scope of test efforts to be conducted C List of items that are planned to be tested, and items that will not be tested - Rationale for not testing what is not going to be tested C Who (organization) that will be responsible for conducting the testing C Which levels of testing are planned to be conducted (e.g. unit, integration, system, function, acceptance) C Physical location(s) where testing is planned to be conducted C List of known requirements for conducting testing activities (e.g. hardware, software, skills, space, equipment) C Who is expected to sign-off and approve the tests Section: 3.2.7 Continuity of Operations Plan/Statement (level 2) The continuity of operations plan (for mission essential applications) or statement (for non-mission essential applications) is reviewed to determine if it includes the following elements. C Requirements for continuity of operation, such as data backup, data recovery, and operation startup C Plans for backup and recovery operations C Training requirements and plans so that the required skills will be in place to execute backup and recovery operations DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-10 Section: 3.2.7a Acceptance Test Plan (draft) (level 2) The acceptance test plan should include the following elements. C The timeframe when the test(s) are being planned to be conducted. C A list of organization(s) from which individuals will conduct the tests. At a minimum, this should contain positions and skills required. C The test cases and scenarios that are planned to demonstrate that the requirements have been satisfied in the new system or application. C A list of, or reference to, requirements that the system should satisfy. C Required hardware, software, documentation, special environmental or operational requirements, and any other special considerations (e.g.travel). Section: 3.2.8 Logical Model (level 2) The logical model is reviewed to ensure it contains the following elements: C Description of the final sources and destination of data C Description of the net flow of data across the system boundary C Complete picture of the system processes, data flow, and data stores C Clear connections between the various pieces of the model Section: 3.2.9 Data Dictionary (level 2) The data dictionary (data model) is reviewed to enure that data elements are documented in detail to include attributes, known constraints, input sources, output destinations, and known formats. The dictionary can include business rules processing statistics, and cross-referencing information.

The reviewer should assure him/herself that techniques have been employed (e.g. structured walkthroughs with appropriate persons) to ensure all data elements have been identified up to this point. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-11 Section: 3.2.10 Traceability Matrix (level 2) Each requirement must be traceable to one or more design entities. The matrix is examined to ensure that it allows for the design entities to be traced back to the project requirements, and for verification that all requirements are satisfied by the design. Section: 3.2.11 Functional Design Document (level 2) The Functional Design Document is reviewed to assure that it describes the functions of the system in user terminology. It should be written from the user/system owner's perspective. It should enable the owner/users to understand how the design will satisfy the requirements, providing an opportunity to give feedback before the design is completed. Under separate covers, or as sections of the Preliminary Design Document, the reviewer should ensure that the following design related work products have also been documented: C The design alternatives including evaluation criteria, alternative descriptions, and recommendations. (Note: A reference to the alternatives study is acceptable). C The operating environment including hardware, software, telecommunications, and interfaces. C System design demonstrating the system architecture, system inputs, outputs, interfaces, and end-user interfaces. C Design method used, design entities, and design dependencies. C Security and control measures that will be incorporated into the software system. C The display conventions that will be followed for the design of all end-user interfaces (such as application screens). C The naming and identification conventions for programs, libraries, data bases, tables, and files. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-12 C The implementation approach, reflecting the necessary planning to take the system through to implementation. Section: 3.2.12 Program Specifications (level 2) This deliverable may consist of one or more documents. The specifications are reviewed to assure inclusion of the following elements. C Program design characteristics including program architecture, and software and common program features. C Standards and guidelines that will govern each program coded for the system. C Program specifications (use sampling as appropriate - good rule of thumb is 15 percent). This should include a description of the logic and other particulars for each program in the system. Section: 3.2.13 Physical Model (level 2) The physical model is reviewed to determine if it includes the following attributes and components: C Contents and organization of the physical data files C Processes and data flows among the files C Modules or groups of modules structured into a hierarchy of sub-systems C Manual tasks with physical input or output characteristics. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-13 Section: 3.2.14 Integration Test Plan (level 3) This document (or section) is reviewed to assure that an incremental integration and test plan exists. It should include the following elements.

C Purpose and scope of test efforts to be conducted C List of items to be tested, and items that will not be tested - Rationale for not testing what is not going to be tested C Responsible individual (or organization) for each activity including sign-offs, management, and acceptance C Schedules C Description of "how" the testing will be conducted Section: 3.2.15 Conversion Plan (level 2) The conversion plan addresses the process of transitioning from the existing manual or automated process to the new application or system. This plan is reviewed to assure that it includes the following elements: C General information. This section should describe, in a concise manner, relevant information about the system and task. C Data conversion. This section should describe the strategy and specific activities required to convert data from the old system to the new one. Section: 3.2.16 System Design Document (level 2) The System Design Document is reviewed to ascertain whether it translates requirements into precise specifications of the software components, interfaces, and data which are necessary before coding and testing can begin. The sequence and conditions inherent within modules should be documented. The requirements cross-reference matrix should be updated and should allow for each requirement to be traced to one or more of the detailed design entities to verify that all of the requirements will be satisfied by the detailed design. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-14 Section: 3.2.17 System Test Plan (level 3) This document is reviewed to assure that it includes the following elements. C Scope of the testing effort, and testing schedules. C Objectives and definition of the test cases, and the hardware and software configuration for each test or set of tests. C A requirements verification matrix mapping individual tests to specific requirements and specifying how each system requirement will be validated. C Identification of test tools and test support needs (e.g. hardware / software to simulate production environment and conditions). C Physical location(s) where testing is planned to be conducted. C Who is expected to sign-off and approve the tests. C How will fixes to defects be handled and how will re-testing be conducted. Section: 3.2.18 Software Baseline (level 1) The reviewer should look for evidence that the software baseline (the system or application) has been completed. This may include a demo of the working application, a library listing showing all the coded programs, evidence of positive test results, and confirmation from the individuals who completed the testing activities. Section: 3.2.19 Acquisition Plan (level 2) An acquisition plan should exist, if hardware, software, or services will need to be acquired at some point during the lifecycle in order to implement the system. Section: 3.2.20 Section Removed DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-15 Section: 3.2.21 Transition Plan (level 2) The Transition Plan should describe the detailed plans, procedures, and schedules that will guide the transition process to full operation of the system or application. It should demonstrate that it has been coordinated with operational and maintenance personnel. Refer to the DOE SEM document, Section 7.7, Plan Transition to Operational Status for more details.

Section: 3.2.22 User's Guide / User Manual (level 2) Note: The User's Guide may be produced as part of the Operating Documentation. The user's guide (or user manual) is reviewed to determine if it contains sufficient information and detailed instructions required to access and use the system functions. For very small systems, a quick reference card may be more appropriate and sufficient. For larger systems, a reference card may also be provided in addition to the users's guide. The extent of the user's guide may also depend on the depth of the online help provided; the lower the level of online help the less extensive the user's guide needs to be. A user's guide should include: C Overview of the system history, background, architecture, and current version C Complete coverage of all system functions, in a logical order C Samples of screens and reports, where appropriate to show examples C Instructions for installing, configuring, and accessing the system C Security features including what is accessible to each category of users C Who to contact for additional information or help Additional note to reviewer: Has it been tested by the documentation team, and have their comments been addressed? DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-16 Section: 3.2.23 Operating Documentation (level 2) The system documentation should be complete, to allow for both ease of operation and maintenance. The following documentation should be produced. C Programmer's Reference Manual. This document should include the information required to allow the programming staff to understand and maintain the system programs, data bases, interfaces, and operating environments. C System Administration Manual. This document should provide information necessary to enable the system administrator to understand and access the system functions required to manage the system. C Data Base Administration Manual. This manual should provide the information necessary to enable the data base administrator (DBA) to understand the logical and physical organization, and the record structure of the system. C Operations Manual. This manual should provide the operations group with a description of the system operation environment and the detailed instructions they need to execute the system functions. Section: 3.2.24 Test Reports (level 2) The project files should be reviewed to ensure that system and acceptance testing results have been documented. Any defects found should have been corrected according to the established procedures that should include the process for assigning, handling, and dispositioning defects. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-17 Section: 3.2.25 Training Plan (level 2) This document is reviewed to determine if it includes the following elements in the level of detail needed for training staff to begin logistical training arrangements. A "draft" level document is not intended to provide complete training plan information. C Background - system description, objectives, curriculum overview C Training requirements - environment, audience, category, skill level of users C Objective - expected results of the training in terms of the increased level of user knowledge C Training strategy - type of training, schedule, duration, sites

C Training resources - resources required, responsibilities of involved parties C Environment - facilities, support from other groups, equipment, actions required C Training materials - types of materials required, e.g., system reference manual Section: 3.2.26 Preacceptance Checklist (level 2) Refer to the DOE SEM document, Section 8.3, Initiate Acceptance Process, for details and a sample of the checklist. Section: 3.2.26a Installation Plan (level 2) This plan is reviewed to assure that it includes the following elements. See SEM section 7.2, Installation Plan, for additional details. C General information relevant to the installation process. C Assumptions and dependencies related to the installation activities. C Strategy and schedule for phasing in the new system and disposing of the old one. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-18 Section: 3.2.27 Acceptance Test Plan (final) (level 2) The acceptance test plan should include the following elements. C The schedule for the test(s) that will be conducted. C A list of the individuals who will conduct the tests and their position/organization. C The test cases and scenarios designed to demonstrate that the requirements have been satisfied in the new system or application. C A list of, or reference to, requirements that the system should satisfy. C Required hardware, software, documentation, special environmental or operational requirements, and any other special considerations (e.g.travel). Section: 3.2.28 Operational System (level 1) The reviewer needs to assure that all the system turnover activities have been completed, including the following items. C Execution and completion of the DOE Acceptance process. C If deemed appropriate, interviews with the client to determine satisfaction with requirements met. C Final acceptance turnover meeting. This is often waived or cancelled. C Acceptance checklist(s) completed as per the Acceptance process. Section: 3.2.29 Acceptance Test Report (level 2) The formal Acceptance Test Report should include a summary of the test procedures executed, any problems detected and corrected, and the projected schedule for correcting any problem reports. DOE G 200.1-1 3.2 SEM Deliverables Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-19 Section: 3.2.30 Maintenance Plan (level 2) Note: This may be part of the Transition Plan. A plan for the support and maintenance of the system after turnover to the user(s) or after installation into its intended production environment should be documented. It should include the names of the person(s) and/or organization(s) that will provide support, a service level agreement, which process will be followed (presumably the DOE SEM) for software maintenance, and any schedules, if appropriate. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-20 Section: 3.3 Project Management Review Guidance Description: Project management includes the set of activities related to planning, organizing, directing, staffing, and controlling available resources to achieve distinct goals and objectives established at the beginning of a project. The key to successful project management is a good project plan, which is developed initially in the planning stage. The plan is dynamic, and should be reviewed and revised in subsequent stages to reflect approved changes.

Guidance: This section is a reference guide to assessing the various components of a project plan. The guidance for the following deliverables is provided in this section. 3.3.1 Project objectives summary 3.3.2 Development approach 3.3.3 Project team 3.3.4 Roles/responsibilities 3.3.5 Problem escalation 3.3.6 Assumptions/constraints/dependencies 3.3.7 Estimates 3.3.8 Stage/project schedule 3.3.9 Status reporting 3.3.10 Resource planning 3.3.11 Budget - plan vs. actual 3.3.12 Sign-offs (prior stage exit) 3.3.13 Configuration management The table on the following page identifies the appropriate level of review to be performed depending on the project management activity and the stage of development. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-21 Project Management Review Level Table SEM STAGE TABLE LEGEND PROJECT PLAN Pl an ni ng R eq ui re m en ts D ef in iti on Fu nc tio na l D es ig n Sy st em D es ig n Pr og ra m m in g So ft w ar e In te gr at io n & T es t In st al la tio n & A cc ep ta nc e Project objectives summary 1 Review Levels - 1 - Verify the existence of the deliverable - 2 - Verify minimum content exists - 3 - Verify content is rational Development approach 3 Project team 3 3 3 3 Roles/responsibilities 2 2 2 2 Problem escalation 1 Assumptions/constraints/ 3 3 3 3 3 3 dependencies Estimates 3 3 3 3 3 3 Stage/project schedule 3 3 3 3 3 3 Status reporting 3 1 1 1 1 1 Resource planning 3 1 1 1 1 1 Budget - plan vs. actual 2 2 2 2 2 2 Sign-offs (prior stage exit) 1 1 1 1 1 CONFIGURATION MGMT. SCI Identification 2 Change initiation 3 Change evaluation 3 Change approval 3 Auditing 2 2 2 2 2 Change control log 3 3 3 Re-baseline requirements 2 2 2 DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-22 Section: 3.3.1 Project Objectives Summary (level 1) This section defines the project objectives, in summary form. It is intended to give the reader a high level overview of the project. It should be in summary form, and therefore brief (e.g., one-half page to 2-3 pages). It can be copied from other project documents where it may already exist. Section: 3.3.2 Development Approach (level 3) This section defines the development environment in terms of the SDLC methodology for the project. This section should include the following elements. C The methodology that is followed for this project. The standard for DOE projects is the SEM. C The variations that are taken (if any) from the standard development model. C If deviations are taken, what is in place to mitigate the increased risk. C Any feature that is unique regarding the SDLC or project management aspects of this project. Section: 3.3.3 Project Team (level 3) The project team section of the project plan is reviewed to assure that the members of a project team have been identified, by name, for the next stage of development and, if possible, for the entire project. The composition of the team should be reviewed and revised as the project progresses through the various stages of development. This section should be reviewed in concert with the section on schedules and estimates to determine if the two sections support each other.

DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-23 Section: 3.3.4 Roles/Responsibilities (level 2) The roles and responsibilities of all team members and the users should be documented initially in the Planning stage. The information should be revised in subsequent lifecycle stages if there are changes, and should be based on the skills and experience of each individual. This section should include, at a minimum, the names and organizations of those individuals responsible for providing a concur/nonconcur position (sign-off) at stage exit. Typically, this would include the project manager's manager, the system owner, the quality assurance representative, and the user point of contact. Other persons having a support or participatory role should also be identified if possible. Section: 3.3.5 Problem Escalation (level 1) All problems should be brought to the attention of the project manager first. Problem management is the instrument used to handle project related conflicts, misunderstandings, and problems that cannot be resolved at the project manager level. This section is reviewed to ensure the following items have been identified. C Escalation point (individual(s) to whom the problem will be brought for resolution if the problem is not resolved at the project manager level. C Timeframe for a problem to be brought to resolution after it was escalated. This could be one day, several days, or week(s). DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-24 Section: 3.3.6 Assumptions/Constraints/Dependencies (level 3) This section should contain all project-wide assumptions, known constraints, and dependencies identified for this project. Assumptions and dependencies should be very specific. Generally known or accepted practices usually do not need to be documented. In many cases, assumptions, constraints, and dependencies will affect project estimates and schedules. This section should be updated and reviewed at every stage of the SDLC. The following is an example of a dependency: The development team is dependent on the availability of user personnel to review deliverables within ten (10) business days. The following statement is an example of an assumption. One senior systems analyst with 4 years of Paradox experience will be on board as planned by August 1, 1994. Section: 3.3.7 Estimates (level 3) Estimates are reviewed to assure they include the following elements. In subsequent stages, changes to the original estimates may be documented. C Description of the methodology or combination of methodologies used to arrive at the estimates (planning stage only). C Line items for all associated project costs including labor months and other direct costs. C Breakout of project stage activities. This should be detailed for the next stage of development and in summary form for the rest of the project. C Revisions from the prior stage of development, if there are approved changes that increase or decrease project cost. C Assumptions upon which the estimates are based. C Factors used to arrive at the contingency numbers. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-25 Section: 3.3.8 Stage/Project Schedule (level 3)

The project schedule for the next stage is reviewed for evidence of the following attributes. Each activity and its associated data (e.g., start and end dates) can be part of a work breakdown structure that also shows relationships. C Date for the stage exit. C Description of each deliverable or activity. C Individual (name) responsible for the deliverable or activity. C Projected start and end dates for each activity. C Impact (if any) that approved changes have on the schedule of record (the one approved at the prior stage exit) and the baselining of the new schedule. C Critical path(s) identified and dependencies, if any. For subsequent stages, at a minimum, milestones (stage exit dates) and SEM deliverables should be documented for each stage. Section: 3.3.9 Status Reporting (level 3) Regular, clear, and effective communication is critical to the success of any project. In the planning stage, this section is reviewed to assure the following elements have been identified. C Method for reporting project progress and problems C Frequency of status meetings C Procedure for tracking actions items to closure In subsequent stages, look for evidence that the established reporting practice is being followed. This might include conference records and meeting minutes. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-26 Section: 3.3.10 Resource Planning (level 3) Resource planning is essential to the success of a project. It should be started as early as possible in the planning stage, and then revised for each subsequent stage of development. It should be based on the known project requirements and should demonstrate when necessary resources and skills need to be on board. The following are examples of resources: Hardware: 8 IBM PS/2s with 10 Mg of RAM and VGA monitors Software: PS/2s must have IBM OS/2 2.1 with communications manager Office space: four offices, 12 by 12 feet (each office will accommodate two programmers People: two senior analysts, four programmers, two junior programmers, one quality assurance analyst (specify experience requirements) Dates: target dates for acquiring specified resources In subsequent stages, look for evidence that action has been taken to ensure resources will be available when needed. Section: 3.3.11 Budget - Plan Versus Actual (level 2) Review this section to verify that it includes the following elements. C Project status from the financial perspective. C Planned/approved expenditure level to date. C Actual expenditure level to date. C The difference (delta) between planned and actual expenditure levels, if any, and to what this difference is attributable. Any deltas should be appropriately reflected in the project plan. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-27 Section: 3.3.12 Sign-Offs (prior stage exit) (level 1) The project files should contain the concurrences (sign-offs) received at the prior stage exit. These should be from the individuals identified as having concurrence rights in the roles and responsibilities section of the project plan. Section: 3.3.13 Configuration Management (level 2) The configuration management section of the project plan has several important sections that need to be reviewed including the following items.

3.3.13.1 SCI identification 3.3.13.2 Change initiation 3.3.13.3 Change evaluation 3.3.13.4 Change approval 3.3.13.5 Auditing 3.3.13.6 Change control log 3.3.13.7 Re-baseline requirements Section: 3.3.13.1 SCI Identification (level 2) Review this section to verify that software configuration item (SCI) baselines have been identified in order to establish control points for various SDLC deliverables. The following elements are required. C Functional baseline (requirements specification) C Design baseline (system/subsystem specifications) C Production baseline (first version of code) C Operations baseline (final version of code) The training baseline is optional. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-28 Section: 3.3.13.2 Change Initiation (level 3) Review this section to determine if it contains the following elements. C The name and organization of the individual(s) who are authorized to approve requests for change. Ideally, this should be one person such as the user point- of-contact for the project. C The form(s) on which the change request must be submitted. A specific form is not required. Quality Assurance can provide a sample of an appropriate form to use. Section: 3.3.13.3 Change Evaluation (level 3) Review this section to assure that the following elements are identified. C The name and organization of the individual(s) who is responsible to evaluate the request for change. In many cases, a change control board is responsible to determine whether a change can be contained within the current project plan. C Change evaluation criteria such as the nature of the requested change, time estimates to perform, and an impact analysis on the current schedule and plan. Section: 3.3.13.4 Change Approval (level 3) This section should define the procedure for change approval including the following items. C The name and organization of the individual(s) who is authorized to make decisions as to the disposition of evaluated changes. C The acceptable disposition of the change request; e.g., make the requested change, hold the change for a future release, or reject the change. DOE G 200.1-1 3.3 Project Management Review Guidance 5-21-97 In-Stage Assessment Process Date: 06/28/94 Appendix D Page Rev Date: October 30, 1995 D.3-29 Section: 3.3.13.5 Auditing (level 2) This section deals with assuring that the SCM plan is being followed. Look for evidence that demonstrates that change against the previously identified baselines is being managed. Typically, all changes should be recorded in the project's change control log. Other complementary documents such as two-way memos, E-mail notes and messages, and internal forms might also exist. Section: 3.3.13.6 Change Control Log (level 2) Review the project's change control log to: C Confirm that change activity is being properly recorded and controlled C Assure that all changes to the project baseline have been evaluated, approved, and noted. C Assure that the impact of accepted changes has been adequately reflected in the project plan, particularly in revised estimates. Section: 3.3.13.7 Re-baseline Requirements (level 2) Review this section to ensure that any changes, additions, or deletion of requirements are properly identified and recorded. The new baseline consists of the agreed-to set of requirements from the prior stage, plus or minus any requirements from the current stage of development. The project plan in terms of schedule and budget should reflect the new baseline.