DOE G 413.3-9A, Project Reviews for Capital Asset Projects

AVAILABLE ONLINE AT: INITIATED BY: www.directives.doe.gov Office of Project Management DOE G 413.3-9A 9-14-2018 Project Reviews for Capital Asset Projects [This Guide describes acceptable, but not mandatory, means for complying with requirements. Guides are not requirements documents and are not to be construed as requirements in any audit or appraisal for compliance with associated rule or directives.] U.S. Department of Energy Office of Project Management NOT MEASUREMENT SENSITIVE http://www.directives.doe.gov/ DOE G 413.3-9A i (and ii) 9-14-2018 FOREWORD This Department of Energy (DOE) guide is for use by all Departmental elements. This guide assists project teams by outlining reviews and assessments that the Department completes in executing capital asset projects to meet the requirements of DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets. This guide provides non-mandatory information for fulfilling requirements contained in rules, regulatory standards, and DOE directives. Guides are not requirements documents and are not to be construed as requirements in any audit or appraisal for compliance. Send citations of errors, omissions, ambiguities, and contradictions found in this guide to PMpolicy@hq.doe.gov. mailto:PMpolicy@hq.doe.gov DOE G 413.3-9A iii (and iv) 9-14-2018 TABLE OF CONTENTS FOREWORD i TABLE OF CONTENTS iii 1.0 PURPOSE AND SCOPE OF THIS GUIDE 1 2.0 NEED FOR PROJECT REVIEWS 1 3.0 REVIEW TYPES 2 3.1 Design Reviews 2 3.2 Earned Value Management System (EVMS) Review 4 3.3 Energy Systems Acquisition Advisory Board (ESAAB) Review 6 3.4 External Independent Review (EIR) 8 3.5 Independent Cost Review (ICR) and Independent Cost Estimate (ICE) 9 3.6 Independent Project Review (IPR) 11 3.7 Mission Validation Independent Review (MVIR) 12 3.8 National Environmental Policy Act (NEPA) Review 14 3.9 Operational Readiness Review (ORR)/Readiness Assessment (RA) 15 3.10 Project Definition Rating Index (PDRI) Assessment 17 3.11 Project Management Risk Committee (PMRC) Review 18 3.12 Project Peer Review (PPR) 20 3.13 Quarterly Project Review (QPR) 21 3.14 Technical Independent Project Review (TIPR) 22 3.15 Technology Readiness Assessment (TRA) 24 Appendix A: Project Reviews and when they are typically performed A-1 Appendix B: Acronyms B-1 Appendix C: References C-1 DOE G 413.3-9A 1 9-14-2018 1.0 PURPOSE AND SCOPE OF THIS GUIDE This Guide is a tool for federal project directors (FPDs), integrated project teams and federal program managers in planning and executing project reviews outlined in DOE O 413.3B. It addresses the reviews that are conducted from outside the project team during the lifecycle of a project based on the critical decision (CD) milestones, complexity, and duration of a project. The guide is organized to summarize each review and provide 5Ws – what, why, when, who, and where – of each review. The guide describes the content of the review (what), the purpose of the review (why), when the review is to be conducted during the project lifecycle (when), the office or entity responsible for executing the review (who), and where the review should take place (where). 2.0 NEED FOR PROJECT REVIEWS

Reviews help assure projects stay on track, within schedule and under budget. They are essential for the Chief Executive for Project Management (CE), Project Management Executive (PME), FPD, project manager, project team, and project stakeholders to maintain confidence that project systems, processes, staffing, and technical efforts are integrated, effectively coordinated and provide the needed information. Reviews also help to ensure that the project is progressing at an effective and acceptable rate, particularly regarding established performance baselines (PBs). Reviews provide peer and subject matter expert opinion and feedback on the project readiness to proceed to the next stage in the project decision-making process. Although reviews are required at key junctures by the CD process, FPDs may also recommend additional project reviews at their discretion at any time if they encounter conditions, such as a project performance deviation, or a significant risk event impacting the project, which warrant a review. Reviews and evaluations are performed by several levels of management at various points in the lifecycle of a project, including the project’s Initiation, Definition, and Execution phases. Reviews and evaluations should be planned and structured using a tailored, or in the case of nuclear safety reviews, graded approach. This means that while the applicable order requirements must be addressed, reviews can be customized in their design depending on project-specific attributes including review/decision objectives, scope, project size, cost, technical complexity, findings from previous reviews, and emerging or intervening issues which may not have been previously presented, defined or evaluated. Where an independent review identifies a project practice that omits or alters an applicable Order requirement, the project team in response should document its rationale for doing so. Reviews and evaluations during the Initiation and Definition phases verify that projects support the Department’s mission, goals, and strategic plans and that the projects can be successfully performed within the funding range given applicable conditions such as site and installation conditions, safety and security requirements, and other applicable regulatory and environmental requirements. During these phases, the review process should evaluate technology alternatives and maturation levels prior to granting approval for the project to proceed into design and execution. This is the time to conduct reviews to verify that the project scope is matured and well-defined prior to baselining the project. 2 DOE G 413.3-9A 9-14-2018 Reviews and evaluations during the Execution phase support ongoing validation of the project technical scope, cost and schedule baseline. They ensure that the project is being successfully executed according to plans and within established cost baselines. These reviews help project teams increase their understanding of the project risks and improve management strategies. They provide recommendations for improving the project’s technical scope, schedule, and cost performance and support the project by developing recommendations and supporting data to proceed or not proceed with subsequent lifecycle phases. Finally, they ensure agreed-upon products are being delivered and performance parameters are being achieved. 3.0 REVIEW TYPES

The reviews listed here are driven by requirements in DOE O 413.3B and are listed in alphabetical order. Appendix A provides a table outlining when projects complete each type of review relative to the CD milestones. This guide does not contain information on reviews of the project documentation that may be needed to identify classified and controlled unclassified information. Requirements for these reviews can be found in the applicable directives listed in Appendix C. 3.1 Design Reviews What: Design reviews are conducted to ensure design quality and maturity. They include assessment of whether the design meets operational and functional objectives and whether interface compatibility is adequately addressed. A design review evaluates design and construction documents to confirm resolution of past issues and identify new issues before committing to further work. A design review typically occurs at the end of each stage of design: (1) conceptual design; (2) preliminary design; and (3) final design. Each accepted solution or alternative produced during the conceptual design stage meets an approved need, is deemed technically achievable and affordable, and provides the best value to the Department. Research, development, testing and other efforts may be required to finalize a concept. At a minimum, the conceptual design review should evaluate the following: • Scope and preliminary Key Performance Parameters (KPPs) required to satisfy the program mission requirements; • Identification of requirements and features; • Attainment of specified performance levels; • Assessment of project risks and identification of appropriate risk handling strategies; • Reliable cost and schedule range estimates for the alternatives considered; • Project criteria and design parameters; • Impact on the site sustainability plan; and • Project feasibility. A preliminary design meets all system requirements within acceptable risk levels and cost and schedule constraints while establishing the basis for detailed design. It will validate the selected design options, identified interfaces, and described verification methods. KPPs and the project scope included have sufficient definition to update the cost estimate and cost range. DOE G 413.3-9A 3 9-14-2018 The final design, completed prior to implementation, produces the final drawings, technical specifications, and contract documents required to obtain quotes and bids for procurement and construction. The final design should include clear testing requirements and acceptance criteria for verifying the safety and functionality of all subsystems. The results of a design review are documented in a report forwarded to the FPD and project team for their information and action as the project progresses through the stages of design. A conceptual design report should clearly and concisely describe the how project’s requirements are met, especially those related to capabilities and capacities. It should also outline the applicable design codes and standards. A preliminary design report should demonstrate how the preliminary design meets all system requirements within acceptable risk levels and cost and schedule constraints while establishing the basis for detailed design. The final design report confirms completion of the design, documents any changes made during the review, ensures that the final design addresses all project requirements including the KPPs, and signals readiness for project execution.

Why: Design reviews confirm the level of maturity a project’s planning and designs have attained. Technical Independent Project Reviews (TIPRs) and External Independent Reviews (EIRs) consider the results of the design reviews. When: As outlined in DOE O 413.3B, the conceptual design review occurs prior to CD-1. The preliminary design review precedes CD-2 approval. For Hazard Category 1, 2, and 3 nuclear facilities, the final design review precedes CD-2 approval. For non-nuclear facilities and less than Hazard Category 3 nuclear facilities, the final design review precedes CD-3 approval. Who: Project Team - The design manager leads reviewers external to the project to conduct design reviews with support from the entire project design team. Where: Design reviews occur at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. 4 DOE G 413.3-9A 9-14-2018 For additional information refer to: DOE-STD-1189, Integration of Safety into the Design Process, https://www.standards.doe.gov/standards-documents/1100/1189-astd-2016 3.2 Earned Value Management System (EVMS) Review What: An EVMS review determines if a contractor executing a capital asset project has an EIA-748 (current edition) compliant EVMS. An EVMS integrates scope, schedule, and cost parameters of high‐value, complex projects to manage and assess the overall performance of the project while providing a basis for forecasting outcomes. The EIA-748 guidelines outline the characteristics of a credible, reliable, and compliant EVMS. The EVMS review will: • Assess the status of the contractor’s project management control system to include the EVMS relative to the requirements of the contract and DOE O 413.3B. • Determine if control systems generate project performance reports, the Federal IPT analyzes the reports and contractor management, and that management actions taken derive from the analysis. • Determine how the project incorporates formal changes, conducts internal re-planning, and retains flexibility to accommodate changes. Determine how well the project documents, justifies, and explains changes, including those accepted retroactively. Changes may correct errors, adjust accounting, or improve the accuracy of the performance measurement data. • Assess if a project contractor with a certified EVMS has an effective self-governance system in place to maintain compliance with EIA-748 as evidenced by the contractor’s EVMS or project control description and surveillance program. • If the project contractor does not have a certified EVMS, assess the likelihood of the EVMS receiving certification by CD-2 but no later than CD-3. o Determine if efforts to prepare for an EVMS certification review, including the review’s scheduling, will likely result in a timely certification. o Where an EVMS certification review has occurred, determine the likelihood that the project will resolve open issues sufficiently before establishment of the PB. The Office of Project Management (PM) engages in four types of EVMS compliance assessments with each described below: Certification Review: A formal review to determine that a contractor’s EVMS, on all applicable projects, fully complies with EIA-748 or as required by the contract, in accordance with either FAR Subpart 52.234-4, EVMS, or another applicable EVMS clause. https://www.standards.doe.gov/standards-documents/1100/1189-astd-2016

DOE G 413.3-9A 5 9-14-2018 Implementation Review: A special type of surveillance performed in lieu of a certification review that extends the certification of a contractor's previously certified system to another facility, from one project to another project after a period of system non-use, from one certified entity to another, or following significant changes to a certified system. Review for Cause: A review of specific elements of the contractor’s EVMS that have displayed a lack of discipline in application or may no longer meet the requirements of the EVMS guidelines. The review determines if the contractor’s EVMS will retain its certification. Surveillance Review: The process of reviewing a contractor's EVMS, on all applicable projects, to assess its continuing compliance with EIA-748, or as required by the contract, in accordance with either FAR Subpart 52.234-4, EVMS, or another applicable EVMS clause. Surveillance begins following implementation of a compliant system. The results of an EVMS review report include the finalized Corrective Action Requests (CARs), Discrepancy Reports (DRs), and Continuous Improvement Opportunities (CIOs), and documents the determination of guideline level compliance. The EVMS review director issues the CARs/DRs to the contractor for the purposes of a factual accuracy review. The contractor may provide comments relating to the accuracy of the facts and exhibits stated in the CAR/DR that led to the determination; however, the intent is not to debate the overall conclusion of non-compliance. After the completion of the factual accuracy review, the final report is issued from PM-1. The final report identifies the contractor’s next steps and requested timeframe for submittal of a corrective action plan. In the case of a Review for Cause, the decision may be to de-certify the contractor’s EVMS without proceeding through the Corrective Action Plan (CAP). After the report with the CARs, DRs, and CIOs has been released, the EVMS review team chief coordinates with the contractor to ensure understanding of the CARs/DRs and the requirements of the CAP. The preparation of the CAP and progression is an iterative process, led by the contractor but monitored by the EVMS review team. The steps for corrective actions are: 1) establish contractor submittal date, 2) receive and review CAP, 3) provide comments, 4) monitor progress. The EVMS team plans, schedules, and approves all verification follow up actions and closure of CARs/DRs. This may be done on site or remotely depending on the nature of the CAR/DR. The contractor provides the evidence package for CAR/DR closeout. The CAR/DR will identify the artifacts or other evidential products needed along with the verification methods that will determine the effectiveness of the corrective actions taken. The EVMS team reviews the evidence packages and discusses any questions with the contractor. CARs/DRs are closed upon verification by the review team that the root cause(s) have been properly identified and corrected. Why: DOE contracts that address capital asset acquisitions with a TPC greater than $50M typically invoke EVMS FAR clauses. A compliant EVMS enables the project team to objectively 6 DOE G 413.3-9A 9-14-2018 monitor scope, schedule, and cost during project execution. In addition, an EVMS allows for forecasting costs based on earned value and actual costs.

When: EVMS compliance assessment is conducted on the contractor’s system at various times, based on contractual requirements, the lifecycle of the project, and implementation concerns. The type of review conducted depends on the situation causing the review. Who: Program Office and PM (if TPC is greater than $100M) - For contracts where there are applicable projects having a TPC of $100M or greater, PM conducts an EVMS review to certify the contractor’s EVMS complies with EIA-748, or as required by the contract. Contractors supporting applicable projects with a TPC between $50M and $100M maintain an EVMS compliant with EIA-748. Although DOE O 413.3B does not require an EVMS review for these projects, PM conducts EVMS reviews on projects with a TPC between $50M and $100M at the request of the Project Management Support Office (PMSO). Where: EVMS reviews are normally conducted at the project site with the contractor providing the EVM system description and data to support the review. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information, refer to: PM’s Earned Value Management System Webpage https://community.max.gov/display/DOEExternal/PM+EVM+Home DOE G 413.3-10A, Earned Value Management System (EVMS) Guide https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-10a-admchg1 3.3 Energy Systems Acquisition Advisory Board (ESAAB) Review What: The ESAAB advises the Secretary, CE, and PMEs on project management policy and issues while advising the CE regarding CD milestones for Major System Projects (MSPs). The Project Management Risk Committee (PMRC) supports the ESAAB. https://community.max.gov/display/DOEExternal/PM+EVM+Home https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-10a-admchg1 https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-10a-admchg1 DOE G 413.3-9A 7 9-14-2018 The ESAAB reviews all capital asset projects with a TPC of $100M or more. Based on analysis provided by the program and other project management organizations, the ESAAB evaluates project scope, schedule, and cost estimates, management oversight processes, technical readiness, and other concerns that may affect a project’s successful delivery. The ESAAB provides recommendations to the CE and PME regarding CDs, Baseline Change Proposals (BCPs), and other project-related matters. The ESAAB may also identify uncertainties and risks affecting successful project execution and advise how best to mitigate the uncertainties and risks. In accordance with requirements outlined in DOE O 413.3B, the ESAAB reviews the following: • All capital asset projects with a TPC of $100M or greater, focusing on projects at risk of not meeting their PB; • CD proposals for MSPs in support of the PME; • BCPs in support of the PME; and • Other topics selected by the Secretary or CE. The results of an ESAAB appear in meeting minutes and memoranda that outline its findings and recommendations to the CE, PME, and other officials. The ESAAB approves its recommendations by a majority vote which ESAAB meeting minutes record. Why: The ESAAB supports the Department of Energy’s strategic objective of achieving and maintaining excellence in project management with its role and responsibilities detailed in DOE O 413.3B.

When: ESAAB reviews are event-driven as described above. The ESAAB meets at least quarterly or when convened by the Chair. Projects requiring an ESAAB review will coordinate scheduling the review through their program office who will make arrangements with the ESAAB executive secretariat. Who: ESAAB - The FPD or Program Office will prepare the necessary briefings and presentations for the ESAAB. The executive secretariat will work with representatives of the CE and PME to schedule the meeting or presentation. 8 DOE G 413.3-9A 9-14-2018 Where: ESAAB meetings occur at DOE headquarters. The FPD and the project team may participate by teleconference. For additional information, refer to: PM ESAAB Webpage https://community.max.gov/display/DOEExternal/PM+ESAAB+Home PM SOP Energy System Acquisition Advisory Board (ESAAB) Standard Operating Procedures (SOP) https://community.max.gov/display/DOEExternal/PM+413.3+SOPs 3.4 External Independent Review (EIR) What: In fiscal year 2000, Congress directed DOE to formalize and implement a process to conduct EIRs. PM outlined this requirement in DOE O 413.3B, for all projects with a TPC of $100M and greater. EIRs review scope, KPPs, schedule, and cost and are required prior to establishing a performance baseline for projects with a TPC of $100M or greater. They validate that: • Schedules and costs are firmly supported with sound underlying planning and technical assumptions; • Designs are sufficiently mature to establish a PB with a high level of confidence (which includes technology readiness, nuclear safety, security and quality assurance); • The composition, skill set, and effectiveness of the integrated project teams (IPT) appropriately match the needs of the project; and • The Acquisition Strategy will position the project for a successful outcome. PM or the PMSO coordinates all review activities through a collaborative process. During the planning phase, project background information is assembled for the review committee or team. A PM analyst, a DOE federal employee, leads an EIR team independent of the project comprised of federal and contractor Subject Matter Experts (SMEs). The results of an EIR appear in a report that PM-1 endorses and forwards to the PME for review and action. The EIR report may include major findings, findings, and observations. PM tracks all major findings and findings for resolution. The project team and FPD report back to PM on how they have addressed major findings and findings. EIRs that occur prior to CD-2 will culminate in PM-1 validating the PB and recommending its approval to the PME. PM-1 will not validate a PB until the project team has satisfactorily addressed all major findings. Why: PM performs EIRs to provide PMEs, senior leaders within DOE, and Congress an unbiased assessment of a project’s potential for meeting proposed scope, KPP, schedule and cost commitments. For MSPs, a second EIR that precedes CD-3 assesses the readiness for construction or execution while reconfirming the completeness and accuracy of the PB. In addition to the review elements employed prior to CD-2, this pre-CD-3 EIR for MSPs considers final drawings, specifications, and construction or execution plans. https://community.max.gov/display/DOEExternal/PM+ESAAB+Home https://community.max.gov/display/DOEExternal/PM+413.3+SOPs DOE G 413.3-9A 9 9-14-2018

When: PM performs an EIR prior to CD-2 for all capital asset projects with a TPC greater than or equal to $100 million. PM performs a construction/execution readiness EIR prior to CD-3 for MSPs per DOE O 413.3B. Prior to a BCP, PM will perform another EIR to validate the BCP PB. Who: PM - PM conducts EIRs for projects with a TPC of $100M or greater in coordination with the FPD and program office representative. The teams consist of senior-level technical personnel and subject matter experts from the project as well as outside experts, as appropriate. Where: EIRs normally occur at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. 3.5 Independent Cost Review (ICR) and Independent Cost Estimate (ICE) What: An ICR is a review of the project team’s estimate to examine the reasonableness of the estimate considering quality, ground rules and assumptions, and risks. An ICE is a new estimate performed by an organization independent of the project sponsor using the same detailed technical and procurement information used by the project team. Prior to CD-0, a DOE PM analyst will lead an ICR for all projects with an anticipated TPC greater than or equal to $750M and for any other projects selected by the CE or PME. This ICR evaluates the reasonableness of the project’s initial rough order of magnitude (ROM) cost estimate based on mission need statement in order to roughly indicate future resource requirements. For projects with an anticipated TPC greater than or equal to $100M prior to CD-1, DOE PM will determine whether to perform an ICR or an ICE based on confidence in the quality of the project team’s range estimate. Either approach will give the Project Management Executive (PME), DOE leadership, and Congress confidence that the project cost range, alternative selected, and processes used in support of CD-1 are reasonable. An ICE supports the following milestone approvals: 10 DOE G 413.3-9A 9-14-2018 • CD-1 (if factors such as risk and complexity create a significant cost exposure for DOE; or as required by appropriations acts (e.g., 50 USC §2537(b) requires an ICE before CD-1 and before CD-2 for each new nuclear facility within the nuclear security enterprise that is estimated to cost more than $500M)) • CD-2 (required by appropriations acts) • CD-3 including CD-3A (required by appropriations acts) • Baseline Change Proposal (BCP) The results of an ICR for projects with a TPC at or above $100M, appear in a report that the director of PM (PM-1) endorses and forwards to the PME for review and action. The ICR report will include findings and recommendations made by the review team regarding the schedule, cost estimate, and risks developed by the project team. The ICR report will recommend to the PME schedule and cost ranges for the project germane to that particular critical decision. The results of an ICE for projects with a TPC at or above $100M, appear in a report that PM-1 endorses and forwards to the PME for review and action. The ICE report will include findings and recommendations regarding the project team’s schedule and cost estimate and will provide an independently generated schedule and cost estimate. If the ICE supports CD-2 approval, the EIR team will base its review of the PB on it.

Why: ICRs and ICEs are required at CD-2 and CD-3 by the Consolidated Appropriations Act of 2012 (Public Law 112-74) and subsequent appropriations. Independent estimates provide PMEs and program managers additional information to help them understand the uncertainty associated with cost and schedule estimates. When: DOE O 413.3B requires the development of an ICR prior to CD-0 for projects with a TPC equal to or greater than $750M and an ICR or an ICE for all projects with a TPC equal to or greater than $100M prior to CD-1, CD-2 and CD-3. ICEs are also performed prior to approving a BCP. Who: Program Office and PM - For all projects greater than $100M, the ICR or ICE required by DOE O 413.3B will be developed by PM in coordination with the program offices. DOE G 413.3-9A 11 9-14-2018 Where: ICEs and ICRs normally are performed at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information refer to: GAO-16-89G, GAO Schedule Assessment Guide: Best Practices for Project Schedules, https://www.gao.gov/products/GAO-16-89G GAO-09-3SP, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, https://www.gao.gov/products/GAO-09-3SP 3.6 Independent Project Review (IPR) What: IPRs are conducted by a non-proponent body to determine whether the scope of programs, projects, or activities can be accomplished within the established cost and schedule baselines. IPRs examine the underlying assumptions regarding technology and management; safety and security; and risks to ensure they are valid and credible. IPRs assist with managing risk by identifying existing and potential problems and recommended resolutions with minimum adverse impacts to the project baselines. IPRs may meet a specific objective such as validating a budget, assessing nuclear safety and security or technology readiness, or fulfilling a CD precondition. The scope of an IPR varies with the complexity, cost, and status of the project. One type of IPR is the Technical Independent Project Review (TIPR) which is conducted to ensure early integration of safety into the design process and discussed separately in this guide. The results of the IPR appear in a report that the PMSO completes and forwards to the PME for review and action. The IPR report will include findings and recommendations. The Program Office tracks findings and recommendations that require follow up actions and then determines the successfullness of implemented corrective actions at the next project review. Findings from IPRs conducted in support of a critical decision may require resolution prior to approval of the critical decision. Why: An IPR identifies technical and programmatic risks and uncertainties along with activities that would mitigate the risks. When: DOE O 413.3B requires IPRs prior to CD-1 (in the form of a TIPR) for Hazard Category 1, 2, and 3 nuclear facilities and prior to CD-2 (to validate the PB) for projects with a TPC less than $100M when the program has a PMSO. Although not required by the Order, PMSOs perform IPRs at other points throughout the project lifecycle according to program policy. https://www.gao.gov/products/GAO-16-89G https://www.gao.gov/products/GAO-09-3SP 12 DOE G 413.3-9A 9-14-2018

Who: Program Office – DOE O 413.3B requires PMSOs to conduct IPRs to validate the PB for projects with a TPC less than $100M and when otherwise directed by the Program Secretarial Officer (PSO). Non-proponents of the project, outside of the project and its program, conduct an IPR. The CE, PSO, PME, site or field office manager, program managers, and federal project directors may authorize an IPR. The teams consist of senior-level technical personnel and subject matter experts from the project as well as outside experts, as appropriate. Where: An IPR normally occurs at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information, refer to: NNSA Business Operating Procedure (BOP)-06.04, Project Reviews https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf DOE Office of Science, Independent Review Process Handbook https://science.energy.gov/~/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf 3.7 Mission Validation Independent Review (MVIR) What: The MVIR is unique to MSPs. It is conducted prior to CD-0 to independently examine the mission need developed by the program. It evaluates whether the project has clear objectives strongly linked to mission; identifies major risks; evaluates the acquisition and conceptual planning relative to those risks; and evaluates the funding request. This review is one of the first steps in the identification and initiation of a DOE MSP and is used, in part, to properly designate the appropriate PME. The review includes an examination of the following: • Mission need statement – to verify that the documented deficiency or capability gap described in the MNS could hinder or prevent the Department from achieving a strategic goal. https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf https://science.energy.gov/%7E/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf DOE G 413.3-9A 13 9-14-2018 • Program/mission requirements – to assess whether high-level requirements are sufficiently defined to support identifying potential alternatives. • Cost range – to review the rough order-of-magnitude (ROM) cost to determine whether this range reasonably bounds the cost and schedule of alternatives under consideration. • Schedule range – to assess consistency with strategic requirements for key milestones including the project’s completion and, when closely linked to other projects, its integration with the other projects. The results of the review are documented in a report forwarded to the program secretarial office (PSO) for consideration in approving the CD-0. If there are any shortfalls, edits, or clarifications required, the reviewing team will outline those issues for the project team to reconcile and correct prior to the project receiving CD-0 approval. Why: The purpose of the MVIR is to provide an independent review of the mission need for MSPs. It is an extra step to ensure that the mission need outlined is valid and that achievement of the mission supports DOE’s overarching strategic goals and objectives. Per DOE O 413.3B all MSPs have this review. When: The MVIR should be completed prior to CD-0 approval. Who: Program Office - The MVIR is completed by an independent review team designated by the PSO. All mission need statements are reviewed by the program office and submitted to the PSO for approval.

Where: The MVIR can be held at the project site or at a location determined by the Program office. For additional information regarding Mission Need Statement document refer to: DOE G 413.3-17, Mission Need Statement Guide. 14 DOE G 413.3-9A 9-14-2018 https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-17-admchg1 NNSA Business Operating Procedure (BOP)-06.04, Project Reviews https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf 3.8 National Environmental Policy Act (NEPA) Review What: The NEPA review provides an early assessment, concurrent with and coordinated with the Analysis of Alternatives (AoA) process, of the actions required to comply with the National Environmental Policy Act of 1969 (NEPA). The review leads to a determination of whether an environmental impact statement (EIS) with a record of decision (ROD), a supplement analysis (with or without an amended ROD), an environmental assessment (EA) with a finding of no significant impact (FONSI), a categorical exclusion (CX), or a determination of “no further NEPA action required,” are necessary to comply with the statute. This review is conducted to fully inform the program of the range of NEPA impacts for each alternative considered. Depending on the actions required, NEPA-driven study and analysis can require long timeframes and incur significant costs. For these reasons, it is important to conduct the review early enough to both inform decision making and allow for adequate planning to complete the required NEPA actions. The results of a NEPA review lead to environmental impact statements (EISs), environmental assessments (EAs), or categorical exclusions (CXs). NEPA reviews are documented according to program policy and vary depending on the level of NEPA review required for the proposed action. Why: The National Environmental Policy Act of 1969 (NEPA) requires federal agencies to analyze the potential impacts of major federal actions prior to deciding whether to move forward with that action. DOE P 451.1 delegates responsibility to Heads of Departmental Elements to comply with NEPA. DOE O 413.3B requires project teams to complete a NEPA Strategy prior to CD-1. The strategy documents the project team’s intended approach to comply with the law. When: The NEPA Review is conducted prior to CD-1 and begins with the development of the project team’s NEPA Strategy. The team works with the appropriate NEPA Compliance Officer (NCO) to assess the proposal and realistic alternatives. Depending on the level of NEPA action required, the required actions may not be completed prior to CD-1. According to the legislation, all NEPA actions must be completed prior to a federal decision whether to approve the project, prior to CD-2, and prior to construction start. https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-17-admchg1 https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf DOE G 413.3-9A 15 9-14-2018 Who: Departmental Elements’ NEPA Compliance Officers (NCOs) perform the NEPA review in conjunction with the program office. Where: NEPA reviews are conducted at the headquarters, site, or other location as determined by the Departmental Element. For additional information refer to: DOE NEPA Guidance, Revised Recommendations for the Preparation of Environmental Assessments and Environmental Impact Statements (the Green Book); https://energy.gov/nepa/downloads/recommendations-preparation-environmental-assessments- a nd-environmental-impact

DOE P 451.1, National Environmental Policy Act Compliance Program https://www.directives.doe.gov/directives-documents/400-series/451.1-APolicy DOE Office of NEPA Policy and Compliance website http://energy.gov/nepa NNSA Policy 451.1, National Environmental Policy Act Compliance Program https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?I D=44 3.9 Operational Readiness Review (ORR)/Readiness Assessment (RA) What: Readiness reviews are grouped into two types: ORRs and RAs. An ORR or RA is conducted in accordance with DOE O 425.1D and DOE-STD-3006-2010. DOE O 425.1D provides criteria to determine whether an ORR or an RA is required. An ORR is an in-depth independent evaluation of the readiness of completed facilities, systems, equipment, procedures, personnel, and supporting and interfacing systems and organizations to begin facility operation. In the case of a facility project, the review focuses on the readiness details associated with turning the facility over to the user, including but not limited to startup, https://energy.gov/nepa/downloads/recommendations-preparation-environmental-assessments-and-environmental-impact https://energy.gov/nepa/downloads/recommendations-preparation-environmental-assessments-and-environmental-impact https://energy.gov/nepa/downloads/recommendations-preparation-environmental-assessments-and-environmental-impact http://www.directives.doe.gov/directives-documents/400-series/451.1-APolicy http://www.directives.doe.gov/directives-documents/400-series/451.1-APolicy http://energy.gov/nepa https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?ID=44 https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?ID=44 16 DOE G 413.3-9A 9-14-2018 testing and balancing mechanical systems. Because of the importance of this activity, ORR planning is initiated early in a project’s lifecycle. An RA is an assessment that uses a graded approach to the tenets of ORR requirements as specified in DOE O 425.1D to determine a facility’s readiness to startup or restart when an ORR is not required or when a contractor’s standard procedures for startup are not judged by the contractor or DOE management to provide an adequate verification of readiness. The results of an ORR or RA are documented in a report that clearly outlines whether startup or restart of the nuclear facility, activity, or operation can proceed safely. The final report contains an adequate level of detail to support its conclusions. A copy of the final ORR report should be forwarded to the contactor, FPD and program office. The closure of ORR findings includes: • Development of corrective action plans to correct the findings. Action plans address programmatic deficiencies and causes. • Creation of a finding closure package which includes a brief description of corrective actions taken, evidence of completion, and reasons for concluding that closure has been achieved. • Receipt of a contractor-prepared Readiness to Proceed Memorandum indicating that readiness to start up or restart nuclear operations has been achieved. The memorandum may include a manageable list of open prestart issues provided that the issues have a well-defined schedule for closure. • Satisfactory contractor resolution of all prestart findings from the DOE and contractor ORRs prior to startup or restart of the facility, activity, or operation.

Why: DOE O 425.1D requires that every startup or restart of a nuclear facility, operation, or activity be evaluated to determine the required level of readiness review (i.e. ORR or RA). The Readiness Review process was developed to provide a high degree of confidence that new and restarted DOE nuclear facility operations will be conducted as intended by the design and safety basis. A graded independent review approach is used. Independence was deemed necessary to avoid conflicts of interest that could compromise reviewer ability to objectively determine the status of the proposed operation. ORRs and RAs are not intended to achieve readiness, but to provide independent verification of readiness. When: DOE O 413.3B outlines that an ORR or RA for Hazard Category 1, 2, and 3 nuclear facilities be conducted in accordance with DOE O 425.1D and DOE-STD-3006-2010 prior to CD-4. DOE G 413.3-9A 17 9-14-2018 Who: Program Office - In cases when an ORR or RA is required, the contractor and DOE program office conduct the review. The DOE ORR may not start until the contractor ORR has been completed, the identified findings are resolved or addressed by an approved corrective action plan, and a formal Readiness to Proceed Memorandum has been submitted to DOE. In the case of an RA, there is more flexibility as to whether it is conducted only by a contractor, jointly (when approved), or both by a contractor and DOE (in that sequence). DOE field office management prepares implementing procedures and concurs with contractor implementing procedures in accordance with the Contractor Requirements Document (CRD) in DOE O 425.1D. The Startup Authorization Authority may approve startup or restart after prestart findings are resolved. Where: ORRs and RAs are conducted at the project site. For additional information regarding ORRs and RAs, refer to: DOE O 425.1D Verification of Readiness to Start Up or Restart Nuclear Facilities https://www.directives.doe.gov/directives-documents/400-series/0425.1-BOrder-d-chg1-admchg DOE-STD-3006-2010 Planning and Conducting Readiness Reviews https://www.standards.doe.gov/standards-documents/3000/3006-astd-2010 3.10 Project Definition Rating Index (PDRI) Assessment What: The PDRI assessment for capital asset projects (both nuclear and non-nuclear) is a project management tool designed to increase the likelihood of project success by evaluating the maturity of project documentation. The PDRI assessment tool provides a numerical score to indicate a capital asset project’s planning and development progress. It serves as a guage by which project teams and executives can mark the project’s progress and decide on its readiness to proceed to the next phase. The score has less importance than the learning process that generates the score. The gap list produced by the assessment helps the team identify, assign, track and monitor action items that when closed would improve the project’s score, signifying a greater readiness for the next critical decision milestone. https://www.directives.doe.gov/directives-documents/400-series/0425.1-BOrder-d-chg1-admchg https://www.standards.doe.gov/standards-documents/3000/3006-astd-2010 18 DOE G 413.3-9A 9-14-2018 The results of a PDRI assessment are documented by the project team by completing the applicable PDRI assessment tool for the project.

Why: As outlined in DOE O 413.3B, projects with a TPC of $100M or greater conduct a PDRI analysis. While not mandated for projects with a TPC less than $100M, the assessment is still recommended to measure planning and design maturity. The Construction Industry Institute has versions of the PDRI appropriate for lower cost projects. When: The PDRI assessment benefits projects during front-end planning which encompasses all activities during conceptual through preliminary design. DOE O 413.3B requires a PDRI assessment prior to CD-2 for projects with a TPC of $100M or greater. PM will review the PDRI assessment as part of the EIR. Who: Project Team - The FPD is responsible for overseeing the PDRI assessment process with input from the project team. Where: PDRI assessments could be conducted at the project site or other locations as determined by the project team. For additional information refer to: DOE G 413.3-12, Project Definition Rating Index Guide for Traditional Nuclear and Non-Nuclear Construction Projects Guide. https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-12-admchg1 Construction Industry Institute (CII) Front-end Planning; https://www.construction-institute.org/resources/knowledgebase/best-practices/front-end-plannin g 3.11 Project Management Risk Committee (PMRC) Review What: The PMRC provides expert advice to the Secretary, CE, PME, and ESAAB on technical, schedule, and cost issues experienced by projects with a TPC of $100M or more. Upon request https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-12-admchg1 https://www.construction-institute.org/resources/knowledgebase/best-practices/front-end-planning https://www.construction-institute.org/resources/knowledgebase/best-practices/front-end-planning DOE G 413.3-9A 19 9-14-2018 of the CE, PME, or ESAAB, the PMRC also addresses projects with a TPC less than $100M at risk of not meeting their PBs. In accordance with requirements outlined in DOE O 413.3B, the PMRC reviews the following: • CD proposals and the maturity of the associated project prior to the CE, PME, or ESAAB considering the CD request for approval; • BCPs prior to their presentation to the CE, PME, or ESAAB; • PPR plans, to confirm a focus on pressing issues and an appropriate composition, results, and corrective actions; • Under Secretary-level project assessment outcomes to advise the CE, PME, ESAAB and other program officials on project performance; • The need for independent assessments to advise the CE, PME or ESAAB accordingly; • DOE Order 413.3B requirement exemption requests presented by programs; and, • Other topics selected by the Secretary or CE. The results of a PMRC review appear as meeting minutes, memoranda, and action items on a tracking sheet maintained by the PMRC Executive Secretariat with their status briefed at future PMRC meetings until closure. Why: The PMRC supports DOE’s strategic objective of excellence in project management by supporting the CE, PMEs, and ESAAB in their project management decision making. PMRC reviews also enable routine sharing of best practices and lessons learned. When: PMRC reviews are event-driven as described above. The PMRC meets biweekly or as often as the Chair deems necessary. A project requiring a PMRC review will coordinate scheduling the review through its program office who will make arrangements with the PMRC executive secretariat.

Who: PMRC - The FPD will prepare the necessary briefings and presentations for the PMRC. The executive secretariat will work with the program office to schedule a presentation or discussion. 20 DOE G 413.3-9A 9-14-2018 Where: PMRC meetings occur at DOE headquarters. The FPD and the project team may participate by teleconference. For additional information, refer to: PM PMRC Website https://community.max.gov/display/DOEExternal/PM+PMRC+Home PM’s Project Management Risk Committee (PMRC) Standard Operating Procedures (SOP) for Planning and Conducting PMRC Meetings. https://community.max.gov/display/DOEExternal/PM+413.3+SOPs 3.12 Project Peer Review (PPR) What: PPRs are part assessment and part assistance. They are conducted to determine whether the following are valid and credible: the scope of programs, projects, or activities; the underlying assumptions regarding supporting technology; the cost and schedule estimates; the contingency provisions; and the management approach. They also facilitate cross feed of best practices and lessons learned between projects, contributing to improvement for the projects being reviewed and the projects from which the reviewers come. The objectives of the PPR may include but are not limited to: • Determining how well the project will meet the mission need; • Evaluating technical approach and technology readiness level (TRL); • Evaluating the readiness of the project to proceed to the next CD; • Determining whether the acquisition strategy represents a technically valid, cost-effective, realistic means of accomplishing its stated objectives; • Assessing the potential for meeting schedule and cost baseline targets; • Evaluating and managing project risks, issues, and challenges; • Assessing the status of the project; • Providing constructive recommendations for alternatives or improvements; • Reviewing corrective action items from previous reviews; • Assessing the management organization’s staffing, work assignment and management processes, project management control systems, risk management, quality management, and environment, safety and health (ES&H) policy compliance; and • Analyzing compliance with federal labor and employment statutes and regulations, including the Davis-Bacon Act, the Service Contract Act, and the Fair Labor Standards Act. The results of a PPR appear in a report the program office completes and forwards to the PME for review and action. The PPR report will include findings and recommendations. The project team will track findings that require corrective action through resolution while reporting progress made to the Program Office. The next PPR will review the status of all findings and recommendations. Why: DOE O 413.3B requires PPRs in order to assist the field in successfully completing the project, as well as identifying areas where programs need to apply additional resources. https://community.max.gov/display/DOEExternal/PM+PMRC+Home https://community.max.gov/display/DOEExternal/PM+413.3+SOPs DOE G 413.3-9A 21 9-14-2018 The United States Government Accountability Office (GAO) has recognized PPRs as best business practices that are valuable in assessing the status of projects. When: For projects $100M and above, a PPR occurs between CD-0 and CD-1, annually between CD-1 and CD-2, and at least annually between CD-2 and CD-4. Also for the most complex projects, those experiencing performance challenges, or as directed by the program office, PPRs occur more frequently.

Who: Program Office - The program office or PME requests the review, establishes the review scope and schedules and selects a team leader as well as the makeup of the review team members. The teams consist of senior-level technical personnel and subject matter experts from the project as well as outside experts, as appropriate. Where: PPRs normally occur at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information, refer to: NNSA Business Operating Procedure (BOP)-06.04, Project Reviews https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf DOE Office of Science, Independent Review Process Handbook https://science.energy.gov/~/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf 3.13 Quarterly Project Review (QPR) What: QPRs provide a snapshot of project performance and progress made as compared to the established PB for the benefit of the PME. They should cover project performance and progress to include the project’s scope, cost, schedule, risk, environmental, safety and health (ES&H), and other project issues. If the project has not been baselined, performance should be measured against the preliminary project execution plan including its milestones. The QPR consolidates or https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf https://science.energy.gov/%7E/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf 22 DOE G 413.3-9A 9-14-2018 expands upon information in project monthly status reports and is specifically intended to inform the PME of the details of the project status and issues of concern. The results of a Quarterly Project Review are documented in meeting minutes and action tracking sheets along with a suspense for corrective action prepared by the reviewing office. Any extant action item should be discussed at the next quarterly project review to verify the corrective actions taken or progress made to resolve the issue. Why: Quarterly reviews are necessary for the PME to monitor performance and progress. These reviews provide a forum to communicate status and garner continued support from senior executives within the Department. When: These reviews are performed at the discretion of the PME, at least quarterly throughout the project lifecycle, or more frequently (such as monthly) when the project complexity, cost, or risks warrant such reviews. After approval of CD-0, begin holding QPRs and continue them through approval of CD-4 for projects with a TPC of $50M and greater. Who: Program Office and PMEs conduct QPRs. The FPD is responsible for preparing the presentation. The Deputy Secretary may delegate quarterly reviews for MSPs to the Under Secretaries. Under Secretaries may delegate quarterly reviews to the PSO for projects for which they are the PME. Other PMEs may delegate QPRs as appropriate prior to CD-2, however they may not delegate QPRs for more than two consecutive quarters after CD-2. Where: QPRs can be held at the project site or at a location determined by the PME and program office. 3.14 Technical Independent Project Review (TIPR) What: TIPRs assess technical risk and uncertainty for projects acquiring or improving Hazard Category 1, 2, and 3 nuclear facilities. A TIPR is not required for non-nuclear facilities.

A TIPR will evaluate safety documentation to assess whether assumptions are reasonably conservative and appropriately bounded and whether administrative controls can be considered reliable as the project proceeds. DOE-STD-1189 was developed to provide the Department’s DOE G 413.3-9A 23 9-14-2018 expectations for incorporating safety early into the design process for new or major modifications to DOE Hazard Category 1, 2, and 3 nuclear facilities. A TIPR ensures that safety aspects of the design will be thoroughly investigated. It also evaluates whether or not the IPT includes personnel appropriately qualified to execute nuclear safety responsibilities and whether those team members have the necessary availability. A technical review is also useful when a process technology or unique equipment developed or adapted for the project is untried, or unproven, and no standards exist against which judgments regarding viability can be made. In such a case, an in-depth review by appropriately trained and knowledgeable peers is encouraged. Other areas covered by the TIPR can include: • Alternative Systems • Constructability • Functions and Requirements • Project Definition (Scope) Assessment • Design (at all stages of design status) • Technology Readiness Assessment (TRA) • System Verification (as part of System Engineering) • Physical Configuration • Test Readiness • Safety and Security • Functional Configuration • Operability and Reliability, Availability, and Maintainability The results of a TIPR are documented in a report that the review team completes and forwards to the PME. The report summarizes and identifies any technical risks and uncertainty associated with the technical scope of the project. The findings and recommendations made by the TIPR team typically address technical, functional, operational, and safety issues in the design. The findings and recommendations are documented, reviewed and tracked for resolution as the project progresses towards CD-2. Why: TIPRs ensure the timely resolution of engineering, system integration, technology readiness assessments, design, quality assurance, operations, maintenance, and nuclear safety issues. Technical reviews are necessary when uncertainty exists concerning the outcome of a key project decision. Reducing technical risks increases the probability of a successful 24 DOE G 413.3-9A 9-14-2018 implementation of the technical scope. DOE O 413.3B requires that a TIPR be conducted for all Hazard Category 1, 2, and 3 nuclear facilities prior to CD-2. When: Conduct a TIPR at or near the completion of the preliminary design prior to CD-2 and prior to the start of any subsequent reviews (e.g. EIR). Identifying the review requirements in a charge memorandum shortly after CD-1 gives the appropriate SMEs time to plan and conduct the review. A TIPR completed as soon as possible after preliminary design maximizes time to make needed corrections. Who: The program office and FPD jointly request the review, establish the review scope and schedule, and select a team leader. The team leader appointed by the program office approves the TIPR review plan and the final review report. Qualified technical personnel external to the project execute TIPRs. Where: TIPRs should be conducted at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information refer to:

NNSA Business Operating Procedure (BOP)-06.04, Project Reviews https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf DOE Office of Science, Independent Review Handbook https://science.energy.gov/~/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf DOE-STD-1189, Integration of Safety into the Design Process https://www.standards.doe.gov/standards-documents/1100/1189-astd-2016 3.15 Technology Readiness Assessment (TRA) What: A TRA examines the maturity of technologies and their readiness for insertion into the project design and execution schedule. This assessment applies to MSPs or first of a kind projects. Through its use, projects may reduce technical risk and technology-driven schedule https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf https://science.energy.gov/%7E/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf https://www.standards.doe.gov/standards-documents/1100/1189-astd-2016 DOE G 413.3-9A 25 9-14-2018 delays and cost increases. Through a methodology adapted from a NASA scale developed in the 1980s, a TRA assigns a technology readiness level (TRL) from 1, basic principles observed, through 9, total system used successfully in project operations. The project team tracks for resolution the findings and recommendations from TRAs. The review process should follow the systems engineering approach to assess proper integration of systems with new technologies into the project. The results of a TRA document the process used to conduct the TRA and provide a comprehensive explanation of the assessed TRL for each Critical Technology Element (CTE). At a minimum, a TRA should provide: • A comprehensive review, using an established program or project work breakdown structure or flow diagram as an outline, of the entire platform or system while identifying the CTEs; • An objective scoring of the technology maturity for each CTE by subject matter experts; • Results that assist the project team in preparing maturation plans for achieving an acceptable maturity for CTEs prior to critical decisions; • A final report documenting the findings of the assessment review team; and, • Lessons learned documented within the TRA report or separately. Why: DOE O 413.3B requires that CTEs associated with MSPs or first-of-a-kind engineering endeavors attain TRL-4 prior to CD-1 and TRL-7 prior to CD-2. When: Conduct TRAs for MSPs or first of a kind projects during conceptual design through preliminary design, at least 90 days prior to CD milestones. For MSPs where a significant technology element modification occurs subsequent to CD-2, conduct another TRA as appropriate prior to CD-3. Who: Program Office - The program office and FPD jointly request the review, establish the review scope and schedule and select a team leader. The teams consist of senior-level technical personnel and subject matter experts from the project as well as outside experts, as appropriate. 26 DOE G 413.3-9A 9-14-2018 Where: Conduct TRAs at the project site. However, they could be performed at an offsite location, if sufficient information and personnel to support the review process are available. For additional information refer to: DOE G 413.3-4A, Technology Readiness Assessment Guide https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-04-admchg1 GAO-16-410G: GAO Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness of Technology for Use in Acquisition Programs and Projects – Exposure Draft, https://www.gao.gov/products/GAO-16-410G

GAO-16-89G, GAO Schedule Assessment Guide: Best Practices for Project Schedules, https://www.gao.gov/products/GAO-16-89G DOE EM SOP Technology Readiness Assessments/Technology /Maturation Plan https://community.max.gov/download/attachments/1321765038/SOPP%2027.pdf?version=1&m odificationDate=1504806757912&api=v2 NNSA Policy Letter NAP-29, Technology Readiness Assessments https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=311 https://www.directives.doe.gov/directives-documents/400-series/0413.3-EGuide-04-admchg1 https://www.gao.gov/products/GAO-16-410G https://www.gao.gov/products/GAO-16-89G https://community.max.gov/download/attachments/1321765038/SOPP%2027.pdf?version=1&modificationDate=1504806757912&api=v2 https://community.max.gov/download/attachments/1321765038/SOPP%2027.pdf?version=1&modificationDate=1504806757912&api=v2 https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=311 DOE G 413.3-9A Appendix A 9-14-2018 A-1 (and A-2) Appendix A: Project Reviews and when they are typically performed Reviews DOE O 413.3B Thresholds Pre- CD-0 Pre- CD-1 Pre- CD-2 and BCP Pre- CD-3 Pre- CD-4 Design Reviews $50M or greater x x x Earned Value Management System (EVMS) Review $100M or greater or at request of PMSO x Energy Systems Acquisition Advisory Board (ESAAB) Review $100M or greater or as directed by chair x x x x External Independent Review (EIR) $100M or greater x Independent Cost Review (ICR) or Independent Cost Estimate (ICE) $100M or greater x x x x Independent Project Review (IPR) less than $100M or as directed by PSO x x x Mission Validation Independent Review (MVIR) $750M or greater x National Environmental Policy Act (NEPA) Review $50M or greater* x Operational Readiness Review (ORR) or Readiness Assessment (RA) initial startup or restart of HAZCAT x Project Definition Rating Index Analysis (PDRI) Assessment $100M or greater x Project Management Risk Committee (PMRC) Review $100M or greater or as requested x x x x Project Peer Review (PPR) $100M or greater x x x x Quarterly Project Review (QPR) $50M or greater x x x x Technical Independent Project Review (TIPR) all high risk, high hazard, or HAZCAT x Technology Readiness Assessment (TRA) $750M or greater or first of a kind x x x * NEPA does not have cost thresholds. DOE O 413.3 applies to projects with a total project cost of $50M or more. DOE G 413.3-9A Appendix B 9-14-2018 B-1 Appendix B: Acronyms AoA Analysis of Alternatives BCP Baseline Change Proposal CAP Corrective Action Plan CAR Corrective Action Request CD Critical Decision CDR Conceptual Design Report CE Chief Executive for Project Management CII Construction Industry Institute CIO Continuous Improvement Opportunities CTE Critical Technology Element CRD Contractor Requirements Document CX Categorical Exclusion DOE U.S. Department of Energy DR Discrepancy Report EA Environmental Assessment EIA Electronic Industries Alliance EIR External Independent Review EIS Environmental Impact Statement EM Environmental Management ESAAB Energy Systems Acquisition Advisory Board ES&H Environment, Safety, and Health EVMS Earned Value Management System FAR Federal Acquisition Regulation FPD Federal Project Director GAO U.S. Government Accountability Office ICE Independent Cost Estimate ICR Independent Cost Review IPR Independent Project Review IPT Integrated Project Team KPP Key Performance Parameter MSP Major System Project MVIR Mission Validation Independent Review NASA National Aeronautics and Space Administration NEPA National Environmental Policy Act NCO NEPA Compliance Officer NNSA National Nuclear Security Administration NQA-1 Nuclear Quality Assurance Standard – 1 (ANSI/ASME standard) ORR Operational Readiness Review PARS Project Assessment and Reporting System PB Performance Baseline PDRI Project Definition Readiness Index PM Office of Project Management

Appendix B DOE G 413.3-9A B-2 9-14-2018 PME Project Management Executive PMSO Project Management Support Office PMRC Project Management Risk Committee PPR Project Peer Review PSO Program Secretarial Office QPR Quarterly Project Review RA Readiness Assessment ROD Record of Decision ROM Rough Order of Magnitude SME Subject Matter Expert SOP Standard Operating Procedure TIPR Technical Independent Project Review TPC Total Project Cost TRA Technology Readiness Assessment TRL Technology Readiness Level DOE G 413.3-9A Appendix C 9-14-2018 C-1 Appendix C: References 10 CFR Part 1017, Identification and Protection of Unclassified Controlled Nuclear Information. Construction Industry Institute (CII) Front-end Planning; https://www.construction-institute.org/resources/knowledgebase/best-practices/front-end-planning. DOE O 413.3B Chg 5, Program and Project Management for the Acquisition of Capital Assets, dated 4-12-18. DOE G 413.3-3A Chg 1, Safeguards and Security for Program and Project Management, dated 10-22-15. DOE G 413.3-4A Chg 1, Technology Readiness Assessment Guide, dated 10-22-15. DOE G 413.3-10A Chg 1, Earned Value Management System (EVMS), dated 10-22-15. DOE G 413.3-12 Chg 1, Project Definition Rating Index Guide for Traditional Nuclear and Non-Nuclear Construction Projects, dated 10-22-15. DOE G 413.3-17 Chg 1, Mission Need Statement Guide, dated 10-22-15. DOE O 425.1D, Chg 1, Verification of Readiness to Startup and Restart of Nuclear Facilities, dated 4-12-13. DOE P 451.1, National Environmental Policy Act Compliance Program, dated 12-21-17. DOE P 470.1B Safeguards and Security Program, dated 2-10-16. DOE PM Earned Value Management System Webpage. https://community.max.gov/display/DOEExternal/PM+EVM+Home. DOE PM ESAAB Webpage. https://community.max.gov/display/DOEExternal/PM+ESAAB+Home. DOE PM PMRC Webpage. https://community.max.gov/display/DOEExternal/PM+PMRC+Home. PM’s Project Management Risk Committee (PMRC) Standard Operating Procedures (SOP) for Planning and Conducting PMRC Meetings. https://community.max.gov/display/DOEExternal/PM+413.3+SOPs. DOE PM SOP Energy System Acquisition Advisory Board (ESAAB) Standard Operating Procedures (SOP). https://community.max.gov/display/DOEExternal/PM+413.3+SOPs. DOE O 471.1B, Identification and Protection of Unclassified Controlled Nuclear Information, dated 3-1-10. https://www.construction-institute.org/resources/knowledgebase/best-practices/front-end-planning https://community.max.gov/display/DOEExternal/PM+EVM+Home https://community.max.gov/display/DOEExternal/PM+ESAAB+Home https://community.max.gov/display/DOEExternal/PM+PMRC+Home https://community.max.gov/display/DOEExternal/PM+413.3+SOPs https://community.max.gov/display/DOEExternal/PM+413.3+SOPs Appendix C DOE G 413.3-9A C-2 9-14-2018 DOE Order 471.3, Chg 1, Identifying and Protecting Official Use Only Information, dated 1-13-11. DOE M 471.3, Chg 1, Manual for Identifying and Protecting Official Use Only Information, dated 1-13-11. DOE O 475.2B, Identifying Classified Information, dated 10-2-14. DOE-STD-1189-2016, Integration of Safety into the Design Process, dated 12-22-16. DOE-STD-3006-2010, Planning and Conduct of Operational Readiness Reviews (ORR), dated 5-6-10. DOE Office of Environmental Management, Technology Readiness Assessment / Technology Maturation Plan Process Guide, dated March 2008. DOE Office of Management, Budget and Evaluation, Reviews, Evaluations, and Lessons Learned, Rev E, dated June 2003.

DOE Office of Science, Independent Review Handbook. https://science.energy.gov/~/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf. GAO-09-3SP, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, March 2, 2009. GAO-16-410G: GAO Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness of Technology for Use in Acquisition Programs and Projects – Exposure Draft, dated August 11, 2016. National Research Council, Progress in Improving Project Management in the Department of Energy, 2001. NNSA Business Operating Procedure (BOP)-06.04, Project Reviews. https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf . NNSA Business Operating Procedure (BOP)-50.003, Establishment of NNSA Independent Project Review Policy, dated June 2007. https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=69. NNSA Policy Letter NAP-29, Technology Readiness Assessments. https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=311. NNSA Policy 451.1, National Environmental Policy Act Compliance Program. https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?I D=44. https://science.energy.gov/%7E/media/opa/pdf/processes-and-procedures/1201_Review_Process.pdf https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/BOP-06.04.pdf https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=69 https://nnsaportal.energy.gov/intranet/na-mb/Active%20Policies/Forms/DispForm.aspx?ID=311 https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?ID=44 https://nnsaportal.energy.gov/collab/na-apm/na-apm-20/Lists/Announcements/DispForm.aspx?ID=44 FOREWORD TABLE OF CONTENTS 1.0 PURPOSE AND SCOPE OF THIS GUIDE 2.0 NEED FOR PROJECT REVIEWS 3.0 REVIEW TYPES 3.1 Design Reviews 3.2 Earned Value Management System (EVMS) Review 3.3 Energy Systems Acquisition Advisory Board (ESAAB) Review 3.4 External Independent Review (EIR) 3.5 Independent Cost Review (ICR) and Independent Cost Estimate (ICE) 3.6 Independent Project Review (IPR) 3.7 Mission Validation Independent Review (MVIR) 3.8 National Environmental Policy Act (NEPA) Review 3.9 Operational Readiness Review (ORR)/Readiness Assessment (RA) 3.10 Project Definition Rating Index (PDRI) Assessment 3.11 Project Management Risk Committee (PMRC) Review 3.12 Project Peer Review (PPR) 3.13 Quarterly Project Review (QPR) 3.14 Technical Independent Project Review (TIPR) 3.15 Technology Readiness Assessment (TRA) Appendix A: Project Reviews and when they are typically performed Appendix B: Acronyms Appendix C: References