DOE G 414.1-4, Safety Software Guide for Use with 10 CFR 830, Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance

DISTRIBUTION: INITIATED BY: http://www.directives.doe.gov Office of Environment, Safety and Health DOE G 414.1-4 Approved 6-17-05 Certified 11-3-10 SAFETY SOFTWARE GUIDE for USE with 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance [This Guide describes suggested nonmandatory approaches for meeting requirements. Guides are not requirements documents and are not construed as requirements in any audit or appraisal for compliance with the parent Policy, Order, Notice, or Manual.] U.S. DEPARTMENT OF ENERGY Washington, D.C. NOT MEASUREMENT SENSITIVE DOE G 414.1-4 i (and ii) 6-17-05 FOREWORD This Department of Energy (DOE) Guide is approved by the Office of Environment, Safety and Health and is available for use by all DOE and National Nuclear Security Administration (NNSA) elements and their contractors. This Guide revises and supersedes earlier guidance identified in Appendix B to include new and updated information. Comments, including recommendations for additions, modifications, or deletions, and other pertinent information, should be sent to the following. Gustave E. Danielson, Jr. Richard H. Lagdon, Jr. (Chip) U.S. DOE U.S DOE Office of Quality Assurance Programs Director, Office of Quality Assurance Programs 1000 Independence Avenue SW 1000 Independence Avenue SW EH-31/270CC EH-31/270CC Washington, DC 20585-0270 Washington, DC 20585-0270 Phone: 301-903-2954 Phone: 301-903-4218 Fax: 301-903-4120 Fax: 301-903-4120 E-mail: bud.danielson@hq.doe.gov E-mail: chip.lagdon@eh.doe.gov Guides are part of the DOE directives system and are used to provide supplemental information regarding DOE/NNSA expectations for fulfilling requirements contained in Policies, Rules, Orders, Manuals, Notices, and Regulatory Standards. Guides are also used to identify Government and non-Government standards and acceptable methods for implementing DOE/NNSA requirements. Guides are not substitutes for requirements nor do they introduce new requirements or replace technical standards used to describe established practices and procedures. mailto:bud.danielson@hq.doe.gov DOE G 414.1-4 iii 6-17-05 CONTENTS BACKGROUND .............................................................................................................................v 1. INTRODUCTION ...............................................................................................................1 1.1 Purpose.....................................................................................................................1 1.2 Scope........................................................................................................................1 1.3 Responsibility for Safety Software ..........................................................................3 1.4 Safety Software Quality Assurance .........................................................................3 1.5 Software Quality Assurance Program......................................................................3 2. SAFETY SOFTWARE TYPES AND GRADING..............................................................5 2.1 Software Types ........................................................................................................5 2.2 Graded Application..................................................................................................6 3. GENERAL INFORMATION..............................................................................................8

3.1 System Quality and Safety Software .......................................................................8 3.2 Risk and Safety Software.........................................................................................9 3.3 Special-Purpose Software Applications.................................................................10 3.3.1 Toolbox and Toolbox-Equivalent Software Applications........................10 3.3.2 Existing Safety Software Applications.....................................................11 3.4 Continuous Improvement, Measurement, and Metrics..........................................11 3.5 Use of National/International Standards................................................................12 4. RECOMMENDED PROCESS..........................................................................................13 5. GUIDANCE.......................................................................................................................13 5.1 Software Safety Design Methods...........................................................................13 5.2 Software Work Activities ......................................................................................17 5.2.1 Software Project Management and Quality Planning.............................17 5.2.2 Software Risk Management .....................................................................19 5.2.3 Software Configuration Management .....................................................21 5.2.4 Procurement and Supplier Management.................................................22 5.2.5 Software Requirements Identification and Management ........................23 5.2.6 Software Design and Implementation .....................................................24 5.2.7 Software Safety ........................................................................................25 5.2.8 Verification and Validation .....................................................................27 iv DOE G 414.1-4 6-17-05 CONTENTS (continued) 5.2.9 Problem Reporting and Corrective Action..............................................30 5.2.10 Training Personnel in the Design, Development, Use, and Evaluation of Safety Software .................................................................30 6. ASSESSMENT AND OVERSIGHT.................................................................................31 6.1 General...................................................................................................................31 6.2 DOE and Contractor Assessment...........................................................................32 6.3 DOE Independent Oversight..................................................................................32 APPENDIX A. ACRONYMS AND DEFINTIONS.............................................................. A-1 APPENDIX B. PROCEDURE FOR ADDING OR REVISING SOFTWARE TO OR DELETING SOFTWARE FROM THE DOE SAFETY SOFTWARE CENTRAL REGISTRY..........................................................B-1 APPENDIX C. USE OF ASME NQA-1-2000 AND SUPPORTING STANDARDS FOR COMPLIANCE WITH DOE 10 CFR 830 SUBPART A AND DOE O 414.1C AND SAFETY SOFTWARE.....................................C-1 APPENDIX D. QUALITY ASSURANCE STANDARDS FOR SAFETY SOFTWARE IN DEPARTMENT OF ENERGY NUCLEAR FACILITIES ................................................................................................. D-1

APPENDIX E. SAFETY SOFTWARE ANALYSIS AND MANAGEMENT PROCESS ......................................................................................................E-1 APPENDIX F. DOE O 414.1C CRITERIA REVIEW AND APPROACH DOCUMENT.................................................................................................F-1 APPENDIX G. REFERENCES ............................................................................................. G-1 DOE G 414.1-4 v (and vi) 6-17-05 BACKGROUND The use of digital computers and programmable electronic logic systems has increased significantly since 1995, and their use is evident in safety applications at nuclear facilities across the Department of Energy (DOE or Department) complex. The commercial industry has increased attention to quality assurance of safety software to ensure that safety systems and structures are properly designed and operate correctly. Recent DOE experience with safety software has led to increased attention to the safety-related decision making process, the quality of the software used to design or develop safety-related controls, and the proficiency of personnel using the safety software. The Department has recognized the need to establish rigorous and effective requirements for the application of quality assurance (QA) programs to safety software. In evaluating Defense Nuclear Facilities Safety Board (DNFSB) recommendation 2002-1 and through assessing the current state of safety software, the Department concluded that an integrated and effective Software Quality Assurance (SQA) infrastructure must be in place throughout the Department’s nuclear facilities. This is being accomplished through the Implementation Plan for Defense Nuclear Facilities Safety Board Recommendation 2002-1, Quality Assurance for Safety Software at Department of Energy Defense Nuclear Facilities. To ensure the quality and integrity of safety software, DOE directives are being developed and revised based on existing SQA industry or Federal agency standards This resulted in the development and issuance of DOE O 414.1C, Quality Assurance, dated 6-17-05, which includes specific SQA requirements, this Guide and the DOE Standard 1172-2003, Safety Software Quality Assurance Functional Area Qualification Standard, dated December 2003. The SQA requirements are to be implemented by DOE and its contractors. Nuclear facility contractors must implement the SQA requirements under their QA program for 10 CFR 830, Subpart A, Quality Assurance Requirements. Thus, the intent of this Guide is to provide instructional guidance for application of DOE O 414.1C safety software requirements. DOE G 414.1-4 1 6-17-05 1. INTRODUCTION 1.1 PURPOSE This Department of Energy (DOE or Department) Guide provides information plus acceptable methods for implementing the safety software quality assurance (SQA) requirements of DOE O 414.1C, Quality Assurance, dated 6-17-05. DOE O 414.1C requirements supplement the quality assurance program (QAP) requirements of Title 10 Code of Federal Regulations (CFR) 830, Subpart A, Quality Assurance, for DOE nuclear facilities and activities. The safety SQA requirements for DOE, including the National Nuclear Security Administration (NNSA), and its contractors are necessary to implement effective quality assurance (QA) processes and achieve safe nuclear facility operations.

DOE promulgated the safety software requirements and this guidance to control or eliminate the hazards and associated postulated accidents posed by nuclear operations, including radiological operations. Safety software failures or unintended output can lead to unexpected system or equipment failures and undue risks to the DOE/NNSA mission, the environment, the public, and the workers. Thus DOE G 414.1-4 has been developed to provide guidance on establishing and implementing effective QA processes tied specifically to nuclear facility safety software applications. DOE also has guidance1 for the overarching QA program, which includes safety software within its scope. This Guide includes software application practices covered by appropriate national and international consensus standards and various processes currently in use at DOE facilities.2 This guidance is also considered to be of sufficient rigor and depth to ensure acceptable reliability of safety software at DOE nuclear facilities. This guidance should be used by organizations to help determine and support the steps necessary to address possible design or functional implementation deficiencies that might exist and to reduce operational hazards-related risks to an acceptable level. Attributes such as the facility life-cycle stage and the hazardous nature of each facility’s operations should be considered when using this Guide. Alternative methods to those described in this Guide may be used provided they result in compliance with the requirements of 10 CFR 830 Subpart A and DOE O 414.1C. Another objective of this guidance is to encourage robust software quality methods to enable the development of high quality safety applications. 1.2 SCOPE This Guide is intended for use by all DOE/NNSA organizations and their contractors to assist in developing site and facility specific safety SQA processes and procedures compliant with 10 CFR 830 Subpart A and DOE O 414.1C. The Department’s objectives for safety software requirements include— 1 DOE G 414.1-2, Quality Assurance Management System Guide for use with 10 CFR 830.120 and DOE O 414.1, dated 6-17-99. 2 See Appendix G for list of consensus standards. 2 DOE G 414.1-4 6-17-05 • grading SQA requirements based on risk, safety, facility life-cycle, complexity, and project quality requirements; • applying SQA requirements to software life-cycle phases; • developing procurement controls for acquisition of computer software and hardware that are provided with supplier-developed software and/or firmware; • documenting and tracking customer requirements; • managing software configuration throughout the life-cycle phases; • performing verification and validation (V&V)3 processes; • performing reviews of software configuration items, including reviewing the safety implications identified in the failure analysis and fault tolerance design; and • training personnel who use and apply software in safety applications. The scope of this Guide includes software applications that meet safety software definitions as stated in DOE O 414.1C. This includes software applications important to safety that may be included or associated with structures, systems, or components (SSCs) for less than hazard category 3 facilities. Safety Software includes safety system software, safety and hazard analysis software and design software, and safety management and administrative control software.

Safety system software is software for a nuclear facility4 that performs a safety function as part of an SSC and is cited in either (1) a DOE approved documented safety analysis or (2) an approved hazard analysis per DOE P 450.4 Safety Management System Policy, dated 10-15-96, and the DEAR clause. Safety and hazard analysis software and design software is software that is used to classify, design, or analyze nuclear facilities. This software is not part of an SSC but helps to ensure the proper accident or hazards analysis of nuclear facilities or an SSC that performs a safety function. Safety management and administrative controls software is software that performs a hazard control function in support of nuclear facility or radiological safety management programs or Technical Safety Requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment as addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause. Additional definitions are included in Appendix A, Acronyms and Definitions. 3 Verification and validation in this Guide includes ASME’s NQA-1 terms design verification and acceptance testing. 4 Per 10 CFR 830, quality assurance requirements apply to all DOE nuclear facilities including radiological facilities (see 10 CFR 830, DOE Std 1120, and the DEAR clause). DOE G 414.1-4 3 6-17-05 Although this Guide has been developed for DOE nuclear facility software, it may also be useful for ensuring the quality of other software important to mission critical functions, environmental protection, health and safety protection, safeguards and security, emergency management, or assets protection. 1.3 RESPONSIBILITY FOR SAFETY SOFTWARE The Assistant Secretary for Environment, Safety and Health has the lead responsibility for promulgating requirements and guidance through the directives system for safety software per DOE O 414.1C. The organizations that use software should determine whether to qualify the software for safety applications. Organizations should coordinate SQA procedures with their respective Chief Information Officers and other appropriate organizations. DOE line organizations are responsible for providing direction and oversight of the contractor implementation of SQA requirements. 1.4 SAFETY SOFTWARE QUALITY ASSURANCE The scope of the Department’s QA Rule, 10 CFR 830 Subpart A, is stated as “This subpart establishes quality assurance requirements for contractors conducting activities, including providing items or services, that affect, or may affect, nuclear safety of DOE nuclear facilities.” The scope of the QA Rule encompasses the contractor’s conduct of activities as they relate to safety software (items or services). Therefore the contractor’s QAP includes safety software within its scope. DOE O 414.1C establishes the safety software QA requirements to be implemented under the Rule. 10 CFR 830 Subpart A and DOE O 414.1C require contractors to perform safety software work in accordance with the applicable criteria. The various sections of this Guide discuss the application of the QA criteria from DOE O 414.1C and 10 CFR 830 Subpart A to the ten SQA work activities. Table 1 provides an illustration of how the SQA work activities satisfy the QA criteria. 1.5 SOFTWARE QUALITY ASSURANCE PROGRAM

It is important that SQA is part of an overall QAP required for nuclear facilities in accordance with 10 CFR 830 Subpart A and DOE O 414.1C. Regardless of the application of the software, an appropriate level of quality infrastructure should be established and a commitment made to maintain this infrastructure for the safety software. An SQA program establishes the appropriate safety software life-cycle practices, including safety design concepts, to ensure that software functions reliably and correctly performs the intended work specified for that safety software. In other words, SQA’s role is to minimize or prevent the failure of safety software and any undesired consequences of that failure. The rigor imposed by SQA is driven by the intended use of the safety software. More importantly, the rigor of SQA should address the risk of use of such software. Effective safety software quality is one method for avoiding, minimizing, or mitigating the risk associated with the use of the software. 4 D O E G 414.1-4 6-17-05 Table 1. An Illustration of Quality Assurance (QA) Criteria (10 CFR 830 Subpart A & DOE O 414.1C) Applicability to Software Quality Assurance (SQA) Work Activities SQA Work Activities 10 CFR 830 QA Criteria and DOE O 414.1C So ftw ar e (s w ) p ro je ct m an ag em en t a nd qu al ity p la nn in g Sw ri sk m gm t Sw c on fig ur at io n m gm t Pr oc ur em en t & su pp lie r m gm t Sw rq m ts id en tif ic at io n & m gm t Sw d es ig n & im pl em en ta tio n Sw sa fe ty V er ifi ca tio n & va lid at io n Pr bl m rp tin g & co rr ec tiv e ac tio n Tr ai ni ng o f … sa fe ty sw Program X X X X X X X X X X Training & Qualific ation X Quality Improvement X X Documents and Records X X X X X X X X X X Work Processes X X X X X X X X X X Design X X X X Procure ment X Inspection & Acceptance Testing X Management Assessment X X X X X X X X X Independent Assessment X X X X X X X X X X Note: This table is only an illustration of QA criteria applicability. Actual application will be described in the organization’s QA program and safety software work process documents. For example, an independent assessment may be performed on any safety software quality element. DOE G 414.1-4 5 6-17-05 The goal of SQA for safety system software is to apply the appropriate quality practices to ensure the software performs its intended function and to mitigate the risk of failure of safety systems to acceptable and manageable levels. SQA practices are defined in national and international consensus standards. SQA cannot address the risks created by the failure of other system components (hardware, data, human process, power system failures) but can address the software “reaction” to effects caused by these types of failures. SQA should not be isolated from system level QA and other system level activities. In many instances, hardware fail-safe methods are implemented to mitigate risk of safety software failure. Additionally other interfaces such as hardware and human interfaces with safety software should implement QA activities. 2. SAFETY SOFTWARE TYPES AND GRADING 2.1 SOFTWARE TYPES Software typically is either custom developed or acquired software. Further characterizing these two basic types aids in the selection of the applicable practices and approaches for the SQA work activities. For the purposes of this Guide, five types of software have been identified as commonly used in DOE applications: (1) custom developed, (2) configurable, (3) acquired, (4) utility calculation, and (5) commercial design and analysis.

Developed and acquired software types as discussed in American Society of Mechanical Engineers (ASME) NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications are compatible with these five software types. Developed software as described in ASME NQA-1-2000 is directly associated with custom developed, configurable, and utility calculation software. Acquired software included in this Guide is easily mapped to that of acquired software in ASME NQA-1-2000. ASME NQA-1-2000 uses acquired and procured software terms interchangeably.5 This Guide includes an additional software type of commercial design and analysis software that is not directly related to either developed or acquired software. Safety software quality requirements can only be specified through work activities described in contractual agreements with the supplier of the facility design and analysis services. Custom developed software is built specifically for a DOE application or to support the same function for a related government organization. It may be developed by DOE or one of its management and operating (M&O) contractors or contracted with a qualified software company through the procurement process. Examples of custom developed software includes material inventory and tracking database applications, accident consequence applications, control system applications, and embedded custom developed software that controls a hardware device. Configurable software is commercially available software or firmware that allows the user to modify the structure and functioning of the software in a limited way to suit user needs. An example is software associated with PLCs. 5 ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, Subpart 2.7 Section 300, American Society of Mechanical Engineers, New York, New York, 2001, p. 105. 6 DOE G 414.1-4 6-17-05 Acquired software is generally supplied through basic procurements, two-party agreements, or other contractual arrangements. Acquired software includes commercial off-the-shelf (COTS) software, such as operating systems, database management systems, compilers, software development tools, and commercial calculational software and spreadsheet tools (e.g., Mathsoft’s MathCad and Microsoft’s Excel). Downloadable software that is available at no cost to the user (referred to as freeware) is also considered acquired software. Firmware is acquired software. Firmware is usually provided by a hardware supplier through the procurement process and cannot be modified after receipt. Utility calculation software typically uses COTS spreadsheet applications as a foundation and user developed algorithms or data structures to create simple software products. The utility calculation software within the scope of this document is used frequently to perform calculations associated with the design of an SSC. Utility software that is used with high frequency may be labeled as custom software and may justify the same safety SQA work activities as custom developed software.6 With utility calculation software, it is important to recognize the difference between QA of the algorithms, macros, and logic that perform the calculations versus QA of the COTS software itself. Utility calculation software includes the associated data sets, configuration information, and test cases for validation and/or calibration.

Commercial design and analysis software is used in conjunction with design and analysis services provided to DOE from a commercial contractor. An example would be where DOE or an M&O contractor contracts for specified design services support. The design service provider uses its independently developed or acquired software without DOE involvement or support. DOE then receives a completed design. Procurement contracts can be enhanced to require that the software used in the design or analysis services meet the requirements in DOE O 414.1C. 2.2 GRADED APPLICATION Proper implementation of DOE O 414.1C will be enhanced by grading safety software based on its application. Safety software grading levels should be described in terms of safety consequence and regulatory compliance. This Guide utilizes the grading levels and the software types (custom developed, configurable, acquired, utility calculations, and commercial design and analysis tools) to recommend how the SQA work activities are applied. The grading levels are defined as follows. Level A: This grading level includes safety software applications that meet one or more of the following criteria. 1. Software failure that could compromise a limiting condition for operation. 2. Software failure that could cause a reduction in the safety margin for a safety SSC that is cited in DOE approved documented safety analysis. 3. Software failure that could cause a reduction in the safety margin for other systems such as toxic or chemical protection systems that are cited in either (a) a DOE approved 6 ASME NQA-1-2000, op.cit., Part 4, Subpart 4.1, Section 101.1, p. 227. DOE G 414.1-4 7 6-17-05 documented safety analysis or (b) an approved hazard analysis per DOE P 450.1 and the DEAR ISMS clause. 4. Software failure that could result in nonconservative safety analysis, design, or misclassification of facilities or SSCs. Level B: This grading level includes safety software applications that do not meet Level A criteria but meet one or more of the following criteria. 1. Safety management databases used to aid in decision making whose failure could impact safety SSC operation. 2. Software failure that could result in incorrect analysis, design, monitoring, alarming, or recording of hazardous exposures to workers or the public. 3. Software failure that could comprise the defense in depth capability for the nuclear facility. Level C: This grading level includes software applications that do not meet Level B criteria but meet one or more of the following criteria. 1. Software failure that could cause a potential violation of regulatory permitting requirements. 2. Software failure that could affect environment, safety, health monitoring or alarming systems. 3. Software failure that could affect the safe operation of an SSC. The grading level criteria should provide for a higher grade level for software in nuclear facilities categorized as Category 1, 2 or 3 and the lower grading level for software in facilities categorized as less than Category 3. Table 2 illustrates the association of grading criteria described above to facility categorization.

Using the grading levels and the safety software types in Table 2, select and implement applicable software quality work activities from the following list to ensure that safety software performs its intended functions. DOE O 414.1C specifies that ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, or other national or international consensus standards that provide an equivalent level of quality assurance requirements as ASME NQA-1-2000 must be used. As specified in DOE O 414.1C, the standards used must be specified by the user and approved by DOE. This Guide provides acceptable implementation strategies and appropriate standards for these work activities. 1. Software project management and quality planning. 2. Software risk management. 3. Software configuration management (SCM). 8 DOE G 414.1-4 6-17-05 4. Procurement and supplier management. 5. Software requirements identification and management. 6. Software design and implementation. 7. Software safety. 8. V&V. 9. Problem reporting and corrective action. 10. Training of personnel in the design, development, use, and evaluation of safety software. Table 2. Grading Criteria and Facility Categorization Illustration Nuclear Facilities 1,2 3 Nuclear Facilities <3 Safety Software A B C A B C Safety System Software X X X Safety & Hazard Analysis Software & Design Software* X X X X X X Safety Management & Admin Controls Software X X X X *Safety and hazard analysis software and design software includes software used to classify facilities. Because this software is used before the facility classification determination, the safety and hazard analysis software and design software type has been identified as being applicable for all grading levels in all categories of facilities. The determination of what constitutes safety software is made by the organization applying the software based upon the requirements in DOE O 414.1C, and 10 CFR 830 Subparts A and B. The application of the software determines whether it is safety related and how it should be graded. Therefore, the organization applying the software is responsible for evaluating and designating the software as safety software and then ensuring that the software development and operations have followed the appropriate QA procedures. 3. GENERAL INFORMATION 3.1 SYSTEM QUALITY AND SAFETY SOFTWARE Maintaining the integrity, safety, and security of all DOE assets and resources is paramount for DOE’s mission. Since software is an integral part of DOE’s resources, the integrity, safety, and security attributes of its software resources are critical to DOE’s mission. All three attributes are interdependent since compromising the security access could obviously present a potential safety hazard. If the integrity of either the data or application itself has been compromised either DOE G 414.1-4 9 6-17-05 accidentally or maliciously, safety could be compromised. Therefore when safety software is being addressed, the integrity and security issues should likewise be addressed. Other system level issues impacting safety software are the availability of trained and knowledgeable personnel to develop, maintain, and use the software; human factor issues such as understandability of the displays or ambient lighting conditions; and potential electromagnetic interference/radio frequency interference, which should be analyzed. Fault tolerance and common cause failure issues, performance requirements, and proper identification and analysis of functional requirements that have safety, security or integrity implications need to be propagated to the safety software.

From the foregoing, it can be seen that there are several interdependencies and tradeoffs that should be addressed when integrating software into safety systems. The necessity for robust software quality engineering processes is obvious when safety software applications are required. However, just ensuring that a “good” software engineering process or that V&V activities exist is not sufficient alone to produce safe and reliable software.7 The life-cycle process should focus upon the safety issues in addition to the basic software quality engineering principles. Both of these concepts are detailed in this Guide. 3.2 RISK AND SAFETY SOFTWARE Software rarely functions as an independent entity. Software is typically a component of a system much in the same way hardware, data, and procedures are system components. Therefore, to understand the risk associated with the use of software, the software function should be considered a part of an overall system function. The consequences of software faults need to be addressed in terms of the contribution of a software failure to an overall system failure. Issues such as security, training of operational personnel, electromagnetic interference, human-factors, or system reliability have the potential to be safety issues. For example, if the security of the system can be compromised, then the safety software can also be compromised. Controlling access to the system is key to maintaining the integrity of the safety software. Likewise, if human factor issues such as ambient lighting conditions and ease of use for understandability are important, the risks need to be addressed either via design or training. For PLCs or network safety software applications, electromagnetic interference could offer potential risks for operation of the safety software system. Once the software’s function within the overall system’s function is known, the appropriate software life-cycle and system life-cycle practices can be identified to minimize the risk of software failure on the system. Rigor can then be applied commensurate with the risk. Managing the risk appropriately is the key to managing a safety software system. Unless risks and trade-offs of either doing or not doing an activity are evaluated, there is not a true understanding of the issues involved regarding the safety software system. Obviously, time and resource constraints should be balanced with the probability of occurrence and the potential consequences versus an occurrence of the worst case scenario. If the safety systems staff zealously and 7 Leveson, Nancy, Safeware: System Safety and Computers, Addison Wesley, 1995. 10 DOE G 414.1-4 6-17-05 religiously inappropriately invokes the strictest rigor for a Level B application, then the application has the potential never to get fielded properly. On the other hand, if the process activities are only minimally or inappropriately performed for a Level A software safety application, then very adverse consequences could potentially occur for which no mitigation strategy exists. Appropriate project management is a risk management strategy and especially so for safety software applications. 3.3 SPECIAL-PURPOSE SOFTWARE APPLICATIONS Several categories of software have a unique purpose in safety-related functions required to support DOE nuclear facility operations. This section contains an overview of the special-purpose software and the additional considerations that should be addressed by SQA programs, processes, and procedures.

3.3.1 Toolbox and Toolbox-Equivalent Software Applications Toolbox codes represent a small number of standard computer models or codes supporting DOE safety analysis. These codes have widespread use and are of appropriate qualification for use within DOE. The toolbox codes are acknowledged as part of DOE’s Safety Software Central Registry. These codes are verified and validated and constitute a “safe harbor” methodology. That is to say, the analysts using these codes do not need to present additional defense as to their qualification provided that the analysts are sufficiently qualified to use the codes and the input parameters are valid. These codes may also include commercial or proprietary design codes where DOE considers additional SQA controls are appropriate for repetitive use in safety applications and there is a benefit to maintain centralized control of the codes. The following six widely applied safety analysis computer codes have been designated as “toolbox” codes. • ALOHA (chemical dispersion analysis) • CFAST (fire analysis) • EPIcode (chemical dispersion analysis) • GENII (radiological dispersion analysis) • MACCS2 (radiological dispersion analysis) • MELCOR (leak path factor analysis) The current designated “toolbox” codes and any software recognized in the future as meeting the “toolbox” equivalency criteria are no different from other custom developed safety software as defined in Section 2.1. Consequently, software of this category should be developed or acquired, maintained, and controlled applying sound software practices as described in Section 5 of this Guide. In the future, new versions of software may be added to the Central Registry while the older versions are removed. Over time, some of the software may be retired and recommended not to be used in DOE safety analysis. Still other software may be added through the formal toolbox-equivalent process, having been recognized as meeting the equivalency criteria. Thus, the Central Registry collection of safety software applications will be expected to evolve as DOE G 414.1-4 11 6-17-05 es re dditional information on the detailed toolbox SQA procedures, criteria, and evaluation plan; the entral Registry software life-cycle phases, usage, and application requirements change. Appendix B address the process for adding new software applications and versions to, and removal of retired softwa from, the Central Registry. A evaluation of the software relative to current SQA criteria (i.e., assessment of the margin of the deficiencies or “gap” analysis); user guidance documentation; description of the toolbox-equivalent process; and code-specific information may be found in the C portion of the DOE SQA Knowledge Portal (http://www.eh.doe.gov/sqa/central_registry.htm). 3.3.2 Existing Safety Software Applications E g proved under a QA program consistent with , xisting safety software should be identified and controlled prior to evaluation using the graded est to 3.4 CONTINUOUS IMPROVEMENT, MEASUREMENT, AND METRICS Lord Kelvin stated “If you can not measure it, you can not improve it.” This truism especially OE O 414.1C criterion 3, Quality Improvement, specifies that processes should be established xistin software that has not been previously ap DOE O 414.1C and has been identified as safety software should be evaluated using the graded approach framework that is described in Section 5. This software is often referred to as legacy software. In many cases this category of software originally met DOE or industry requirements but SQA for existing software was not updated as the SQA standards were revised.

E approach framework in this Guide. The evaluation performed should be adequate to address the correct operation of the safety software in the environment it is being used. This evaluation should include (1) identification of the capabilities and limitations for intended use, (2) any t plans and test cases required demonstrating those capabilities, and (3) instructions for use within the limitations.8 One example of this evaluation is a posteriori review9 as described in American Nuclear Society (ANS) standard, ANS 10.4. Future modifications to existing safety software should meet all safety software work activities in DOE O 414.1C associated with the changes the safety software. 10 applies to safety software systems. Metrics used throughout the life-cycle should bolster the confidence that the software applications will achieve their mission in a safe and reliable manner. If design, testing, or software reliability measures are unknown, then there is no assurance that the safety software has sufficient robustness to minimize the risks. D and implemented to detect and prevent problems. Measurements and the metrics developed from these measures can be indicators for potential future problems, and thus, steps can be initiated to prevent the occurrence. For long term avoidance of problems, continuous improvement methods should be implemented to determine the root causes and eliminate the events that could lead to a 8 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 302, p. 105. 9 American National Standards Institute (ANSI)/American Nuclear Society (ANS) 10.4-1987 (R1998), Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry, ANS, 1998, Section 11, pp. 29–32. 10 Lord Kelvin ( Sir William Thomson, 1824–1907) http://www.eh.doe.gov/sqa/central_registry.htm 12 DOE G 414.1-4 6-17-05 e dards to develop and xperts ME is the nationally accredited body for the development of nuclear part A he 11 the requirements generally apply to safety software l, r rk ce other standards useful in achieving compliance ll the applications. reoccurrence. Metrics further provide qualitative or quantitative indicators of the improvements or lack thereof when a process or work activity has been modified. Metrics are the evidence that an improvement has occurred. Both Institute of Electrical and Electronic Engineers (IEEE) Standards 982.1 and 982.2 provide recommendations for what metrics to use and when in th software life-cycle phase applying the metric is most appropriate. 3.5 USE OF NATIONAL/INTERNATIONAL STANDARDS Title 10 CFR 830 Subpart A and DOE O 414.1C require the use of stan implement a QAP. National/international standards facilitate a common software quality engineering approach to developing or documenting software based upon a consensus of e in the particular topic areas. Many national and international standards bodies have developed software standards to ensure that the recognized needs of their industry and users are satisfactorily met. In the United States, AS facility QA standards. DOE O 414.1C cites ASME NQA-1-2000 or other national or international consensus standards that provide an equivalent level of quality assurance requirements as ASME NQA-1-2000 as the appropriate standard for QAPs applied to nuclear-related activities (e.g., safety software). The ten QA criteria in 10 CFR 830 Sub and DOE O 414.1C are mapped to ASME NQA-1-2000 in Appendix C. DOE O 414.1C also requires that additional standards be used to address specific work activities conducted under t QAP, such as safety software.

In the case of ASME NQA-1-2000, Part I, work activities. For example, Requirements 3, 4, 7, 11, 16, and 17 for Design Control, Procurement Document Control, Control of Purchased Items and Services, Test Contro Corrective Action, and Quality Assurance Records (respectively) can have specific safety software applicability. In addition, ASME NQA-1-2000, Part II, Subpart 2.7, and Part IV, Subpart 4.1, specifically address “quality assurance requirements for computer software fo nuclear facility applications” and “guide on quality assurance requirements for software” (respectively). As stated in the introduction of Part II, Subpart 2.7, this subpart “provides requirements for the acquisition, development, operation, maintenance, and retirement of software.” Table 3 provides a cross reference of ASME NQA-1-2000 with the ten SQA wo activities in DOE O 414.1C. Although ASME NQA-1-2000 provides excellent process guidan for a software quality engineering process for managing a software development, maintenance, or procurement or otherwise acquiring software, the detailed guidance for safety software is not provided within this standard. Appendix D of this Guide includes references to with 10 CFR 830 Subpart A and DOE O 414.1C for safety software work activities. It should be noted that the use of the standards discussed can aid in the development of a robust safety software quality engineering process and a resulting software product that is adequate for a safety software applications. Use of consensus standards can promote a robust safety software quality engineering process and a resulting software product that is adequate for safety software 11 ASME NQA-1-2000, op.cit., Part I. DOE G 414.1-4 6-17-05 13 Recognizing that there are five OE listed in Section 2.1, the safety software analyst needs a defined process to enable a determination of w fety 5.1 SOFTWARE SAFETY DESIGN M Safety should be designed into a system, just as quality should be built into the system. Safe n nt, uses two primary approaches: ( h ces is the first approach to developing high quality software systems. These practices can be a e e more integrated model, Capability Maturity M 12 Leveson, op. cit., p. 398. 4. RECOMMENDED PROCESS safety software applications types within D hat needs to be accomplished for each of the respective software safety applications. In addition, the safety software analyst needs a process to support the integration of software sa into the system safety process to improve system and software design, development and test efforts. Lastly, the process to manage each of the five application types should support the planning and coordination of the software safety tasks based on established priorities. Appendix E of this Guide presents the details of a risk-based graded approach for the analysis and safety software management process for (1) custom developed, (2) configurable, (3) acquired, (4) utility calculations, and (5) commercial design and analysis tools. 5. GUIDANCE ETHODS desig of a system, in which safety software is a subcompone 1) applying good engineering practices based upon industry proven methods and (2) guiding design through the results of hazard analysis. Identifying and assessing the hazards is not enoug to make a system safe. The information from hazard analysis needs to be factored in the design.12 Applying industry accepted software engineering and software quality engineering practi generally pplied to safety software to improve the quality and add a level of assurance that the software

performs its safety functions as intended. DOE O 414.1C requires SQA work activities, referred to as work activities, to be performed for safety software. Many national and international consensus standards, such as ASME NQA-1-2000, ANS 10.4, and the IEEE software engineering series provide detailed guidance for performing the work activities. Section 3.5 of this Guide describes some of these standards. Software process capability models such as the Software Engineering Institute’s legacy Softwar Capability Maturity Model (SW-CMM) and th odel Integration (CMMI), are proven tools to assist in the selection of practices to perform for achieving a level of assurance that the processes performed will produce the desired level of quality for safety software. The CMMI has two approaches: staged and continuous. For organizations introducing a software process improvement program, these models should be considered. 14 D O E G 414.1-4 6-17-05 Table 3. ASME NQA-1-2000 Cross Reference to DOE Software Quality Assurance (SQA) Work Activities SQA Work Activities ASME NQA-1 So ftw ar e (s w ) pr oj ec t m an ag em en t & q ua lit y pl an ni ng Sw ri sk m an ag em en t Sw c on fig ur at io n m an ag em en t Pr oc ur em en t & su pp lie r m an ag em en t Sw re qu ire m en ts id en tif ic at io n & m an ag em en t Sw d es ig n & im pl em en ta tio n Sw sa fe ty V er ifi ca tio n & va lid at io n Pr bl m re po rti ng & co rr ec tiv e ac tio n Tr ai ni ng o f … sa fe ty sw Organization Req. 1, 1A-1, 200 2A-2, 301 Quality Assurance Program SP 2.7, 400 2A-2, 300 1A-1, 200 SP 2.7, 400 2A-2, 301, 502 Req. 1 Req. 2, 100 SP 2.7, 400 SP 4.1, 400 Req. 2 SP 2.7, 402 1A-1, 303 Req. 2A-2 2, , 600 Design Control SP 2.7, 400 SP 4.1, 101, 200, 404, 406 Req. 3, 802 SP 2.7, 203 SP 4.1, 203 SP 2.7, 300 SP 4.1, 300 Req. 3, 800 SP 2.7, 400 SP 4.1, 400 Req. 3, 800 SP 2.7, 400 SP 4.1, 400 Req. 3, 800 SP 2.7, 402 SP 4.1, 100 Req. 3, 801.4, 801.5 Req. 11, 400 SP 2.7, 402.1, 404 SP 4.1, 402.1, 404 Req. 15 Req. 16 SP 2.7, 204 SP 4.1, 204 Procurement Document Control Req. 4 Instructions, Procedures, and Drawings Req. 5 Req. 5 Document Control Req. 6 SP 2.7, 201 SP 4.1, 201 Req. 6 SP 2.7, 201 SP 4.1, 201 D O E G 414.1-4 15 6-17-05 SQA Wo Activities SME NQA-1 w ) ng nt n nt nt nt & n on … rk A So ftw ar e (s pr oj ec t m an ag em en t & q ua lit y pl an ni Sw ri sk m an ag em e Sw c on fig ur at io m an ag em e Pr oc ur em en t & su pp lie r m an ag em e Sw re qu ire m en ts id en tif ic at io n & m an ag em e Sw d es ig n im pl em en ta tio Sw sa fe ty V er ifi ca tio n & va lid at io n Pr bl m re po rti ng & co rr ec tiv e ac ti Tr ai ni ng o f sa fe ty sw Cont Purc ms S 00 SP 4.1, 300 rol of hased Ite and Services Req. 7, P 2.7, 3 Identific Control of It ation and ems eq. 3, 802 SP 2.7, 203 SP 4.1, 203 R Control of Special Processes Inspection , 402.1, 404 Req. 3, 801.4, 801.5 Req. 11 400 SP 2.7, 402.1, 404 SP 4.1, Test Control Req. 3, 801.4, 801.5 Req. 11, 400 SP 2.7, 402.1, 404 SP 4.1, 402.1, 404 16 D O E G 414.1-4 6-17-05 SQA Work Activities ASME NQA-1 So ftw ar e (s w ) pr oj ec t m an ag em en t & q ua lit y pl an ni ng Sw ri sk m an ag em en t Sw c on fig ur at io n

m an ag em en t Pr oc ur em en t & su pp lie r m an ag em en t Sw re qu ire m en ts id en tif ic at io n & m an ag em en t Sw d es ig n & im pl em en ta tio n Sw sa fe ty V er ifi ca tio n & va lid at io n Pr bl m re po rti ng & co rr ec tiv e ac tio n Tr ai ni ng o f … sa fe ty sw Control of Measuring and Test Equipment SP 4.1, 101.3 Handling, Storage, and Shipping Inspection, Test and Operating Status Control of Nonconforming Req. 15 SP 2.7, 04 SP 4.1, 04 Items 2 2 Corrective Action Req. 16 SP 2.7, 204 SP 4.1, 204 Quality Assurance Records SP 2.7, 201 SP 4.1, 01 2 Audits Req. 1 8 DOE G 414.1-4 6-17-05 For safety s a s 17 ystems, hazards and accident analyses are performed at the system level and then for ny subcomponent of the system that potentially could have an adverse effect on safety. Since oftware is a subcom , hazard analys afety software is performed. Hazard analysis is best performed periodically throughout the life-cycle of the safety software development and operations to reassess h s safety of the system and its software. The information from these hazard an ke design decisions related to the safety software and its associated safety system. 5.2 SOFTWARE WORK ACTIVITIES Software should be lled in a traceable, planned, and orderly manner. The safety software quality work activities defined in this section provide the basis for planning, implementing, maintaining, and operating safety software. The work activities for safety software include tasks such as software project planning, SCM, and risk analysis that cross all phases in the life-cycle. Additionally, the work activities include tasks that are specific to a life-cycle phase. These work activities cover tasks during the development, maintenance, and operations of safety software. The work activities should be implemented based upon the graded level of the safety software and the applicable software type. Table 4 provides a summary of the mapping between software type, the grading levels, and the ten SQA work activities. Not all work activities will be applicable for a particular instance of safety software. This Guide indicates where these work activities may be omitted. However, the best judgment of the software quality engineering and safety system staffs should take precedence over any optional work activities presented in this Guide. 5.2.1 Software Project Management and Quality Planning As with any system, project management and quality planning are key elements to establishing the foundation to ensure a quality product that meets project goals. For software, project management starts with the system level project management and quality planning. Software specific tasks should be identified and either included within the overall system planning or in separate software planning documents. These tasks may be documented in a software project management plan (SPMP), an SQA plan (SQAP), a software development plan (SDP), or similar documents. They also may be embedded in the overall system Typically the SPMP, SQAP, and/or SDP are the controlling documents that define and guide the processes necessary to satisfy project requirements, including the software quality requirements. These plans are initiated early in the project life-cycle and are maintained throughout the life of the project.

The software project management and quality planning should include identifying all tasks associated with the sof devel ent and ent of services, ponent of the system is specific to the s the yses aza is rd used to m and aal contro level planning documents. 13 tware opm procurement, including procurem ration (CM ng (CM Mellon U 13 Capability Maturity M m ability Matu tegration, Version 1.1, CMMI for Softw , Stag resentatio -2002-TR-029, ESC-TR-2002-029, C e Eng ng Institu . odel Integ are Engineeri arnegie MI) Pro MI-S niversity duct Tea W, V1.1) , Softwar , Cap ed Rep ineeri rity Mod n, CMU te, 2002 el In /SEI 18 D O E G 414.1-4 6-17-05 Table 4. Mapping Safety Software Types and Grading Levels to Software Quality Assurance (SQA) Work Activities SQA Work Activity Level A Level B Level C C us to m D ev el op ed C on fig ur ab le A cq ui re d U til ity C al cs C om m er ci al D & A C us to m D ev el op ed C on fig ur ab le A cq ui re d U til ity C al cs C om m er ci al D & A C us to m D ev el op ed C on fig ur ab le A cq ui re d U til ity C al cs C om m er ci al D & A Software (Sw) project management & quality planning Full Full Grade Grade n/a Full Full Grade Grade n/a Grade Grade Grade Grade n/a Sw risk management Full Full Full Full n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a Sw configuration management Full Grade Grade Grade Grade Full Grade Grade Grade Grade Grade Grade Grade Grade Grade Procurement & supplier management Full Full Full Full Full Full Full Full Full Full Full Full Full Full Full Sw requirements identification & management Full Full Full Full Full Full Full Full Full Full Full Full Full Full Full Sw design & implementation Full Grade n/a Grade n/a Full Grade n/a Grade n/a Full Grade n/a Grade n/a Sw safety Full Full Full n/a n/a Grade Grade Grade n/a n/a Grade Grade Grade n/a n/a Verification & Validation Full Full Full Grade n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a Problem reporting & corrective action Full Full Full Grade Full Full Full Full Grade Full Full Grade Grade Grade Grade Training of … safety Sw Full Full Full Full n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a DOE G 414.1-4 19 6-17-05 estimate of the duration of the tasks, resources allocated to the task, and any dependencies. The planning sh n of the tasks and y rele t i A-1-20 nsensus nda s14 planning documents that are d resources to ass in the id fic n and curem quality and tware de op nt ning ntifies and th ftwa hases grad QA and software developme ctivit pe d during software elopment or maintenance. The software quality and softwa ng ng ivitie d rigor lem e dependent on the identified ading el ty software and the ity of DOE or its contractors to build quality in and assess the quality of the safety software. ause SQ are ove basic quality and ftware gi vities, suc risk ma , p lem r rting co e ons, V&V, uding software reviews and testing, may be further detailed in separate plans. These plans ac ed in th pl will be discussed la id tware project management and quality planning fully apply custo ev ped a figura pes for b L l A d Lev saf o F evel nd B acquired and utility calculation and all Level C software applications, software project ent and qua plannin sk n rade his grading i de th catio g of all if t ware tasks. W i s e so re -cycle may include little or n ftware development activities, the software project and

pl st likely be part of the overall system el o ility nning. s work not apply co er l design and analysis software because the ject man quality ni ac ties as ciated th lysis softw med by the service supplier. DOE controls the SQA activities of that ware through procurement agreements and specifications. .2 So anage t tware risk manag nt provides a disciplined environmen r proa e ision tinuou t can go n ete ine w risks im t t dres d ent actions to address those risks.16 Because risk management is such a fundamental tool t managem an gr ar softw proje a en lthou mes associated with safety analysis of tentia ilures ftw k nagement ses on the risks to the successful c pletion of the software project. The risks addressed by work ject ma em ri associated with the successful completion of a ware ap hereas Section 5.2.7 addresses the risks associated with the potential modes of the software. ould include a descriptio 00, several an van nformation. In addition to NQ goo pro Sof and dev of i abil Bec acti incl and Sof con Lev ma ide life qua Thi pro ana soft 5.2 Sof con imp for som focu this soft fail co sta rd ,15 provide details of ist enti atio and description of the software development ent tasks. tware any sof e S vel me plan ide nt a gui rfor des me e so re p ing of th ies re e lev ine of eri safe act s an mp entation will b gr AP and S h as SC DP M, rall nagem so epo en and nee rre ring plans, some quality ctivent rob acti and the tivities identifi ese ans ter in this Gu e. to ety s m d re. elo or L nd A able software ty oth eve an el B ftwa el nagem ntifi lity ckin g ta sign s ca ican be g soft d. T sh nsta ould nce nclu of th e ftwan and tra here o so to lity anning will activity do mo es lev project r fac pla mm cia agement a are are nd perfor plan ng tivi so wi commercial design and ftware Risk M men eme ha t fo are ctiv rtan dec o ad making to s, ansly assess w wro g, d rm hat po lem projec eti ent, it is inte al p t of po are l fa ct m , so nag ar em e ris t. A ma gh om ent activity are pro nag sks plication w ure nd E ctronic E EE, 1998. EE Standard f eference Doc ers Software Q 14 In M 15 IE 16 S stitu anagem EE Std QAS21.0 nergy Q te of Electrical a le n St So ent Plans, IE 730-2002, IE 1.00-1999 ( E uality Mana gine or Soft ument) uality Assu ers (IEE ware Q , Softw ra E), d 1058 uality Ass are Risk M nce Subco -1998, urance Pla anagemen mmittee, d IEEE S ns, IEE t: A Pr ated 2- tand E, 20 acti 200 ard for 02. cal Gu 0. ftware ide, Depart Project ment ofR g 20 DOE G 414.1-4 6-17-05 k g isks throughout all phases of the project’s life-cycle should include special mphasis on tracking the risks associated with costs, resources, schedules, and technical aspects r clude— • using unproven computer and software technologies such as programming languages not i ; The risks associated with the safety software applications need to be understood and documented. The above bulleted list identifies a few potential risks associated with safety software applications. Each risk should be evaluated against its risk thresholds. Different Risk assessment and risk control are two fundamental activities required for project success. Ris assessment addresses identification of the potential risks, analysis of those risks, and prioritizin the risks to ensure that the necessary resources will be available to mitigate the risks. Risk control addresses risk tracking and resolution of the risks. Identification, tracking, and management of the r e of the project. Several risk identification techniques are described and detailed in standards and literature.17,18

Risk resolution includes risk avoidance, mitigation, or transference. Even the small risks during one phase of the safety software application’s life have the potential to increase in some other phase of the application’s life with very adverse consequences. In addition, mitigation actions fo some risks could create new (secondary) risks. Examples of potential software risks for the safety software application might in • incomplete or volatile software requirements; • specification of incorrect or overly simplified algorithms or algorithms that will be very difficult to address within safety software; • hardware constraints that limit the design; • potential performance issues with the design; • a design that is based upon unrealistic or optimistic assumptions; • design changes during coding; • incomplete and undefined interfaces; ntended for the target application • use of a programming language with only minimal experience using the language; • new versions of the operating system; • unproven testing tools and test methods; • insufficient time for development, coding, and/or testing; • undefined or inadequate test acceptance criteria; and • potential quality concerns with subcontractors or suppliers. Christensen, Mark J., and Richard17 H. Thayer, The Project Manager’s Guide to Software Engineering’s Best , pp. 417–447. Practices, Institute of Electrical and Electronics Engineers Computer Society Press, 2001 18 Society of Automotive Engineers (SAE) JA1003, Software Reliability Program Implementation Guide, SAE 2004, Appendix C4.6. DOE G 414.1-4 21 6-17-05 isks. ritized, resolved to an acceptable level of risk, and tracked through the life of the fety software. For Level B or Level C software applications, the granularity for the risks to be n determine a graded approach for resolving the sks and the process for tracking the risks. This work activity does not apply to commercial design and analysis safety software because this alysis services provided to DOE from a rform procure n or analysis requirements. ng risk management is provided by IEEE andar nce regarding the risk management of isting organizational k ma 1.00-1999, Software Risk Management: A Practical risk transference, and risk avoidance that may be of erest 2.3 ines ange control process.20 The following four areas anagement: control, (3) configuration status accounting, E NQA-1-2000 software audits and reviews.23 techniques may be used to evaluate the risks. Examples of these techniques include decision trees, scenario planning, game theory, probabilistic analysis, and linear programming. Various treatment alternatives to addressing risk should be considered to avoid, reduce, or transfer r Flexibility may need to be applied regarding risk management based upon the risk categorization of the safety software application. For a Level A safety software, all apparent risks known at the time, whether large or small, should be identified, analyzed for impact and probability of occurrence, prio sa identified, analyzed, prioritized, resolved to an acceptable level of risk, and tracked should be determined by the safety system staff and can be graded. The safety system staff should focus o the adverse events that would dominate the risk and assess these in a qualitative manner. The safety system staff also has the responsibility to ri

software is used in conjunction with the design and an commercial contractor. The risk management work activity associated with that software is pe ed by the service supplier. DOE controls the SQA activities of that software through ment agreements and specifications of desig Further guidance beyond that in NQA-1-2000 regardi St d 16085-2004.19 This standard provides guida acquired, developed, operational, or maintained systems to support the ex ris nagement processes. SQAS21.0 Guide, also discusses a risk taxonomy, int to the safety software analyst. 5. Software Configuration Management SCM activities identify all functions and tasks required to manage the configuration of the software system, including software engineering items, establishing the configuration basel to be controlled, and software configuration ch 21of SCM should each be addressed when performing configuration m (1) configuration identification, (2) configuration and (4) configuration audits and reviews. This Guide extends ASM 22configuration management tasks by including configuration s (IEEE) EEE, 20 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 203, p. 105. 22 EE, 2003, Section 5.3.5. 19 International Organization for Standardization (ISO)/Institute of Electrical and Electronics Engineer Std 16085, IEEE Standard for Software Engineering: Software Life Cycle Processes—Risk Management, I 2004. 21 IEEE Std 828-1998, IEEE Standard for Software Configuration Management Plans, IEEE, 1998, Section 4.3. ASME NQA-1-2000, op. cit., Part I, Section 802, p. 16. 23 IEEE Std 7-4.3.2-2003, IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations, IE 22 DOE G 414.1-4 6-17-05 of each nd configuration reviews and audits. are nted udits or reviews should be conducted to verify that the software product is consistent with the Level A or Lever B, all four areas of SCM noted a ap re graded as Level A or Level B and all Level C ion as re are a variety of approaches for software procurement and supplier management based upon— • service being procured and The methods used to control, uniquely identify, describe, and document the configuration version or update of software and its related documentation should be documented. This documentation may be included in a SCM plan or its equivalent. Such documentation should include criteria for configuration identification, change control, configuration status accounting, a During operations, authorized users lists can be implemented to ensure that the software use is limited to those persons trained and authorized to use the software. Authorized users lists access control specifications that are addressed in Section 5.2.5, Software Requirements Identification and Management. A baseline labeling system should be implemented that uniquely identifies each configuration item, identifies changes to configuration items by revision, and provides the ability to uniquely identify each configuration. This baseline labeling system is used throughout the life of the software development and operation. Proposed changes to the software should be documented, evaluated, and approved for release. Only approved changes should be made to the software that has been baselined. Software verification activities should be performed for the change to ensure the change was impleme correctly. This verification should also include any changes to the software documentation.

A configuration item descriptions in the requirements and that the software, including all documentation, being delivered is complete. Physical configuration audits and functional configuration audits are examples of audits or reviews that should be performed.24 SCM work activities should be applied beginning at the point of DOE’s or its contractor’s control of the software. For custom developed safety software graded at bove ply. For all other types of safety softwa safety software, this work activity may be graded by the optional performance of configurat audits and reviews. 5.2.4 Procurement and Supplier Management Most software projects will have procurement activities that require interactions with suppliers regardless of whether the software is Level A, B, or C. Procurement activities may be as basic the purchase of compilers or other development tools for custom developed software or as complicated as procuring a complete safety system software control system. Thus, the the level of control DOE or its contractors have on the quality of the software or software • the complexity of the software. 24 Institute of Electrical and Electronics Engineers (IEEE) 1042-1987, IEEE Guide to Software Configuration Management, IEEE, 1987, Section 3.3.4. DOE G 414.1-4 23 6-17-05 rocurement documentation should include the technical25 and quality26 requirements for the g and validating the software, including any documentation to be delivered; n e e supplier, on the complexity of the software and its importance to safety. e em requirements should be translated into requirements specific for the s re be documented in system level requirements docume ocurement contracts, and/or other acquired P safety software. Some of the specifications that should be included are— • specifications for the software features, including requirements for safety, security, functions, and performance; • process steps used in developin • requirements for supplier notification of defects, new releases, or other issues27 that impact the operation; and • mechanisms for the users of the software to report defects and request assistance i operating the software. These requirements should be assessed for completeness and to ensure the quality of the softwar being purchased. There are four major approaches for this assessment: • performing an assessment of th • requiring the supplier to provide a self-declaration that the safety software meets the intended quality, • accepting the safety software based upon key characteristics (e.g., large user base), and verifying the supplier has obtained a certification or accreditation of the software product • quality or software quality program from a third party (e.g., the International Organization for Standardization, Underwriters Laboratories, and Software Engineering Institute). It is important to note that while Levels A, B, and C software applications are required to fully meet this work activity, the implementation detail and assessment method of the supplier can vary based 5.2.5 Software Requirements Identification and Management Safety system requirements provide the foundation for the requirements to be implemented in th software. These syst oftwa . The identified software requirements may nts, software requirements specifications, pr software agreements. These requirements should identify functional; performance; security, including user access control; interface and safety requirements; and installation considerations and design constraints where appropriate. The requirements should be complete, correct, consistent, clear, verifiable, and feasible.28

25 NQA-1-2000, op. cit., Part I, Req NQA-1-2000, op. cit., Part I, Requir ASME uirement 4, Section 202, p. 18. 26 ASME ement 4, Section 100, p.18. 27 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 301, p. 105. ware 28 Institute of Electrical and Electronics Engineers (IEEE) Std 830-1998, IEEE Recommended Practice for Soft Requirements Specifications, IEEE, 1998, Section 4.3. 24 DOE G 414.1-4 6-17-05 an important aspect to ensuring only authorized users can operate the system or use the software for design or analysis tasks. Controlling access is a fications as part of the 29 Once th been defined and documented, they should be managed to nimi ies to ensure the corr to operations. Software requirements should be traceable This work activity has no grading associated with its performance. Software requirements ns and sho rement. However, the detail and format of the safety software requirements may vary with the software type. Custom developed software most likely will ore formal document may be applicable. During software design and implementation the software is developed, documented, reviewed, require escribe how th function internally. Data structure requirements and layouts may be necessary to fully understand the internal operations of the software. e ionships between data lements, interfaces with external components, and basic database table structures may be all that are need m developed software, User access control during operations is software safety and/or security requirement that can be associated with training or qualification to operate the system. ASME NQA-1-2000 addresses access control speci operations phase. e software requirements have mi ze conflicting requirements and maintain accuracy for later validation activit ectness of the software placed in throughout the software life-cycle.30 identification management and traceability applies to Level A, B, and C software applicatio uld fully meet this requi contain a larger number of software requirements than configurable, acquired, utility calculation, or commercial design and analysis tool software, and thus, a separate m 5.2.6 Software Design and Implementation and controlled. The software design elements should identify the operating system, function, interfaces, performance requirements, installation considerations, design inputs, and design constraints. The software design should be complete and sufficient to meet the software ments.31 The design activities and documentation should be adequate to fully d e software will interface with other system components32 and how the software will Custom developed software will require more formality in the documentation and review of th design than configurable or utility calculations. Simple process flows, relat e ed for configurable or utility calculations, whereas for custo complete functional and logical designs of the software components, the input and output data, and pseudo code may be required to fully understand the safety software design. The software design description may be combined with the documentation of the software requirements or software source code.33 During implementation, static analysis, clean room inspections, and reviews are common techniques to ensure the implementation remains consistent with the design and does not add 29 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 405, p. 106. 6.

30 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 401, p. 106. 31 ASME NQA-1-2000, op. cit., Part I Introduction, and Section 801.2, p. 1 32 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402, p. 106. 33 ASME NQA-1-2000, op. cit., Part I Introduction and Section 801.2, p. 16. DOE G 414.1-4 25 6-17-05 omplexity or functions which could decrease the safe operation of the software. Many tools he software developer should perform unit testing prior to system level V&V techniques, a graded or tailored approach to ensure the known risks are ng; ign performed. Additionally, formal developer testing that includes functional, structural, tim ctors testing should be planned, performed and ed as ple at includes safety functions, security, and performance ign strategies to eliminate or mitigate those hazards. Hence, it is commended that the software safety process address the mitigation strategy for the components that have potential safety consequences if a fault occurs, whereas the software design and im re of the safety software application. c exist to evaluate the complexity and other attributes of the source code design structure. Walkthroughs and more formal inspections, such as Fagan inspections, can be used to identify defects in source code, as well as design descriptions and other software development process outputs. T including acceptance testing. Developer testing can be very structured and formal, using automated tools or less formal methods. In addition to unit testing, functional, structural, timing (performance testing), stress, security, and human-factors testing are useful testing methods. These methods can be applied using mitigated appropriately. Other techniques34,35 such as error seeding; equivalence class testi branch and path testing; statistical-based, boundary value testing; and code coverage analysis may all be beneficial testing techniques to ensure robust and reliable software. The software design and implementation work activity for Levels A, B, and C custom developed software applications should fully meet this requirement. For this software type, the design, including interfaces and data structures, should be completely documented; reviews of the des and code should be ing, stress, security, and human-fa the results documented. It is recommended that the complexity of the custom developed safety software be evaluated and analysis performed to reduce the complexity of the source code modules. Configurable and utility calculation for Levels A, B, and C software applications may be grad for this work activity. This grading should include fully performing the design work activities with custom developed software. However, less formal design and code reviews, such as sim desk checks by another individual other than the developer, may be performed. Developer testing should be performed and documented th testing. This work activity does not apply to acquired or commercial design and analysis safety software types since the design and implementation activities associated with commercial design and analysis software are performed by the service supplier. DOE controls the SQA activities of that software through procurement agreements and specifications. 5.2.7 Software Safety The development of software applications requires identification of hazards (i.e., abnormal conditions and events) that have the potential for defeating a safety function and the mplementation of desi re

plementation process addresses the architectu 34 Pressman, Roger S., Software Engineering: A Practitioner’s Approach, McGraw Hill, 1992, pp. 595–629. fety and Reliability, Lawrence Livermore 35 Sparkman, Debra, Techniques, Processes, and Measures for Software Sa National Laboratory, UCRL-ID 108725, 1992. 26 DOE G 414.1-4 6-17-05 C of level. at se a system failure. uring the initial concept and requirement analysis phases for the software, potential failures ut the requirements and design structure. These techniques include failure modes and ffects analysis, fault-tree modeling, event-tree modeling, cause-consequence diagrams, hazard horoughly. Separation of the safety features also allows for more rigorous software development and verification practices to be applied to the safety components while providing the appropriate and cost effective level of uld upt Software is only one component of the overall safety system. It may be embedded in an I& system, it may be a custom control system for hardware components, or it may be standalone software used in safety management or support decisions. In any of these or other applications software important to safety, analysis of the software application occurs first at the system The analysis should then be performed at the software component level to ensure adequate safeguards are provided to eliminate or mitigate the potential occurrence of a software defect th could cau Methods to mitigate the consequences of software failures should be an integral part of the software design.36 Specific software analysis and design methods for ensuring that safety functions are well thought out and addressed properly should be performed throughout the software development and operations life-cycles. These methods include dynamic and static analyses. The techniques and methods described in this section are only a selection of those available. Several resources are available to assist in the selection and use of these methods. A few are listed in the reference section of this Guide. D need to be identified and evaluated for their consequences of failure and probability of occurrence. Some potential problems are (1) complex or faulty algorithm, (2) lack of proper handling of incorrect data or error conditions, (3) buffer overflow, and (4) incorrect sequence of operations due to either logic or timing faults. There are several hazard analysis techniques that may be used for this purpose. Many of these techniques are performed as preliminary analyses and later updated as more information is known abo e and operability analysis, and interface analysis. Techniques such as these should be applied and appropriately documented to understand and assess the impact of software failures on the system. The design of the software is critical to ensuring safe operation of the system. The software design should consider principles of simplicity, decoupling, and isolation to eliminate the hazards.37 Complexity of the software design, including the logic and number of data inputs, has proven to increase the defect density in software components. The safety features should be separate from nonsafety modules, minimizing the impact of failure of one module on another.38 The interfaces between the modules need to be defined and tested t SQA applied to the nonsafety components. Software engineering safety design practices sho include process flow analysis, data flow analysis, path analysis, interface analysis, and interr analysis during the design phase.

38 3. 36 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402, p. 106. 37 Leveson, op. cit., pp. 400–412. IEEE Std. 7-4.3.2-2003, op. cit., Section 5.6, p. 1 DOE G 414.1-4 27 6-17-05 uced e the apabilities of the overall system that may not be immediately detectable by the system. In these e ely ory functionality and integrity tests, such as checksums and watch dog timers for software processes, including operating system processes.41 Additionally, ing for d. For Level A custom developed safety ftware, the design concepts that include simplicity of modules that perform safety functions f ault Level B or Level C software applications may be raded. This grading may include fully performing the safety analysis activities for the software ethods described above could add undue burden to the development of these applications and e at e development or acquisition processes to ensure the software meets the intended requirements. When hazards related to software functions cannot be eliminated, the hazard should be red and/or monitored. Additionally, software can experience partial failures that can degrad c instances, other design techniques, such as building fault detection and self-diagnostics into th software, should be implemented. Using external monitors (safety bag) for the software safety functions, n-version programming, and Petri nets are examples of techniques39,40 that can ensure the software design adequately addresses safety issues and minimizes failure modes by adding fault tolerant concepts. Self-diagnostics detect and report software faults and failures in a tim manner and allow actions to be taken to avoid an impact on the system operating safety. Some of these techniques include mem software control functions can be performed incrementally rather than in a single step, reduc the potential that a single failure of a software component would cause an unsafe state. The software safety work activity for Level A custom developed, configurable, and acquired safety software should fully meet this requirement. For this software type the safety analysis the software components should be performed. This analysis may be part of the overall safety system analysis if detailed software failures are include so and isolation of those modules should be part of the design considerations. Where the design o the software modules still presents an unacceptable risk to failure of the safety system, f tolerant and self-diagnostics designs should be implemented. Custom developed, configurable, and acquired g components to ensure the safety aspects are being addressed. The design concepts of simplicity and isolation and fault tolerance and self-diagnostics may not apply to Level B or Level C software applications and, thus, can optionally be applied. This work activity does not apply to utility calculation or commercial design and analysis safety software types. Utility calculations are typically simple calculations where techniques and m not increase the assurance that any software failure would not impact safety. However, if the safety analysis determines that complexity of the utility calculation warrants the use of thes techniques, they should be applied. For commercial design and analysis software, the software safety activities are performed by the service supplier. DOE controls the SQA activities of th software through procurement agreements and specifications.

5.2.8 Verification and Validation V&V is the largest area within the SQA work activities. Verification is performed throughout th life-cycle of the safety software. Validation activities are performed at the end of the software . 39 Sparkman, op. cit. 40 SAE JA1003, op. cit., Appendix C 41 IEEE Std 7-4.3.2-2003, op. cit., Section 5.5.3, p. 13. 28 DOE G 414.1-4 6-17-05 activities include reviews, inspections, assessments, bservations, and testing. This Guide expands on ASME NQA-1-2000 acceptance testing n, n may upplier assessments are important aspects of V&V. Assessments are covered in Section 5.2.4, and all test activity deliverables laced under configuration management. e g strategies that may be appropriate for acceptance testing include equivalence class testing, branch and path testing, statistical-based e V&V activities should be performed by competent staff other than those who developed the item being verified or validated.42 V&V o activities to include more extensive V&V activities of reviews, inspections, assessments, and observations as described in other consensus standards. Reviews and inspections of software deliverables requirement specifications, procurement documents,43 software design, code modules, test results, training materials, user documentatio and processes that guide the software development activities should be performed. The software deliverables may be combined with other software or system documents. Traceability of the software requirements to the software design should be performed.44 As mentioned in the development practice section, inspections can be formally implemented Fagan inspections, walkthroughs, or desk checks. Verification of the software design, using one of the above methods, should be completed prior to approval of the software for use.45 This verificatio be performed as part of the software development and implementation activity. S Procurement and Supplier Management, and Section 6, Assessment and Oversight. Observations and testing can be performed during the development, factory or site acceptance testing, installation, and operation (i.e., in-use testing)46 of the software. Observations and testing during development is discussed in Section 5.2.6, Software Design and Implementation. Software testing activities should be planned and documented. Test cases and procedures, including expected results, should be created. All test activity deliverables should be under configuration management. Test results should be documented 47p Acceptance testing should include functional testing, performance testing, security testing, stress testing, and load testing. Users’ guides, use cases, and operational profiles are instrumental in identifying and detailing the positive test cases and procedures. Failure mode analyses can b used for defining negative test cases and procedures. Testin and boundary value testing. Additionally, the system should continually be monitored to estimate its continuing reliability and safety. Periodic testing of the operational system should be performed to detect any degradation.48 If testing is not possible, monitoring using quantitative measurements should b performed. 42 ASME NQA-1-2000, op. cit., Part I, Requirement 3, Section 801.1, p. 16. 43 ASME NQA-1-2000, op. cit., Part I, Requirement 4, Section 300, p. 18. 44 ASME NQA-1-2000, op. cit, Part II, Subpart 2.7, Section 402.1, p. 106. 45 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402.1, p. 106.

0, op. cit., Part I, Requirement 11, Section 400, p. 29. Requirement 11, Section 200, p. 29. ction 400, p. 30. 46 ASME NQA-1-200 47 ASME NQA-1-2000, op. cit., Part I, 48 ASME NQA-1-2000, op. cit., Part I, Requirement 11, Se DOE G 414.1-4 29 6-17-05 When a new version of a software product is obtained, predetermined and ad-hoc test cases and y users with nearly the same capabilities and that are reused 50 deliverables ing tance test cases and procedures, including expected y or Level A software, continual monitoring of safety software operations based upon historical failure data and results of periodic reassessment of hazards should be performed. For Level A, B, n developed, reviews and e performed. procedures should be performed to validate that the system meets the requirements and does not perform any unintended functions.49 If the system is operational, only positive testing may be possible. In those instances, it is important to perform analysis of failure modes for the software to understand the consequences if the software or system should get into an abnormal operational state. Modern utility calculation applications, such as spreadsheet programs, have grown dramaticall in power, with a corresponding growth in risk. The addition of macro programming languages nd the ability to incorporate “add-in” programs provide a as code developed with traditional programming tools. Utility calculation applications are installed on virtually every desktop, and user files containing algorithms and data can be easily modified by users. Section 101.1 of ASME NQA-1-2000, Subpart 4.1, provides useful guidance on V&V of utility calculations. Calculations performed using applications such as commercial spreadsheet programs may be treated in either of two ways. In the case of relatively straightforward calculations, the calculation result may be checked and verified in the same manner as a hand calculation. For more complex or extensive calculations, where checking verification of calculation results are impractical or undesirable, the user files containing the calculation formulas, algorithms, or macros should be subject to the entire software life-cycle rocess. The latter approach may also be expedient for calculation applications p frequently. ustom developed software will most likely have a larger number and more detailedC than would utility calculations. For Level A safety software all deliverables should be reviewed using V&V methods. Additionally for Level A, traceability of the requirements to the design and from requirements to test cases should be performed. For Level B safety software, deliverables that include requirements, test plans and procedures, and test results should be reviewed us V&V methods. For all Level A safety software except utility calculations, acceptance testing work activities hould be planned and documented; acceps results should be created; test results should be documented; and all test activity deliverables should be under configuration management. Level A utility calculations and Level B and C custom developed, configurable, acquired, and utility calculations can use a graded approach b applying less formality in the documentation. Simple check lists for acceptance test cases and procedures may be used in place of more detailed test cases and procedures. Test results should be documented and all test activity deliverables placed under configuration management. F

or C software, when new releases of the safety software have bee bacceptance testing of changed documents and software should 49 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 404, p. 106. 50 ASME NQA-1-2000, op. cit., Section 101.1, Subpart 4.1. 30 DOE G 414.1-4 6-17-05 e ed by and specifications. enting, evaluating and correcting software problems; (2) an evaluation process for etermining whether a reported problem is indeed a defect or an error; and (3) the roles and and ems to the s to need not be separate from the other problem reporting and corrective action rocesses if the existing process adequately addresses the items in this work activity.53 that reduces the formality of documenting problem reports and approving orrective actions taken may be applied for Level A and B utility calculation safety software and SQA This work activity does not apply to commercial design and analysis safety software types sinc the V&V activities associated with commercial design and analysis software are perform the service supplier. DOE controls the SQA activities of that software through procurement agreements 5.2.9 Problem Reporting and Corrective Action Coupled with the configuration management of the software system, the problem reporting and corrective action process should address the appropriate requirements of the QAP corrective action system. The reporting and corrective action system will cover (1) methods for docum d responsibilities for disposition of the problem reports, including notification to the originator of the results of the evaluation.51 If the noted problem is indeed an error, the problem reporting and corrective action system should correlate the error with the appropriate software engineering elements; identify the potential impacts and risks to past, present, and future developmental operational activities; and support the development of mitigation strategies. After an error has been noted, all users should be apprised to ascertain any impacts upon safety basis decisions. Procurement documents should identify the requirements for suppliers to report probl supplier, any required supplier response, and the method for the purchasers to report problem the supplier.52 Maintaining a robust problem reporting and corrective action process is obviously vital to maintaining a reliable and vital safety software system. This problem reporting and corrective action system p This work activity should be fully implemented for all Level A and B software types (custom developed, acquired, configurable, and commercial design and analysis) and for Level C custom developed. This formal implementation should include documentation and tracking to closure of any problems reported for the software and authorization to perform the corrective action. A graded approach c all Level C software applications except custom developed. This less formal implementation may include interoffice communications describing the problem identified and the corrective actions planned. 5.2.10 Training Personnel in the Design, Development, Use, and Evaluation of Safety Software Training personnel in designing, developing, testing, evaluating, or using the safety software application is critical for minimizing the consequences of software failure. Although other 204, p. 229. 51 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 204, p. 105. 52 ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 301, p. 105. 53 ASME NQA-1-2000, op. cit., Part IV, Subpart 4.1, Section

DOE G 414.1-4 31 6-17-05 ork activities may indicate that the software satisfies its operational objective, improper or ation users, and perations staff. The analyst and developers may need training in fault tolerant methodologies, ed, that proper options and menus are selected, and that the results of the software can be interpreted correctly. A trained and knowledgeable staff is e ia to ensure the proper levels of quality and d te in For Level B and C software applications, this work activity can be graded to onal, g n and d management and control issues. w invalid use of the software may negate the safety mitigation strategies included within the software. Training may be necessary for the analyst, development and test teams, applic o safety design methodologies, user interface design issues, testing methodologies, or configuration management to ensure delivery of a robust software application. Meanwhile, the software application users and operations staff may need training specific to the software to ensure that proper data are enter ssent l to assess and evaluate the SQA requirements safety exists in the software. Training should be commensurate with the scope, complexity, and importance of the tasks an the education, experience, and proficiency of the individual. Indoctrination as described in ASME NQA-1-200054 meets this work activity requirement. Personnel should also participa continuing education and training as necessary to improve their performance and proficiency and ensure that they stay up-to-date on changing technology and new requirements.55 Completion of training, education, and/or qualification requirements for all staff involved in the development, testing, use, and evaluation of custom developed or configurable software graded as Level A, B, or C should be documented and reviewed periodically. This may include a osition description, qualification criteria, or a list of training courses along with verification of p successfully meeting the knowledge requirements. Completion of training, education, and/or qualification requirements for all staff involved in the procurement, testing, use, and evaluation of acquired or utility calculation software graded as Level A should be documented and reviewed eriodically.p include periodic evaluation by the appropriate supervising authority of the training, educati or qualification requirements for performing assigned tasks associated with using and evaluatin acquired or utility calculation software. This work activity does not apply to commercial desig and analysis safety software since the training activities associated with commercial design nalysis software are performed by the service supplier. DOE controls the SQA activities of that a software through procurement agreements and specifications. 6. ASSESSMENT AND OVERSIGHT 6.1 GENERAL DOE assessment requirements in 10 CFR 830 Subpart A and DOE O 414.1C should be applie to safety software 54 ASME NQA-1-2000, op. cit., Part I, Requirement 2, Section 200, p. 10. ea Qualification Standard, dated 12-03. 55 DOE-STD-1172-2003, Safety Software Quality Assurance Functional Ar 32 DOE G 414.1-4 6-17-05 6 D e d DOE ine ES&H Oversight Policy, dated 5-31-01, contains guidance on independent and odel for the work will ensure that the SQA implementation process ctions related safety software issues. AP .2 OE AND CONTRACTOR ASSESSMENT

DOE should assess the effectiveness of its actions in resolving issues related to safety softwar management and controls. DOE also evaluates the adequacy and implementation effectiveness of DOE and contractor safety software management and controls. DOE G 414.1-1, Management Assessment and Independent Assessment Guide for Use with 10 CFR, Part 830, Subpart A, an DOE O 414.1A, Quality Assurance; DOE P 450.4, Safety Management System Policy; and 450.5, LP management assessment. Contractors are expected to assess the adequacy and effectiveness of their safety software controls in accordance with DOE O 414.1C and this Guide. A model criteria review and approach document (CRAD) is provided in Appendix F. This m contains software qualification assessment criteria for assessing the safety software used for safety analysis and design of safety SSCs and I&C systems in the defense nuclear facilities. The organization responsible addresses the processes presented in this Guide. 6.3 DOE INDEPENDENT OVERSIGHT The DOE Office of Independent Oversight and Performance Assurance, and the Office of Inspector General are responsible for conducting independent oversight of DOE a to The DOE/NNSA SQA responsible person will verify that the SQA implementation process meets the intent of this Guide throughout the entire software life-cycle as described in the Q and procedures. DOE G 414.1-4 APPENDIX A 6-17-05 A-1 APP NS A.1. ACRONYMS al Standards Institute ASME American Society of Mechanical Engineers r Quality Control Defense Nuclear Facilities Safety Board DoD U.S. Department of Defense gy Order IEC International Electrotechnical Commission lectrical and Electronics Engineers National Aeronautics and Space Administration NNSA National Nuclear Security Administration quality assurance program QARD quality assurance requirements document RSICC Radiation Safety Information Computational Center SAE Society of Automotive Engineers SC safety class SCM software configuration management SDD software design description SDP software development plan ENDIX A. ACRONYMS AND DEFINTIO ANS American Nuclear Society ANSI American Nation ASQC American Society fo CFR Code of Federal Regulations CMMI Capability Maturity Model Integration COTS commercial off-the-shelf CRAD criteria review and approach document DCS distributed control system DNFSB DOE G U.S. Department of Energy Guide DOE O U.S. Department of Ener DOE U.S. Department of Energy DSA documented safety analysis HMI human-machine interface I&C Instrumentation and control IAEA International Atomic Energy Agency IEEE Institute of E ISO International Organization for Standardization M&O management and operating NASA PLC programmable logic controller QA quality assurance QAP APPENDIX A DOE G 414.1-4 A-2 6-17-05 SEI Software Engineering Institute SG safety guide SMS safety management system SPMP software project management plan SQA software quality assurance SQAP software quality assurance plan SRS software requirement specification SS safety significant SSC structure, system, and component SW-CMM Software Capability Maturity Model TR technical report TSR technical safety requirement USQ unreviewed safety question USQD unreviewed safety question determination V&V verification and validation VV&A verification, validation, and accreditation WIPP Waste Isolation Pilot Project DOE G 414.1-4 APPENDIX A 6-17-05 A-3

The following definitions are included with this Guide for convenience and clarification. DOE O 414.1 nitions sh ll take pre in this appendix. Acceptance Testing. T process o stem component by manual or automated means to ensu and to identify differen een exp cted and a NQA-1-2000. Admini Contr . The pro agement, procedures, record k assessm safe operation of a facility. Source: 10 CFR 830. Assessm view, valuation, dit, to determine and doc ether ems, proce requirements and perform ively. So ce: DOE O Configuration Management. The the configuration items in a system software d hardwa d change of these items throughout the system’s ife cycle, he status of configuration items and change requests. So rce: ASM Consequence. An outcome of an ev n. Source: IEEE Std 1540-2001. Firmwa combin tion of a h a that reside as read-only software on that devic only to the hardware device or only to the com e deprecated. (2) The on surro nding this t that it be avoided altogether. Source: d 610.1 -1990. Functional Configuration Audit. An audit conducted to verify that the development of a configuration item has been comple as achieved the performance and func l characte stics speci al or allocated configuration identification, and that rational d support d satisfactory. Source: IEEE Std-610.12-1990. Graded Approach. Th process of d actions used to comply ith require • the relative importance to safety, safeguards, and security; • t nitude o any hazard • t -cycle sta e of a faci • the programmatic mission o • the particular characteristics of a facility or item; A.2. DEFINITIONS C defi a cedence over those included he f exercising or evaluating a system or sy re that it satisfies the specified requirements ces betw e ctual results in the operating environment. Source: ASME strative ols visions relating to organization and man eeping, ent, and reporting necessary to ensure ent. A re wh e inspection, test, check, surveillance, or au t specified ument effect it ur sses, systems, or services mee 414.1C. process of identifying and defining (i.e., an re), controlling the release an l u and recording and reporting t E NQA-1-2000. ent, hazard, threat, or situatio re. The a ardware device and computer instructions and dat e. Notes: (1) This term is sometimes used to refer puter instructions or data, but these meanings ar confusi u term has led some to sugges IEEE St 2 ted satisfactorily, that the item h tiona ri fied in the function its ope an documents are complete an e ensuring that the level of analyses, documentation, an w ments are commensurate with— he mag f involved; he life g lity or item; f a facility; APPENDIX A DOE G 414.1-4 A-4 6-17-05 • t ative imp tance to ra azards; and • any other relevant factors. Source: 10 CFR 830. Hazard Controls. Me ures to eli workers, the public, or the envi nt, includ g— 10 CF (1) physical, design, structural, (2) safety structures, systems an (3) safety managem nt program (4) Technical Safety Requireme (5) o sary to p rds. Source: 10 CFR 830. Item. An all-inclusive term used in bly, component, equipment, material le, part, ructure, pr , subsystem, system, unit, or support Source 10 CFR 8 Nuclear Facility. A rea tor or a no re an activity is conducted for or on behalf of DOE and includes any to the extent necessary to ensure proper impleme tablished in CFR, part 10, ction 830. Source: 10 CFR 830.

Physical Configuration Audit. An audit conducted to verify that a configuration item, as built, conforms to the technical documentation that defines it. IEEE Std-610.12-1990. Process. A series of actions that achieves an end result. Source: 10 CFR 830. Quality. The condition achieved when an item, service, or process meets or exceeds the user’s requirements and expectations. Source: 10 CFR 830. Quality Assurance. All those actions that provide confidence that quality is achieved. Source: 10 CFR 830. Quality Assurance Program. The overall program or management system established to assign responsibilities and authorities, define policies and requirements, and provide for the performance and assessment of work. Source: 10 CFR 830. Risk. The likelihood of an event, hazard, threat, or situation occurring and its undesirable consequences; a potential problem. Source: IEEE Std 1540-2001. Safety. An all-inclusive term used synonymously with environment, safety, and health to encompass protection of the public, the workers, and the environment. Source: DOE O 414.1C. Safety-class structures, systems, and components (SC SSCs). Structures, systems, or components, including portions of process systems, whose preventive and mitigative function is he rel or diological and nonradiological h as minate, limit, or mitigate hazards to ronme in R 830 and engineering features; d components e s; nts; and ther controls neces rovide adequate protection from haza place of appurtenance, assem , modu st oduct, software, subassembly systems. : 30. c nreactor nuclear facility whe related area, structure, facility, or activity ntation of the requirements es se DOE G 414.1-4 APPENDIX A 6-17-05 A-5 necessary to limit radioactive hazard a ined from the safety analyses. Source: 10 CFR 830. or mitigative nction is a major contributor to defense in depth and/or worker safety as determined from diological or chemical exposure to workers. Source: DOE G 420.1-1 the ards analysis of nuclear facilities or an SSC that performs a safety nction. Source: DOE O 414.1C. d r radiological safety management programs or echnical Safety Requirements or other software that performs a control function necessary to ment as afety Management Program. A program designed to ensure a facility is operated in a manner ic such as: ; maintenance of safety systems; personnel training; conduct of operations; advertent criticality protection; emergency preparedness; fire protection; waste management; n as part component and is cited in either DOE approved documented safety nalysis or an approved hazard analysis per DOE P 450.4, Safety Management System Policy, tion, environmental medi software development/ ous m terial exposure to the public, as determ Safety-significant structures, systems, and components (SS SSCs). Structures, systems, and components which are not designated as safety-class SSCs, but whose preventive fu safety analyses [10 CFR 830]. As a general rule of thumb, safety-significant SSC designations based on worker safety are limited to those systems, structures, or components whose failure is estimated to result in a prompt worker fatality or serious injuries (e.g., loss of eye, loss of limb) or significant ra Safety and Hazard Analysis Software and Design Software. Software that is used to classify, design, or analyze nuclear facilities. This software is not part of an SSC but helps to ensure proper accident or haz fu

Safety Management and Administrative Controls Software. Software that performs a hazar control function in support of nuclear facility o T provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environ addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause. Source: DOE O 414.1C. S that adequately protects workers, the public, and the environment by covering a top quality assurance in or radiological protection of workers, the public, and the environment. Source: 10 CFR 830. Safety Software. Includes safety system software, safety and hazard analysis software and design software and safety management and administrative controls software. Source: DOE O 414.1C. Safety Structures, Systems, and Components. Both safety class structures, systems, and components and safety significant structures, systems, and components. Source: 10 CFR 830. Safety System Software. Software for a nuclear facility1 that performs a safety functio of a structure, system or a dated 10-15-96, and the DEAR clause. Source: DOE O 414.1C. Service. Work, such as design, construction, fabrication, decontamina ple analysis, safetyre ation, waste management, laboratory sam 1 P CFR 830, quality assurance requirements apply toer 10 clear facilities including radiological facilities e 10 C all DOE nu (se FR 830, DOE Std 1120, and the DEAR clause). APPENDIX A DOE G 414.1-4 A-6 6-17-05 qualification, uipm ssessment, repair, and installation or the like. Source: o : NQA-1-2000 lated actions that establish the entified in the documents safety analysis for the e and ell as a bases CFR 830. m the previous phase, and the final system or component tal , ards and security; or data collection and analysis. Source: validation/testing, inspection, nondestructive examination/testing, environmental eq ent qualification, training, a 10 CFR 830. Software. Computer programs, procedures, and associated documentation and data pertaining t the operation of a computer system. Source rols, and reTechnical Safety Requirements. The limits, cont ns for the safe operation of a nuclear facility and include, specific parameters and requisite actio as appropriate for the work and the hazards id facility: safety limits, operating limits, surveillance requirements, administrativ use and application provisions, and design features, as wmanagement controls, ppendix. Source: 10 a Verification and Validation. The process of determining whether the requirements for a syste or component are complete and correct, the products of each development phase fulfill the equirements or conditions imposed by r complies with specified requirements. Source: IEEE Std-610.12-1990. Work. A defined task or activity; such as research and development; operations; environmen remediation; maintenance and repair; administration; safety software development, validation esting, and use; inspection; safegut DOE O 414.1C. DOE G 414.1-4 APPENDIX B 6-17-05 B-1 (and B-2) APPENDIX B. PROCEDURE FOR ADDING OR REVISING SOFTWARE TO .............B-3 3 .......B-4 B.2.1 Adding Software ApplicatioNs to the Central Registry ......................................B-4 0 13 14 OR DELETING SOFTWARE FROM THE DOE SAFETY SOFTWARE CENTRAL REGISTRY CONTENTS B.1 INTRODUCTION ...........................................................................................................B-3

B.1.1 Purpose.................................................................................................... B.1.2 Scope....................................................................................................................B-3 B.1.3 Functions..............................................................................................................B- B.2 PROCESS ................................................................................................................. B.2.1.1 Evaluation Process .............................................................................B-7 B.2.1.2 Submittal to the Central Registry ......................................................B-1 B.2.2 Revisions to Software Applications in the Central Registry .............................B-12 B.2.3 Removal of Software Applications from the Central Registry.........................B- B.2.4 Issue Resolution and Action Communication....................................................B-13 B.3 REFERENCES ..............................................................................................................B- DOE G 414.1-4 APPENDIX B 6-17-05 B-3 PROCEDURE FOR ADDING OR REVISING SOFTWARE TO OR DELETING SOFTWARE FROM THE DOE SAFETY SOFTWARE CENTRAL REGISTRY B.1.1 PURPOSE manage Registr the Centr DOE O determine s also pre software from More detailed softwar http://w B.1 INTRODUCTION Toolbox codes represent a small number of standard computer models or codes supporting DOE safety analysis having widespread use and of appropriate qualification that are maintained, d and distributed by DOE’s Safety Software Central Registry (referred to as the Central y). The purpose of this appendix is to outline the procedure for adding new software to al Registry that is consistent with software quality assurance (SQA) requirements of 414.1C, Quality Assurance,1 Criteria are referenced for demonstrating compliance with applicable SQA requirements, and are recommended for use in an evaluation process to uitability of candidate software for inclusion in the Central Registry. Information is sented in brief on the procedures to (1) revise or update toolbox software and (2) remove the Central Registry due to retirement by the software developer. information of the SQA requirements and criteria that are applicable to safety e as a basis for consideration to the Central Registry is found at ww.eh.doe.gov/sqa/central_registry.htm. B.1.2 SCOPE safety-related purpose that is proposed for inclusion in the Central Registry. B.1.3 FUNCTIONS Procedures to identify, document and submit additional software applications to the Central Registry are based on the process followed to evaluate the six initial toolbox codes.2 Following this precedent, three principal entities described below perform the major tasks. Software Sponsor—either the originator of the software (developer) or the primary user (site organization) who is requesting the software to be placed in the Central Registry or a combination of the two. In either case, this party is responsible for documenting SQA programs, procedures and processes associated with development of the software, maintaining and configuration controlling the software, developing new versions of the subject software, addressing user questions, and resolving technical and programmatic issues. The software sponsor is responsible for documenting the rationale for adding the subject software to the Central Registry.

The scope of this procedure includes any software application used by DOE or its contractors for a 1 DOE O 414.1C, Quality Assurance, dated 6-17-05. 2 U.S. Department of Energy, Implementation Plan for Defense Nuclear Facilities Safety Board Recommendation 2002-1: Quality Assurance for Safety Software at Department of Energy Nuclear Facilities, Report, dated 3-13-03. APPENDIX B DOE G 414.1-4 B-4 6-17-05 SQA Evaluator—an independent reviewer of the computer software, who is not affiliated with the software developing organization. It is required that the review organization or individuals have a thorough understanding of the applicable SQA requirements, expert level knowledge and application experience with the software in question, and an awareness of the overall context for the use of the subject software as part of the DOE safety process. The SQA evaluator is responsible for documenting the SQA evaluation of the candidate software, and based on this evaluation, confirms that the software SQA satisfactorily meets requirements for inclusion to the Central Registry. DOE Office of Environment, Safety and Health, Office of Quality Assurance Programs— reviews the candidate software SQA evaluation and decides whether the candidate software should be included in the Central Registry. Independence between the evaluator and the sponsor is critical for completion of a formal SQA evaluation, and should be maintained throughout the Central Registry submittal process. Ideally, the two participants should be based out of different organizations. In addition, while the SQA evaluator and the sponsor can be collocated at the same site, they should be functionally separated. Before a software application and an independent evaluation are transmitted to DOE for consideration to the Central Registry, it is recommended that the software sponsor notify DOE Office of Quality Assurance Programs of its intentions. The notification will allow DOE to review usage characteristics of the software and the credentials of the designated evaluator. A screening review of this nature will minimize software and evaluation submittals that are not likely to be successful. B.2 PROCESS B.2.1 ADDING SOFTWARE APPLICATIONS TO THE CENTRAL REGISTRY Submittal of a software application for consideration as toolbox-equivalent is a two-phase documentation effort, consisting of strategic benefits and SQA technical basis phases. In principle, the first phase should be prepared by the software sponsor, and needs to establish the basis or rationale for including the software in the Central Registry. At minimum, the discussion in the first phase should establish the following. • Widespread use of the software across the DOE complex for safety related applications. • Methods to ensure proper software information, error reporting, configuration control and other SQA management interface with the Central Registry. • Demonstrated and quantifiable benefit for designating the software for the Central Registry. The second phase, the SQA technical basis phase, is initiated with completion of an independent SQA evaluation, and is performed by the software evaluator. The evaluation should demonstrate satisfactory compliance with toolbox-equivalency criteria and requirements established for the DOE G 414.1-4 APPENDIX B 6-17-05 B-5 DOE progr ct software. An input template for this purpose has been developed, and is recommended as a starting point mechanism to solicit b m the software sponsor. An electronic copy may be obtained from SQA Knowledge Portal under the SQA Library (Software

), http://www.eh.doe.gov/sqa/doc_library.htm Central Registry. The software sponsor is requested to provide information on the ams and procedures associated with the development, maintenance, and use of the subje asic SQA information fro Information Template . . Software design and related documents action reports anuals, and training packages/user qualification documents red because they compliance with the primary criteria. Furthermore, formal documents in verifying completion of an action. ssociated with lso Central Registry that best matches the DOE O 414.1C SQA work activity. For many of the SQA work activities, the match is only partial, (i.e., not all of a specific work practice is covered by the SQA toolbox requirement). The input template seeks the following set of documents from the software developer. 1. Software project management and SQA plans 2. Software risk management documents 3. Software configuration management plan 4. Procurement and supplier management documents 5. Software requirements specifications 6. Software design, model description, programmer’s reference, and related documents 7 8. Verification and validation, test report, and other documents 9. Software error notification and corrective 10. User instructions, user m Files, reports, telephone conferences, and other documented communications can provide confirmatory indications that actions have been performed in SQA, and these can be used in lieu of the availability of formal documents. However, formal documents are prefer explicitly demonstrate reduce the uncertainty Software practices discussed in Section 5 of this Guide and the corresponding documents for assessing compliance are listed in Table B-1, and are similar to those used in the evaluation of he initial software applications designated for the Central Registry. The details at each work activity are discussed in detail in the SQA plan and criteria document at the SQA Central Registry Web site.3 Because current and potential Central Registry software is best described under the custom developed category, requirements for evaluation of software should be consistent with the grading approach for custom developed software. Table B-2 lists SQA work activities discussed in Section 5 of this Guide for custom developed software at both A and B grading levels. A shown is the SQA requirement from those used to evaluate the initial software applications designated for the 3 U.S. Department of Energy, Software Quality Assurance Plan and Criteria for the Safety Analysis Toolbox Codes, Revision 1, dated 11-03. APPENDIX B DOE G 414.1-4 B-6 6-17-05 Table B-1. Software Quality Assurance Work Activity and Corresponding Documentation for Demonstrating Compliance DOE O 414.1C SQA Work Activity SQA Documents 1. Software Project Management and Quality Planning - Software Project Management Pl (SPMP) and/or an - Software Quality Assurance Plan (SQAP) - Software Safety Plan 2. Software Risk Management - Various document types can be used to cover risk management 3. Software Configuration Management - Software Configuration Management Plan (SCMP) or related documents 4. Procurement and Supplier Management - Contractual documents or other Software procurement and use agreement documentation 5. Software Requirements Identification and Management - Software Requirements Specifications (SRS) or related document 6. Software Design and Implementation odel Description, Programmer’s

Reference Manual, or other related - Software design description (SDD) M documents 7. Software Safety SDD - - Software Safety Analysis documentation 8. Verification and Validation - Verification and Validation Report - Test Case Description and Outcome Report; Other testing documents 9. Problem Reporting and Corrective Action - Software Error Notification and Corrective Action Report 10 ftware - Training Packages and User Qualification . Training of Personnel in the Design, Development, Use and Evaluation of Safety So - User Instructions or User Manuals DOE G 414.1-4 APPENDIX B 6-17-05 B-7 the and ation. Examples of alternative information are previous reviews, older documentation from the code developer, ne individual or a team of subject matter valuation size, those involved should be experienced he evaluation of the software work activities equately evaluate the constituent f compliance was used with the designated toolbox pliance conditions: Yes (meets requirement), No completion of the evaluation of each of the iew results as a whole and render an overall a verifiable, objective nner in is B.2.1.1 Evaluation Process The SQA evaluator performs and documents a review of the software, using the inputs from code developer, including the responses in the Software Input Template or the equivalent, other communications. In cases where the software developer is unable to supply requested inputs, the SQA evaluation may consider alternative sources of inform 4 technical and journal articles, and previous software comparison studies. The size of the actual SQA evaluation effort, whether o experts, depends on the complexity of the software application. Regardless of SQA e team in use of the software, but also knowledgeable of the evaluation criteria. It is recommended that t covered in Table B-1 use a sub-matrix of finer criteria to ad parts of the requirement. Qualitative ranking o software, applying the four terms defining com (does not meet requirement), Uncertain (insufficient information available to evaluate), and Partial (some but not all criteria are met). Upon SQA work activities, the SQA evaluator can rev assessment. The process leads to a firm basis to document findings in ma . Table B-3 contains a procedure for evaluating toolbox-equivalent candidate software, defined the custom developed category for most safety applications. The overall evaluation process shown schematically in Figure B-1. Input information for the evaluation is based on receipt of a Software Information Template. An electronic copy may be obtained from SQA Knowledge Portal under the SQA Library (Software Information Template), http://www.eh.doe.gov/sqa/doc_library.htm. While grading Level C cases can be postulated, it is believed that most software application candidates for the Central Registry are categorized best under grading Levels A and B. The SQA evaluation (gap analysis) reports performed on the six initial toolbox codes are a reasonable level of detail for SQA evaluation documentation. While the SQA requirements and criteria used for the toolbox codes are similar to those described in this Guide, they differ in ped. emphasis and extent of coverage. Thus, the gap analysis reports are illustrative, but not directly applicable models. Instead, a software evaluation template for this purpose has been develo

The toolbox-equivalent software input and evaluation templates, as well as, copies of the gap analysis reports and the full SQA evaluation plan and criteria document, can be downloaded from the Central Registry Web site (http://www.eh.doe.gov/sqa/central_registry.htm). 4 If previous reviews are used in whole or in part, it is required to confirm that the older review results are still applicable. APPENDIX B DOE G 414.1-4 B-8 6-17-05 Tab ng Leve ities le B-2. Software Quality Assurance (SQA) Requirements by Software Gradi l and Matching DOE O 414.1C SQA Work Activ Software Grading Level DOE O 414.1C SQA Work Activity* Level A Custom Level B Custom Corresponding SQA Toolbox Software Requirement* (a) Software project management & quality planning Full** Full 2. SQA Procedures and Plans (b) Software risk management Full Grade ot addressed in the list of SQA d*** N requirements. (c) Software configuration manage Configuration Control ment Full Full 12. 14. Access Control (d) Procurement and supplier manage Full Fu ment ll 3. Dedication (e) Software requirements identification and management FuFull ll 5. Requirements (f) Software design and implem Full Full entation; 6. Design 7. Implementation (g) Software safety Full Gr dea d 6. Design (h) Verification and validation Full Grade est enance d 8. Testing 10. Acceptance T 11. Operation and Maint (i) Problem reporting and corrective action Full Graded 13. Error Impact (j) iTra ning of personnel in the design, development, use and evaluation of safety software FuFull ll 9. User Instructions *The SQA requirements used for evaluation of the initial set of s tions designated for the Central Regi are m orresponding SQA work activi ro O 414.1C. See Table 3-3 of U.S. Dep re Quality Assurance Plan an r the Safety Analysis Toolbox Codes, Revision 1, (November 2003) for details on the requirement d e. **Required for the computer software *** d d based on judgm oftware applica stry atched to the c artment of Energy, Softwa ty f m DOE d Criteria fo s an the labeling (numbering) schem Gra ed depending on the application an ent of SQA evaluator. DOE G 414.1-4 APPENDIX B 6-17-05 B-9 Evaluation of Candidate Software for Central Registry Table B-3. Plan for Step Procedure 1. Review Documentation Determine that sufficient information is provided by the software developer to allow proper classification of the softwa Review developer reports, previous evaluations, and conference and journal submittals, etc. re. Interview software developer. 2. Evaluate Justification (Rationale) for Including Software in Central Registry Review software sponsor’s document: Widespread use of the software across DOE complex for safety related applications? Methods to ensure proper software information, error reporting, configuration control and other SQA management interfaces with the Central Registry? Demonstrated and quantifiable benefit for designating the software for the Central Registry? 3. Process Software Information Template Download template from Software Quality Assurance Web sit http://www.eh.doe.gov/sqa/doc_library.htm e, . Confirm graded level determination. 4. Assess Software Project Management and Software Quality Assurance Plans Review software project management plan (SPMP) and software quality assurance plan (SQAP) for— • required activities, documents, and deliverables and • level and extent of reviews and approvals, including interna

and independent review. Confirm that a l ctions and deliverables (as specified in the SQAP) have been completed and are adequate. d uments; • software error notification and corrective action reports; and • user instructions, user manuals, and training packages/user qualification documents. Review engineering documentation identified in the SPMP an SQAP, including— • software risk management documents; • software configuration management plan; • procurement and supplier management documents; • software requirements specifications; • software design, model description, programmer’s reference, and related documents; • software design and related documents; • verification and validation, test report, and other doc APPENDIX B DOE G 414.1-4 B-10 6-17-05 Step Procedure 5. Assess SQA Work Activity Review aga the Software Evaluation Tem late for DOE O 414.1C SQA Work A • software project management & quality planning, • software risk m ment, software configuration management, procurement and supplier ma • software requirements identi nagement, • software design and implem • software safety, verification and validation, • problem reporting and corrective action, and • tr of pers in the d t, use and evaluation of safety software. SQA documentation inst detailed criteria found in p ctivities: anage • • nagement, fication and ma entation, • aining onnel esign, developmen 6. Document Evaluation Using Software Evaluation Template. Use g alysis re as examap an ports ples. B.2.1.2 Submittal to the Central Registry O s been c cted and nted be submitted to t th wing ca 1 ncludes in the software evaluation (gap analysis) that software ite criteria in the ten SQA work activities, and no criterion is o 2 dentified one or more criteria not compliant for the subject or can document a compelling technical basis for submitting the software as “toolbox-equivalent” to the Central Registry. Part of the technical basis should include a software application guidance report that points out specific limitations and weaknesses and provides instructions to the user on informed use of the subject software despite the identified gaps and other vulnerabilities. Examples of guidance reports prepared for the initial six codes designated for the Central Registry may be downloaded from the DOE SQA Web site at http://www.eh.doe.gov/sqa/doc_library.htm nce the SQA evaluation ha ondu docume , the software may he Central Registry under one of . The software sponsor co has met all major requis e follo ses. evaluated as “No (=not met).” In other words, all significant improvement actions are completed before the software is submitted for consideration as “toolbox-equivalent” t the Central Registry. . The software sponsor has i software based on the gap analysis. However, the software spons . DOE G 414.1-4 APPENDIX B 6-17-05 B-11 e If ith ftware sponsor may move forward wi t message should be sent to sqa@eh Figur all substantive issues in e B-1. Flow Sheet for Software Evaluation er Case 1 or Case 2 are satisfactorily dispositioned, the so th he toolbox software submittal process. An electronic mail .doe.gov, requesting a review of the evaluation and designation of the software as should be transmitted as attach The DOE Office of Quality As Table B-4 lists several of the k on to include the candidate software in the Cent e candidate software as a toolbox software applicati w oper and evaluator organizations. If the decision i software in question, and a general noti w eb site. Additional notification methods may be im Central Registry software coll io

If, on the other hand, issues w t n the software sponsor is advised not to proce examine continued use of the s tw software, such as software curr application. a toolbox software application. All supporting documentation ments. surance Programs will review the submittal in a timely manner. aey cceptance criteria for rendering a decisi ral Registry. A decision on designation of th on ill be communicated to the software devel s favorable, the appropriate links will be provided for the ce ill be posted on the Central Registry W plemented to ensure broad notification of the changes in the ect n. ith he subject software are irreconcilable, the ed further with the submittal process. It may be prudent to of are at the site in question, and explore use of alternative ently contained in the Central Registry, for the specific safety 1. Review Sw umentation & Doc Interview Sw Developer • Software development reports • Previous Sw evaluations • Journal & conference documents 2. Evaluate Justification for Including Sw (or New Sw Version) in Central Registry 3. Process Software Information Template (Includes graded level) 4. Assess Software Project Management and Software Quality Assurance Plans 5. Assess Software Quality Assurance Work Activities Software configuration management ware design and implementation d validation (a) Software project management & quality planning (b) Software risk management (c) (d) Procurement and supplier management (e) Software requirements identification and management (f) Soft (g) Software safety (h) Verification an (i) Problem reporting and corrective action (j) Training of personnel in the design, development, use and evaluation of safety software 6. Document Outcome of Evaluation Software Evaluation Report nt r including • Compliant areas • Areas for improveme • Overall assessment fo software in Central R • Determine whether su Central Registry alon eport egistry itable for g with Code Guidance R APPENDIX B DOE G 414.1-4 B-12 6-17-05 Table B-4. Primary Criteria for Deciding on Inclusion of Software to the Central Registry Phase Criterion* 1. Rationale for Adding Software to Central Registry a. fety ns. b. reporting, rance entral Registry. c. esignating the Widespread use of the software across DOE complex for sa related applicatio Methods to ensure proper software information, error r software quality assuconfiguration control, and othe (SQA) management interfaces with the C Demonstrated and quantifiable benefit for d software to the Central Registry. 2. SQA Technical Basis a. ly demonstrates that are has met all major requisite criteria, and o (=not met).” If remedial tasks it is or b. entified one or more analysis. However, a compelling technical basis is made for submitting the software as “toolbox-equivalent” to the Central art of the technical basis should include a guidance oints out specific limitations, weaknesses, and The SQA evaluation document adequate the candidate softw n criterion is evaluated as “No were cited before all criteria are considered met, determined that these have been completed. The SQA evaluation document has id criteria not compliant for the subject software based on the gap Registry. P report that p provides instructions to the user on informed use of the subject software despite identified gaps and other vulnerabilities. *Th artial list—others may be added as the process for software addition matures.

REVISIONS TO SOFTWARE APPLICATIONS IN THE CENTRAL REGISTR ypical life-cycle processes associated with most software applications, updates, ements, and modificat is is a p B.2.2 Y In the t improv ions will be made. Similar to software that is being considered for irst tim r The sam outline 1. 2. 3. Upon conclusion of the evaluation and issuance of the SQA evaluation report (the gap analysis), the software sponsor decides whether software has satisfactorily met all requisite criteria for the ten SQA work activities, the revised software may be submitted to the Central Registry. the f e, revised software in the form of a new software version may also be submitted fo inclusion in the Central Registry, with accompanying removal of the older version. e process is followed for revised software to be placed in the Central Registry as is d above for new software applications. The steps may be summarized as follows. The software sponsor identifies the SQA evaluator organization. The evaluator performs a complete evaluation over all aspects of the new software version, emphasizing new and revised aspects of the software application. DOE G 414.1-4 APPENDIX B 6-17-05 B-13 4. As noted earlier for new software applications to the Central Registry, an electronic mail message should be sent to sqa@eh.doe.gov, requesting a review of the evaluation and designation of the software as a toolbox software application. All supporting documentation should be transmitted as attachments. 5. The DOE Office of Environment, Safety and Health, Office of Quality Assurance Programs will review the submittal and decide on designation of the candidate software as a replacement version to existing toolbox software. Upon reaching a favorable determination, the appropriate links will be provided for the software version, and a general notice will be posted on the Central Registry Web site regarding a new software revision. In parallel with this action, the older software version will be removed from the Central Registry and designated as an “archived toolbox version.” .2.3 REMOVAL OF SOFTWARE APPLICATIONS FROM THE CENTRAL REGISTRY oftware applications are also subject to being removed from the Central Registry. Several auses for this action include but are not limited to the following. . The software developer indicates that older versions will no longer be supported and elects to retire the software. . New survey information indicates that few if any sites are using the software and that other software applications . The DOE Office of the Web site for a comment period of 60 days, and no revision, and removal of software or when it is necessary ken B S c 1 2 are being used for the specified safety applications. Environment, Safety and Health, Office of Quality Assurance 3 Programs may make a decision to formally remove the software due to accumulated evidence of unsatisfactory SQA events. Significant software errors in the subject software or other factors may lead to this outcome. Regardless of the basis, the subject software application may be removed from the Central egistry after notification is posted onR compelling evidence is received that conflicts with the planned removal action. The notification should cite the basis or bases for the removal along with supporting documentation. Upon reaching the end of comment period, the software application is then removed from the Central Registry and designated as an “archived software application.”

B.2.4 ISSUE RESOLUTION AND ACTION COMMUNICATION Actions will be taken on the addition, to communicate information about software contained in the Central Registry. Several communication mechanisms may be used to alert DOE staff, DOE software user, and stakeholder groups. The extent of the communication will be commensurate to the action ta or the importance to safety of the issue, and will be decided by the DOE Office of Environment, Safety and Health, Office of Quality Assurance Programs. APPENDIX B DOE G 414.1-4 B-14 6-17-05 more of the following. 1 vir ffi .doe.gov/ tm Several of the primary mechanisms may include, but are not limited to, announcements to one or . DOE Office of En within the DOE O (http://www.eh onment, Safety and Health Central Registry Web site home page ce of Environment, Safety and Health Knowledge Portal sqa/central_registry.h ) 2. Safety Analysis Wor Accident Analysis Subg king Group in EFCOG, particularly its Steering Committee, and its roup (http://www.efcog.org/workgroups/sawg-aa/) 3. Radiation Safety Information Computational Center (RSICC), (http://www- rsicc.ornl.gov/rsicc.html) 4 R 5. Formal Letter Notificati 1. American Society of nal Nuclear Quality Assuran Oversight, Letter to Lint 2. ASME NQA-1-2000, Qu s, American Society of Me 3. ASME NQA-1A-1999, A f Mechanical ment of del 9. assurance standards—Part 3: Guidelines for the application of . Use of the Central egistry e-mail distribution list on to the Program Secretarial Officers B.3 REFERENCES Mechanical Engineers, Re: Comments on the Benefits of Natio ce Standards for NNSA and DOE Nuclear Activities and on F. Brooks, NNSA (2002). ality Assurance Requirements for Nuclear Facility Application chanical Engineers, 2001. ddenda to ASME NQA-1-1997, Quality Assurance Requirements for Nuclear Facility Applications, American Society o Engineers, 1999. 4. 10 CFR 830, Nuclear Safety Management. 5. 10 CFR 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants. 6. DNFSB/TECH-25, Quality Assurance for Safety-Related Software at Depart Energy Defense Nuclear Facilities, Defense Nuclear Facilities Safety Board Technical Report, January 2000. 7. DNFSB Recommendation 2002-1, Quality Assurance for Safety-Related Software, Defense Nuclear Facilities Safety Board, September 2002. 8. International Organization for Standardization (ISO)9001-1994, Quality Systems—Mo for Quality Assurance in Design, Development, Production, Installation and Servicing, ISO, 1994. ISO 9001-2000, Quality Management Systems—Requirements, ISO 9000-3, ISO Quality management and quality DOE G 414.1-4 APPENDIX B 6-17-05 B-15 (and B-16) 10. afety and Health, Designation of 11. ISO 9001:1994 to the development, supply, installation and maintenance of computer software. U.S. Department of Energy, Office of Environment, S Initial Safety Analysis Toolbox Codes, Memorandum to Linton Brooks, Defense Programs and Jessie Hill Roberson, Office of Environmental Management, March 28, 2003. DOE-STD-3009-94, Preparation Guide for U.S. Department of Energy Nonreactor Nuclear Facility Documented Safety Analyses, dated 7-94. DOE G 414.1-4 APPENDIX C 6-17-05 C-1 APP COMPLIANCE WITH DOE 10 CFR 830 SUBPART A AND DOE O 414.1C AND SAFETY SOFTWARE This ap Program for Nuclear Facilities, and supporting standards for compliance with the Department of Energy’s (DOE’s) quality assurance (QA) requirements (10 CFR 830 Subpart A and DOE

C.1. PURPOSE This guidance may be used by organizations adopting ASME NQA-1-2000 as a national consensus standard for development and implementation of a quality assurance program (QAP) that meets the DOE QA requirements and includes safety software within its scope. This appendix describes how ASME NQA-1-2000 addresses the DOE QA requirements and identifies DOE QA requirements that are not addressed by ASME NQA-1-2000. Selected standards from other standards bodies are included where emphasis or detail for safety software quality is necessary. C.2. INTRODUCTION DOE QA requirements for activities that affect, or may affect, quality, nuclear safety or other site specified criteria are established by 10 CFR Part 830 Subpart A, Quality Assurance Requirements. DOE also has equivalent requirements for all other federal and contractor activities in DOE O 414.1C. The DOE QA requirements and Guides are available for review at http://tis.eh.doe.gov/nsps/quality.html. The DOE’s objective of the QA Rule and Order is for organizations to establish effective integrated management systems (i.e., QAPs) for the performance of DOE nuclear-related work. The objective is accomplished through performance oriented QA criteria, coupled with appropriate technical standards to manage, perform, and assess work activities. The DOE Rule requires the use of voluntary consensus standards in the development and implementation of the QAP. The ASME NQA-1-2000 standard is a national consensus standard, and as indicated in DOE O 414.1C, ASME NQA-1-2000 or other national or international consensus standards that provide an equivalent level of quality assurance requirements as NQA-1-2000 must be used for providing the essential implementing methods for a QAP, including details for effective and reliable supporting processes and procedures, as presented in this subpart. C.3. DOE RULE AND ORDER GENERAL ADMINISTRATIVE QAP REQUIREMENTS The DOE Rule and Order include both administrative and regulatory quality requirements. Those administrative requirements relating to QAP approval authority, change control authority, and compliance should not be relevant to the scope of ASME NQA-1-2000. Other administrative quality related requirements that are relevant are addressed in Table C-1. ENDIX C. USE OF ASME NQA-1-2000 AND SUPPORTING STANDARDS FOR pendix provides guidance on the use of ASME NQA-1-2000, Quality Assurance O 414.1C, Quality Assurance, dated 6-17-05) and their application to safety software. APPENDIX C DOE G 414.1-4 C-2 6-17-05 C.4. DOE RULE AND ORDER QA CRITERIA The DOE Rule and Order include ten QA criteria that are used to develop and implement a QAP. Table C-2 identifies each of the ten DOE Rule and Order QA criterion and how they are addressed by the ASME NQA-1-2000, Part I requirements. Differences in the documents and topics that should be addressed independently of the ASME NQA-1-2000 criteria to meet the DOE criteria are described. In some cases, the ASME NQA-1 Part II, “QA Requirements for Nuclear Facility Applications,” and Part IV, “Non-mandatory Guidance in ASME NQA-1-2000,” are also appropriate to address the DOE requirements and describe how the QA criteria will be implemented. Table C-2 also includes selected standards from other standards bodies (IEEE and IAEA) where they add emphasis or detail for safety software quality. D O E G 414.1-4 A ppendix C 6-17-05 C -3 TABLE C-1 10 CFR 830 S d uubpart A, date Jan ary 10, 2001

§830.121 a P Quality Assur nce rogram DOE O 414.1C DOE Genera ql Re uirements (Summarized) AS R iME NQA-1-2000 equ rements Grad ( 7 Wher e, appro m t docu s h subm e i ion, nPart I, Introduct Requirement 1 a d Requirement 2 sing on activ manner con , however a is applied an ed Approach e appropriat ach to imple ment the basi it that docum 830. a co ent of t ntat ) ntractor must use a graded he requirements of this Part, e graded approach used, and on to DOE. provid r to y focu ities af h plic nts in a sistent m tanc ivity. t does all proach DOE describe roach d to meet t nt. t 3, 801. es for fecting with QAP a graded app quality and t the relative i The cited tex will need to documented Requiremen oach e ap por achieving quality b ation of requireme e of the item or act ow for a graded ap how the graded app he DOE requireme 4 provides grading -2 rela endix 2A tive to software. Part II, App Nonmandatory idance on this topi Guida ty A cludes gu c. , 101 nce on Quali ssurance Programs in Part IV, 4.1 nical Rep 97, ApIAEA Tech ort (TR) Series 3 pendix 1 QAP Development & Implementation how the DOE QA criteria are The s satisf NQA-1 re ly m t 2 The ASME Requiremen quirements partial eet the DOE requirement. QAP must de ied. cribe requi ted QA d, and mai es the for th accompl es affe t 5 res that a documen ntained; and requir ishment of activiti P be planne QAP provide cting quality. implemented planning and Requiremen e requi affect d serv escrib ocum procedur t inclu ce ri uantitati ceptan r deter es ed results ctorily atta OE will need t b w the DOE crit are satisfied res that “Activities ed by and performed in acco es, or drawings tha ve or qualitative ac have been satisfa o descri e ho ing quality an rdance with d de or referen ce criteria fo ined.” eria ices ented mining . should be pr instructions, approp ate q that pr crib A D QAP A ppendix C D O E G 414.1-4 C -4 6-17-05 TABLE C-1 10 CFR 830 Subpart A, dated January 10, 2001 §830.121 Quality Assurance Program DOE O 414.1C DOE General Requirements (Summarized) ASME NQA-1-2000 Requirements Integrated Management Systems The QA program must integrate the QA criteria with the Safety Management System (SMS) or describe how the QA criteria apply to the SMS. The ASME NQA-1 requirements do not e A DOE QAP will need to address integr r address the DOE requirem ation to meet the DOE crite nt. ion. Ensuring Subcontractor & Supplier Quality The QAP must describe how the contractor responsible for the nuclear facility ensures that subcontractors and suppliers satisfy the QA criteria. Requirements 1, 2, 4, 7 and 18 The ASME NQA-1 requirements meet t establishment of quality interfaces betwe inclusion of applicable QA requirements , supplier evaluation activities and audits A DOE QAP will need to describe how i the DOE criteria. he DOE requirement by the en organizations, by the in procurement documents of suppliers. subcontractors/suppliers sat sfy D O E G 414.1-4 A ppendix C 6-17-05 C -5 TABLE C-2 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria DOE Quality Assurance Criteria ASM Comments, Software RequiremeE NQA-1-2000 Requirements nts & Other Standards (documents noted in bold italic) Criterion 1 - Management/Program NQA Requirements 1 and 2 Part IV, 4.1, 400 The ASME NQA-1 requirements meet the DOE criterion, as noted. IEEE 730-2002 IAEA Nuclear Safety Guide

IAEA TR 397, 2.2 (NS-G) NS-G-1.1, 4.11 (1) Establish an organizational structure, functional responsibilities, levels of authority, and interfaces for those managing, performing, and The ASME NQA-1 r the DOE criterion. assessing work. equirements satisfy this element of None (2) Establish management processes, including planning, NQA Requiremen Basic meet the D scheduling, and providing resources for the work. t 1, OE senior management to effective implementatio and is responsible for This implies that adeq desired results. to describe 201 General and Requirement 2, 100 criterion. ASME NQA-1 requires establish overall expectations for n of the quality assurance program A DOE QAP will need the management process for providing resources. obtaining the desired end result. uate resources are provided to obtain A ppendix C D O E G 414.1-4 C -6 6-17-05 TABLE C-2 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other ASME N irements QA-1-2000 RequCriteria Standards (documents noted in bold italic) Criterion 2 - Management/Personnel Training and Qualification NQA Requirement 2 The ASME NQA-1 requirements meet the DOE criterion. (1) Train and qualify personnel to be capable of performing their The ASME the DOE cr assigned work. (2) Provide continuing training to personnel to maintain their job proficiency. NQA-1 r of iterion. DOE-STD-1172-2003 Safety Software Quality Assurance 2.4 equirements satisfy these elements Functional Area Qualification Standard IAEA TR 397, IAEA NS-G-1.1, 4.9&10 Criterion 3 - Management/Quality Improvement NQA Requirements 2, 15, and 16 The ASME NQA-1 requirements partially meet the DOE criterion. Part II, 2.7, 204 Part IV, 4.1, 204 IAEA TR 397, 2.5 A DOE QA Program will need to extend the requirements of ASME NQA-1 to ALL conditions adverse to quality not just significant conditions adverse to Quality. TABLE C-2 D O E G 414.1-4 A ppendix C 6-17-05 C -7 10 CFR 830 Subp anuary 10, 2001 art A, dated J §830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other Standards ASME NQA-1-2000 Requirements Criteria (do lic) cuments noted in bold ita (1) Establish and implement etect and prevent The ASME NQA-1 requirements partially meet the DOE ming conditions from causing quality problems. his is accomplished through various controls, inspections, and tests. Requirement 16 includes criteria to prevent recurrence of identified problems. processes to d quality problems. criterion. ASME NQA-1 provides a system of establishing quality requirements and monitoring compliance to prevent nonconfor T (2) Identify, control, and correc items, services, and p do not meet established requirements. t rocesses that The ASME NQA-1 requirements satisfy this element of the DOE criterion. (3) Identify the causes of problems and work to prevent ting The ASME NQA-1 requirements partially satisfy this element of the DOE criterion for “significant” or “generic” recurrence as part of correc the problem. nonconformances. (4) Review item characteris process implem tics, entation, and other quality-related information to identify items, services, and processes needing improvements. The NQA requirements partially address this element of the DOE criterion for known deficiencies. A ppendix C D O E G 414.1-4 C -8 6-17-05 TABLE C-2 10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other ASME NQA-1-2000 Requirements Criteria Standards (documents noted in bold italic) Criterion 4 - Management/Documents and Records NQA Requirements 5, 6 and 17 The ASME NQA-1 requirements meet the DOE criterion. (1) Prepare, review, approve, establish design. The ASME NQA-1 requirements satisfy these elements of Part I, Requirement 3, 801 397, 2.6 & 3.1 issue, use, and revise documents to prescribe processes, specify requirements, or (2) Specify, prepare, review, approve, and maintain records. the DOE criterion. Part II, 2.7, 201 & 802 Part IV, 4.1, 201 IAEA TR IEEE 730, 829 Criterion 5 - Performance/Work NQA Requirements 5, 8, 9, 12, 13, and 14 and the Part I, Introduction The ASME NQA-1 requirements meet the DOE criterion, as noted. Processes (1) Perform work consistent with technical standards, administrative controls, and other hazard controls adopted to meet regulatory or contract requirements, using approved instructions, procedures, or other appropriate means. The ASME NQA-1 requirements address “work” as activities affecting quality. (2) Identify and control items to ensure their proper use. The ASME NQA-1 requirements satisfy this element of the DOE criterion. will need to Part I, Requirement 3, 802 Part II, 2.7, 203 & 404 A DOE QA program address “work” as broadly as the DOE criterion, since the requirements for “work” are derived from multiple sources in the DOE Rule and Order. TABLE C-2 D O E G 414.1-4 A ppendix C 6-17-05 C -9 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other ASME NQA-1-2000 Requirements Criteria Standards (documents noted in bold italic) (3) Maintain items to prevent damage, loss, or deterioration. their The ASME NQA-1 requirements satisfy this element of the DOE criterion. (4) Calibrate and maintain equipment used for process monitoring or data collection. The ASME NQA-1 requirements satisfy this element of Part IV, 4.1, 203 & 405 IAEA TR 397, 3.1 & 3.2 IEEE 828-1998 & 1219-1998 the DOE criterion. Criterion 6 - Performance/Design NQA Requirement 3 The ASME NQA-1 requirements meet the DOE criterion. (1) Design items and processes c applicable asis in s. . approval and implementation of the design. ents of Part II, 2.7, 401 & 402 Part IV, 4.1, 401 & 402 ANS-10.4 IAEA TR 397, 3.2 & 3.4 IEEE 1012-1998 & 1012A-1998 using sound engineering/scientifi principles and appropriate standards. (2) Incorporate requirements and design b design work and design change (3) Identify and control design interfaces. (4) Verify or validate the adequacy of design products using individuals or groups other than those who performed the work (5) Verify or validate work before The ASME NQA-1 requirements satisfy these elem the DOE criterion. A ppendix C D O E G 414.1-4 C -10 6-17-05 TABLE C-2 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other ASME NQA-1-2000 Requirements Criteria Standards (documents noted in bold italic) Criterion 7 - Performance/Procurement NQA Requirements 4 and 7 The ASME NQA-1 requirements meet the DOE criterion. (1) Procure items and services that and processes to ensure that approved s and services. The ASME NQA-1 requirements satisfy these elements of Part II, 2.7, 300 meet established requirements perform as specified.

(2) Evaluate and select prospective suppliers on the basis of specified criteria. (3) Establish and implement suppliers continue to provide acceptable item the DOE criterion. Part IV, 4.1, 300 IAEA TR 397, 3.3 Criterion 8 - Performance/Inspection and Acceptance Testing NQA Requirements 8, 10, 11, and 12 The ASME NQA-1 requirements meet the DOE criterion. (1) Inspect and test specified items, services, and processes using established acceptance a performance criteria. (2) Calibrate and maintain equipment used for inspections nd ents satisfy these elements of the DOE criterion. IAEA TR 397, 3.4 IEEE 1008 and tests. The ASME NQA-1 requirem Part II, 2.7, 404 Part IV, 4.1, 404 ANS-10.4 TABLE C-2 D O E G 414.1-4 A ppendix C 6-17-05 C -11 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria Comments, Software DOE Quality Assurance Requirements & Other ASME NQA-1-2000 Requirements Criteria Standards (documents noted in bold italic) Criterion 9 - Assessment/Management Assessment NQA Requirement 2 and 18 The ASME NQA-1 requirements partially meet the DOE criterion, as noted Ensure managers assess their management p identify and correct pro hinder the organization from rocesses and blems that While ASME NQA-1, Requirement 2, 100 Basic, requires y assess the adequacy and effective nce, the DOE criterion is Part II, 2.7, 202 art IV, 4.1, 202 IAEA TR 397, 4.1 of NQA provide an input to this will need G 414.1-1A, Management Assessment and Independent Assessment Requirements of 10 CFR 830.120 and DOE O- 414.1, Quality Assurance, to meet the DOE criterion. achieving its objectives. management to regularl implementation of quality assura broader in scope and intent. P IEEE 1028-1997 While audits per Req. 18 requirement, a DOE QAP to align with the intent, focus, and concepts described in DOE A ppendix C D O E G 414.1-4 C -12 6-17-05 TABLE C-2 10 CFR 830 Subpart A, dated January 10, 2001 §830.122 Quality Assurance Criteria DOE Quality Assurance Criteria ASME NQA-1-2000 Requirements Comments, Software Requirements & Other Standards (documents noted in bold italic) Criterion 10 - Assessment/Independent Assessment NQA Requirements 1, 2, 10, 11, 15, 16, and 18 The ASME NQA-1 requirements meet the DOE criterion. (1) Plan and conduct in to measure item and service quali m dependent assessments ty, to easure the adequacy of work performance, t. and freedom group sments. m in alifi e areas to be assessed. DOE defines assessment as a general term that includes a variety of evaluation methods (i.e.; reviewing, evaluating, inspecting, testing, checking, surveillance, auditing, or otherwise determining and documenting). As such, several ASME NQA-1 requirements may be necessary to address the various DOE independent assessment methods. These with the NQA Assessment as a DOE activity for a DOE QAP will need to align with the intent, focus and concepts described in DOE G-414.1-1A, Management Assessment and Independent Assessment Requirements of 10 CFR 830.120 and DOE- O-414.1 Quality Assurance. and to promote improvemen (2) Establish sufficient authority, from line management, for the performing independent asses (3) Ensure persons who perfor assessments are technically qu knowledgeable in th dependent activities when combined ed and corrective action requirement have the intent of the DOE criterion, to “promote improvement.” IAEA TR 397, 4.2 DOE G 414.1-4 APPENDIX D 6-17-05 D-1

APPENDIX D. QUALITY ASSURANCE STAN R SAFETY SOFTWARE IN DEPARTMENT OF ENERGY NUCL ACILITI D.1. INTRODUCTION AND R The Department of Energy (DOE) stablishes quality assurance requirements for n may affect nuclear safety of DOE n requirement that consensus standar software is included in the scope o i d ensus standards should be used for apply le and consistent with contractual and regulatory requirements. This appendix describes practicable standards for safety softw QA th used to satisfy the QA Rule. D.2. REGULATORY AND QA PROGRAM COMPLIANCE The ultimate responsibility for com th the QA Rule, and for selecting standards for safety software that falls er the contractor. Nuclear facil ontractors with DOE-approved QA Programs should ensure that any changes to their QA Pro are m cordance with the QA Rule and any supplemental DOE direction prov ugh co means. D.3. QA PROGRAM STANDARDS VERSUS SOFTWARE STANDARDS Dozens of consensu hav oped that address every aspect of software. In the broadest sense of Q l se s ld be interpreted as “QA standards.” To develop a useful report, it is lim n of standards to those that directly support compliance with the DOE QA Rule and development of a QA Program that includes safety software. There are other documents (e.g., technical reports, agency directives, and industry guides) that may be useful as exam developed through an accredited consensus standards process. D.4. NDAR Many of the standards loped a phases of software development rather than a QA program that enco sses safe some cases the standards do cover a single criterion within the QA program, such as training. Where this type of standard is used, it should be in the context of r QA ncludes all criteria necessary for effective QA. This report will diff etwe standards and standards that address a specific criterion. DARDS FO EAR F ES EGULATORY BASIS ulation, 10 CFR 830 Subpart A, e g providing items or services that affect or lop and implement QA Programs. Safety by the QA Rule. Therefore cons oftware activities where practicab nucl activ ucle ds b f act ing Q ear s ities, inc ar fa e use vitie A t afety cilities. The Quality Assurance (QA) Rule includes a d to s co o saf reg ludi deve vere ety s are und ity c gram thro ndards l of the ssary to at may be plying wi scope of the QA Rule, rests with the nuclear facility ade in ac ntractual e been devel tandards cou it discussio ples for application of the standards, but they are not ided s sta A, a nece STA deve mpa the broade erentiate b DS USE IN A QA PROGRAM CONTEXT ddress specific ty software. In program that i en QA program APPENDIX D DOE G 414.1-4 D-2 6-17-05 D.5. QA P G AND SOFTWARE I A RD REQUIREMENTS Identification of QA program standards for saf ider the following: • co th the DOE QA Rule; • rel lear facility safety • applicability to software develope h ed; • appli to entire software li y • inclusion of monly accepted w a . NA NA AN LEAR CILITY Q LIT Th omp nsive lear pro afety software is the Am Soci f Me ical inee Assurance Program for Nuclear Facilities. A dix f thi t are compatible OE Q ule, be in ated rds, and is directly to sa softw Mo portantly, ASME NQA-1-2000 expands upon the DOE m re emen spe ally e quality, thus, placing ware quality in con th fic software quality • ME -1, P , Re e • t II, Subpart 2 l r oftware for Nuclear ility Applications; and • Part IV, Subpart 4.1, Guide on Qu A ran equirem s for Software.

ASME NQA-1-2000 with Subpart 2.7 is also a practicable choi enting the DOE QA Rule for safety software because it— • is easily supplem d with other IAEA, IEC, a t • provides independence for develo r • support ded i mentation; • is wide ed am DOE contra g • is accredited as the American Nat d a on. Table C-1 in A dix C Guide de A A ligns with DOE QA criterion and includes other standards that n e E NQA-1 requirements for safety software. RO mpatibility evance to cable RAM wi nuc the com QUAL TY ST NDA ety software should cons ouse, purchased, or modifi cle; and ctivities for software QA. ; d in- fe-c ork D.6 rehe ety o A R fety quir NQA TIO nuc chan ppen can are. ts to the art I .7, Qua L ST UA QA Eng C o tegr st im cific text of quirem ity Assu DARD FOR NUC FA Y AND SOFTWARE gram standard for application to s rs ASME NQA-1-2000, Quality s Guide includes SQA requirements tha /supplemented with other standa address requirements for softwar e overall QA program. These speci nt 3, Section 800, Design Control; ance Requirements for Computer S e m eri h th lica pr ety uire ost c can e D ble ogra soft ments are discussed in— AS Par Fac wit app QA saf req ality ssu pment and ve ctor QA pro ional Standar scribes how further expa ce R ce nd IEEE s ification; rams; and for nucle SME NQ d the cont ent for implem andards; r applicati -1-2000 a nt of ASM ente mple ong of this s gra ly us ppen DOE G 414.1-4 APPENDIX D 6-17-05 D-3 I Computers in Saf omputer specific requirements addressing firmware, software, and hardware alike for the development process in an integr f functional and design requirements for computer components of a safety system employed in nuclear power ng as s D.7. IN ARE including software. The requirements and guidance for nuclear facility quality are addressed in a 1996 Safety Series “Code” No. 50-C-Q, Quality Safety G parallel the DOE QA Rule. t to d IAEA programs for safety software, • Appendix • Appendix III, considerations before acquisition of computerized tools; nstitute of Electrical and Electronic Engineers (IEEE) Std 7-4.3.2-20031, Criteria for Digital ety Systems for Nuclear Power Generating Stations, describes c ated approach. This standard recommends a minimum set o generating stations. Additionally IEEE Std 1228, Standard for Software Safety Plans provides requirements for the development of a management plan and performance of safety software activities. American Nuclear Society (ANS) standard, ANSI/ANS-10.4-19872 is supplemental to the IEEE Std 7-4.3.2-2003 since it targets activities to improve the reliability of scientific and engineeri computer applications while mitigating the risk of incorrect applications. Additionally IEEE h a complete series of software and systems engineering standards to provide detail requirement and guidance for the development of safety software. TERNATIONAL STANDARDS FOR QUALITY AND SOFTW D.7.1 INTERNATIONAL ATOMIC ENERGY AGENCY (IAEA) The responsibility for international standards for nuclear safety is assigned to the International Atomic Energy Agency (IAEA). The IAEA has a significant number of standards, guides, and requirements for all aspects of nuclear facility safety, Assurance for Safety in Nuclear Power Plants and other Nuclear Installations, and uides 50-SG-Q1–Q14, respectively. The IAEA Code quality requirements closely

IAEA safety software guidance is detailed in Technical Reports (TR) Series No. 397, Quality Assurance for Software Important to Safety. This TR provides information and guidance for defining and implementing QA programs covering the entire life-cycle of software importan safety. TR 397 was developed using a large amount of available information and standards an offers implementation guidance that is tied to the QA program requirements found in the Code. The application guides are useful aids for developing QA specifically: I, illustration of a graded software quality assurance program; • Appendix IV, functions of computer program understanding and reverse engineering tools; 1 IEEE Std 7-4.3.2-2003, IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations, Institute of Electrical and Electronic Engineers, 2003. 2 American National S for the Verification a tandards Institute (ANSI)/American Nuclear Society (ANS) 10.4-1987 (R1998), Guidelines nd Validation of Scientific and Engineering Computer Programs for the Nuclear Industry, ANS, 1998. APPENDIX D DOE G 414.1-4 D-4 6-17-05 • • Appendix VII, characteristics of defect prevention process; ign input documentation for monitoring, control, onitoring, d procedures handbooks applicable to • Appendix XII, recommendations on the content of software requirements specifications for moni • Appendix XIII, recomm iptions for monitoring, control, and safety system software; ndations on programming of monitoring, control, and safety • ations on verification reports and activities for monitoring, • Appendix XX, recommendations on commissioning monitoring, control, and safety TR 397, and IAEA Safety Guide (SG) Series No. NS-G-1.1, Software for Computer Based ormation that can be e DOE QA Rule to produce an ir relationship to the DOE QA Rule criteria and ASME e for f Nuclear Power Stations, IEC 987 Programmed Digital Appendix V & VI, general training guideline and proposed outlines for training; • Appendix VIII, examples of software development life-cycle models; • Appendix IX, recommendations for des and safety system software; • Appendix X, recommendations for software development plans applicable to m control, and safety system software; • Appendix XI, recommendations for standards an monitoring, control, and safety system software; toring, control, and safety system software; endations on software design descr • Appendix XIV, recommendations on design and development documents for design, engineering, and analysis software; • Appendix XV, recommendations on application documents for design, engineering, and analysis software; • Appendix XVI, suggested good coding practices for design, engineering, and analysis software; • Appendix XVII, recomme system software; • Appendix XVIII, discussion of verification and validation methods; Appendix XIX, recommend control, and safety system software; and system software. Systems Important to Safety in Nuclear Power Plants, provide expanded inf fully integrated with the ASME NQA-1-2000 requirements and th effective quality program for safety software. Relevant portions of TR 397 are referenced in Appendix C of this Guide to illustrate the NQA-1 requirements. D.7.2 INTERNATIONAL ELECTROTECHNICAL COMMISSION (IEC) The IEC is responsible for several software standards in the nuclear power plant arena. These standards are referenced in the IAEA TR 397. Those standards include IEC 880 Softwar Computers in the Safety Systems o

DOE G 414.1-4 APPENDIX D 6-17-05 D-5 omputers Important to Safety for Nuclear Power Stations, IEC 1226 Nuclear Power Plants— 7 s and, to maximize the reliability of the safety systems within a nuclear power plant. O is not chartered to develop standards for nuclear safety applications (this is the domain of the IAEA) and consequently lacks sufficie that face high hazards and high mission/political risk similar to DOE (e.g., aerospace, telecom, O 9001 for application AEA d standards body for that subject, and (3) the IAEA offers safety ftware quality standards compatible to DOE and ASME NQA-1, ISO should not be considered , g systems” of Level A applications. This standard recommends particular detailed outputs for the software life-cycle processes, but is not prescriptive in how the outputs should be C Instrumentation and Control Systems Important for Safety—classification, IEC 61508 Parts 1- Functional Safety of Electrical/Electronic/Programmable Electronic Safety Related System IEC 61511 Parts 1-3 Functional Safety – Safety Instrumented Systems for the Process Industry Sector. The IEC Standard 8803 is applicable to Level A highly reliable safety systems of nuclear power plants. Like its Canadian counterpart, CE-1001-STD, IEC 880 standard advises various approaches D.7.3 INTERNATIONAL ORGANIZATION FOR STANDARDIZATION (ISO) The International Organization for Standardization (ISO) is responsible for ISO 9001-2000, Quality management systems – requirements. The ISO 9001 standard is designed for use internally or as a contractual requirement for generic quality systems. ISO 9001 does not specifically address computer software. More importantly, IS nt focus (and rigor) to address DOE nuclear facility hazards. Commercial industries chemical) have each issued supplemental requirements to improve on IS to their industry. Although ISO has a guide for applying a previous version of ISO 9001 (1994) to software (ISO 9000-3, ISO Quality management and quality assurance standards—Part 3: Guidelines for the application of ISO 9001:1994 to the development, supply, installation and maintenance of computer software), this guide is not focused on nuclear safety. Given that (1) the ISO standards are not developed for nuclear facility applications, (2) the I is the internationally chartere so a practicable choice for standards in this subject area. D.7.4 ATOMIC ENERGY CANADA LTD (AECL) The Canadian standard4 CE-1001-STD specifically recommends a minimum set of processes for the software quality engineering of “safety critical systems used in real-time protective, control and monitorin obtained.5 3 IEC 880, Software for Computers in the Safety Systems of Nuclear Power Stations, International Electrotechnical Commission, 1986. d Ontario Power Generation, Inc., 5 Computer Society, 2000. 4 CE-1001-STD, Rev. 2, Standard for Software Engineering of Safety Critical Software, CANDU Computer Systems Engineering Centre for Excellence, Atomic Energy of Canada Ltd. an 1999. Herrmann, Debra S., Software Safety and Reliability: Techniques, Approaches, and Standards of Key Industrial Sectors, IEEE APPENDIX D DOE G 414.1-4 D-6 6-17-05 REQUIREMENTS AND PROCEDURES hese cume n, maintenance, ampl (VV&A ide. The guide also describes methods for assuring software • been previously accredited based on verification and validation data which is available; • • tion and validation available. implem efinitions, roles and responsibilities for an e fety

Safety G sses to establish and implement standards and procedures. es the DO ing 40 CFR 194. The 40 CFR 194 Rule and the IPP Q E comple contracts for cleanup of certain Superfund sites. For these projects EPA has used the national s D.8. EXAMPLE APPLICATION GUIDES, FEDERAL AGENCY D.8.1 DEPARTMENT OF DEFENSE (DOD) The DoD software project requirement is Directive 5000.61 and related guidance. T do nts address software development, verification, validation, accreditatio review, and management. The documents also refer to national and industry standards. For ex e, independent review is addressed in the Verification, Validation and Accreditation ) Recommended Practices Gu using a graded approach depending on whether the software has— been previously accredited based on historical use; • not been previously accredited, but some verification and validation data available; or not been previously accredited, with little or no verifica DoD MIL-STD-882D6 Appendix A is particularly useful because it supplies guidance for enting a system safety effort, and the d organization undertaking a new system safety effort. Similarly, the NASA Standard “Softwar Sa NASA Technical Standard”7 provides general guidance for a software safety effort. D.8.2 NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA) The NASA software document is the NASA 8739.8, Standard for Software Assurance. Also reference NASA STD 8719.13B, Software Safety; and NASA-GB-8719.13 NASA Software uidebook. The NASA standard includes proce requirements and procedures, as well as, evaluating software products against requirements D.8.3 ENVIRONMENTAL PROTECTION AGENCY (EPA) The EPA uses at least two standards for software in environmental safety projects. EPA regulat E Waste Isolation Pilot Project (WIPP) us W A program requirements influence many other waste generation sites across the DO x. The regulation adopts ASME NQA-1, 1989, and NQA-2A-1990 Part 2.7. EPA also standard Quality Systems for Environmental Data and Technology Programs - Requirement with Guidance for Use, ANSI/ASQC E4-1994. This standard is currently undergoing revision and includes requirements for software quality that parallel ASME NQA-1-2000. The standard also parallels the DOE QA Rule criterion. 6 DoD MIL-STD-882D, Standard Practice for System Safety, U.S. Department of Defense, dated 2-10-00. 7 NASA-STD-8719.13A, Software Safety, National Aeronautics and Space Administration, 1997. DOE G 414.1-4 APPENDIX D 6-17-05 D-7 . PRACTICABLE STANDARDS FOR DOE QA RULE IMPLEMENTATION The tables in Appendix C describe how ASME NQA-1-2000 aligns with DOE QA criterion. It ents for safety software to address appropriate elements for safety software quality. are A-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, in A Geologic Repository at Yucca Mountain, Nevada. ty Systems, Defense Nuclear Facilities 7. DOE Letter and Report, Quality Assurance for Safety-Related Software at Department of D.8.4 DOE PROGRAM REQUIREMENTS AND PROCEDURES The Department and its contractors have a variety of program requirement documents and implementing procedures for safety software in use for nuclear facilities. However, the Yucca Mountain Project’s quality assurance requirements document (QARD) DOE/RW-0333P has been evaluated by an external regulatory body and found acceptable. The QARD and software quality supplements describe a rigorous graded approach to safety software suitable for review by other DOE organizations for use in developing their QA programs for safety software.

D.9 D.9.1 QA RULE & STANDARDS ALIGNMENT also includes other standards that further expand the content of ASME NQA-1 requirem D.9.2 STANDARDS LISTING Appendix G of this Guide contains a listing of standards that may be applied to safety softw to ensure quality. D.10. REFERENCES . ASME NQ1 American Society of Mechanical Engineers, 2001. 2. 10 CFR 830, Nuclear Safety Management. 3. 10 CFR 63, Disposal of High-Level Radioactive Wastes 4. DNFSB/TECH-25, Quality Assurance for Safety-Related Software at Department of Energy Defense Nuclear Facilities, Defense Nuclear Facilities Safety Board Technical Report, January 2000. 5. DNFSB/TECH-31 Engineering Quality into Safe Safety Board Technical Report, March 2001. 6. DNFSB Recommendation 2002-1, Quality Assurance for Safety-Related Software, Defense Nuclear Facilities Safety Board, September 2002. Energy Defense Nuclear Facilities, DOE Response to TECH-25, U.S. Department of Energy, October 2000. APPENDIX D DOE G 414.1-4 D-8 6-17-05 ommendation 2002-1: Quality Assurance for Safety Software at Department of Energy Nuclear Facilities, U.S. Department of Energy, March 13, 2003. 8. DOE Report, Implementation Plan for Defense Nuclear Facilities Safety Board Rec DOE G 414.1-4 APPENDIX E 6-17-05 E-1 APPENDIX E. SAFETY SOFTWARE ANALYSIS AND MANAGEMENT PROCESS The following diagrams provide a recommended process flow for the analysis and management of safety software applications. Note: everything above dotted line must be done per DOE-STD 3009-94 Apply Level A grading practices Develop documented safety analysis in accordance with DOE- STD-3009-94 and DOE- STD-3011-94 Graded safety application– E C using DO O 414.1 Apply Level B grading practices Apply Level C grading practices Safety software analysis process Elicit & analyze software requirements Review documented report) safety analysis (safety analysis 1 11 N Y Typical organizational software policies apply Does the software meet the definition of safety software? Does the software meet the definition of safety software? APPENDIX E DOE G 414.1-4 E-2 6-17-05 Are performance or quality requirements required for safety software application? N Define performance and/or quality requirements Y Y N Are data structures or database design important for safety software application? Ensure data integrity with fault tolerant data designs Is safety software task an upgrade to existing software? Y Assess impact of new maintenance safety software requirements upon existing application N 2 1 DOE G 414.1-4 APPENDIX E 6-17-05 E-3 (and E-4) Does a product safet verification and validation (V&V) strategy exist? y Does the strategy conform to Level A (or B e) software? DOE G 414.1-4 or C if appropriat Verif softw y and validat are functionality e safety Does software conform to requisite safety goals? Y Develop safety software V&V strategy N A Notify software developer of safet software deficiencies for corrective action. y N Y Y N Determine required safety software quality assurance activities and document in gap analysis A B Place base and softwa configuratio line documents re under n control. Safety software is ready for use End safety software analysis process B 2 Develop strategy to eliminate deltas noted in gap analysis DOE G 414.1-4 APPENDIX F 6-17-05 F-1 (and F-2) DOE G 414.1-4 APPENDIX F 6-17-05 F-1 (and F-2) F-3 F-3 F-4 F-7 F-7 F-8 F-9 F-10 F-14 F-18

APPENDIX F. DOE O 414.1C CRITERIA REVIEW AND APPROACH DOCUMENT CONTENTS F.1 INTRODUCTION ........................................................................................................... F.2 PURPOSE AND SCOPE................................................................................................. F.3. GUIDING PRINCIPLES................................................................................................. F.4. ASSESSMENT METHODOLOGY................................................................................ F.5. CRITERIA AND APPROACH ....................................................................................... F.5.1 Software Project Management and Quality planning .................................... F.5.2 Software Risk Management........................................................................... F.5.3 Software Configuration Management.......................................................... F.5.4 Software Procurement and Supplier Management ......................................F-11 F.5.5 Software Requirements Identification and Management.............................F-12 F.5.6 Software Design and Implementation..........................................................F-13 F.5.7 Software Safety............................................................................................ F.5.8 Software Verification and Validation ..........................................................F-15 F.5.9 Software Problem Reporting and Corrective Action ...................................F-16 F.5.10 Training Personnel in the Design, Development, Use, and Evaluation of Safety Software .....................................................................F-17 F.6. REPORT FORMAT....................................................................................................... DOE G 414.1-4 APPENDIX F 6-17-05 F-3 DOE O 414.1C CRITERIA REVIEW AND APPROACH DOCUMENT This document contains software qualific t criteria and guidelines for assessing the safety system software used in Department of Energy (DOE) defense nuclear facilities. • • . oftware includes safety analysis; design of structures, systems, and components (SSCs); human- machine interface software; network interface software; programmable logic controller (PLC) programming language software; and safety management software in DOE defense nuclear facilities. DOE O 414.1C includes the definitions for safety software. The assessment criteria and guidelines ensure that the software being used in DOE’s nuclear facilities is adequate. The primary set of baseline SQA criteria for evaluating safety software are based on the following: • 10 CFR 830, Nuclear Safety Management; • DOE O 414.1C, Quality Assurance, dated 6-17-05; • American Society of Mechanical Engineers (ASME) NQA-1-2000, Quality Assurance Program for Nuclear Facilities; and • applicable Institute of Electrical and Electronics Engineers (IEEE) standards. F.1 INTRODUCTION ation assessmen This document is organized as follows. • The Assessment Guidelines section covers the purpose, scope, guiding principles, and assessment methodology for assessing the processes currently in use for ensuring the adequacy of safety software.

• The Criteria and Approach section presents the objective, criteria, approach, and tailoring for the following work activities: (1) Software Project Management and Quality Planning, (2) Software Risk Management, (3) Software Configuration Management (SCM), (4) Software Procurement and Supplier Management, (5) Software Requirements Identification and Management, (6) Software Design and Implementation, (7) Software Safety, (8) Software Verification and Validation, (9) Software Problem Reporting and Corrective Action, (10) Training of Personnel in the Design, Development, Use, and Evaluation of Safety Software. The Report Format section provides a suggested report format. The References section lists selected references relevant to software quality assurance (SQA) F.2 PURPOSE AND SCOPE The purpose and scope of the criteria review and approach document (CRAD) is to provide a set of consistent criteria and guidelines for the assessment of the safety software. The safety s APPENDIX F DOE G 414.1-4 F-4 6-17-05 The CRAD is not intended to be prescriptive enough to evaluate the overall QA program associated with the safety software but rather to focus on the safety software application/product. Individual sites should tailor the scope of the assessment to suit the specific usage software in their safety systems. The CRAD could be used for assessment of the following types of software: • custom software developed by DOE, its contractors, or subcontractors for use with safety systems or safety class (SC) SSCs; • configurable, such as PLCs; • acquired software, including commercial off-the-shelf (COTS) software; • utility calculation software, such as spreadsheets and math programs (along with their associated user files), used to perform safety analysis and design calculations; and • commercial design and analysis tools, such as proprietary facility design and accident analysis software. These software types are used in the following safety software applications: • safety management and administrative database programs and associated user files to maintain control of information that has nuclear safety implications; • instrumentation and control software, including embedded microprocessors, distributed control systems, supervisory control and data acquisition systems, PLCs, and other related software; • networking and interface applications; • safety accident analysis; and • design and analysis of SSCs. Should an issue arise that casts doubt on the validity of software previously used to support design or development, it should be resolved using the unreviewed safety question (USQ) determination (USQD) process. Generic USQs will be used to the extent possible to preclude multiple facilities’ developing separate USQDs for the same issue. F.3. GUIDING PRINCIPLES The following principles should guide the conduct of the assessment. The assessment team leader, with assistance from the DOE site manager responsible for these assessments, should ensure that these guiding principles are incorporated in the tailoring process for assessing safety software applications. • The team should review any previous assessments and reviews of safety software. This review will enable the team to understand previous assessments, software qualification processes, associated requirements and performance criteria, assumptions concerning system operations, and the role of safety software in operations. DOE G 414.1-4 APPENDIX F 6-17-05 F-5

• Th tware applications and include any propriate in the assessment plan. • The review of SQA processes for existing safety software should follow the guidance r uld be agreed upon by DOE, the contractor line r to the start of the assessment and should • entation, against the demonstrated effect on result • of failure of the software and the associated effects on documented evidence to the team that the appropriate SQA standards were applied to software development, procurement, or use; and provide a staff point of contact for further information. • Procedures and records for software design, implementation, procurement, verification y personnel should ill be necessary to tailor the assessment criteria and guidelines to focus rmined to be appropriate for the assessment nsure that the assessments are conducted in ppropriate and applicable to each re and its ability to operate safely on a continued ntly e team should review any lessons learned from past events associated with sof additional attributes as ap provided in the DOE G 414.1-4 Section 3.3.2 Existing Safety Software Applications. The physical boundaries of the software within the safety system or subsystem level o• portions thereof under review sho management, and the assessment team lead prio be documented in the assessment report. Care should be taken to balance the effort invested during the assessment in verifying the SQA processes and their supporting docum improving the software quality and safety, and on eliminating the costly errors that from misunderstood requirements. The assessment of specific software applications should begin with gaining an understanding of the overall system, and documenting the system safety functions, the performance criteria that the system should meet to successfully accomplish its safety functions, and the role of the software in ensuring that these functions and criteria are met. The potential consequences system operability should be understood and documented. • The facility staff should assist the team in understanding the associated SQA process; provide and validation (V&V), testing, and maintenance should be evaluated for adequacy and to determine whether they are appropriate and are being used to verify that software requirements and performance criteria described in the software requirements documentation are satisfied. • If the team identifies a condition that poses an imminent threat to personnel or facilit safety, line management should be notified immediately. Team immediately point out the imminent threat condition to their points of contact or appropriate facility manager and notify the assessment team leader as soon as practical. • In some cases it w the assessment to address those aspects dete scope. The tailoring process is intended to e accordance with the criteria and guidelines that are a specific situation. These assessment criteria and guidelines are intended to be flexible, and may be tailored to allow the most cost-effective use of resources in determining the operational readiness of safety softwa basis. The tailoring process may take into account considerations, such as rece APPENDIX F DOE G 414.1-4 F-6 6-17-05 . The tailoring and its associated rationale should be agreed upon prior to the start of the level of modification to the software when evaluating the es. Acquired software, such as COTS, may not be modified software is d ts tances

as custom developed software for the specific application. The grading approach in this Guide assists in this effort. • The assessment should consider the effectiveness of SQA processes that are separate e rease the safe • hould r any of the documentation, such as a problem statement, requirements lan, or test results, is available. In situations al documents do not exist, sufficient information may be entation. • used in analysis or design for several years, iew pment d responses. The level of a posteriori review may range from a simple demonstration that the software produces reasonable results for a representative sample of inputs or test cases, to a rigorous verification of program r coverage, and evaluation of test results. The team may consider using documented engineering judgments (including their ot be found, the assessment may consist primarily of a review of system is completed assessments, evaluations, studies, inspectio