DOE N 203.1, Software Quality Assurance

DISTRIBUTION: INITIATED BY: All Departmental Elements Office of the Chief Information Officer DOE N 203.1 U.S. Department of Energy NOTICE Washington, D.C. Approved: 10-02-00 Expires: 06-02-01 SUBJECT: SOFTWARE QUALITY ASSURANCE 1. OBJECTIVES. To define requirements and responsibilities for software quality assurance (SQA) within the Department of Energy (DOE) to ensure that— a. all software owned or maintained by DOE, as referenced in paragraph 3c, Applicability, is subjected to formal quality assurance; b. all DOE software engineering follows identified standards and best practices throughout the project and product lifecycle; c. due to the spectrum of requirements, the degree of SQA is risk-based; and d. personnel are capable of correctly developing, using, and managing software. 2. CANCELLATION. None. 3. APPLICABILITY. a. DOE Elements. This directive applies to Departmental elements that acquire, develop, modify, or maintain computer software. b. Contractors. The Contractor Requirements Document, Attachment 1, sets forth the requirements to be applied to all management and operating and other contracts that require the acquisition, development, modification, or maintenance of computer software, as provided by contract and as implemented by the appropriate contracting officer. Compliance with the Contractor Requirements Document will be required to the extent set forth in the contract. c. DOE Software. The provisions of this Notice apply to all DOE software or software customized for DOE use, proposed for use, under development, or being maintained and used, whether that software was developed in-house, licensed from a commercial vendor for customized use, obtained from another organization, or otherwise acquired. The type of software includes, but is not limited to (a) administrative/business-oriented software, (b) scientific/engineering software except as identified in paragraph 3.d. below, (c) manufacturing-oriented software, and (d) process control; (e.g., Programmable Logic Control instructions). 2 DOE N 203.1 10-02-00 d. Basic Research Activities. The requirements of this Notice are not mandatory for basic scientific research and development activities conducted to support the Office of Science mission unless those activities are governed by the requirements in 10 CFR part 830. However, line management is encouraged to consider all or part of the Notice requirements in meeting its responsibilities to ensure the quality of the software developed for basic research. Business systems that support basic research are not exempted from the Notice requirements. e. Exclusion. Executive Order 12344 (set forth in Public Law 106-65 of October 5, 1999 [50 U.S.C. 2406]) establishes the responsibilities and authority of the Director, Naval Nuclear Propulsion Program, for all facilities and work that comprise the Program, which is a joint Navy/DOE organization. The Director’s responsibilities include the operating practices and procedures applicable to Naval nuclear propulsion plants. The Director must establish the quality assurance requirements implemented within the Program. Accordingly, this Notice does not apply to the Naval Reactors Program. 4. REQUIREMENTS. a. This directive is effective upon issuance.

b. SQA Program. Each Departmental element shall develop, document, and implement an SQA program. Each SQA program will consist of an identified focal point of contact, defined authorities, policies, procedures, training, adopted standards, and conventions tailored to local needs. Each program will treat SQA initiatives appropriately, commensurate with their size, complexity, cost, degree of external impact, degree of customization, functions performed, and other factors important to local management. The SQA program will describe how project SQA plans are to be developed and implemented. c. Risk-Based, Graded Approach. All software, which is owned or maintained by DOE, must be subjected to a degree of formal SQA commensurate with the safety, security, and risk involved in developing and using the software. This approach allows all software, including that which may be categorized as "research and development", to be assessed for and receive an appropriate and commensurate amount of SQA. d. Lifecycle-Based SQA Processes and Procedures. The SQA processes and procedures used must be software product and project lifecycle based; documented to provide a baseline for auditing; and applied in a consistent, repeatable, and predictable manner. The adequacy of selected processes and practices, as well as their oversight, is the responsibility of each individual Departmental element. e. Project SQA Plans. Project SQA plans will be developed and address testing (e.g., unit, integration, system, acceptance), verification and validation, structured walkthroughs, peer DOE N 203.1 3 10-02-00 reviews, inspections, audits and any other requirements specified for an application (e.g., by contract). Each plan should be commensurate with the level of the size, complexity, and scope of the software project. f. Oversight. Each Departmental element will conduct systematic reviews to ensure that the requirements of this directive and DOE O 414.1A, QUALITY ASSURANCE, are met and determine the need to update its own SQA program. Relative to software, these reviews should also ensure that appropriate safety and security controls are in place, are effective, and reflect currently accepted industry practices. For line management assessment of an SQA program, the principles and guidelines in DOE P 450.5, LINE ENVIRONMENT, SAFETY AND HEALTH OVERSIGHT, will apply and should be followed. g. Training. Sites are responsible for ensuring the adequacy of training programs to meet current and future personnel skill needs in the areas of SQA, software engineering, and software user training. h. Integration. Sites must integrate the SQA program planning process with the strategic planning, Safety Management System, and budget process, as appropriate, to ensure that SQA program decisions are made, adequately funded, and executed to support DOE organizational and site missions and priorities. 5. RESPONSIBILITIES. a. Office of the Chief Information Officer. (1) Establishes and maintains Departmentwide direction and guidance for SQA management processes. (2) Periodically reviews the results of internal and external compliance assessments and determines if the Departmentwide direction and guidance need to be improved or assistance provided. b. Power Marketing Administrations. Execute program office responsibility, accountability, and oversight for SQA management process compliance within their respective program areas.

c. Departmental Elements. Implement the appropriate level of management effort, and assume responsibility, accountability, and oversight for continued SQA management process compliance within their respective program areas. Specifically— (1) Establish and document SQA programs. (2) Identify a focal point of contact. 4 DOE N 203.1 10-02-00 (3) Ensure that the SQA programs conduct risk assessments and determine the level of SQA to be applied. (4) Ensure that the level of SQA is tailored to the site needs. (5) Oversee development and implementation of SQA processes and procedures. (6) Ensure the production and delivery of quality software products. (7) Ensure that SQA programs are reviewed. (8) Ensure SQA plans are approved. (9) Relative to software, ensure that appropriate safety and security controls are in place, are effective, and reflect currently accepted industry practices. (10) Ensure the adequacy of training programs for SQA, software engineering and software user training. (11) Ensure that any SQA program related to safety is developed and implemented in a manner that is consistent with DOE P 450.4, SAFETY MANAGEMENT SYSTEM POLICY, and associated standards and manuals. (12) Ensure that any nuclear software program related to safety is developed and integrated with existing nuclear safety policies and standards. (13) Ensure that all SQA programs are developed and implemented in a manner that is consistent with applicable classified and/or unclassified policy. d. Assistant Secretary for Environment, Safety, and Health (EH-1), acting as DOE's independent element responsible for safety aspects relative to public and worker health, and safety and environmental protection, shall provide advice and assistance to the Chief Information Officer concerning policy requirements and guidance necessary to implement this directive on software used for safety applications. e. Deputy Assistant Secretary for Oversight, acting as the Department's independent element responsible for the oversight of environment, safety, and health has the following responsibilities. (1) Assess and report to the Secretary of Energy on all aspects of safety related to implementation of this directive, including performance of the Secretarial Offices, field elements and contractors. (2) Review and comment on proposed SQA policy, regulations, standards and requirements to assess their potential effects on the safety of operations at DOE facilities. DOE N 203.1 5 10-02-00 f. Director, Office of Independent Oversight and Performance Assurance, acting as the Department's independent element responsible for the oversight of safeguards and security has the following responsibilities. (1) Assess and report to the Secretary of Energy on all aspects of safeguards and security related to implementation of this directive, including performance of the Secretarial Offices, field elements and contractors. (2) Review and comment on proposed SQA policy, regulations, standards and requirements to assess their potential effects on the security of operations at DOE facilities. 6. IMPLEMENTATION. Implementation of this directive is site-specific. An implementation plan that describes the actions necessary to comply with this directive and the expected date for completing those actions must be submitted to the applicable Program Secretarial Office (PSO) or Power Marketing Administration management 90 days after the approval date of this directive. Where there are multiple programs, coordination should be implemented by the Lead Program Secretarial Officers. SQA program plans should be approved by PSOs within 120 days of receipt.

7. ASSESSMENTS OF SQA IMPLEMENTATIONS. Assessments of SQA implementations of this directive will be forwarded to the Office of the Chief Information Officer. 8. REFERENCES. a. 10 CFR part 830, Nuclear Safety Management. b. DOE O 414.1A, QUALITY ASSURANCE, dated 9-29-99. c. DOE O 5480.23, NUCLEAR SAFETY ANALYSIS REPORTS, dated 4-10-92. d. DOE P 450.4, SAFETY MANAGEMENT SYSTEM POLICY, dated 10-15-96. e. DOE P 450.5, LINE ENVIRONMENT, SAFETY AND HEALTH OVERSIGHT, dated 6-26-97. f. DOE S 1027-92, HAZARD CATEGORIZATION AND ACCIDENT ANALYSIS TECHNIQUES FOR COMPLIANCE WITH DOE ORDER 5480.23, NUCLEAR SAFETY ANALYSIS REPORTS, updated 9-97. g. DOE G 200.1-1, DEPARTMENT OF ENERGY SOFTWARE ENGINEERING METHODOLOGY, dated 5-21-97. 6 DOE N 203.1 10-02-00 h. DOE G 414.1-2, QUALITY ASSURANCE MANAGEMENT SYSTEM GUIDE FOR USE WITH 10 CFR 830.120 AND DOE O 414.1, dated 7-17-99. i. Quality Criteria (QC-1), invoked via reference in DOE/AL Supplemental Directive 56XB (Nuclear Weapon Development and Production Manual). 9. CONTACT. For additional information or assistance in interpreting or implementing this directive, please contact the Office of the Chief Information Officer at 202-586-0166. 10. DEFINITIONS. To promote a common understanding of SQA and systems engineering concepts, the following definitions are provided. a. Acceptance Testing. Formal testing conducted to determine whether or not a software product or system satisfies its acceptance criteria and to enable the system owner to determine whether or not to accept the product or system. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. b. Configuration Management (CM). A discipline applying technical and administrative direction and surveillance to identify and document the functional and physical characteristics of a configuration item, control changes to those characteristics, record and report change processing and implementation status, and verify compliance with specified requirements. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. c. Departmental Element. A Departmental Element is defined as a first-tier organization at Headquarters and in the Field. First-tier at Headquarters is the Secretary, Deputy Secretary, Under Secretary, and Secretarial Officers (Assistant Secretaries and Staff Office Directors). First-tier in the Field is Managers of the eight Operations Offices, Managers of the three Field Offices, and the Administrators of the Power Marketing Administrations. Headquarters and Field Elements are described as follows: (1) Headquarters Elements are DOE organizations located in the Washington Metropolitan Area; and (2) “Field Elements” is a general term for all DOE sites (excluding individual duty stations) located outside of the Washington, DC, Metropolitan Area. DOE Glossary in the Directives System. d. Information System. A combination of information, computer, and telecommunications resources and other information technology and personnel resources that collects, records, processes, stores, communicates, retrieves, and displays information. DOD Directive #7920.1, Life Cycle Management of Automated Information Systems, 1988. e. Integration Testing. Testing in which software components, hardware components, or both are combined and tested to evaluate the interaction between them. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. DOE N 203.1 7 10-02-00

f. Project Planning. The planning of project technical and management activities that are documented in a project plan. The plan typically describes the work to be done, the resources required, the methods to be used, the procedures to be followed, the schedules to be met, and the way the project will be organized. It includes a list of deliverables, actions required, and other key events needed to accomplish the project. DOE Software Quality and Systems Engineering support team, 1999. g. Project Tracking and Oversight. The tracking and reviewing of accomplishments and results against documented estimates, commitments, and plans. Includes the adjusting of plans based on actual accomplishments and results. DOE Software Quality and Systems Engineering support team, 1999. h. Quality Assurance. (1) A planned and systematic pattern of all actions necessary to provide adequate confidence that the item or product conforms to established operational, functional, and technical requirements. (2) A set of activities designed to evaluate the process by which products are developed or manufactured. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. i. Quality Control. (1) The process by which product correctness is determined and action is initiated when nonconformance is detected. (2) A line function; the work done within a process to ensure that the work product conforms to standards/requirements. Effective Methods for Software Testing by William Perry, John Wiley & Sons, 1995. j. Requirements Management. In system/software system engineering, the process of controlling the identification, allocation, and flowdown of requirements from the system level to the module or part level, including interfaces, verification, modifications, and status monitoring. Software Requirements Engineering, edited by Thayer & Dorfman, IEEE Computer Society Press, 1997. k. Risk Management. An approach to problem analysis that is used to identify, analyze, prioritize, and control risks. DOE Software Engineering Methodology, March 1999. l. Software Design. In software engineering, the process of defining the software architecture (structure), components, modules, interfaces, test approach, and data for a software system to satisfy specified requirements. Software Requirements Engineering, edited by Thayer & Dorfman, IEEE Computer Society Press, 1997. 8 DOE N 203.1 10-02-00 m. Software Engineering. (1) The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software. (2) The study of approaches as in (1). IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. n. Software Quality Assurance. See Quality Assurance. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. o. System Testing. Testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. p. Unit Testing. Testing of individual hardware or software units or groups of related units. The isolated testing of each flowpath of code with each unit. The expected output from the execution of the flowpath should be identified to allow comparisons of the planned output against the actual output. DOE Software Engineering Methodology, March 1999.

q. Validation. The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. r. Verification. (1) The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. (2) Formal proof of program correctness. IEEE Standard Glossary of Software Engineering Terminology, Std. 610.12-1990. BY ORDER OF THE SECRETARY OF ENERGY: T.J. GLAUTHIER Deputy Secretary DOE N 203.1 Attachment 1 10-02-00 Page 1-1 CONTRACTOR REQUIREMENTS DOCUMENT DOE N 203.1, SOFTWARE QUALITY ASSURANCE The requirements in this Contractor Requirements Document must be applied to all management and operating and other contracts that require the acquisition, development, modification, or maintenance of computer software, as provided by contract and as implemented by the appropriate contracting officer. Compliance with this Contractor Requirements Document will be required to the extent set forth in the contract. 1. The provisions of this Contractor Requirements Document apply to DOE software or software customized for DOE use, proposed for use, under development, or being maintained and used, whether that software was developed in-house, licensed from a commercial vendor for customized use, obtained from another organization, or otherwise acquired shall be subjected to formal quality assurance. The type of software includes, but is not limited to— (a) administrative/business-oriented software, (b) scientific/engineering software within the context of considerations identified in number 2, (c) manufacturing-oriented software, and (d) process control (e.g., Programmable Logic Control instructions). 2. The provisions of this Contractor Requirements Document are not mandatory for basic scientific research and development activities conducted to support the Office of Science mission unless those activities are governed by the requirements in 10 CFR part 830. However, as directed, contractor line management is encouraged to consider all or part of the Notice requirements in meeting its responsibilities to ensure the quality of the software developed for basic research. Business systems that support basic research are not exempted from the Contractor Requirements Document provisions. 3. The contractor must develop, document, and implement an SQA program for projects under its contract. Each SQA program will consist of an identified focal point of contact, defined authorities, policies, procedures, training, adopted standards, and conventions tailored to local needs. Each program will treat SQA initiatives appropriately, commensurate with their size, complexity, cost, degree of external impact, degree of customization, functions performed, and other factors important to the site's management. 4. The contractor must ensure all software, which is owned or maintained by DOE, is subjected to a degree of formal SQA commensurate with the safety, security, and risk involved in developing and using the software. This approach allows all software, including that which may be categorized as "research and development", to be assessed for and receive an appropriate and commensurate amount of SQA. Attachment 1 DOE N 203.1 Page 1-2 10-02-00

5. The contractor must ensure the SQA processes and procedures are software product and project lifecycle based; documented to provide a baseline for auditing; and applied in a consistent, repeatable, and predictable manner. The contractor must ensure the adequacy of selected processes and practices, as well as their oversight. 6. The contractor must develop project SQA plans and address testing (e.g., unit, integration, system, acceptance), verification and validation, structured walkthroughs, peer reviews, inspections, audits and any other requirements specified for an application (e.g., by contract). The contractor must ensure that each plan is commensurate with the level of the size, complexity and scope of the software project. As appropriate, a standard SQA plan may be adopted and/or adapted for subsequent projects within a program. 7. The contractor must conduct systematic reviews to ensure that the requirements of this directive and DOE O 414.1A, QUALITY ASSURANCE, are met and determine the need to update its own SQA program. Relative to software, these reviews should also ensure that appropriate safety and security controls are in place, are effective, and reflect currently accepted industry practices. 8. The contractor must ensure the adequacy of training programs to meet current and future personnel skill needs in the areas of SQA, software engineering, and software user training. 9. The contractor must ensure the integration of the SQA program planning process with DOE strategic planning, Safety Management System, and budget process, as appropriate, to ensure that SQA program decisions are made, adequately funded, and executed to support DOE organizational and site missions and priorities.