DOE G 440.2B-1, Implementation Guide - Aviation Program Performance Indicators (Metrics) for use with DOE O 440.2B, Aviation Management And Safety

DOE G 440.2B-1 12-10-02 IMPLEMENTATION GUIDE AVIATION PROGRAM PERFORMANCE INDICATORS (METRICS) for use with DOE ORDER 440.2B, Aviation Management and Safety [This Guide describes suggested nonmandatory approaches for meeting requirements. Guides are not requirements documents and may not be construed as requirements in any audit or appraisal for compliance with the parent Policy, Order, Notice, or Manual.] U.S. Department of Energy Washington, D.C. 20585 DISTRIBUTION: INITIATED BY: All Departmental Elements Office of Aviation Management/OMBE/CFO i FOREWORD This Department of Energy (DOE) Aviation Program Performance Indicators interim guide is approved by the Office of Aviation Management (ME-2.4) and is available for use by all DOE and National Nuclear Security Administration (NNSA) organizations and their contractors. This Interim Guide is applicable to DOE Order 440.2B, "AVIATION MANAGEMENT AND SAFETY." Beneficial comments (recommendations for changes, additions, or deletions) should be sent to the Director, Office of Aviation Management, U.S. Department of Energy, 1000 Independence Avenue, S.W., Washington, D.C. 20585, by letter or by sending the self-addressed Standardization Document Improvement Proposal (DOE F 1300.3) in Attachment A. This Guide provides information regarding the expectations of the Department on specific provisions of DOE 440.2B. It identifies acceptable methods of implementing certain requirements of the Order regarding Aviation Program Performance Indicators (Metrics). It identifies relevant principles and practices by referencing Government and non-Government standards. The discussions on methods and approaches and other information are intended to be useful in understanding and implementing performance indicators (metrics) required by the Order. The use of this Interim Guide will facilitate consistency in implementing the Order and help ensure that all of the Aviation Program Performance Indicator’s (Metrics) provisions of the Order are addressed. This Interim Guide will not supersede any requirements of the Order. The statements in this Interim Guide are not substitutes for requirements. If a statement or provision from this Interim Guide is explicit in a contract or a plan required by a DOE Rule, an enforceable obligation is created by those documents. Additionally, implementation plans that reference a procedure as the intended methodology to accomplish an action cause the referenced parts to become mandatory. DOE G 440.2B-1 12-10-02 ii TABLE OF CONTENTS FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 1.2 Characteristics of Good Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 2. OVERVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 2.1 Defining the Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 2.2 Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

3. AVIATION OPERATIONS PERFORMANCE INDICATORS . . . . . . . . . . . . . . . . . 3-1 4. AIRCRAFT MAINTENANCE PERFORMANCE INDICATORS . . . . . . . . . . . . . . . 4-1 4.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1 4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1 4.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 4.4 Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 4.5 Maintenance Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 5. AIRCRAFT SUPPLY PERFORMANCE INDICATORS . . . . . . . . . . . . . . . . . . . . . . 5-1 6. MISSION CREW INDICATORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1 7. MISSION EQUIPMENT MAINTENANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1 8. SAFETY PROGRAM INDICATORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1 8.1 Prelude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1 8.2 Measuring Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1 8.3 Normalizing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2 9. COST PERFORMANCE INDICATORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1 APPENDIX A — ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 APPENDIX B — DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 APPENDIX C — REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 DOE G 440.2B-1 12-10-02 Page 1-1 1. INTRODUCTION 1.2 Background The Department of Energy, Office of Aviation Management (OAM) determined in 1999 that DOE/NNSA Aviation Program managers needed to develop and implement a system of Performance Measures or Indicators. These performance indicators provide managers a structured approach to understanding and measuring key processes that lead to the production of aircraft, flight and mission crews, and mission equipment that is ready to meet all program requirements. Indicators highlight processes that are not functioning optimally and identify where management attention is required. In the end, performance indicators provide managers with quantifiable information to use as a basis for management decisions. After conducting studies and reviews of the DOE/NNSA aviation program and data systems deployed in the Field, it was determined that the Department captured data but did not have an effective process to turn the data into information useful to managers. Working with the Field elements, a working group developed a core set of Performance Indicators (Metrics) for implementation at each site. In addition, it was determined that most of the existing data systems or records kept by the Aviation Managers and the Contract organizations could be utilized to implement the performance indicators. This guidance document will assist Field elements in implementing the Aviation Program Performance Indicators.

What is a Performance Indicator? Simply stated, a performance indicator is a value or process to measure output and outcome, or, with respect to a goal, course and tempo. The purpose of performance indicators is to provide aviation managers with a tool for improving the effectiveness and efficiency of the processes involved with safely delivering aircraft services. In addition, performance measurement can provide “leading indicators” so that actions can be taken early on in any one of the work processes to improve the end product and provide information needed by senior managers to support aviation program goals. Performance indicators will also provide the Office of Aviation Management information necessary to promote and support aviation program goals throughout DOE and NNSA. 1.2 Characteristics of Good Indicators Good indicators measure only what is important and focus only on key information that is of real value for managing production quality, quantity, timing, and cost. Such as: • Inputs • Processes • Outputs Good indicators must be quantitative and be a measure that can be expressed as an objective value such as: • Cardinal, Ordinal, Ratio • State, Condition, Rate, Trend DOE G 440.2B-1 12-10-02 Page 1-2 One of the quantitative terms the Department will be using is “Fully Mission Capable,” which is defined later in the guide. In addition, Aviation Program Performance Indicators in this document have been defined and mutually understood and agreed to by all involved in the process that is being measured. Some examples of quantitative and mutually understood Performance Indicators are: Quantitative “Supply Response Time (SRT) begins when the requisition document is date/time stamped by the aircraft supply clerk and ends when the issue document is date/time stamped by supply clerk and part issued to maintenance personnel.” Defined and Mutually Understood “Departure deviation occurs when actual departure time is + or -15 minutes from the published departure time.” The measure conveys at a glance what it is measuring and how it is derived, which is the goal for all of the Aviation Program Performance Indicators. The Concept of Customer Wait Time and What Does it Mean During the development of the DOE/NNSA Aviation Program Performance Indicators, senior DOE managers expressed their collective agreement that new measures must be developed. The new measures, where practical, would reflect the time from order to receipt when customer requirements are satisfied. This new measuring process was based on a concept in use by the DoD and was included in the DoD Logistics Strategic Plan as Customer Wait Time (CWT). DOE/NNSA adopted the definition of the process and incorporated it throughout its performance indicators process, where practical. Although DOE did not adopt the term “Customer Wait Time” the concept was incorporated into many of the indicators, such as Mean Time to Repair, Mean Supply Response Time, etc. Good indicators must also encourage appropriate behavior, the measure is balanced to reward productive behavior and discourage “game playing.” In addition, good indicators use economies of effort and the benefits of the measure outweigh the costs of collection and analysis. This means that performance indicators should use existing data, where possible; be a one-time entry of data; and integrated with the work processes.

In summary, good Aviation Program Performance Indicators will facilitate trust and the measure validates the participation among the various parties. Indicators are not a “horse race.” They are a process that respects the diversity of our organizations and provides a management process for a common commitment to continuous improvement. The hallmark of our Aviation Program Performance Indicators is the systematic identification and measurement of key processes that lead to the production of our ultimate product: an aircraft with flight crew, mission crew and mission equipment/proper configuration available to meet its mission. This is the goal of implementing Aviation Program Performance Indicators to provide managers at all levels, quantifiable information to be used for decision making purposes. DOE G 440.2B-1 12-10-02 Page 2-1 2. OVERVIEW 2.1 Defining the Product What is DOE/NNSA’s aviation product? The answer is readily available safe, reliable, and efficient aviation services. This means the Department will deliver an airworthy aircraft, operated by qualified, current flight and mission crews, configured for the mission, and if applicable, mission equipment (Me) installed and operationally ready to meet customer needs. 2.2 Processes To deliver this product, within the goals established by each Field element, will take dedicated, trained and proficient pilots, mechanics, and mission crews. We also need logistics personnel that are able to execute purchases and contracts that keep parts and components flowing in a timely manner, to support the pace of flight operations for each Field element. In addition, effective scheduling and coordination between customers, service providers, flight and mission crews, and maintenance support are essential to efficiently deliver our services. Most of the indicators in the following chapters are based on “Assigned Hours.” Assigned hours are measured monthly by model and by total aircraft (Fleet). An example: The month of May– 31 days x 24 hours x 2 (B-200s) = 1488 Assigned Hours; By fleet– 31 days x 24 hours x 7 (total aircraft) = 5208 Assigned Hours. The following chart depicts the framework of the work processes that must be measured to determine the effectiveness and efficiency of an aviation organization. From this framework the next chapters will define the indicators within each of these groups and provide information about each of the processes. DOE G 440.2B-1 12-10-02 Page 3-1 3. AVIATION OPERATIONS PERFORMANCE INDICATORS Aviation operations within DOE/NNSA are diverse in terms of complexity of missions and types of aircraft. The programs, the Department’s aviation services support, are also very complex, with many factors that may effect performance, some within the Aviation Manager’s control and some not. The following terms and indicators were developed to identify the areas in which the aviation manager can measure the performance of the processes under the control of the aviation manager. Term: Mission Capable (MC)– Organization’s End Product Definition: An airworthy aircraft with a readily available flight crew and mission crew, with the aircraft properly configured, including operable mission equipment, if applicable, to meet the primary or secondary mission requirement.

Thresholds: Mission Capable time does not include Not Mission Capable (NMC) hours. NMC hours include: aircraft not airworthy hours ( NAH) [maintenance downtime and supply downtime], Flight crew unavailability (FCNA), Mission Crew unavailability (MCNA), mission equipment maintenance downtime (MeNAM), mission equipment supply downtime (MeNAS), and/or aircraft not configured for mission, e.g. cargo configured versus personnel transport configuration, etc. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, Chief Pilot records, Mission Scientist or Mission Crew records, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Aircraft reliability, mission equipment reliability, logistics (supply), training schedules of flight and mission crews, maintenance, availability of personnel, etc. DOE G 440.2B-1 12-10-02 Page 3-2 Term: Non-Mission Capable (NMC) Definition: Any time the aircraft can not meet its primary or secondary mission requirements due to aircraft availability (maintenance downtime and supply downtime (NAH)), Flight crew is unavailable (FCNA), Mission Crew unavailable (MCNA), mission equipment maintenance downtime (MeNAM), mission equipment supply downtime (MeNAS), and/or aircraft not configured for mission, e.g. cargo configured versus personnel transport configuration, etc. Thresholds: The clock starts when Management or Dispatch becomes aware the aircraft is Not Mission Capable until the aircraft returned to Mission Capable Aircraft status. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, Chief Pilot records, Mission Scientist or Mission Crew records, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Aircraft reliability, mission equipment reliability, logistics (supply), training schedules of flight and mission crews, maintenance, availability of personnel, etc. DOE G 440.2B-1 12-10-02 Page 3-3 Total Mission Capable Hrs. - Total Not Mission Capable Hrs. Total Assigned Hours 2700 (MC) Hrs. - 250 (TNMC)Hrs. 31 days X 24 hours X 4 (Fleet) = 2976 (Total Assigned Hours) Jan Feb Mar Apr May Jun Jul Performance Indicator: Mission Capable Aircraft Rate (MCR)– Trailing Indicator Definition: The proportion of assigned hours an aircraft is Mission Capable to meet its assigned primary or secondary mission over a defined period of time (Assigned hours) minus the time the aircraft is Non-Mission Capable due to aircraft maintenance downtime, downtime due to aircraft supply, mission equipment maintenance downtime, downtime due to mission equipment supply, mission crew not available, or flight crew not available divided by total assigned hours x 100. Goals: Site specific goals. The goal should be to increase the MCR by as much as economically possible and the trend should be upward over time. X 100 = MCR X 100 = 82 % MCR 90% 80% 70% 60% 50% 40% 30% MCR Trend DOE G 440.2B-1 12-10-02 Page 3-4 Jan Feb Mar Apr May Jun Jul Performance Indicator: Non-Mission Capable Rate (NMCR)--Trailing Indicator Definition: The proportion of assigned hours an aircraft is non- mission capable to meet its primary or secondary mission requirements. From the previous example on page 3-3, if the MCR is 82% then the NMCR is 18%. Goals: Site specific goals. The goal should be to reduce the NMCR by as much as economically possible and the trend should be downward over time.

Data Location: Data records may be found at the AvM's office, Operations, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the aircraft, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, insufficient mission or flight personnel, etc. Note: The Aviation Manager’s focus will be on the organization’s processes impacting the NMCR. What portion of the eighteen percent NMCR is due to scheduling, pilot availability, aircraft reliability (failure rates), mission crew availability, mission equipment maintenance or supply, or aircraft configuration control. Is the NMCR rate due to aircraft reliability of a particular system or operational factors? Only by looking at the indicators in the following chapters will the Aviation Manager be able to determine what corrective actions are necessary to reduce the NMCR. 20 % 10 % 1% 2001– NMCR Trend DOE G 440.2B-1 12-10-02 Page 3-5 Total Scheduled Commitments. Total Customer Requests 240 (Total Scheduled Commitments). 300 (Total Customer Requests) Performance Indicator: Customer Scheduling Effectiveness (CSE)–Leading Indicator Definition: CSE is defined as the proportion of customer requirements for aviation services that result in schedule commitments for aircraft services as requested, and as modified through negotiation, or in cancellation. Thresholds: Since many organizations receive several request per day for aircraft services, many of which do not meet DOE or Federal Regulations for the use of government aircraft, a threshold was established to determine which customer request would be counted and which ones would not. The threshold for determining when a customer’s request is valid, is when a firm commitment was made by a customer for a valid request for specific aviation services by a qualified customer. Goals: Site specific goals. The goal should be to increase the CSE and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, scheduling office, contractor's operation office, or the contractor's aircraft dispatch office. Factors That Effect The Measure: Customer cancellations, aircraft availability, flight crew availability, cost justification, weather, emergencies, etc. X 100 = CSE X 100 = 80% CSE May 2001– CSE Report DOE G 440.2B-1 12-10-02 Page 3-6 Example: 950 (Flts. Accomplished) 1000 (Scheduled) Jan Feb Mar Apr May Jun Jul = .95 X 100 = 95% OSE Performance Indicator: Operations Scheduling Effectiveness (OSE)– Leading Indicator Definition: The number of scheduled missions accomplished divided by the number of missions scheduled multiplied by 100. Goals: Site specific goals. The goal should be to increase the OSE and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, Central Scheduling office, Contractor's Operation's office, or Contractor's dispatch organization. Factors That Effect The Measure: Weather cancellations, customer cancellations, aircraft availability, maintenance, etc. 100% 90% 80% 70% 60% 50% 40% 2001– OSE Trend DOE G 440.2B-1 12-10-02 Page 3-7 Example: 1750 (On-time) 2100 (Total departures) Jan Feb Mar Apr May Jun Jul X 100 = 83% Performance Indicator: Departure (Dispatch) Reliability (DR)– Trailing Indicator

Definition: The difference between the total departure times per the planned schedule and the actual departure times. Divide the on-time departures by the total departures. Thresholds: Any flight delayed by more than 15 minutes from planned departure time. Goals: Site specific goals. The goal should be to increase the DR and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, Central Scheduling office, Contractor's Operation's office, or Contractor's dispatch organization. Factors That Effect The Measure: Weather cancellations, customer cancellations, aircraft availability, maintenance, Air Traffic Control delays, etc. Departure (Dispatch) Reliability 100% 90% 80% 70% 60% 50% 40% 2001--DR Trend DOE G 440.2B-1 12-10-02 Page 3-8 Performance Indicator: Pilot Readiness - Proficiency– Leading Indicator Definition: For an individual pilot, the number of required proficiency activities accomplished divided by the number of proficiency activities required. Goals: Site Specific Goals should be 100% (Exception noted, if not) Data Location: Data records may be found at the AvM's office, Chief Pilot's office, Training office, Contractor's Operation's office, pilot's records, or Contractor's dispatch organization Factors That Effect The Measure: Pilot availability, aircraft availability, program funds, etc Note: This measure can be depicted as a percent of completion or as shown in the above graph. The graph depicts whether the pilot is on track to meet the organization’s proficiency goals which gives the manager a visual cue to determine if a pilot is on track to meet organizational goals. DOE G 440.2B-1 12-10-02 Page 3-9 Performance Indicator: Pilot Readiness - Training– Leading Indicator Definition: For an individual pilot, the number of required training activities accomplished divided by the number of training activities required. Goals: Site specific goals should be 100%; exception noted, if not. Data Location: Data records may be found at the AvM's office, Chief Pilot's office, Training office, Contractor's Operation's office, pilot's records, or Contractor's dispatch organization. Factors That Effect The Measure: Pilot availability, aircraft availability, program funds, etc. Note: This measure can be depicted as a percent of completion or as shown in the above graph. The graph depicts whether the pilot is on track to meet the organization’s proficiency goals, which gives the manager a visual cue to determine if a pilot is on track to meet organizational goals. DOE G 440.2B-1 12-10-02 Page 3-10 Example: June 1, 2002 12 Pilots available 15 Pilots ( # of pilots employed) Jan Feb Mar Apr May Jun Jul Performance Indicator: Pilot Availability Rate– Leading Indicator Definition: The percentage of time that the minimum required number of qualified and current pilots are available to meet defined (primary) mission requirements. Goals: Site specific goals. The goal should be to maintain the PA to meet primary and secondary mission needs, while controlling payroll costs. Data Location: Data records may be found at the AvM's office, Chief Pilot's office, Training office, Contractor's Operation's office, pilot's records, or Contractor's dispatch organization. Factors That Effect The Measure: Vacation time, vacancies, duty-time limitations, sickness, operational pace, etc. = .80 X 100 = 80% Pilot Availability for June 1, 2002 2001--PA Rate--Trend

100% 90% 80% 70% 60% 50% 40% Note: The horizontal bar in the graph depicts the minimum percentage of available pilots required to meet mission requirements. DOE G 440.2B-1 12-10-02 Page 3-11 F L t. H r s. Quarters Performance Indicator: Pilot Utilization Effectiveness– Trailing Indicator Definition: For an organization, the proportions of total flying hours accomplished by individual pilots by make and model per quarter. Goals: Site specific goals. The goal should be to maintain the PUE, to meet pilot proficiency goals and utilization is balanced. Data Location: Data records may be found at the AvM's office, Chief Pilot's office, Training office, Contractor's Operation's office, pilot's records, or Contractor's dispatch organization. Factors That Effect The Measure: Pilot availability, training requirements, proficiency requirements, aircraft availability, program funds, etc. Pilot Utilization Chart Jones—— Smith—• Hawker— Blanchard— DOE G 440.2B-1 12-10-02 Page 4-1 4. AIRCRAFT MAINTENANCE PERFORMANCE INDICATORS 4.1 General Aircraft maintenance organizations are an integral part of an organization’s aviation program. The single purpose of aircraft maintenance organization is aircraft readiness or commonly referred to as “Availability.” From this single purpose come two primary objectives that Aircraft maintenance organizations are responsible for: • Providing safe, flyable (airworthy) aircraft, in the proper configuration, when and where needed to satisfy an organization’s program requirements. • Maintaining a level of aircraft availability at some point beyond that of the organization’s program requirements to provide aircraft for surge capacity or to meet other mission requirements. 4.2 Introduction An aircraft is considered available, if it is airworthy and ready for flight. The difference between an aircraft that is available and one that is mission capable is, an aircraft that is mission capable, is one that is available (airworthy), with a qualified and current flight and mission crew, configured for the mission, and if applicable, mission equipment installed and operational. Most of the indicators in the following chapter are based on “Assigned Hours.” Assigned hours are measured monthly by model and by total aircraft (Fleet). An example: The month of May– 31 days x 24 hours x 2 (B-200s) = 1488 Assigned Hours; By fleet– 31 days x 24 hours x 7 (total aircraft) = 5208 Assigned Hours. Maintenance indicators, where applicable, should be measured monthly. In addition, Aircraft Availability Rates should be calculated quarterly and annually by model and by fleet. The Maintenance Manager should determine what the quarterly and annual average availability rates are, by taking the sum of the preceding three or twelve monthly reports and dividing by three or twelve, as applicable, to determine the average Aircraft Availability Rate. The first task for the Maintenance Manager will be to separate the Available Hours (AH) from Not Available Hours (NAH). NAHs are the total hours in a month an aircraft or the fleet was not airworthy for flight due to a maintenance or supply (awaiting parts) problem. The Maintenance Manager will focus on NAHs (See Table 1) to determine where the manager should focus to improve work processes. These measures will be discussed in this chapter. DOE G 440.2B-1 12-10-02 Page 4-2 MAINTENANCE INDICATORS Aircraft Availible Hours Maintenance Scheduling

Effectiveness Top Five Descrepencies Mean Time Between Failures Recurring Descrepency Rate Not Airworthy Maintence Mean Supply Response Time Cannibilazation Rate Not Airworthy Supply Not Airworthy Hours Aircraft Availibilty Table 1 4.3 Overview The maintenance performance indicators chapter is designed to show managers what type of processes and data should be reviewed, and where to get the data. This chapter will provide you a broad overview of aircraft maintenance and some associated indicators that will make a managers job easier to understand how well his or her organization is functioning and what processes require improvement. This does not mean that the performance indicators in this chapter are all of the Aviation Performance Indicators that can measure the inputs, processes, or outputs of a maintenance organization. Each manager should evaluate their organization and determine if other measures should be incorporated to provide the information a manager needs to determine the effectiveness and efficiency of the maintenance organization. 4.4 Mission The aircraft maintenance team exists to provide safe, reliable aircraft and equipment to the organization and to optimize availability in a cost effective manner. All team members play a vital role in this process—from the newest mechanic in the maintenance organization to the quality control inspector—the focus is to provide a safe, reliable aircraft, in the right configuration, and on time to meet all mission and contingency requirements, cost effectively. 4.5 Maintenance Products Maintenance products are the major elements maintenance organizations produce. They are divided into six product areas as follows: 1) Mission Capable Aircraft; 2) Serviceable Aircraft and Components; 3) Serviceable Engines; 4) Serviceable Mission Equipment; 5) Trained Technicians, and 6) Other Services. Each of these product areas are divided into processes we track to ensure the health of our maintenance organization. DOE G 440.2B-1 12-10-02 Page 4-3 Example: Assigned Hours - Non-Airworthy Hours = # of Aircraft Available Hours Assigned Hours [31 days x 24 hours x 1 (Bell 206)] - 20 NAH = 724 Aircraft Available Hours Term: Aircraft Available Hours (AAH) Definition: The time an aircraft is available for use (airworthy). Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Aircraft maintenance downtime, downtime due to aircraft supply, mechanic availability, operational pace, age of aircraft, etc. Term: Non-Airworthy Hours (NAH) Definition: The time an aircraft is unavailable for use (unairworthy). Thresholds: The moment an aircraft, aircraft system, engine, propeller, avionic system, navigation system, or any component of or part of the aircraft, aircraft system, engine, propeller, avionic system, navigation system does not function or becomes damaged, worn, or deteriorates to cause an unsafe condition for flight or when an aircraft does not meet the regulatory equipment requirements for the type of operation being conducted. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Operational pace, operational environment, age of

aircraft, operator errors, improper maintenance, etc. DOE G 440.2B-1 12-10-02 Page 4-4 Assigned Hours - NAH Total Assigned Hours 31 days x 24 hrs x 7 (Fleet acft) 5208 hrs. - 520 hrs. (NAH) 5208 Jan Feb Mar Apr May Jun Jul Performance Indicator: Aircraft Availability Rate (AAR)--Trailing Indicator Definition: The proportion of time an aircraft is available for use, minus total NAH, divided by the assigned hours multiplied by 100. Goals: Site specific goals. The goal should be to increase the AAR by as much as economically possible and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the aircraft, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, etc. X 100 = AAR = .90 x 100 = 90% AAR 90% 80% 70% 60% 50% 40% 30% AAR Trend DOE G 440.2B-1 12-10-02 Page 4-5 Jan Feb Mar Apr May Jun Jul Performance Indicator: Non-Airworthy Rate (NAR)– Trailing Indicator Definition: The proportion of time an aircraft is not available for use. From the above example, if the AAR is 90% then the Non-Airworthy Rate is 10%. Goals: Site specific goals. The goal should be to reduce the NAR by as much as economically possible and the trend should be downward over time. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the aircraft, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, etc. Note: The Maintenance Manager’s focus will be on the organization’s processes impacting the NAR. What portion of the ten percent NAR is due to scheduling, mechanic availability, reliability (failure rates), maintenance or supply. Is the NAR rate due to aircraft reliability of a particular part or system? Only by looking at the following indicators will the manager be able to determine what corrective actions are necessary to reduce the NAR. 20 % 10 % 1% NAR Trend DOE G 440.2B-1 12-10-02 Page 4-6 Performance Indicator: Time Left to Inspection– Trailing Indicator Definition: The health of the aircraft fleet is a very important issue. In order to keep aircraft availability high, it is important to properly manage your inspection flow to preclude several inspections coming due at the same time causing backlogs or grounding aircraft and impacting mission capability. Time left to inspection is graphically depicted by aircraft tail number. This information can be obtained from maintenance plans, scheduling and maintenance personnel. Fleet average time left to inspection should be close to 50% of the inspection interval and should be evenly staggered along a 45 degree slope. Goals: Site specific goals. Data Location: Data records may be found at the AvM's office, Maintenance Office, Production Control office, or Contractor's Maintenance office. Factors That Effect The Measure: Operational pace, mechanic availability, etc. N1455—— N2345—• N65477— N8956DE—

Note: The graph depicts whether the Maintenance Manager or Production Control Manager is maintaining a steady flow of product into the maintenance organization. If scheduling is not managed well, the maintenance organization could be overwhelmed and severely impact the organization’s mission readiness. DOE G 440.2B-1 12-10-02 Page 4-7 Total Maintenance Actions Accomplished as Scheduled Total Maintenance Actions Scheduled 25 (Total Maintenance Actions Accomplished as Scheduled) 30 (Total Maintenance Actions Scheduled) Jan Feb Mar Apr May Jun Jul Performance Indicator: Maintenance Scheduling Effectiveness– Leading Indicator Definition: The number of scheduled maintenance actions accomplished as scheduled for each quarter. Thresholds: A scheduled inspection that is accomplished within 5 working days or one week of its scheduled inspection time is considered as scheduled. Goals: Site specific goals. The goal should be to increase the number of maintenance actions accomplished as scheduled, as much as economically possible, and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, Maintenance Office, Production Control office, or Contractor's Maintenance office. Factors That Effect The Measure: Operational pace, mechanic availability, etc. X 100 = MSE X 100 = 83.3 % MSE 90% 80% 70% 60% 50% 40% 30% DOE G 440.2B-1 12-10-02 Page 4-8 Example: June 1, 2002 12 Mechanics available 15 Mechanics (Required # of mechanics) Performance Indicator: Mechanic Availability– Leading Indicator Definition: The percentage of time that the minimum required number of qualified and current mechanics are available to meet maintenance schedules. Goals: Site specific goals. The goal should be to maintain the MA to meet program needs, while controlling payroll costs. Data Location: Data records may be found at the AvM's office, Direct Of Maintenance's office, Training office, Contractor's Maintenance office, or employee’s records. Factors That Effect The Measure: Vacation time, duty-time limitations, job vacancies, sickness, operational pace, etc. X 100 = 80 % MA Rate Performance Indicator: Top 5 Reported Discrepancies– Leading Indicator Definition: The top five discrepancies as reported by pilots or maintenance. The discrepancies should be converted to the Air Transport Association (ATA) Aircraft System/Component code. As an example: Write up– Fuel system leaking at filter. For data collection, convert write up to ATA 7310– Engine Fuel Distribution. Goals: Site specific goals. The goal should be to reduce the number of discrepancies reported as much as economically possible. Data Location: Data records may be found at the AvM's office, Direct Of Maintenance's office, Training office, Contractor's Maintenance office, or employee’s records. Factors That Effect The Measure: Quality problems with a certain component, part or appliance within a system, age of aircraft, a lack of trained maintenance technicians, etc. DOE G 440.2B-1 12-10-02 Page 4-9 Note: This data should be used to focus on aircraft, engines, propellers, and systems that may be experiencing higher than normal reliability problems. It also may reflect a quality problem with certain components, parts or appliances within a system, poor troubleshooting, or a lack of trained maintenance technicians. This is raw data tracked by each aircraft and system for a particular month and plotted on a bar graph. This data should be used to reduce the number of failures or Recurring Discrepancies (RD), so the numbers may initially be high at first. The goal is to steadily lower the number.

DOE G 440.2B-1 12-10-02 Page 4-10 1st Failure Time + 2nd Failure Time + 3rd Failure Time Total Failures Performance Indicator: Mean Time Between Failure (MTBF)–Leading Indicator Definition: The average elapsed time between failures of an aircraft, engine, propeller, or appliance or any component or part of an aircraft, engine, propeller, or appliance. Thresholds: Any product is considered failed if it does not meet its design life limit or if no design life limit is established, fails to meet its intended function, form or fit. Data Location: Maintenance records, component historical records, Service Difficulty Reports, or maintenance data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Aging aircraft, improper operation, improper inspection, improper maintenance, design or manufacturing defects, etc. = MTBF DOE G 440.2B-1 12-10-02 Page 4-11 NAM hours Total Assigned Hours 150 hrs. (NAM) 31 days x 24 hrs x 7 (Fleet acft) 5208 hrs. Performance Term: Non-Airworthy Maintenance (NAM) Definition: Occurs when a maintenance action, including an inspection, is required on the aircraft, engine, propeller, or any component of or part of the aircraft, engine, or propeller that renders the aircraft unairworthy. Thresholds: The clock starts for NAM when the maintenance organization is notified or becomes aware that the aircraft, engine, propeller, or any component of or part of the aircraft, engine, or propeller is unairworthy because of a maintenance action; the clock stops when the aircraft, engine, or propeller is returned to service. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the aircraft, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, operational environment, etc. Performance Indicator: NAM Rate (NAMR)--Trailing Indicator Definition: The NAM rate is derived by dividing your NAM hours by your Not Airworthy Hours and multiplying by 100 (NAM hours/Total Assigned Hours x 100). Goals: Site specific goals. The goal should be to reduce the NAMR, as much as economically possible, and the trend should be downward over time. X 100 = NAMR X 100 = 2.8% NAMR DOE G 440.2B-1 12-10-02 Page 4-12 TTR+ TTR+TTR+TTR Total Repairs Performance Term: Time to Repair (TTR) Definition: The elapsed time it takes a person, shop, or vendor to make a repair on an aircraft, engine, propeller, or appliance, or part or component thereof, that places the item in an airworthy condition, less the time spent waiting for parts. TTR is a specific value to be used in computing Mean Time To Repair (MTTR). Thresholds: A qualified and current mechanic or repairman is on- hand, who has the proper tools, repair or inspection data, and parts in-hand, if applicable, to do the work. Data Location: Data records may be found in Work Orders, maintenance records, supply records, maintenance office data bases, Quality Control office, etc. Factors That Effect The Measure: Maintenance and inspection personnel availability, aircraft reliability, age of aircraft, operating conditions, improper maintenance procedures, etc. Performance Indicator: Mean Time to Repair (MTTR)-Leading Indicator

Definition: Used in computing the maintainability of an aircraft, engine, propeller, or appliance, or any component of or part of an aircraft, engine, propeller, or appliance. The sum of TTRs divided by the total number of repairs. Goals: Site specific goals. The goal should be to reduce the MTTR, as much as economically possible, and the trend should be downward over time. = MTTR DOE G 440.2B-1 12-10-02 Page 4-13 Jan Feb Mar Apr May Jun Jul RD + RD + RD+ RD Total Discrepancies Mean Time To Repair Trend (B-200) 50 MHrs. 40 MHrs. 30 MHrs. 20 MHrs. 10 MHrs. 5 MHrs. Performance Term: Aircraft: Recurring Discrepancy (ARD) Definition: When the same discrepancy occurs on two or more flights, within the span of five flights. Thresholds: R & D activity of new systems or equipment excluded. Data Location: Operations or Maintenance records may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Poor troubleshooting of original discrepancy, unreliable parts, inadequate diagnosis of problem, etc. Performance Indicator: Aircraft: Recurring Discrepancy Rate (RDR)– Leading Indicator Definition: The RD rate is derived by dividing your total number of RDs by the total number of discrepancies reported and multiplying by 100. Goals: Site specific goals. The goal should be to reduce the RDR, as much as economically possible, and the trend should be downward over time. X 100 = RDR DOE G 440.2B-1 12-10-02 Page 4-14 Total Cannibalization Total Number of Parts Requested Performance Term: Cannibalization Definition: The act of taking a serviceable part from an aircraft, engine, propeller, appliance, or assembly to replace an unserviceable part on another aircraft, engine, propeller, appliance, or assembly rather than using stores from supply. Thresholds: A part removed from one product used for troubleshooting purposes only and returned to the original product is not considered cannibalized. Data Location: Maintenance records may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Procurement process, Not Later Than (NLT) dates, long lead time OEM parts, older aircraft (parts not being manufactured), cost considerations, etc. Performance Indicator: Cannibalization Rate (CR) Definition: The percentage of time parts requirements are satisfied/met by cannibalization. Goals: Site specific goals. The goal should be to reduce the CR, as much as economically possible, and the trend should be downward over time. = CR DOE G 440.2B-1 12-10-02 Page 5-1 NAS Hours Total Hours Aircraft Not Available 250 hours (NAS) 750 hrs. NAH 5. AIRCRAFT SUPPLY PERFORMANCE INDICATORS Performance Term: Aircraft Non-Airworthy: Supply (NAS) Definition: NAS occurs when parts are needed to complete a maintenance action on an aircraft, engine, propeller, or appliance, or any component of or part of an aircraft, engine, propeller, or appliance. Thresholds: The clock starts for NAS from the time a maintenance action is stopped due to a lack of parts; the clock stops when the part is issued. Data Location: Maintenance or supply records or maintenance or supply data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Procurement process, Not Later Than (NLT) dates, long

lead time OEM parts, older aircraft (parts not being manufactured), cost considerations, etc. Performance Indicator: NAS Rate (NASR)– Leading Indicator Definition: The NAS rate is derived by dividing your NAS hours by your total NAH and multiplying by 100 (NAS hours/Total NAH x 100). Goals: Site specific goals. The goal should be to reduce the NASR, as much as economically possible, and the trend should be downward over time. X 100 = NASR x 100 = 33.3% NASR DOE G 440.2B-1 12-10-02 Page 5-2 Total Number of Supply Response Times Total Number of Supply Responses 30 min. + 25 min.+40 min.+20 min. +50 min. = 165 min. 5 Performance Term: Supply Response Time (SRT) Definition: A value used in computing Mean Supply Response Time, the elapsed time between issuance of a customer request (order) and satisfaction of that order. Thresholds: From the time a parts request is approved or initiated until the request is filled. Data Location: Maintenance or supply records or maintenance or supply data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Procurement process, Not Later Than (NLT) dates, long lead time OEM parts, older aircraft (parts not being manufactured), cost considerations, etc. Performance Indicator: Mean Supply Response Time (MSRT)– Leading Indicator Definition: Used in computing the effectiveness of supply for an aircraft, engine, propeller, or appliance or any component of or part of an aircraft, engine, propeller, or appliance. The sum of supply response times divided by the total number of supply responses. Goals: Site specific goals. The goal should be to reduce the MSRT, as much as economically possible, and the trend should be downward over time. = MSRT = 33 min. MSRT DOE G 440.2B-1 12-10-02 Page 5-3 Total Number of Items Over and Short Total Number of Record of Balanced Inventory 200 800 Jan Feb Mar Apr May Jun Jul Performance Indicator: Inventory Accuracy Rate (IAR) Definition: Compute the indicator by adding the number of items over and short divided by the total record balance inventoried, then subtract the results from 100. Physically counting the assets allows the operator to correct errors in processing and storing these assets. Inventory processes are determined by DOE policies. Goals: Site specific goals. The goal should be to 95% or better and the trend should be upward over time. Data Location: Supply records or supply data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: DOE property rules and Federal Property Management Regulations. = N X 100; 100 - N = IAR = .25 X 100 = 25% ; 100 -25= 75% IAR IAR Trend 100 % 90 % 80 % Note: The horizontal line indicates the 95% IAR target. DOE G 440.2B-1 12-10-02 Page 5-4 Total Number of Items without Issue for 12 Months Total Number of Assets 16 550 Jan Feb Mar Apr May Jun Jul Performance Indicator: Excess in Inventory (EI) Definition: Compute the potential excess by dividing those items in stock that have not had an issue for more than 12 consecutive months. The EI indicator is computed using the potential excess divided by the total assets multiplied by 100. Inventory processes are determined by DOE policies. Goals: Site specific goals. The goal should be to reduce the EI, to 3 % or less and the trend should be downward over time.

Data Location: Supply records or supply data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Procurement process, Not Later Than (NLT) dates, long lead time OEM parts, older aircraft (parts not being manufactured), cost considerations, etc. X 100 = EI X 100 = 2.9% EI Excess in Inventory (EI) 10 % 5 % 1 % DOE G 440.2B-1 12-10-02 Page 6-1 6. MISSION CREW INDICATORS Performance Indicator: Mission Crew Readiness - Proficiency– Leading Indicator Definition: For individual mission crew, the number of required proficiency events accomplished divided by the number of proficiency events required. Goals: Site specific goals should be 100%; exception noted, if not. Data Location: Operations records or data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether or not the mission crews are under the operational control of the aviation program management, frequency of flying, utilization of available flight time, scheduling considerations and supervision. Performance Indicator: Mission Crew Readiness - Training–Leading Indicator Definition: For individual mission crew, the number of required training activities accomplished divided by the number of training activities required. Goals: Site specific goals should be 100%; exception noted, if not. Data Location: Operations records or data bases may be at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether or not the mission crews are under the operational control of the aviation program management, frequency of flying, utilization of available flight time, scheduling considerations and supervision. DOE G 440.2B-1 12-10-02 Page 6-2 Performance Indicator: Primary Mission Crew Availability– Trailing Indicator Definition: The percentage of time that the minimum required number of Mission Crew are available to meet defined (primary) mission requirements, such as Emergency Response, Security, etc. Goals: Site specific goals. The goal should be to maintain the MeMCA to meet program needs, while controlling payroll costs. Data Location: Operations records or data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether or not the mission crews are under the operational control of the aviation program management. DOE G 440.2B-1 12-10-02 Page 7-1 Example: Assigned Hours - Me Not Operational Hours = # of Me Operational Hours Assigned Hours [31 days x 24 hours x 1 (Bell 206)] - 20 MeNOH = 724 Me Operational Hours 7. MISSION EQUIPMENT MAINTENANCE Term: Mission Equipment (Me) Operational Hours– MeOH Definition: The time mission equipment is available for use (operational). Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Mission equipment maintenance downtime, downtime due to supply, mechanic availability, operational pace, age of equipment, etc. Term: Me Non-Operational Hours (MeNOH) Definition: The time mission equipment is unavailable for use (nonoperational).

Thresholds: The moment mission equipment or any component of or part of the mission equipment does not function or becomes damaged, worn, or deteriorates to cause the equipment to malfunction or not operate to specifications. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Product: Operational pace, operational environment, age of equipment, operator errors, improper maintenance, etc. DOE G 440.2B-1 12-10-02 Page 7-2 Assigned Hours - MeNOH Total Assigned Hours 31 days x 24 hrs x 7 (Fleet acft) 5208 hrs. - 520 hrs. (MeNOH) 5208 Jan Feb Mar Apr May Jun Jul Performance Indicator: Me Availability Rate (MeAR) (Trailing Indicator) Definition: The proportion of time an aircraft is available for use, minus total MeNOH, divided by the assigned hours multiplied by 100. Goals: Site specific goals. The goal should be to increase the MeAR by as much as economically possible and the trend should be upward over time. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the mission equipment, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, etc. X 100 = MeAR = .90 x 100 = 90% MeAR 90% 80% 70% 60% 50% 40% 30% MeAR Trend DOE G 440.2B-1 12-10-02 Page 7-3 Jan Feb Mar Apr May Jun Jul Performance Indicator: Me Non-Operational Rate (NOR)– Trailing Indicator Definition: The proportion of time mission equipment is not available for use. From the above example, if the MeAR is 90% then the MeNon-Operational Rate is 10%. Goals: Site specific goals. The goal should be to reduce the MeNOR by as much as economically possible and the trend should be downward over time. Data Location: Data records may be found at the AvM's office, Maintenance Manager’s or Contractor's Operation's office, maintenance records, or Contractor's dispatch organization. Factors That Effect The Measure: Age of the equipment, availability of parts, manufacturer defects, lack of qualified maintenance or inspection personnel, operational pace too high, poor maintenance scheduling, etc. Note: The Aviation Manager’s focus will be on the organization’s processes impacting the MeNOR. What portion of the ten percent MeNOR is due to scheduling, mechanic availability, reliability (failure rates), maintenance or supply. Is the MeNOR due to mission equipment reliability of a particular part or system? Only by looking at the following indicators will the manager be able to determine what corrective actions are necessary to reduce the MeNOR. 20 % 10 % 1% MeNOR Trend DOE G 440.2B-1 12-10-02 Page 7-4 MeNOM hours Total Assigned Hours 200 hours (MeNOM) 5208 Assigned Hours Performance Term: Mission Equipment (Me): Non-Operable Maintenance (MeNOM) Definition: Occurs when a maintenance action is required on the mission equipment or any component of or part of the mission equipment when the equipment is unable to meet mission performance requirements. Thresholds: The clock starts for MeNOM when mission equipment becomes unable to meet mission performance requirements and stops when returned to service.

Data Location: Maintenance records or maintenance data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Availability of technicians to commence repair, parts not on-hand, technical factors effecting the payload or equipment, or whether the equipment is owned by DOE/NNSA or by some other organization or agency. Performance Indicator: MeNOM Rate (MeNOMR) Definition: The MeNOMR is derived by dividing the MeNOM hours by the total Assigned Hours and multiplying by 100 (MeNOM hours/Total Assigned Hours x 100). Goals: Site specific goals. The goal should be to reduce the MeNOMR, as much as economically possible, and the trend should be downward over time. X 100 = MeNOMR X 100 = 40 % MeNOMR DOE G 440.2B-1 12-10-02 Page 7-5 RD + RD + RD+ RD Total Discrepancies Performance Term: Me: Recurring Discrepancy (MeRD) Definition: When the same discrepancy occurs on two or more flights, within the span of five flights. Thresholds: R & D activity of new systems or equipment excluded. Data Location: Maintenance records or maintenance data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether the equipment is owned by DOE/NNSA or by some other organization or agency. Poor troubleshooting of original discrepancy, unreliable parts, inadequate diagnosis of problem, etc Performance Indicator: MeRD Rate (MeRDR) Definition: The MeRDR is derived by dividing your total number of RDs by the total number of discrepancies reported and multiplying by 100 (Total RDs / Total Discrepancies x 100). Goals: Site specific goals. The goal should be to reduce the MeRDR, as much as economically possible, and the trend should be downward over time. X 100 = MeRDR Performance Term: Me: Non-Operable Supply (MeNOS) Definition: MeNOS occurs when parts are needed to complete a maintenance action on mission equipment or any component of or part of mission equipment. DOE G 440.2B-1 12-10-02 Page 7-6 MeNOS Hours Total MeNOH Thresholds: The clock starts for MeNOS from the time maintenance action is stopped due to a lack of parts; clock stops when the part is issued. Data Location: Maintenance records or maintenance data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether the equipment is owned by DOE/NNSA or by some other organization or agency. Performance Indicator: MeNOS Rate Definition: The MeNOS rate is derived by dividing the MeNOS hours by the total MeNOH and multiplying by 100 (MeNOS hours/MeNOH x 100). Goals: Site specific goals. The goal should be to reduce the MeNOS rate, as much as economically possible, and the trend should be downward over time. X 100 = MeNOSR Performance Term: Me: Supply Response Time (MSRT) Definition: A value Used in Computing Mean Supply Response Time. The elapsed time between issuance of a customer request (order) and satisfaction of that order. Thresholds: From the time a parts request is approved or initiated until the time the request is filled. Data Location: Operational reports, maintenance or supply records, or data bases may be found at the Contractor's site or in the Aviation Program Manager's organization (Federal). Factors That Effect The Measure: Whether the equipment is owned by DOE/NNSA or by some other organization or agency.

DOE G 440.2B-1 12-10-02 Page 7-7 MeSRT + MeSRT+ MeSRT+MeSRT Total Supply Responses 1hour + 24 hours + 5 hours + 18 hours 4 Performance Indicator: Me: Mean Supply Response Time (MSRT)– Leading Indicator Definition: Used in computing the effectiveness of supply for mission equipment / configuration. The sum of supply response times divided by the total number of supply responses. Goals: Site specific goals. The goal should be to reduce the Me MSRT, as much as economically possible, and the trend should be downward over time. = MeMSRT = 12 hours MeMSRT DOE G 440.2B-1 12-10-02 Page 8-1 8. SAFETY PROGRAM INDICATORS 8.1 Prelude The commitment to safety must start at the top of an organization. The single most important element of a successful safety program is the commitment of senior management. Safety cannot be dictated – it must be practiced. A successful safety program must be built on a foundation of trust between the Safety Officer, employees and management. Personnel at all levels must know that the reporting of incidents, near misses, occurrences, etc., can be accomplished without fear of reprisal or management playing a blame game. Safety program indicators differ slightly from the previous indicators discussed in this guidance, in that, the safety manager is trying to gather leading information (indicators) to predict trends and make corrective actions on the work processes before a major accident occurs. Trailing indicators such as the Fatal Accident rate per 100,000 hours of operation or Accident rate per 100,000 hours of operations may provide the analytical figures to make someone feel safe, but do not provide the necessary information needed to prevent accidents from occurring in the first place. There are many requirements and thresholds for reporting and recording incidents and accidents established by DOE, FAA, and NTSB that each organization must report. DOE requirements use many of the same definitions established in Title 49 CFR Part 830, which provides for uniformity between DOE and the outside aviation community. DOE already requires Field elements and contractors to develop safety program measures, but do not address any specific work process category. The aviation safety program indicators discussed in this chapter are meant to provide leading indicators for the aviation safety professional to prevent or eliminate accidents and incidents. In some cases, the indicators will mirror statistics that are already being tracked in the Field and at Headquarters. Increased safety information (data) availability and accessibility will create opportunities for educating program managers on the use and interpretation of aviation safety data, as well as for describing how DOE’s flight crews, mechanics, and others work together to promote safety. A significant question examined during the development of this chapter is what aviation safety information would be useful in informing management and personnel about DOE’s aviation safety program. While concern about safety is most acute immediately following an accident or incident, safety also reflects the concerns of the customers that use DOE aircraft, and Senior management’s view of DOE’s stewardship of its aviation programs. Whether or not aviation management and personnel believe that aviation safety concerns are justified given the high absolute levels of aviation safety, the Senior management’s concerns are real and are likely to have a large impact on the discourse about DOE aviation safety.

DOE G 440.2B-1 12-10-02 Page 8-2 1 Villareal, Carmen Teresa. "Uses and Misuses of Risk Metrics in Air Transportation," in Public-Sector Aviation Issues--1986-1987 2 Weener, Earl F. and Peter B. Wheeler. "Key Elements of Accident Avoidance," Logistics and Transportation Review 28(1): 49-60, 1992 8.2 Measuring Safety OAM believes it is possible to identify or compile "safety indicators" that provide insights as to whether an organization is more or less likely to undertake unsafe practices. DOE is focused on three broad aspects of aviation operations that are believed to be important to safe operations: pilot competence, maintenance quality, and management attitude. This data provides useful information about current aviation program safety practices, the reporting of this data could provide a positive incentive for the level of effort organizations put into aviation safety. The degree of compliance with DOE and Field level policies, in addition to FAA regulations, might be an indicator of an organization’s diligence in the safety arena. However, it must be noted that there may be no relationship between appraisal and inspection results and the probability that an organization will have an accident in the future, especially if the aviation programs improve as a result of the appraisal and inspection findings. 8.3 Normalizing Data Computation of an accident or incident rate requires normalizing information about the level of exposure to risk. For comparative purposes, it is essential that accident and incident data be normalized in some way, due to the diversity of DOE’s aviation program and exposure to risk changes over time. One organization's exposure to risk in a particular time period will likely differ from that of another, because different organizations have different levels and types of activity. Measures of exposure to risk commonly used to normalize event data include number of flights, hours flown, passenger enplanements, and passenger miles flown. Villareal1 discusses advantages and disadvantages of the various exposure measures used for normalizing safety research data. Most researchers prefer to use the number of flights (measured as departures) for normalizing data, rather than hours or miles flown, because the risk of accident for an aircraft is greatest during takeoff and landing. For customers, the most relevant measure is also likely to be flight or a round trip. Although a commercial aircraft spends only about six percent of its flight time in the takeoff, initial climb, final approach, and landing components of its flight, around 70 percent of "hull loss" accidents have occurred during these stages.2 Because of this, using an hours flown-based measure or a mileage-based measure of risk can be misleading. This is especially true at DOE when comparisons are being made between organizations that have different average flight lengths. Using a hourly-based measure will make a commuter type operation, such as Bonneville Power Administration (BPA) with very short average flight times, look more risk prone relative to a major jet carrier flying longer stage lengths, such as Albuquerque Operations Office (AL) on average. (This occurs because BPA with shorter DOE G 440.2B-1 12-10-02 Page 8-3 average flights will make more takeoffs and landings per hour flown, and aircraft are most exposed to the risk of an accident or incident during takeoff and landing.)

DOE aviation operations should use "percentage of departures" format for normalization of its data. The following pages of this section will describe the aviation safety performance indicators that each organization should establish. Performance Indicator: Incident Rate per 1,000 departures Definition: An occurrence, other than an accident, associated with the operation or maintenance of an aircraft, which affects or could affect the safety of operations or maintenance. Use total departures as a basis to determine rates. Goals: Site specific goals. The goal should be to reduce or eliminate incidents and the trend should be downward over time. Data Location: Data records may be found at the Aviation safety office, GSA Aviation Accident and Incident Reporting System (AAIRS), DOE ORPS reports, and Aviation Operations office. Factors That Effect The Measure: Personnel qualifications and experience, age of aircraft, quality of maintenance, operational pace, operating environment, etc. Performance Indicator: Accident Rate per 1,000 departures Definition: An occurrence associated with the operation of an aircraft which takes place between the time any person boards the aircraft with the intention of flight and all such persons have disembarked, and in which any person suffers death or serious injury, or in which the aircraft receives substantial damage. Use total departures as a basis to determine rates. Goals: Site specific goals. The goal should be to reduce or eliminate accidents and the trend should be downward over time. DOE G 440.2B-1 12-10-02 Page 8-4 Data Location: Data records may be found at the Aviation safety office, GSA Aviation Accident and Incident Reporting System (AAIRS), DOE ORPS reports, and Aviation Operations office. Factors That Effect The Measure: Personnel qualifications and experience, age of aircraft, quality of maintenance, operational pace, operating environment, etc. Performance Indicator: Fatality Rate per 1,000 departures Definition: Any injury associated with the operation or maintenance of an aircraft which results in death within 30 days of the accident. Use total departures as a basis to determine rates. Goals: Site specific goals. The goal should be to reduce or eliminate fatalities and the trend should be downward over time. Data Location: Data records may be found at the Aviation safety office, GSA Aviation Accident and Incident Reporting System (AAIRS), DOE ORPS reports, and Aviation Operations office. Factors That Effect The Measure: Personnel qualifications and experience, age of aircraft, quality of maintenance, operational pace, operating environment, etc. Performance Indicator: Serious Injury Rate Per 1,000 Departures Definition: Any injury associated with the operation or maintenance of an aircraft which: (1) requires hospitalization for more than 48 hours, commencing within 7 days from the date the injury was received; (2) results in a fracture of any bone (except simple fractures of fingers, toes, or nose); (3) causes severe hemorrhages, nerve, muscle, or tendon damage; (4) involves any internal organ; or (5) involves second- or third-degree burns, or any burns affecting more than 5 percent of the body surface. Use total departures as a basis to determine rates. DOE G 440.2B-1 12-10-02 Page 8-5 Goals: Site specific goals. The goal should be to reduce or eliminate injuries and the trend should be downward over time.

Data Location: Data records may be found at the Aviation safety office, GSA Aviation Accident and Incident Reporting System (AAIRS), DOE ORPS reports, and Aviation Operations office. Factors That Effect The Measure: Personnel qualifications and experience, age of aircraft, quality of maintenance, operational pace, operating environment, etc. Performance Indicator: Program Audit Findings Rate Definition: Total number of findings divided by the total number of audited items. Goals: Site specific goals. The goal should be to reduce or eliminate the number of findings and the trend should be downward over time. Data Location: Data records (checklists, audit forms, etc.) may be found at the Aviation safety office and Aviation operations office. Factors That Effect The Measure: Supervisory controls, management, personnel qualifications and experience, maintenance program complexity, complexity of operational rules, etc. Performance Term: Lost Workday Aviation Incidence Rates Definition: Anytime an employee losses eight hours of work due to an injury sustained during an aircraft operation (ground or flight), maintenance activity, or other work associated with aircraft. Goals: Site specific goals. The goal should be to reduce or eliminate lost work day incidents and the trend should be downward over time. DOE G 440.2B-1 12-10-02 Page 8-6 Data Location: Data records may be found at the Human Resources office and Facilities safety office. Factors That Effect The Measure: Supervisory controls, management, personnel qualifications and experience, program complexity, complexity of operational rules, etc Performance Term: Aviation Property Loss Rates - Excluding Fires Definition: Any property loss over $500 associated with the operation or maintenance of an aircraft that resulted in operation, maintenance, or facilities management action. Goals: Site specific goals. The goal should be to reduce or eliminate aviation property losses and the trend should be downward over time. Data Location: Data records may be found at the property office and Facilities safety office. Factors That Effect The Measure: Location and size of facilities, pace of operations, training, personnel qualifications and experience, etc. DOE G 440.2B-1 12-10-02 Page 9-1 9. COST PERFORMANCE INDICATORS Performance Term: Cost Per Flight Hour Definition: Total Aviation program costs divided by total hours flown for each aircraft. Goals: Site specific goals. The goal should be to reduce the cost per hour and the trend should be downward over time. Data Location: Data records may be found at the AvM’s office, Finance records, and Contractor’s office. Factors That Effect The Measure: Age of aircraft, payroll costs, unscheduled maintenance costs, overhead, overhead charge rates, accident or incident damages and repairs, fuel prices, etc. Performance Term: Cost Per Mile Definition: The total Aviation program costs divided by total miles (statute miles) flown. Goals: Site specific goals. The goal should be to reduce the costs per mile and the trend should be downward over time. Data Location: Data records may be found at the AvM’s office, Finance records, and Contractor’s office. Factors That Effect The Measure: Age of aircraft, payroll costs, unscheduled maintenance costs, accident or incident damages and repairs, fuel prices, etc. Performance Term: Cost Per Pound (Cargo operations) Definition: The total Aviation program costs divided by total number of pounds transported.

Goals: Site specific goals. The goal should be to reduce the costs per pound and the trend should be downward over time. Data Location: Data records may be found at the AvM’s office, Finance records, and Contractor’s office. Factors That Effect The Measure: Whether or not cargo is carried, special handling requirements, area and scope of operations, etc. DOE G 440.2B-1 12-10-02 Page 9-2 Performance Term: Cost Per Seat Definition: The total variable and fixed costs from personnel transport flights divided by the total number of personnel flown. Goals: Site specific goals. The goal should be to reduce the costs per seat and the trend should be downward over time. Data Location: Data records may be found at the AvM’s office, contractor’s office. dispatch office, and passenger manifest. Factors That Effect The Measure: Whether or not personnel are carried, overhead, overhead charges, area and scope of operations, etc. Performance Term: Program Cost Savings Definition: The total number of dollars saved by using the aircraft over conventional means to accomplish a task or mission. The cost savings may include per diem expenses, lodging expenses, and reduced overtime. In addition, this cost measure may include analysis to show reductions in corporate lost revenues. This can be calculated using the difference between down time in infrastructure such as powerlines by comparing time between repair using aircraft and conventional means. Example 1– Scenario 1: Dispatch is aware of a line fault at 9:00 AM. The aircraft is despatched at 9:15 AM to locate fault. At 10:00 AM aircraft crews identify location and type of fault and the equipment and personnel needed to correct fault. Crews arrive at 12:00 PM and fix the fault. Total elapsed time is 3 hours. Scenario 2: Dispatch is aware of a line fault at 9:00 AM. A crew is dispatched by truck at 9:15 AM to locate fault. At 12:00 PM truck crews identify location and type of fault and the equipment and personnel needed to correct fault. Crews arrive at 2:00 PM and fix the fault. Total elapsed time is 5 hours. Scenario 1 lost revenue 3 X $ 1M= $3M Scenario 2 lost revenue 5 X $ 1M=$ 5M $5M - $3M = $2M in costs savings (Total cost benefit) DOE G 440.2B-1 12-10-02 Page 9-3 Example 2– Scenario 1: The aircraft takes two hours to arrive on scene and can survey a ten square mile area by air in four hours. The aircraft and crew cost $ 1400 per hour. The total cost to the organization is $ 8,400.00. Scenario 2: A crew of 50 persons with hand sensors takes 6 hours to arrive on scene and it takes 14 hours to survey the site. The average cost per hour for each person is $ 38.50. The total cost of surveying the site by ground is $38,500.00. Scenario 1 Government costs= $ 8,400.00 Scenario 2 Government costs= $38,500.00 $38,500 - $8,400 = $ 30,100.00 in costs savings (Total cost benefit) Goals: Site specific goals. The goal should be to maximize program costs savings through use of aircraft and the trend should be upward over time. Data Location: Data records may be found at the program managers office, financial records, etc. Factors That Effect The Measure: TBD DOE G 440.2B-1 12-10-02 Page A-1 APPENDIX A — ACRONYMS

AAH Aircraft Available Hours AAR Aircraft Availability Rate ARD Aircraft: Recurring Discrepancy AvM Aviation Maintenance CR Cannibalization Rate CSE Customer Scheduling Effectiveness CWT Customer Wait Time DoD Department of Defense DOE Department of Energy EI Excess in Inventory FCNA Flight Crew Unavailability IAR Inventory Accuracy Rate MCA Mission Capable Aircraft MCNA Mission Crew Unavailability MCR Mission Capable Aircraft Rate Me Mission Equipment ME-2.4 DOE/Office Aviation Management MeOH Me: Operational Hours MeNOH Me: Non Operational Hours MeNOM Me: Non Operable Maintenance MeNOS Me: Non Operable Supply MeRDR Mission Equipment: Recurring Discrepancy Rate MSRT Mean Supply Response Time MTBF Mean Time Between Failure MTTR Mean Time to Repair NAH Non-Airworthy Hours NAM Non-Airworthy Maintenance / Maintenance Downtime NAMR Non-Airworthy Maintenance Rate NAS Non-Airworthy Supply / Supply Downtime NASR Non-Airworthy Supply Rate NMC Non-mission Capable NLT Not Later Than (dates) NNSA National Nuclear Security Administration OAM Office of Aviation Management RDR Aircraft: Recurring Discrepancy Rate SRT Supply Response Time TTR Time to Repair DOE G 440.2B-1 12-10-02 Page B-1 APPENDIX B — DEFINITIONS Aircraft accident Means an occurrence associated with the operation of an aircraft which takes place between the time any person boards the aircraft with the intention of flight and all such persons have disembarked, and in which any person suffers death or serious injury, or in which the aircraft receives substantial damage. Appliance Any instrument, mechanism, equipment, part, apparatus, appurtenance, or accessory, including communications equipment, that is used or intended to be used in operating or controlling an aircraft in flight and is not part of an airframe, engine, or propeller. Assigned Hours (AH) The number of hours in a month, quarter, or year that the aircraft has been assigned to the Field organization. For example the month of May: 31 days x 24 hours x (# same make and model) = Assigned Hours. Cannibalization The act of taking a serviceable part from an aircraft, engine, propellor, or assembly to replace an unserviceable part on an aircraft, engine, propellor, or assembly, rather than using stores from supply. Conditional Inspection An inspection that is required as a result of unusual events, such as an overspeed, hard landing, overtorque, etc. Customer Wait Time (CWT) The total elapsed time between issuance of a customer request (order) and satisfaction of that order. Fatal Injury Means any injury which results in death within 30 days of the accident. Flight From the time an aircraft leaves the surface until it touches down at its destination. Ground Abort A condition that occurs before flight that results in the aircraft not completing its intended mission. Inflight Abort A condition that occurs during flight that results in the aircraft not completing its intended mission. DOE G 440.2B-1 12-10-02 Page B-2 Incident An occurrence, other than an accident, associated with the operation or maintenance of an aircraft which affects or could affect the safety of operations or maintenance. Inspection A method of qualifying the condition or status of the appliance, its systems, and/or accessories to specific standards and requirements. Maintenance Inspection, overhaul, repair, preservation, and the replacement of parts, but excludes preventative maintenance. Maintenance Scheduling Effectiveness Maintenance scheduling effectiveness is obtained by

dividing the total number of scheduled maintenance events that occurred on the date due by total number of scheduled maintenance events, and multiplying by 100 (total on-time events/total events scheduled x 100). Mean Supply Response Time (MSRT) Used in computing the effectiveness of supply for an aircraft, engine, propeller, or appliance or any component of or part of an aircraft, engine, propeller, or appliance. The sum of supply response times divided by the total number of supply responses. Mean Time Between Failures (MTBF) Used in computing the reliability of aircraft and equipment, it is the average elapsed time between failures of an aircraft, engine, propeller, or appliance or any component or part of an aircraft, engine, propeller, or appliance. The total elapsed time between failures of an aircraft, engine, propeller, appliance, or any component or part of an aircraft, engine, propeller, or appliance. Mean Time To Repair (MTTR) Used in computing the maintainability of an aircraft, engine, propeller, or appliance, or any component of or part of an aircraft, engine, propeller, or appliance. The sum of TTRs divided by the total number of repairs. Mission The government function for which the aircraft was dispatched. DOE G 440.2B-1 12-10-02 Page B-3 Mission Capable (MC) The aircraft is airworthy; the flight crew is available, certified, trained, and current; the mission crew is available, trained, and current; the mission equipment is operable and installed or the aircraft is configured properly for the mission, i.e cargo, passenger, etc. Mission Capable Rate The proportion of assigned hours an aircraft, flight crew, mission crew, and mission equipment/configuration is available to meet its assigned primary or secondary mission over a defined period of time (Assigned hours) minus the time the aircraft is not available due to aircraft maintenance downtime, downtime due to aircraft supply, mission equipment maintenance downtime, downtime due to mission equipment supply, mission crew not available, or flight crew not available divided by total assigned hours x 100. Non-mission Capable (NMC) The aircraft is un-airworthy; or the flight crew is not available, certified, trained, or current; or the mission crew is not available, trained, or current; or the mission equipment is not operable or installed or the aircraft is configured improperly for the mission, i.e cargo, passenger, etc. Non-mission Capable Rate (NMCR) The proportion of assigned hours an aircraft is not mission capable to meet its primary or secondary mission requirements. From the above example, if the MCR is 82% then the NMCR is 18%. Preventative Maintenance Simple or minor preservation operations and the replacement of small standard parts not involving complex assembly operations. Repeat/Recur When the same discrepancy occurs on two or more consecutive flights, it is a recurring event. Reportable Incident Estimated damage of $500 or more that involves Federal Government-owned, -rented, or -leased aircraft, equipment, or privately owned aircraft operated while on official business shall be considered a "reportable case" and shall be reported. DOE G 440.2B-1 12-10-02 Page B-4 Return to Service An entry made in the appropriate record by a qualified individual certifying that the appliance, system, or accessory is airworthy after accomplishing the required inspection, test, preventative maintenance or maintenance, IAW manufacturer’s maintenance instructions, or instructions for continued airworthiness.

Scheduled Inspection / Maintenance An inspection or maintenance performed on a calendar, cycle, or hourly basis or a combination of calendar / hourly / cycle basis. Serious injury Means any injury which: (1) Requires hospitalization for more than 48 hours, commencing within 7 days from the date of the injury was received; (2) results in a fracture of any bone (except simple fractures of fingers, toes, or nose); (3) causes severe hemorrhages, nerve, muscle, or tendon damage; (4) involves any internal organ; or (5) involves second- or third-degree burns, or any burns affecting more than five percent of the body surface. Special Inspection An inspection that is performed after completing other maintenance, such as installation of a major component (e.g. replacement of binocular assembly, monocular assembly, intensifier tube, etc.). Substantial damage Means damage or failure which adversely affects the structural strength, performance, or flight characteristics of the aircraft, and which would normally require major repair or replacement of the affected component. Engine failure or damage limited to an engine if only one engine fails or is damaged, bent fairings or cowling, dented skin, small punctured holes in the skin or fabric, ground damage to rotor or propeller blades, and damage to landing gear, wheels, tires, flaps, engine accessories, brakes, or wingtips are not considered "substantial damage" for the purpose of this part. Unscheduled Maintenance An inspection, overhaul, repair, preservation, and the replacement of parts, but excludes preventative maintenance, that occurs between scheduled inspection/maintenance. DOE G 440.2B-1 12-10-02 Page C-1 APPENDIX C — REFERENCES DOE G 120.1-5 (Guide, 06/30/1996, PO/HR), Guidelines for Performance Measurement DOE G 151.1-1 V2 (Guide, 08/21/1997, SO), Hazardous Survey and Hazards Assessments DOE O 200.1 (Order, 09/30/1996, SO), Information Management Program DOE O 210.1 Chg 2 (Order, 05/01/1996, EH), Performance Indicator and Analysis of Operations Information DOE O 224.1 (Order, 12/08/1997, FM), Contractor Performance-Based Business Management Process DOE O 224.2 (Order, 03/22/2001, IG), Auditing of Programs and Operations DOE O 231.1 Chg 2 (Order, 11/07/1996, EH), Environment, Safety, and Health Reporting DOE O 232.1A (Order, 07/21/1997, EH), Occurrence Reporting and Processing of Operations Information DOE M 231.1-1 Chg 2 (Manual, 01/28/2000, EH), Environment, Safety and Health Reporting Manual DOE P 413.1 (Policy, 06/10/2000, CR), Program and Project Management Policy for the Planning, Programming, Budgeting, and Acquisition of Capital Assets DOE O 413.1A (Order, 04/18/2002, ME), Management Control Program DOE O 430.1A (Order, 10/14/1998, FM), Life Cycle Asset Management DOE O 440.2B (Order, 11/27/2002, ME), Aviation Management and Safety DOE O 534.1A (Order, 07/05/2001, CR), Accounting Government Performance and Results Act of 1993 (GPRA)