HQ NORTH AMERICAN AEROSPACE DEFENSE COMMAND (NORAD) NORAD REGULATION 10-601
HQ UNITED STATES SPACE COMMAND (USSPACECOM) USSPACECOM REGULATION 10-601
Peterson Air Force Base Colorado 80914-4100 23 December 1993

Operations

THE INTEGRATED TACTICAL WARNING AND ATTACK ASSESSMENT (TW/AA) SYSTEM CERTIFICATION PROCESS

This regulation establishes the policy and procedures for the Commander in Chief, North American Aero-space Defense Command (CINCNORAD), and the Commander in Chief, United States Space Command (USCINCSPACE) Integrated Tactical Warning and Attack Assessment (TW/AA) System Certification process. It applies to HQ NORAD and its supporting commands and responsible agencies, and HQ United States Space Command (USSPACECOM) and its components.

1. Introduction:

1.1. References:

1.1.1. NR 57-1, NORAD Statement of Operational Requirement.

1.1.2. USSPACECOMR (UR) 57-1, Operational Requirements System.

1.1.3. NR/UR 102-1, NORAD/USSPACECOM Integrated Command and Control System Mission Systems Integration Process.

1.1.4. MCM 172-91, Integrated Tactical Warning and Attack Assessment System Integrity.

1.1.5. USSPACECOM System Management Directive (SMD) for the Integrity of the Integrated Tactical Warning and Attack Assessment System.

1.1.6. Integrated Tactical Warning and Attack Assessment (TW/AA) Configuration Control System Directive (CCSD), Volume 1, Policies and Procedures.

1.1.7.Integrated Tactical Warning and Attack Assessment (TW/AA) Configuration Control System Directive (CCSD), Volume 2, Configuration Baseline.

1.1.8. USSPACECOM Integrated TW/AA Architectural Concept.

1.2. Terms and Definitions. Terms and definitions are listed in attachment 1.

1.3. Acronyms. Acronyms used in this regulation are listed in attachment 2.

________________________
No. of Printed Pages: 14
OPR: J6CCT (Major Joseph Snaufer)
Approved by: Maj Gen Phillip E. Bracher
Editor: Pam Gatson
Distribution: F;X (Joint Staff/J6, Washington, DC 20318-6000........................................................................1
Naval Space Command, Dahlgren, VA 22448-5170........................................................................................5
Army Space Command, Colorado Springs, CO 80916-2749...........................................................................5)
2 NR/UR 10-601 23 December 1993

2. Background:

2.1. USCINCSPACE is assigned responsibility to manage and control the integrity of the Integrated TW/AA System, by Memorandum in the Name of the Chairman, Joint Chiefs of Staff (JCS) (MCM) 172-91, Integrated Tactical Warning and Attack Assessment (TW/AA) System Integrity, 1 Oct 91. USCINCSPACE is responsible to the President and the Secretary of Defense, through the Chairman, Joint Chiefs of Staff, for the integrity of the Integrated TW/AA System.

2.2. While "total system" integrity includes functions performed by several organizations belonging to NORAD and USSPACECOM, this regulation focuses on the integrity and technical performance aspects of that responsibility. The term, CINCNORAD/USCINCSPACE indicates that the Integrated TW/AA System is spread across these two organizations sharing one commander.

2.3. CINCNORAD/USCINCSPACE accomplishes these responsibilities through mission systems integration oversight; the establishment of direction and guidance in technical (engineering, interface, and testing) standards; issuing configuration management oversight, guidance and policy; developing and applying architectures; system/element testing oversight and certification; and system-wide technical integrity testing. CINCNORAD/USCINCSPACE has review and approval authority for the addition of new equipment or software within or interfacing with the Integrated TW/AA System. CINCNORAD/USCINCSPACE may delegate this authority to either the HQ NORAD Director of Combat Operations (J3) or the USSPACECOM Director of Operations (J3) as appropriate. CINCNORAD/USCINCSPACE works directly with Office of the Secretary of Defense, the Joint Staff, unified and specified commands, services, defense agencies, non-DoD agencies, and foreign governments on these matters.

3. Policy. USCINCSPACE designated the Director for Command Control Systems and Logistics (HQ USSPACECOM/J4-J6) as the office of primary responsibility for the integrity of the Integrated TW/AA System. The Deputy Director for System Certification (HQ USSPACECOM/J6C) is responsible for System Certification, testing oversight, and integrated end-to-end testing of the Integrated TW/AA System.

4. Scope:

4.1. This regulation applies to HQ NORAD and NORAD responsible agencies, USSPACECOM components and other commands and agencies that participate as members of the Integrated TW/AA System community.

4.2. The implementation of this regulation in other commands and agencies not under CINCNORAD operational control or USCINCSPACE Combatant Command will be done through Memorandums of Agreement (MOA). Service component commanders obtain review and approval from CINCNORAD/USCINCSPACE for the addition of new equipment or software, or modifications to existing equipment, hardware or software, within or interfacing with the Integrated TW/AA System to ensure integrity is maintained. CINCNORAD/ USCINCSPACE may delegate this authority to either the HQ NORAD/J3 or the HQ USSPACECOM/J3 as appropriate.

4.3. The focus of this regulation is the certification of the Integrated TW/AA System of systems after assessing the impact of any changes to the System or changes to interfaces to the System that may affect its integrity and overall technical performance. Certification stipulates proper Integrated TW/AA System evolution, consistent performance, and nondegradation of the system.

NR/UR 10-601 23 December 1993 3

4.4. The Integrated TW/AA System provides support to some of the seven missions assigned to CINCNORAD/USCINCSPACE. The missions are Integrated Tactical Warning and Attack Assessment, Air Defense, Air Sovereignty, Force Enhancement, Force Application, Space Control, and Space Support. The NORAD/USSPACECOM Integrated Command and Control System (NUICCS) is a worldwide, multinational, multicommand, multimission integrated command and control "system of systems" supporting CINCNORAD/USCINCSPACE. Expansion of the System Certification process to include NUICCS missions other than Integrated TW/AA may be considered at a future date.

4.5. System Certification is normally required before a change to the Integrated TW/AA System or a change to an interfacing system is permitted to begin unrestricted operational use. The exception is in the event of an overriding operational need to activate a capability that has been determined to be uncertifiable. The responsible operational authority makes such a decision. In that case, the System Certification process serves in an advisory role to identify the deficiencies and risks, so operational workarounds or other means may be employed to mitigate those risks.

5. System Certification. The following paragraphs outline the process for certifying changes to the Integrated TW/AA System. It explains the purpose of System Certification, System Certification procedures, requirements of the System Certification process, and the requirements for documenting this process.

5.1. Purpose of System Certification. The purpose of System Certification is to independently confirm to CINCNORAD/USCINCSPACE that the Integrated TW/AA System, or, subsystem or element is technically capable of accomplishing its assigned functions and does not adversely impact the Integrated TW/AA System, or degrade Integrated TW/AA System performance below operationally required performance levels.

5.1.1. This involves independently assessing integrity, to include communications interfaces; processing and display activities; conformance to established and required technical standards; interoperability with associated systems; and technical performance requirements.

5.1.2. This independent assessment is a continuous process encompassing all aspects of the change process. This assessment is based on information gathered through reviews of validated requirements documentation; acquisition, technical and programmatic reviews; observation of development; participation in test planning activities; reviews of test plans and procedures; and independent assessment of test results and program or project documentation.

5.1.3. HQ USSPACECOM/J6C asks for technical assessments of the change under consideration for System Certification from the appropriate Integrated TW/AA System community members when necessary.

5.2. System Certification Process Overview. The System Certification process involves essentially four phases: identification , classification, assessment, and decision.

5.2.1. Identification phase. HQ USSPACECOM/J6C is notified of an approved change to the Integrated TW/AA System.

5.2.2. Classification Phase. A HQ USSPACECOM/J6C Action Officer (AO) is assigned to staff the change. The AO prepares a System Certification Class recommendation for System Certification Board approval. The assigned System Certification Class determines the level of risk associated with the change and determines USSPACECOM/J6C's level of involvement in certifying the change. Next, the AO prepares the System Certification Plan for System Certification Board approval. The System Certification Plan establishes and documents the certification strategy.

4 NR/UR 10-601 23 December 1993

5.2.3. Assessment Phase. The assessment phase extends from approval of the System Certification Plan through change development, Development Test and Evaluation (DT&E), Operational Test and Evaluation (OT&E), and ends with Operational Approval. This phase involves the actual monitoring of the change and its implementation and integration into the Integrated TW/AA System.

5.2.4. Decision Phase. The System Certification process ends with a HQ USSPACECOM certification decision. There are four possible Certification actions: certify, certify with caveats, refuse to certify, and decertification.

5.3. Changes Excluded From System Certification. The following classes of changes are excluded from the HQ USSPACECOM/J6C System Certification:

5.3.1. Hardware maintenance changes not involving a corresponding software/firmware change.

5.3.2. Communications maintenance changes not involving a corresponding software/firmware change.

5.3.3. Temporary changes reviewed by HQ USSPACECOM/J6C to assess their potential integrity and technical performance impact, and to ensure adequate safeguards are in place prior to change implementation. Temporary changes are also configuration controlled in accordance with the appropriate configuration control processes. Temporary changes that become permanent are evaluated for certification.

5.3.4. Off-line products such as simulators, diagnostic routines, data reduction software, and test and development software.

5.4. Change Identification:

5.4.1. HQ USSPACECOM/J6C involvement begins with notification of a change to the Integrated TW/AA System.. For new system acquisitions, HQ USSPACECOM/J6C notification must occur immediately after Milestone 0 approval. For changes to existing systems, notification is in accordance with the appropriate configuration management processes.

5.4.2. For Integrated TW/AA Systems where HQ NORAD or a responsible or supporting agency is the user or user representative, HQ USSPACECOM/J6C reviews requirement and certain other acquisition documentation in accordance with the applicable MOA. HQ USSPACECOM/J6C reviews requirements and certain other acquisition documentation in accordance with USSPACECOMR 57-1 for Integrated TW/AA Systems where HQ USSPACECOM or a service component is the user or user representative.

5.4.3. USSPACECOM service component commands (through USSPACECOMR 57-1), NORAD responsible agencies or other commands/agencies, in accordance with their respective MOAs, provide USSPACECOM/J6C access to program/change documentation.

5.5. System Certification Board:

5.5.1. The System Certification Board encompasses the classification phase and has two functions. First, the board assigns a System Certification Class (Risk Assessment) to each new change brought before the board. Second, the board evaluates and approves a System Certification Plan. Each change requires a System Certification Plan.

NR/UR 10-601 23 December 1993 5

5.5.2. The System Certification Board considers changes to systems currently in the System Certification Baseline or systems being added to the baseline. The decision to add a system to the System Certification Baseline is done on a case-by-case basis. For acquisition systems, once the decision to add the system to the System Certification Baseline is made, the integrity and technical performance requirements for certifying the system will be provided to the affected agencies for inclusion in the appropriate requirements documentation. HQ USSPACECOM/J6C makes the decision to add a system to the System Certification Baseline and notifies the applicable agencies.

5.5.3. The basis for the board's class decision is a subjective assessment of the change. A System Certification Class I, II, or III (risk assessment) is assigned to the change. The Class I assessment is considered high risk, Class II moderate, and Class III low risk, based on the potential to impact overall integrity or technical performance.

5.5.4. The only formal HQ USSPACECOM/J6C System Certification requirement for a Class III certification is for the Operational Approval Authority to notify HQ USSPACECOM/J6C of the Operational Approval date.

5.5.5. System Certification Board Membership. HQ USSPACECOM/J6C chairs the board. The HQ USSPACECOM Chief, Sensor Systems and Testing Division (HQ USSPACECOM/J6CC); HQ USSPACECOM Chief, Command and Correlation Centers Division (HQ USSPACECOM/J6CD); and the HQ USSPACECOM/J6C Technical Advisor comprises the board membership. Representatives from NORAD, USSPACECOM, the service components, and responsible/supporting agencies are invited to serve as advisors to the board.

5.5.6. System Certification Board Minutes. The board secretariat prepares board minutes for HQ USSPACECOM/J6C approval. HQ USSPACECOM/J6C approval of the minutes constitutes formal System Certification Class assignment for approved changes presented to the board and approval of System Certification Plans for previously briefed changes.

5.5.7. Integrated TW/AA System Community Notification. Following approval of the board minutes, the board secretariat distributes a copy of the minutes to the board members and affected members of the Integrated TW/AA System community.

5.5.8. Review of Out-of-Cycle Changes. Out-of-cycle changes, emergency and non-emergency, require System Certification Board review for risk assessment.

5.6. System Certification Plan:

5.6.1. HQ USSPACECOM/J6C prepares a System Certification Plan for each change and coordinates it with other organizations as appropriate prior to briefing the System Certification Board. The plan establishes and documents the certification strategy, and identifies the major criteria, elements, and milestones for System Certification. This plan may require updating as the change evolves. The board must approve major changes to the plan.

5.6.1.1. System Certification Criteria is derived from the following: conformance with approved and documented requirements; conformance with Integrated TW/AA System architecture; conformance with Integrated TW/AA System engineering and integration standards; conformance with Integrated TW/AA System configuration management policy and directives; conformance with Integrated TW/AA System certification policy and procedures; and other appropriate established policy and guidance as indicated in the System Certification Plan.

5.6.1.2. System Certification Elements are divided into two categories: Integrity Elements and Technical Performance Elements.
6 NR/UR 10-601 23 December 1993

5.6.1.2.1. Integrity Elements must be from approved requirements documents and may include, but are not limited to, accuracy, availability, credibility, display correlation, reliability, timeliness, and unambiguous.

5.6.1.2.2. Technical Performance Elements must be from approved requirements documents and may include, but are not limited to, message accountability, message accuracy, message handling, message integrity, message processing, message throughput, and system response time.

5.6.1.3. System Certification Suitability Assessment Milestones, as defined in the certification plan, may include, but are not limited to, Preliminary Design Reviews (PDR), Critical Design Reviews (CDR), DT&E entry or completion, OT&E entry or completion, and others as needed. The System Certification Plan will list any other requirements or criteria as needed.

5.7. Level of Involvement:

5.7.1. System Certification Class (Risk Assessment) determines the level of future HQ USSPACECOM/J6C involvement in the change certification. The System Certification Plan documents the actions required.

5.7.1.1. For System Certification Class I, HQ USSPACECOM/J6C involvement focuses on monitoring change or acquisition development, participating or coordinating in the test planning process, monitoring DT&E and OT&E, providing System Certification Suitability Assessments, and either certifying or refusing to certify the change before operational approval.

5.7.1.2. For System Certification Class II, HQ USSPACECOM/J6C focuses on monitoring change or acquisition development, monitoring DT&E and OT&E as required, providing System Certification Suitability Assessments, and either certifying or refusing to certify the change before operational approval.

5.7.1.3. For System Certification Class III, HQ USSPACECOM/J6C involvement consists of documenting the change content and implementation details when the change has received operational approval.

5.7.1.4. For System Certification Class II or III changes, HQ USSPACECOM/J6C may delegate the evaluation of a change to a USSPACECOM component command, NORAD responsible agency, or another organization with an established Integrated TW/AA System memorandum of agreement with USSPACECOM. Such delegations must be approved by HQ USSPACECOM/J4-J6, and would be to an organization already performing an independent test and evaluation role. Designations of such delegations will be in writing and in coordination with the designated organization.

5.7.1.4.1. In the event an evaluation is delegated, the designated organization develops the System Certification Plan for System Certification Board approval, and recommends appropriate System Certification Suitability Assessments for HQ USSPACECOM/J6C approval. HQ USSPACECOM/J6C makes the final System Certification decision based on information provided by that organization, augmented by any information independently gathered by HQ USSPACECOM/J6C.

5.8. Change Assessment:

5.8.1. The change assessment phase extends from approval of the System Certification Plan through change development, DT&E, OT&E and ends with Operational Approval. For the Integrated TW/AA System, this period is divided into an OT&E phase and a Trial Period (TP).

5.8.2. USSPACECOM service component commands (through USSPACECOMR 57-1), NORAD responsible agencies or other commands or agencies in accordance with their respective MOA provides HQ USSPACECOM/J6C access to schedules and meetings, DT&E/OT&E plans and procedures, test schedules and access to tests on a noninterference basis, and DT&E/OT&E test results.
NR/UR 10-601 23 December 1993 7

5.8.3. HQ USSPACECOM/J6C participates in DT&E/OT&E.

5.8.4. If applicable, protocol conformance testing is conducted by HQ USSPACECOM/J6C during DT&E.

5.8.5. For Class I and II changes, HQ USSPACECOM/J6C assists the implementing command or agency to ensure DT&E/OT&E planning and testing satisfies integrity and technical performance requirements.

5.8.5.1. For Class I changes, HQ USSPACECOM/J6C reviews and coordinates on the Test and Evaluation Master Plan (TEMP) and DT&E/OT&E test plans.

5.8.5.2. Prior to the start of OT&E of a Class II change, the OT&E command or agency provides HQ USSPACECOM/J6C access to OT&E test documentation for review.

5.8.6. Following successful OT&E, Operational Integration Working Group (OIWG) and/or the Operational Integration Review Board (OIRB), if appropriate, decides if the changed system is ready to enter TP. HQ USSPACECOM/J6C acts as an advisor to these boards.

5.8.6.1. For System Certification Class I changes, HQ USSPACECOM/J6C provides a System Certification Suitability Assessment to the operational community before the change enters TP. If the OIWG and/or OIRB is the approval authority for TP entry, the HQ USSPACECOM/J6C representative presents the assessment. For changes the OIWG and/or OIRB does not review, or when an assessment is provided to an agency not represented by the OIWG and/or OIRB, the assessment is sent to the appropriate Integrated TW/AA System community members.

5.8.7. The conduct of TP is the responsibility of the operating command. The operating command provides notification to HQ USSPACECOM/J6C prior to the change entering TP.

5.8.7.1. During TP, HQ USSPACECOM/J6C performs a final evaluation of the change for System Certification.

5.8.8. The System Certification Suitability Assessment addresses any deficiencies discovered. Negative assessments may require corrective action by the implementing agency prior to final System Certification.

5.9. System Certification Actions:

5.9.1. There are four possible HQ USSPACECOM/J6C System Certification actions: certify (no caveats), certify with caveats, refuse to certify, and decertification.

5.9.1.1. Certify (no caveats): HQ USSPACECOM/J6C certifies when review of the change implementation indicates the changed element's integrity and technical performance requirements are met, and the change does not degrade system end-to-end technical integrity.

5.9.1.2. Certify with caveats: HQ USSPACECOM/J6C certifies with caveats when review of the change implementation reveals problems with the element's integrity or technical performance that, in the assessment of HQ USSPACECOM/J6C, do not warrant refusal of certification. HQ USSPACECOM/J6C tracks actions to resolve caveats, and reassesses the certification status as appropriate. Caveats are normally restricted to capabilities that are implemented but have not been fully tested.

5.9.1.3. Refuse to certify: HQ USSPACECOM/J6C refuses to certify when review of the change implementa-tion reveals serious deficiencies with the element's integrity or technical performance that in the assessment of HQ USSPACECOM/J6C warrant refusal of certification. The system will be reconsidered for certification when the identified serious deficiencies have been corrected and successfully tested.

8 NR/UR 10-601 23 December 1993

5.9.1.4. Decertification: Only applies to systems which are currently certified and to a condition in a certified system which could adversely affect technical integrity or technical performance.

5.9.1.4.1. HQ USSPACECOM J4-J6 provides written notification of decertification actions to CINCNORAD/ USCINCSPACE, the appropriate NORAD/USSPACECOM agencies, and applicable members of the Integrated TW/AA System community.

5.9.1.4.2. The decertified system is recertified when the proper corrective action is successfully tested/demonstrated.

5.9.2. Class I or II System Certification, Refusal to Certify, and Decertification recommendations are approved by HQ USSPACECOM/J6C and validated by HQ USSPACECOM/J4-J6 before promulgation to the Integrated TW/AA System community.

5.9.3. Class III System Certification recommendations are approved by the appropriate HQ USSPACE-COM/J6C division chief, validated by HQ USSPACECOM/J6C, and provided to the appropriate Integrated TW/AA System community members for information.

5.9.4. The designated operational authority will notifies HQ USSPACECOM/J6C of TP exit for the change.

5.10. Emergency Change Notification:

5.10.1. Emergency changes require System Certification, but the certification process can be modified to accommodate the operational urgency. The following process is used:

5.10.1.1. Notification of the emergency change is accomplished in accordance with the appropriate configuration management processes.

5.10.1.2. The implementing agency tests and installs the change, and provides test procedures and test results immediately following change implementation. A HQ USSPACECOM/J6C TP assessment is not required, but the test is monitored and the certification decision is based upon test results.

5.10.1.3. Simultaneously, the System Certification Board reviews the change, assigns a System Certification Class, and determines when the final certification decision can be made based on the implementing agency's testing results.

5.11. Caveats and Serious Deficiencies:

5.11.1. HQ USSPACECOM/J6C tracks caveats and serious deficiencies until the problems are resolved. HQ USSPACECOM/J6C also requests the appropriate NORAD/USSPACECOM agency identify an Office of Primary Responsibility (OPR) for each caveat. This OPR should ensure that HQ USSPACECOM/J6C is notified of the disposition of the problem. If the operational community decides to correct the problem, HQ USSPACECOM/J6C monitors the implementation and removes the caveat or serious deficiency in a subsequent certification action.

5.12. Waivers:

5.12.1. Requests for waivers to Integrated TW/AA standards required for System Certification for a specific change or program are considered on a case-by-case basis.

NR/UR 10-601 23 December 1993 9

5.12.2. Agencies wishing to apply for waivers to this regulation or to System Certification requirements as outlined in a specific System Certification Plan must request such waivers in writing to HQ USSPACECOM/J6C. Requests for waivers to established guidance and standards to the Integrated TW/AA System must be presented to the Integrated TW/AA System Configuration Control Board for approval/disapproval.

5.12.3. Requests for waivers are acted upon promptly by HQ USSPACECOM/J6C.

OFFICIAL CHARLES A. HORNER
General, USAF
Commander in Chief

ROBERT F. KOENIG, JR. 2 Attachments
Colonel, USAF 1. Terms and Definitions
Director of Information Management 2. Acronyms
10 NR/UR 10-601 23 December 1993

DEFINITIONS

A1.1. Accuracy. System products (event reports, displays, etc.) provided to the user contain information reflective of ground truth to the degree required by operations.

A1.2. Additional System Interface. Interface allowing Integrated TW/AA System to either receive informa-tion from sources outside the system or provide system data to non-Integrated TW/AA System users. This interface is part of Integrated TW/AA System and will be considered for HQ USSPACECOM/J6C System Certification.

A1.3. Availability. The system meets the operationally required availability factor.

A1.4. Caveat. A technical integrity deficiency which, by itself, does not warrant decertification or withholding System Certification. Caveats may be specified in conjunction with a certification action. They may be specified for changes which could not be tested.

A1.5. Change. Any temporary or permanent modification, addition (acquisition), or deletion to an Integrated TW/AA System configuration item, static data base, or a Integrated TW/AA System interface to a Non-Integrated TW/AA System.

A1.6. Communications Maintenance Change. A form, fit, and function change which does not change the current configuration or effect the operability of the communications system/subsystem, (e.g., replacing a failed card with a like card, or replacing a failed modem with a like modem, or replacing a communications link with a like communications link). These changes are not reported to or certified by HQ USSPACE-COM/J6C.

A1.7. Correctly. The system will produce the same exact results each and every time the same exact inputs are provided to the system and those results are the operationally required and expected results.

A1.8. Credibility. The system demonstrates a high degree of accuracy and consistency at all levels of operation thus promoting user confidence that the system will provide the operationally required and expected results.

A1.9. Data Bases. In general, system data bases can be divided into three types--static, dynamic, and transient.

A1.9.1. Static data bases do not change or change very infrequently (e.g., files describing message formats, message routing, or display formats) and are not changed by mission application programs, messages, or operator input.

A1.9.2. Dynamic data bases remain in the system over time but may change as the result of operator input (e.g., location of threat points).

A1.9.3. Transient data bases are generated, maintained, and deleted by the system (including messages and applications programs).

A1.10. Decertification. Formal HQ USSPACECOM/J6C declaration that a condition exists in a certified system which could adversely affect technical integrity or technical performance.

A1.11. Developmental Test and Evaluation (DT&E). Testing during development that demonstrates that design and development is complete, that the design risks have been minimized, and that performance is as required and specified.
NR/UR 10-601 Attachment 1 23 December 1993 11

A1.12. Display Correlation. All required system display devices at all locations accurately, unambiguously, and concurrently display visual information related to specific system stimuli in a manner which precludes misinterpretation.

A.1.13. Emergency Change. A priority assigned to prevent one or more of the following: a serious compromise of national security; fatal or serious injury to personnel; or extensive loss or damage of equipment. Emergency change determination and notification will be accomplished in accordance with the appropriate configuration control processes.

A1.14. End-to-End System. From the face of the sensor to the face of the user display, i.e., the point at which an input/stimulus is received at the sensor, through the intermediate communications processing facilities, the correlation/assessment centers, to the displays or message receptacles at the command centers and forward user sites.

A1.15. Engineering Guidance. Direction for developers/maintainers describing what to build into a system(s) or part(s) thereof to satisfy requested characteristics and capabilities. This guidance provides a framework for building open systems and/or interoperable system components. Some examples of engineering guidance are system/segment architectures, style guides, tenets, and principles.

A1.16. Hardware Maintenance Change. A form, fit and function, hardware only, change which does not affect the current configuration or operability of the hardware and is transparent to operations (e.g., replacing a failed card with a like card). These changes are not reported to or certified by HQ USSPACECOM/J6C.

A1.17. Integrated TW/AA System: A "system of systems" consisting of ballistic missile, space, and atmospheric warning sensors; intelligence indicators; associated communications links; operations and command centers, correlation nodes; forward users; and standard processing and display equipment.

A1.18. Integrated TW/AA System Configuration Control Baseline. The subset of the Integrated TW/AA System controlled under the Integrated TW/AA Configuration Control System Directive (CCSD). Volume II of the Integrated TW/AA System CCSD is also called the Integrated TW/AA Operational Baseline.

A1.19. Integrity. That condition of a system which implies that it will perform its assigned function correctly, reliably, and within the timeframe required by the operational mission. When more than one system provides the same or similar result, those results must agree to the extent that they may be interpreted by the next system(s) in the flow, or the end user, without any ambiguity in meaning.

A1.20. Integrity Certification Elements. The elements of integrity used to support the system certification process (accuracy, availability, credibility, display correlation, reliability, timeliness, and unambiguity).

A1.21. Message Accountability. All messages transmitted from the source are received at the specified destination unless intentionally discarded.

A1.22. Message Accuracy: Transmitted messages contain the required information for each field and each field's contents contain information reflective of ground truth to the degree required by operations.

A1.23. Message Handling: Proper conveyance of messages once they leave the application processor and while en route to a distant application processor, which ensures no messages are lost during transmission and ensures no corrupted messages are processed at the distant end.

A1.24. Message Integrity. Format and content of received messages are consistent with the format and content of transmitted messages.

12 NR/UR 10-601 Attachment 1 23 December 1993

A1.25. Message Processing. Messages are correctly processed and provide the operationally required outputs.

A1.26. Message Throughput. The system is able to process and transmit messages at the operationally required rate.

A.1.27. Operational Test and Evaluation (OT&E). Programs structured to determine the operational ef-fectiveness and suitability of a system under realistic combat conditions and to determine if the minimum acceptable operational performance requirements as specified in the Operational Requirements Document have been satisfied.

A1.28. Out-of-Cycle Change. Any change implemented outside the normal change process described in the appropriate configuration management processes.

A1.29. Reliability. The system will continue to function under all projected loads and processing conditions, and will respond in a predictable manner to all loads or processing conditions (exception conditions) not projected to be encountered. Exception processing will ensure the assigned function or mission can be continued within operationally required parameters.

A1.30. Serious Deficiency. A technical integrity or technical performance deficiency which warrants decert-ification or withholding System Certification. Generally, problems affecting data integrity, data accountability, or which prevent unrestricted operational use would be considered showstoppers. Showstoppers will not be caveated.

A1.31. System Certification. Formal J6C declaration that a change to the Integrated TW/AA System, or to an interface to a system external to the Integrated TW/AA System, satisfies technical integrity and technical performance requirements and does not degrade the present Integrated TW/AA System. System Certification warrants to the CINCNORAD/USCINCSPACE that the system will continue to operate as required after the change becomes operational--in short, that they can have confidence that it works as it is supposed to.

A.1.32. System Certification Baseline. The subset of the Integrated TW/AA Configuration Control Baseline currently certified by J6C.

A1.33. System Certification Class I. Any change which has significant potential to impact technical integrity or technical performance. Generally, changes which have an interoperability or mission processing impact are assigned a System Certification Class I.

A1.34. System Certification Class II. Any change which has potential to impact technical integrity or technical performance. Generally, significant system-unique changes are assigned a System Certification Class II.

A1.35. System Certification Class III (Risk Assessment). Any change which has minimal potential to impact technical integrity or technical performance. Generally, data base changes are considered System Certification Class III.

A1.36. System Certification Plan. The formal plan that identifies the requirements, key milestones, and criteria used to certify the change. The complexity of the plan increases with the impact of the change and the assigned System Certification Class.

A1.37. System Response Time. The amount of time to detect, process, and report an event.

NR/UR 10-601 Attachment 1 23 December 1993 13

A1.38. Technical Performance. Measurement of the system's capability to fulfill its operational functions under all required conditions. Technical performance must be demonstrated during OT&E testing and lack of conformance with Integrated TW/AA System Standards may preclude System Certification.

A.1.39. Technical Performance Certification Elements. The elements of technical performance used to support the system certification process (message accountability, message accuracy, message handling, message integrity, message processing, message throughput, and system response time).

A1.40. Temporary Change. A change implemented for a short time, typically one to two weeks, which is not intended to be implemented permanently in a subsequent version. Temporary changes are normally used for test data capture.

A1.41. Timeliness. System products (event reports, displays, etc.) are provided to the user in the operationally required time frames.

A1.42. Trial Period. A formally scheduled period of time to verify and validate the mission capabilities and operational suitability of a system in an operational environment. This period begins after the successful conclusion of Operational Test and Evaluation (OT&E). The operational command conducts this event using production and organic resources. The period of time for this event depends on the complexity of the system or equipment acquired or changed.

A1.43. Unambiguous. System products (event reports, displays, etc.) provided to the user are clearly presented and are properly defined.

A1.44. Version. An identified and documented collection of hardware, software, firmware, or communication media configuration changes scheduled to be released at a specific date.
14 NR/UR 10-601 Attachment 2 23 December 1993

ACRONYMS

AFSPACECOM Air Force Space Command

CCSD Configuration Control System Directive

CDR Critical Design Review

CI Configuration Item

CINCNORAD Commander in Chief, North American Aerospace Defense Command

CMAFB Cheyenne Mountain Air Force Base

CONOPS Concept of Operations

DT&E Development Test and Evaluation

JCS MCM Memorandum in the Name of the Chairman, Joint Chiefs of Staff

MOA Memorandum of Agreement

MNS Mission Need Statement

NORAD North American Aerospace Defense Command

NUICCS NORAD/USSPACECOM Integrated Command and Control System

OIRB Operational Integration Review Board

OIWG Operational Integration Working Group

ORD Operational Requirement Document

OT&E Operational Test and Evaluation

PDR Preliminary Design Review

TEMP Test and Evaluation Master Plan

TP Trial Period

TW/AA Tactical Warning and Attack Assessment

USCINCSPACE Commander in Chief, United States Space Command

USSPACECOM United States Space Command