Index

February 3, 2000

TO: Honorable Senate Committees and House Committees

and Honorable Government Officials

FROM: Dr. Nira Schwartz

2800-187 Plaza Del Amo

Torrance, CA. 90503

Email: nira7@aol.com

Tel: 310-787-8557

Subject:

    1. Qui-Tam Case CV96-3065 RAP Federal Court Los Angeles Related to Exoatmospheric Kill Vehicle (EKV) Contract # DASG60-90-C-0165.
    1. Government Fraud and False Project/Technologies
    2. Please have the Government intervene in my Qui-Tam Case
    3. Please investigate the "Crime" and "covering-up" created by BOEING/TRW/NRC and some Government employees

 

Dear Honorable Officials,

I need your help to prevent the Government fraud and false EKV project/technologies. Please intervene in my Qui-Tam case. So far the Government declined to intervene in this Qui-Tam Case. I do believe that the Government employees that elected not to intervene in my case are not representing the true interest of the Government and/or our Nation.

  1. Nutshell Description
  2. For over ten years Contractor BOEING/TRW/NRC provided fraud and false technologies relative to a project known as Exoatmospheric Kill Vehicle (EKV). Contract # DASG60-90-C-0165. Exo-atmospheric-kill-vehicle.
  3. BOEING/TRW/NRC provided to the Government false EKV performance reports, false test results, false test procedures, false robustness evaluation, false Risk Reduction test results and analysis. False discrimination performance that was based on alleged prior knowledge that was stated by the Government Technical Requirement Document (TRD) not to be available.
  4. Mr. Roberson at GBI/ARMY Huntsville Alabama sabotaged the DCIS/SCHWARTZ investigation, and concealed the Kalman Filter (KF) discrimination technology test results that pointed to performance that did not comply with the Government TRD performance requirements. This to cover up for the BOEING/TRW/NRC milking the Government for over ten years. Also to cover up for his wrong doing, of not reading the performance reports properly.
  5. Mr. Boster at GBI/ARMY Huntsville Alabama sabotaged the DCIS/SCHWARTZ investigation and openly refused to read the DCIS reports created by Special Agent Samuel Reed. Mr. Boster authorized false flight tests of the EKV project, and removed from the attacking scenarios deployed lethal objects that he became aware that the BOEING/TRW/NRC technologies will not discriminate, while reporting to his superiors of no problems.
  6. BOEING/TRW/NRC/POET reported false excellent EKV discrimination performance with the IFT-1A flight data. While their test result show that the PAT (Probability to Assign a Target) was less than 40% of what they falsely reported to the Government, with no way to correct it.
  7. Mr. Kith Englander and Mr. Charles Rando BMDO/DOD at Defense Pentagon in Washington knowingly approved and accepted false test performance results that were based on after the fact analysis. Approved discrimination of technology that must relay on parameters and their values not available in combat. Mr. Englander approved discrimination technology for over 10 years that must get close to the object within xx seconds to make discrimination but too late to maneuver to another object if it was the wrong object. Both approved kinetic interception knowingly that will not perform when the deployed objects are with xx attachment and/or yy attachment and/or with zz deployed motions. All defined within the TRD. Both ignored any conclusions from the Chinese missiles.
  8. LtCol. John B. Hoffman, United Sates Army Legal Services, Arlington VA, and Mr. Dennis Egan DOJ (Tel: 202-307-0240) wrote a letter on behalf of the Government decline intervention in SCHWARTZ Qui-Tam case that was based on lies.
  9. The project was originated in and about 1990. DCIS/SCHWARTZ investigation lasted since 1996 until 1999. Schwartz personally met the people listed above. For years SCHWARTZ also wrote to Secretary of Defense Honorable William S. Cohen, DIRKSEN, Secretary of Defense, 1000 Defense Pentagon, Washington, DC 20301-1000, and to Senate members but was ignored.
  10. Even though Government officials and Army Generals are aware of the False BOEING/TRW/NRC EKV technology, million of dollars are still spend on flights testing this fraud. False interceptions with alleged missiles are provided to the public, while they were ‘arranged’ and that in real life with the use of enemy missile/deployment hardware/conditions these interceptions will not occur.

  11. Why not to develop a true technology. Why spending the public time and money on technology known not to work?
  12. The Result of the covering up by Government officials is that BOEING was awarded the EKV Lead System Engineer (LSI) and TRW was awarded the System Engineer of the EKV program. Both companies were awarded nine billion dollars as a price for their "crime" and "covering up" as Special Agent Reed of DCIS reported on 1999 (enclosed) and was forced to retire, and the Government Officials harassed and retaliated against SCHWARTZ, and declined intervention in her case based on lies. [Government and Government officials are endangering the Public safety for their own political benefits].
  13. The Problem is that the Government officials that are suppose to bring the fraud and false companies and technologies to Justice are part of the fraud.
  14. The performance reports of BOEING/TRW/NRC of the EKV program were false. To continue the cover up, Mr. Englander of DOD/BMDO falsely elected POET companies (such as Lawrence Livermore, MIT, and Aerospace Corporation) to be independent evaluator of the EKV BOEING/TRW/NRC. While Mr. Englander carefully selecting these companies to assure their final evaluations will be within his "request", since all of these companies were retained by BOEING/TRW to perform work on the same EKV project, or worked intensively with BOEING/TRW. Special Agent Reed requested to perform independent evaluation of the EKV technology and was denied, by lack of money.
  15. The EKV discrimination based on an IR sensor does not work and will not work. The BOEING technology will not discriminate the lethal object deployed out of the decoys. The EKV will not destroy/intercept with an object that has an "xxx" (classified shape that the enemy use), or a "yyy" (classified shape that the enemy use) or posses zzz motion, that the enemy missiles deployed objects have, or have the enemy deployed mechanism, in a kinetic impact. All are a large percentage of the enemy lethal deployed objects inventory, and identified within the Government TRD (Technical Requirement Document).
  16. A Qui-Tam case #CV96-3065 Honorable Judge Paez, federal Court Los Angeles. Where the Government declined to intervene.
  17. The Honorable Judge Paez Los Angeles Federal Court dismissed the Qui-Tam case unless SCHWARTZ finds a lawyer within 30 days from today.
  18. Local Rule 2.1 does not allow SCHWARTZ as a Pro se to represent the Government in a Qui-Tam case.
  19. SCHWARTZ may be connected at Tel: 310-787-8557, fax: 310-782-9074, Email: nira7@aol.com, or at 2800-187 Plaza Del Amo, Torrance, CA. 90503.
  20. SCHWARTZ has DCIS/DOD-IG documents to support her allegations. SCHWARTZ has tests result she performed, the POET performed, and BOEING/TRW performed.
  21. SCHWARTZ requests to have the Government intervene in the Qui-Tam Case. Investigate and stop the fraud, and punish the Companies and Government officials that knowingly supported the fraud.
  22. Please initiate a congressional investigation reflecting these above issues.
  23. No point writing the Inspector General of the Defense Department DOD-IG requesting a renewed investigation. They are aware of the problem, and were part of causing/forcing Special Agent Reed to retire. The only person that was honest and lost his job within the Government to protect our Nation.
  24. For ten years we tested technologies that do not work, and will not work. The enemy is improving his Deployed objects, while we are playing interception with out-dated deployed objects, and with technologies that will not work, and known not to be working since 1979.
  25. Enclosed are three documents. Mr. Roy Danchick Declaration, Special Agent Reed reports.

Please intervene in my Qui-Tam case. Please investigate and stop the fraud in the EKV project/program.

 

Thanks,

Dr. Nira Schwartz

CC: Mr. Egan – DOJ Fax: 202-514-7361

Mr. William S. Zanca-DOD Fax: 703-614-5407

FBI- Mr. Kerry Hadaway Fax: 310-996-3359

 

 

Here are computer-scanned documents for your information. The documents are unclassified.

1. First documents, Mr. Danchick declaration.

_________________________________________

I Roy Danchick, hereby declare as follows:

1. I am a retired TRW senior staff engineer. My retirement date

was September 1, 1998. I have personal knowledge of facts stated

herein and could and would testify competently thereto if sworn as a

witness.

2. I have a B.A. in Mathematics from U.C.L.A granted in February

1959. 1 have an M.A. in Mathematics from U.C.L.A granted in June

1 963. 1 have published a number of papers in numerical analysis,

tracking, and astrodynamics in reviewed scientific and engineering

journals as well as presented papers at local, national, and

international conferences. I am a member of the national honorary

mathematical society, Pi Mu Epsilon, and belong to the American

Mathematical Society and the Society for Industrial and Applied

Mathematics.

3. 1 have worked in the aerospace industry for forty years,

almost sixteen of which were at TRW, and finished a recent contract

assignment at Boeing in Downey, California. As an applied

mathematician and aerospace systems engineer my areas of

technical competence include estimation theory, statistics,

tracking, data fusion, and discrimination. I have successfully

developed a number of Kalman Filters for attitude determination,

electrooptical surveillance sensor, electrooptical interceptor, and

radar systems applications. I was designated by TRW's Systems

Integration Group as 1991 Technologist of the Year for my design,

development, and implementation of the Backup Attitude

Determination System for the Defense Support Program. I also hold

a patent for an Infrared Surveillance Tracking System.

Prior to my participation on the Exo-atmospheric

Kill Vehicle (EKV) program I had developed effective ballistic

missile typing algorithms on both Internal Research and

Development ([R&D) programs and Department of Defense contracts

such as Brilliant Pebbles and Brilliant Eyes. To develop these

algorithms I familiarized myself with and became competent in

the physics and phenomenology underlying discrimination

algorith'ms.

4. My work on the EKV program began in or about June, 1 992

prior to the contract awarded to the Boeing/TRW team. I wrote those

parts of the proposal technical volume that dealt with tracking and

data fusion. After the contract w a s awarded I worked part-time on

the EKV program for about two years. From about January, 1994 1

worked f u II time on the EKV program f u II time.

5.. I was the chief tracking and data fusion algorithm designer

on the EKV program. I was also responsible for the interfaces

between the tracking and fusion functions and the discrimination

function. In these capacities I supervised and monitored the

development, implementation, integration, and testing of a

tracking and fusion software. I also did hands-on work in the

integration and testing of two discrimination algorithms, the Bulk

Classifier and the Kalman Feature Extractor (KFE).

6. It was in these supervising, monitoring, testing, and

integration duties that I first developed strong suspicions about the

the effectiveness of both the KFE and the Baseline Algorithms (BLA)

for discriminating genuine targets from decoys. These suspicions

arose even while TRW was telling Boeing and the U.S. government in

various reports and Technical interchange Meetings (TIMS) that the

KFE was a technological breakthrough that would work for near and

far-term threats and that the BLA would meet a II near-term threat

discrimination requirements.

7. My doubts about the KFE's underlying theory arose for three

reasons. First, on phenomenological grounds, it is well- known that

intensity signals are characterized by a continues band-limited

spectrum. The underlying KFE model assumes a discrete two-point

spectrum arising from a pure sinusoid plus d.c. term. Second, the

KFE does its time-invariant parameter estimation recursively while

my experience shows that batch estimation is far more effective in

extracting such constants from the signal. Third, the KFE developers

were vague about initialization, handling data gaps, outlier

detection,. and divergence control. These are critical issues in the

development of any effective estimation algorithm.

8. My suspicions were heightened at a TRW internal TIM in

September, 1995 at which Dr. John Tang and Mr. Wallace Beitzel

September, 1995 at which Dr. John Tang and Mr. Wallace Beitzel

gave a presentation on the KFE. The presentation compared the

performance of the KFE and two competing approaches, one based on

the Fast Fourier Transform, the other on the Maximum Entropy

Method. Dr. Schwartz was one of the meeting attendees. I found the

presentation both technically unconvincing and too much of a high-

pressure sales job. Dr. Raymond Maddalone urged me to sign-off with

my approval as a technical peer on the effectiveness of the KFE as a

discrimination algorithm. I declined. After the meeting I had several

discussions with Dr. Schwartz in which we shared our doubts about

KFE effectiveness and whether it would work at a II with the more

complex signal spectra which would be encountered in real world

operation.

9. I became sure that the KFE was a failure despite contrary

claims made by Mr. Robert Hughes, Dr. Maddalone, Dr. Clifford

Freiler, Mr. Beitzel, Dr. Henry Crowder, et al, when TRW terminated

Dr. Schwartz's employment in February, 1996. Subsequently, I

read her letter to TRW management in which she described how the

KFE had f ailed when she had subjected it to a more rigorously valid

scientific testing. This letter was circulated by TRW EKV program

management to several of its senior staff.

10. During the Spring of 1996 1 was involved in the integration

of the KFE into TRW's Tracking, Fusion, and Discrimination (TFD)

software and testing the KFE. The testing personnel also included

Dr. George Newnam and Mr. Norman Maclean. Our unanimous

conclusion was that the KFE required speed and memory resources

which would not be available in the EKV onboard processor. No

determination could be made as to its effectiveness for

discrimination because only one simulated test case was provided

by Dr. Freiler, the algorithm's original designer.

11. Shortly thereafter TRW officially removed the algorithm

from its TFD baseline software and re-inserted the original BLA.

12. Dr. Schwartz contacted me by phone in or about March,

1996. 1 told her that I believed that she had been wrongfully

terminated and that I shared her conclusions about the

ineffectiveness of the KFE. We stayed in periodic phone contact

thereafter.

13. 1 was interviewed by the Department of Defense Criminal

Investigation Service (DCIS) in the late Spring or early Summer of

1996. I told the investigators that I was sure that the KFE would not

work but I couldn't be sure whether the BLA would work or not. I

suspected that it wouldn't.

14. My suspicions about the BLA were borne out in the Fall of

1996 after TRW received telemetry data from the June seeker f light

test. I was heavily involved with the reduction and processing of the

data with the TFD prototype software. I observed at first-hand TRW

discrimination personnel (Mr. Beitzel et al) attempts to demonstrate

the effectiveness of the BLA in selecting the correct object by both

impermissibly manipulating the Mission Data Load (MDL) and

censoring the test data. This MDL manipulation and data censoring

was the case even though TRW had contractually certified that the

MDL and discrimination software were flight-ready.

15. After my retirement at the end of August, 1998 I supported

Dr. Schwartz's analysis of the BLA performance on the flight test

data as a voluntary member of a DCIS team. I reviewed and analyzed

TRW reports over a two month period. I concluded that the BLA does

not meet contractual performance requirements as specified in the

government's Technical Requirements Document and that TRW

knowingly and impermissibly manipulated and censored the flight

test data to hide the BLA's failure.

16. I also reviewed several reports produced by the Phase One

Engineering Team ( POET) in their review of the BLA and KFE as well

as a Nichols Research Corporation KFE test report. I concluded that,

while there ports in general state in their executive summaries

that TRW's discrimination algorithms work, this assertion is not

supported at all by a careful reading of the bodies of the reports.

I declare under penalty of perjury that the foregoing is true and

correct.

Executed this 18th day of May, 1999 at Los Angles , California.

Roy Danchick

__________________________________

 

 

1. Second document, Special Agent Reed document (DCIS/DOD-IG).

 

DEFENSE CRIMINAL INVESTIGATIVE SERVICE

WESTERN FIELD OFFICE

26722 PLAZA ST., SUITE 130

MISSION VIEJO, CA 92691-6300

February 01, 1999

Mr. Keith Englander,

In our last correspondence we provided what we feel is absolute irrefutable scientific

proof that TRW's discrimination technology does not cannot, and will not work, and falls

far-short of meeting the TRD/contract requirements. After reviewing the POET draft report

we feel even stronger about that statement. We have just received the final version of-the

POET report but have not had time to review it in detail.

I would like to start by highlighting a few areas of the four-page "Executive

Summary" from the POET draft report which are extremely supportive in exposing the

serious flaws in the TRW discrimination technology. These statements fully corroborate

our findings and reiterate what we have been saying for over two years.

Starting with the first paragraph of the "Executive Summary" (page iii), last

sentence, it states "Target discrimination and selection must be done with or without

handover information about the target from the rest of the NMD system " The second

paragraph, first sentence, states "TRW has developed an onboard autonomous

discrimination architecture and a set of algorithm for the Boeing EKV "

The first indented paragraph on page iii, #1, states "Overall, the Baseline

Algorithm(s) (BLA) are well designed and) work properly, with only sonic refinement or

redesign required to increased robustness of the overall discrimination functions. " We

feel this statement is misleading and incomplete, and contradictory of later material in the

report. The indented paragraph #4, page iv, states "The Performance of the discrimination

architecture may be fragile. Training, (data sets that closely match actual threats are

crucial to the performance of the multimodal Bayesian quadratic classifier (MBOO for

the fine (i.e., precision) discrimination function. Performance may degrade significantly

if incorrect prior knowledge regarding target signatures is used in the classifier database.

The target signatures are heavily influenced by threat type, target characteristics, sensor-

to-target geometry, and engagement timeline. Therefore, unexpected targets in the threat

may challenge the performance of the MBOC It would be desirable to expand the

current discrimination architecture to make it more robust by including, for example, real-

time adaptation capability and certain threat-type-based system-level discrimination.

We definitely agree with the above statement which totally supports and

corroborates what we have been saying, and contradicts indented paragraph 91 which states

"Overall the BL4 are well designed and work properly, etc" along with the statement that

TRW's discrimination architecture is autonomous. The Technical Requirements Document

(TRD) requires that the discrimination technology be capable of operating with or without

knowledge of threat types. The POET clearly indicates that TRW's discrimination BLA is

not robust, and it is crucial that the training data sets closely match actual threats for the

fine discrimination function.

The POET's statement of "Overall, the BLA(s) are well designed and work

properly" would be accurate if stated that TRW's BLA's are well designed and work

properly if and only if the training data sets, loaded in the Mission Data Load (MDL),

closely match the actual threats for the fine discrimination function. We have always

contended that prior knowledge can improve algorithm performance to a certain

extent but it is critical that a discrimination algorithm perform independently, to the

maximum possible extent, with limited or no prior knowledge. A discrimination

algorithm has to be capable of adjusting to unpredictable and unknown target objects

from classes that are not expected. It has to be capable of adjusting to the situations in

which an object from an expected object class is missing. The TRD specifically states

that "Included in the scenarios are accidental and unauthorized launches by the

Commonwealth of Independent States (CIS, formerly the Soviet Union) and China, and

limited strikes by China and the Rest of World (ROW) countries" (page A-2). Also, the

BLA must perform when an object from an expected object class does not perform

nominally. TRW's discrimination algorithms have never been able to function in this

situation. Their algorithms are heavily dependent on prior knowledge, or they adjust

their discrimination results by using, Post Flight data in order to show acceptable

results. It is scientifically impermissible to use unknowable information to correct

experimental results after the fact.

On pace vi of the Executive Summary, the last sentence on the page states "If targets

are deployed nominally in IFT-3, it is the POET team's assessment that the TR WBLA

will successfully select the MR V as the intercept taro et, even for the most stressing target

suite. " It also states in the body of the same paragraph "The most stressing target suite for

EKV discrimination contains the same ten targets as those deployed in IFT-IA. " This

contention seems questionable because TRW requested the removal of one of the objects

from the target suite for the EFT-3. This information surfaced at the July 21 & 22, 1998,

meeting with TRW at the Aerospace Corporation. The reason for the removal of the object

was that its off-nominal deployment caused TRW's BLA significant problems. This

confirms the POET's statement "The performance of the discrimination architecture may

be fragile".

On page vi of the Executive Summary, third paragraph, it states "One of the most

important criticisms raised by DCIS is that during the IFT-IA postmission analyses TRW

unjustifiably shifted the preflight feature distribution ellipses between the 45-day report

and the 60-day report in order to increase the BL4 's discrimination Performance. As

previously noted, TRW’s explanation that an improperly constituted GFA caused the

discrepancies appears to be correct." In TRW's report titled "(U) POET Review Of EKV

Discrimination Algorithms" dated July 21 & 22,1998, they state "The primary differences

 

 

between the 45 and 60-Day MDL was in the settings for the available track length".

"Gap Filling did not affect the measured signatures because of the start time of 1752

for the hand-tracked object reports. Features are calculated from the measured

signatures and compared with the MDL." The explanation by TRW that an

improperly constituted GFA caused the discrepancies referring to the shifting of the

ellipses between the 45 and 60-Day reports is totally incorrect (pages 156 and 173

refer). The real difference between the discrimination performance result (PAT)

provided by TRW in the 45-Day Report, compared to the 60-Day PAT, was the

shortening of available flight data track length to 16 seconds for the 60-Day, versus 47

seconds used for the 45-Day Report. Additionally, it appears the training of the OSC

code by TRW with the actual IFT-lA flight data, was used to generate the 60-Day

MDL".

The very high PAT provided in the 60-Day Report is a direct result of the

extremely close match between the OSC code trained with EFT-IA flight test data and the

IFT-IA flight test data results. It is certainly clear that the IFT-3 actual results will not be

available as a training set until the flight test has been completed.

In our March 25, 1998, report we provided our views regarding- the Gap Filling

Algorithm (GAP) and the serious adverse of using this consequence algorithms We related

that both NRC and GBI representatives informed us, during our December 1997 trip to

Huntsville, that they do not understand how TRW's GFA for the signatures works and that

TRW has never explained or validated the method. We explain that TRW's GFA attempts

to fill in missing data points in the EKV sensor signal output by using values from the

OPTISIG library. However, the intended program inadvertently used zero values for the

data points. This resulted in the "anomalistic" behavior of the signal outputs analyzed by

'INRC during their testing of TRW's discrimination technology, wherein every signature had

an unexplained dip. Additionally, TRW artificially chopped the amplitude of the sensor

signal output which results in arbitrarily forcing the signal mean to a predetermined value.

This resulted from the sensor output noise level being too high for the baseline

discrimination algorithm to process. It has been shown by TRW that the GFA manipulates

data in such a way to give artificial results. TRW showed this in their correction to the 60-

Day Status Report by changing the location of the center of the ellipses, changing the shape

of the ellipses, and by changing two of the features. This allowed an artificial improvement

in the Probability of assigned Target (PAT). The PAT should be solely dependent on the

actual measurement of data from the real signature(s), versus creating a signature(s)

with gaps of missing data and filling in this data artificially to increase the value of the

calculated PAT. This GFA process is done outside of accepted statistical scientific

methods. Prior td the down-select TRW stated they were not going to use the GFA for the

IFT-3 which is also stated in the POET's report. This can only raise the question of how

would TRW handle any gaps in the actual IFT-3 signatures.

 

On page vi the POET states "Another important issue concerns the "improper"

concept of ranking probability used by TRW for selecting an intercept target. It is clear to

the POET team that all points raised concerning this issue were based on

misunderstanding of Bayes rule and misinterpretation of TRW’s approach. " Our

understanding of TRW's ranking scheme is that it selects the object with the highest

probability of belonging to the RV class given the object's extracted feature vector. This is

the case even though it is possible that the particular object selected may have a higher

posterior probability of belonging to another class. It can also be the case, that another

object with lower PAT than the highest-ranking object could actually have the largest

probability of being the RV among the classes. The table below is a simple illustration of

this possibility:

OBJ-ECT OBJ-ECT CLASSES

RV MB LB

1 0.40 0.60 0.00

2 0.39 0.305 0.305

 

We do not misunderstand Bayes rule. We apply it differently than TRW does in its

MBQC. Rather than assuming that each tracked object surely belongs to one of five

different classes with a given prior hypothesis probability per object class, we assume the

equally likely prior hypotheses that one of the tracked objects is surely the RV. In addition

we view the set of extracted independent feature vectors as comprising the compound

posterior event.

Given our hypothesis and event structure, our Monte Carlo analysis (1000 samples)

yielded an estimated probability of correct RV selection that is within a few percentage

points of the POET'S. The Confusion Matrix, displayed on page 18 of the POET draft

report, shows their probability of correct RV selection. The POET results and ours are

consistent and mutually supportive. The difference between the two is well within the

Monte Carlo sampling error.

The last paragraph, page vi, of the Executive Summary the POET states "It is a

legitimate concern that the robustness of the BLA's discrimination function could be

compromised by the assumption that threat-typing, information would be available for use

in the MDL database and that target characteristics, motion parameters, and engagement

geometry would be known for a given threat system. Reliable prior knowledge on many Of

the threat types included in the TRD cannot be obtained. The POET study did not evaluate

to what extent the BLA may be used to meet the general requirements described in the YRD

beyond the ability to determine which object among a threat cloud is lethal (given that the

threat type is known). However, the POET team's concerns regarding the potential lack

of robustness of the current BLA have been indicated earlier. The POET team also

 

 

recognizes that the concept of threat typing can have significant impact on EKV

discrimination performance and that such impact should be examined carefully in the

context of overall NMD architecture design and threat assessment Within the EKV

scenario setting it may be important to structure the discrimination architecture to take

advantage of threat-typing information while at the same time reducing potential risk "

We agree with the sense of this paragraph even more strongly than as stated by the POET.

In reviewing the POET report we noted that part of the report focused on the Kalman

Filter (KF) performance. We reviewed the Statement of Work (SOW) generated by Dr.

Frank Handler for the POET but did not see anything about the KF. Our concern is when

did this become part of the SOW? Why weren't we notified? Why didn't we get a

copy of the new SOW? and Why didn't we get a chance to provide some input? In

reviewing the reference list for the POET report we noted that the "I-SETAC ANALYSIS

OF ROCKWELL KALMAN FILTER FEATURE EXTRACTION CONCEPT (LD" dated

November 14, 1996 was not identified. The Nichols Research Corporation (NRC), which is

the technical advisor to the Ground Base Interceptor (GBI) Office in Huntsville, Alabama

authored this report. We feel that before the POET made any comments regarding the KF

they should have had all immediately available reports concerning the matter.

From reviewing the POET draft report, the TRD requirements, and all the other

accumulated relevant documents we will summarize a number of points on why the TRW

BLA is not, nor has it ever been basically sound or capable of meeting the contract

requirements:

(1). Almost all features are only descriptive statistics which wash out any intrinsic

physics and spectral information contained in the intensity signatures. Moreover, the

features are not functionally independent.

(2). The features, for all object classes, are assumed to be constant over time; they

are not. Feature time variability is artificially compensated by lumping it into the apriori

feature error covariance matrices. Lumping feature time varying effects into these

covariance matrices creates large regions of overlap in the feature spaces of different object

classes. These overlap regions grow as mission elapsed time increases because of the

secular trends in the features. Such trends, reflected in the extracted features, are highly

likely to cause one detected object's extracted feature vector to wander close to the center of

the apriori error ellipse family for a different object class. Depending on the actual

engagements object-by-object materials, temperatures, aspect angle temporal variation, etc.

the RV's extracted feature vector will, with uncomfortably high probability, travel out of the

apriori likely RV class region into the likely region of another object class; conversely the

extracted feature vector for another object class is likely to "walk" close to the RV apriori

feature vector mean. We can predict with some certainty that such confusion will grow

overtime as more object-by-object data is accumulated. This is exactly what a sound

 

algorithm should not do. A sound algorithm should be able to select the RV with greater

and greater confidence as the object-by object data spans increase.

(3). TRW's BLA, while computing a feature extraction error covariance matrix

for each tracked object, does not use this matrix in its Baysean Likelihood calculations.

This omission is both statistically incorrect and computationally inefficient. We can only

speculate that the required use of this matrix as a summand was left out because to include it

would generate larger total error overlap and selection performance degradation.

(4). The strictly recursive feature extraction approach is questionable. Recursive

computations have appreciable startup transient times and preclude the exploitation of

spectral methods, which require a batch of data over an appropriate time span. Batch

estimation is more appropriate than recursive estimation for truly time invariant parameters.

(5). With one exception the total feature set has nothing to do with the underlying

physics, thermodynamics and mechanics.

(6). To paraphrase one of the POET's main finding's, apriori threat identification

data will not always be available. Even if accurate missile typing data is transmitted from

surveillance assets to the GBI via the BMC 3 element, there is no assurance whatsoever that

the corresponding MDL would faithfully represent the set of objects, RV and decoys,

carried by the missile's PBV, nor all likely variations in the actual object-by-object

composition, temperature distribution, deployment geometry, rotational dynamics

properties, etc.

(7). Given (1) to (6) one can expect that the BLA performance will be exquisitely

sensitive to feature extraction start times, data spans, non-nominal object deployments,

irradiance data gaps, dips, and spikes of whatever origin. Such dips and spikes can arise

from particle and gamma ray events, sensor electronic anomalies, or actual rapidly time

varying aspect angles as well as the individual objects projected area, shape, and body

element material composition changes.

(8). As for robustness, we assert that the BLA is, instead, extremely fragile, This

contention is borne out in both the POET's and our results as shown in the table on page

66 of the draft report titled "Independent Review of TRW Discrimination Techniques", 7-8

December 1998, by Dr. Ming Tsai et al (page 67 of the POET final report refers).

(9). It is clear from our reading and analysis of the documentation cited in the

POET's bibliography that TRW heavily censored the IFT-IA signature data, deleting

approximately the first 20 seconds and the last I I seconds to go from poor results in its 45-

Day Report, to excellent results in its 60-Day Report Addendum. Removal of the last I I

seconds is especially critical because we have evidence to believe that in this interval of

time the RV's PAT fell dramatically and dropped below another object's rising PAT.

 

Additionally, TRW generated the MML used for the 60-Day Report analysis by training the

OSC code with the detailed knowledge from EFT- IA flight data since that information was

used in constructing the MDL. The very high PAT achieved in the 60-Day Report is a

direct result of this close match and has nothing to do with the GFA. Such after-the-fact

censoring, is impermissible as any kind of empirical evidence that the BLA meets TRD

specified near term discrimination performance requirements.

(10). We also suspect that early data was censored to hide the difficulty the BLA

was having with real signature spikes, not Boeing/GBI sensor-induced spikes. These

signature spikes were actually induced by object attitude motion as verified by the_

Airborne Sensor Testbed (AST) data. In addition the use of impermissible after the fact

knowledge by TRW to switch features in mid-data stream was also noted, and previously

reported to the POET.

(11). TRW's BLA cannot be well designed if it is lacking, a critical component

such as an effective Gap Filling Algorithm.

Putting together all these instances of impermissible use of after-the-fact

knowledge, early data censoring to remove difficult data, late data censoring to make sure

that the RV was selected, and a missing key algorithm, can only cast grave suspicion about

the ineffectiveness and fragility of TRW's BLA's capability to meet contract requirements.

There is no crime in producing a failed algorithm during a Research and

Development project. That's all part of the acceptable risk in order to eliminate the non-

viable approaches. The crime is in producing a failed algorithm and knowingly covering up

its failure.

In conclusion, we are looking forward to meeting with you and the POET this week.

I request that this correspondence not be disseminated outside your office or members of the

POET. I want to thank for your time and interest in this matter.

Sincerely,

Samuel W. Reed - Group Manager, DCIS

1. Third document, Special Agent Reed document (DCIS/DOD-IG).

_________________________________________

DEFENSE CRIMINAL INVESTIGATIVE SERVICE

WESTERN FIELD OFFICE

26722 PLAZA ST., SUITE 130

MISSION VIEJO, CA 92691-6300

 

9610701Z-18-JUN-96-50LA-E0QT March 15, 1999

TRW CORPORATION

Redondo Beach, CA

RECEIPT OF ARMY CORRESPONDENCE: On March 8, 1999,

Department of Justice (DOJ) Attorney Dennis Egan faxed

reporting agent a copy of a letter from the Department of

the Army, United States Army Legal Services Agency,

Arlington, VA. The letter is addressed to Mr. Michael F.

Hertz, head of the Commercial Litigation Branch, Civil

Division, DOJ Washington, D.C. The letter is dated February

26, 1999, and recommends against DOJ intervening in this

case. One of the reasons for not intervening was based upon

conversations with the investigators handling the case.

This statement has no factual basis as reporting agent is

the only investigator assigned to the case, and has not

spoken with any Army representative regarding not

intervening in the case.

 

Attachment

1. Army Letter dated February 26,1999.

Prepared by SA Samuel W. Reed, SO-Western Field Office

APPR: SWR