Title: Anomaly Detection on Graphs (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-05-8440, 2005.

Abstract:

This paper describes the results of some initial experiments that explore the problem of anomaly detection on graphs.

Back to the Home Page for James Howse

Title: Simple Classifiers (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-03-0193, 2003.

Abstract:

In this paper we introduce simple classifiers as an example of how to use the data dependent hypothesis class framework in (Cannon, Ettinger, Hush and Scovel 2002) to explore the performance/computation trade-off in the classifier design problem. We demonstrate that simple classifiers have many remarkable properties. For example they possess computationally efficient learning algorithms with favorable bounds on estimation error, admit kernel mappings, are particularly well suited to boosting, and are fully parallelizable. In addition they are robust to the choice of learning problem which we demonstrate with the error minimization, Neyman-Pearson and min-max problems. Our experiments with synthetic and real data suggest that simple classifiers are competitive with powerful alternative methods.

Back to the Home Page for James Howse

Title: Learning with the Neyman-Pearson and min-max criteria (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-02-2951, 2002.

Abstract:

We study two design criteria for classification: the Neyman-Pearson criterion and a min-max criterion. For each we prove a lemma bounding estimation error in terms of error deviance. We then show how these lemmas can be used to determine probabilistic guarantees on estimation error.

Back to the Home Page for James Howse

Title: Linking Learning Strategies and Performance for Support Vector Machines (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-02-1933, 2002.

Abstract:

We develop a formal representation of the technique introduced in (Shawe-Taylor, Bartlett, Williamson and Anthony, 1998), (Shawe-Taylor and Cristianini, 1998) for bounding the generalization error of support vector machines. As a consequence we provide a framework that can be utilized to link learning strategies to their performance bounds in such a way that the bounds are expressed in terms of the structural properties of the learning strategy (e.g. characterizations of the optimum classifier in terms of the structure of the finite sample optimization criterion and its value at optimum). We use this framework to provide performance bounds for a class of support vector machines that includes the soft margin learning strategies commonly used in practice. We also show how to eliminate the effects of the center and scale of the data in the learning theorem. We apply this framework to improve results obtained in (Shawe-Taylor and Cristianini, 1998) for the 2-norm soft margin learning strategy by exploiting a relationship between covering numbers of classes of linear functions and covering numbers of linear operators. This result is expressed in terms of the finite sample criterion value at optimum. Finally we show how this bound can be expressed in terms of the random process.

Back to the Home Page for James Howse

Title: Least Squares Estimation Techniques for Position Tracking of Radioactive Sources (PDF)

Citation:

This paper has appeared in the journal Automatica. The full reference is Automatica, Vol. 37, No. 11, pp. 1727-1737, 2001.

Abstract:

This paper describes least squares estimation algorithms used for tracking the physical location of radioactive sources in real-time as they are moved around in a facility. We present both recursive and moving horizon nonlinear least squares estimation algorithms that consider both the change in the source location and the deviation between measurements and model predictions. The measurements used to estimate position consist of four count rates reported by four different gamma ray detectors. There is an uncertainty in the source location due to the large variance of the detected count rate, and the uncertainty in the background count rate. This work represents part of a suite of tools which will partially automate security and safety assessments, allow some assessments to be done remotely, and provide additional sensor modalities with which to make assessments.

Back to the Home Page for James Howse

Title: Comparison of Recursive Estimation Techniques for Position Tracking Radioactive Sources (PDF)

Citation:

This paper was presented at the 2001 American Control Conference in June 2001. The full reference is Proceedings of the 2001 American Control Conference, pp. 1656-1660, 2001.

Abstract:

This paper compares the performance of recursive state estimation techniques for tracking the physical location of a radioactive source based on radiation measurements obtained from a series of detectors at fixed locations. Specifically, the first order, iterated, and a second order Kalman filter performance is compared to nonlinear least squares estimation. The results of this study indicate that least squares estimation significantly outperforms the extended Kalman filter implementations in this application due to the nature of the model nonlinearities.

Back to the Home Page for James Howse

Title: Recursive Estimation for Tracking Radioactive Sources (PDF)

Citation:

This paper was presented at the 1999 American Control Conference in June 1999. The full reference is Proceedings of the 1999 American Control Conference, pp. 1905-1909, 1999.

Abstract:

This paper describes a recursive estimation algorithm used for tracking the physical location of radioactive sources in real-time as they are moved around in a facility. The algorithm is a nonlinear least squares estimation that minimizes the change in the source location and the deviation between measurements and model predictions simultaneously. The measurements used to estimate position consist of four count rates reported by four different gamma ray detectors. There is an uncertainty in the source location due to the variance of the detected count rate. This work represents part of a suite of tools which will partially automate security and safety assessments, allow some assessments to be done remotely, and provide additional sensor modalities with which to make assessments.

Back to the Home Page for James Howse

Title: Detection and Location of Radioactive Sources using a Suite of Slab Detectors (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-99-908, 1999.

Abstract:

This report describes several experiments used to characterize and test a suite of four radiation sensors. The sensors are scintillation counters composed of plastic connected to an amplifier. The purpose of these tests is to assess the feasibility of using these sensors to detect and track radioactive sources in a large room. The tests are also used to compare a number of different detection and tracking algorithms.

We describe real-time algorithms for both detecting the presence and tracking the position of radioactive sources in a facility in the presence of measurement noise. We formulate the detection problem as a nonparametric hypothesis testing problem. This problem is solved by comparing a statistic computed over some window(s) of the data to a threshold value. If this threshold is exceeded then we decide that a source is present. We formulate the tracking problem as a state estimation problem and solve it recursively using a constrained nonlinear optimization method. The optimization simultaneously minimizes the change in source position and disagreement between measurements and a sensor model. The sensor model is a fairly complex function relating position to detected count rate.

The overall purpose of this work is to enhance both security and safety by automating part of the assessment process, allowing remote assessment, and introducing new sensor modalities into the assessment process. We present detection and tracking results based on experiments done with one source in a single room. Our results indicate that a source can be detected and tracked quite well with these algorithms in spite of fairly poor signal to noise ratios, and rather high measurement noise levels. In short, we demonstrate the capability to detect and track a single source in real-time with high accuracy in spite of a complex mapping from source position to detected count rate, an unknown background signal, and high measurement noise.

Back to the Home Page for James Howse

Title: Solving a Thermal Regenerator Model using Implicit Newton-Krylov Methods (PDF)

Citation:

This paper has appeared in the journal Numerical Heat Transfer. The full reference is Numerical Heat Transfer: Part A - Applications, Vol. 38, No. 1, pp. 23-44, 2000.

Abstract:

In this paper we discuss the use of an implicit Newton-Krylov method to solve a set of partial differential equations representing a physical model of a blast furnace stove. Blast furnace stoves are thermal regenerators used to heat the air injected into the blast furnace, providing the heat to chemically reduce iron oxides to iron. The stoves are modeled using a set of partial differential equations which describe the heat flow in the system. The model is used as part of a predictive controller which minimizes the fuel gas consumption during the heating cycle, while maintaining a high enough output air temperature in the cooling cycle to drive the chemical reaction in the blast furnace. The discrete representation of this model is solved with a preconditioned implicit Newton-Krylov technique. This algorithm uses Newton's method, in which the update to the current solution at each stage is computed by solving a linear system. This linear system is obtained by linearizing the discrete approximation to the PDE's, using a numerical approximation for the Jacobian of the discretized system. This linear system is then solved for the needed update using a preconditioned Krylov subspace projection method.

Back to the Home Page for James Howse

Title: Implicit Newton-Krylov Methods for Modeling Blast Furnace Stoves (PDF)

Citation:

This paper was presented at the 1998 ASME conference on Thermophysics and Heat Transfer in June 1998. The full reference is Proceedings of the 1998 AIAA/ASME Joint Thermophysics and Heat Transfer Conference, pp. 283-290, 1998.

Abstract:

In this paper we discuss the use of an implicit Newton-Krylov method to solve a set of partial differential equations representing a physical model of a blast furnace stove. The blast furnace stove is an integral part of the iron making process in the steel industry. These stoves are used to heat air which is then used in the blast furnace to chemically reduce iron ore to iron metal. The simulation of the stove's behavior is the first step in a program to reduce the cost of operating these stoves by minimizing the natural gas consumption during the heating cycle, while still maintaining a high enough output air temperature in the cooling cycle to drive the needed chemical reaction in the blast furnace. The formulation and solution of this optimal control problem will also be discussed. The solution technique used to solve the discrete representations of the model and control PDE's must be robust to linear systems with disparate eigenvalues, and must converge rapidly without using tuning parameters. The disparity in eigenvalues is created by the different time scales for convection in the gas, and conduction in the brick; combined with a difference between the scaling of the model and control PDE's. A preconditioned implicit Newton-Krylov solution technique was employed. The procedure employs Newton's method, where the update to the current solution at each stage is computed by solving a linear system. This linear system is obtained by linearizing the discrete approximation to the PDE's, using a numerical approximation for the Jacobian of the discretized system. This linear system is then solved for the needed update using a preconditioned Krylov subspace projection method.

Back to the Home Page for James Howse

Title: Optimal Operation and Control of the Blast Furnace Stoves (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-99-5051, 1999.

Abstract:

This report documents the Phase One effort for the project "Advanced Control of Operations in the Blast Furnace" sponsored by the Department of Energy Office of Industrial Technology, Energy Efficiency and Renewable Energy, under Work Authorization ED/18019/AL04, and performed by Los Alamos National Laboratory and Ispat Inland Steel. The first phase of this project involved improving the thermal efficiency of blast furnace stoves using advanced control technology. This technology was developed at Los Alamos National Laboratory and implemented on the No. 7 blast furnace stoves at the Ispat Inland Steel facility in East Chicago, Indiana. A post-audit of the performance of the technology was carried out by Ispat Inland personnel in January, 1999. The result was a five percent reduction in the energy used by the stoves attributable to the advanced control technology. This reduction was achieved with the advanced control technology operating for approximately eighty percent of this test period. Similar results have been obtained for subsequent blast furnace stove operation with advanced control.

The advanced control technology developed for this phase of the project consists a model-based control strategy that uses a detailed heat transfer model of the hot blast stoves. The control strategy determines the minimum amount of fuel necessary to achieve the blast air energy requirements based on the blast furnace stove model predictions. State and parameter estimation is used to update the model predictions and adapt to changes in the blast furnace stove system and operation. The advanced control software consists of a FORTRAN 77 program that runs on a workstation computer. Implementation of the technology was accomplished by interfacing the workstation computer to the Ispat Inland process monitoring and control computer system network.

Back to the Home Page for James Howse

Title: Predicting Chilled Hearth Conditions: A Classification Approach using Bayes Error Estimation (PDF)

Citation:

This paper is a Los Alamos technical report, LA-UR-99-6189, 1999.

Abstract:

A blast furnace is used to produce molten iron from iron oxides, coke, and flux. Ordinarily a blast furnace is controlled so that the molten iron temperature has some nominal operating value within some operating range (e.g., approximately 1500 C +/- 30 C for Ipsat Inland's No. 7 blast furnace). In spite of the control procedures, the iron temperature sometimes deviates from this operating range. When it does so for several ladles in succession, the condition is called a chilled hearth. When this situation arises, manual corrective actions must be taken to restore the iron temperature to its proper operating range. The corrective actions usually lead to a decrease in the production rate of molten iron. Furthermore, the lower iron temperatures usually indicate a reduction in iron quality because the trace element chemistry in the iron changes. The combination of these two factors means that iron-making companies seek to avoid chilled hearth conditions because they are economically expensive.

The overall aim of this project is to construct an automated system which detects the presence of conditions which could lead to a chilled hearth condition in a blast furnace. Reliably detecting the onset of these conditions before a chilled hearth occurs would allow corrective measures to be instituted that would probably prevent the chilled hearth. Since serious chilled hearth conditions are fairly rare in practice, this problem can be thought of in the context of anomaly detection. Intuitively, an anomaly detector takes measurements of a system as inputs, and produces a decision about whether the measurements are unusual and a confidence in that decision, as outputs. The approach to cold hearth detection discussed in this report is called the classification method. Roughly speaking, the classification method consists of designing a system which distinguishes between ``normal'' and ``abnormal'' measurements and then assigning the current measurements to either the ``normal'' or the ``abnormal'' category. The category to which the current measurements are assigned represents the current state of the blast furnace.

This report is produced in two parts. The first part is an executive summary that overviews chilled hearth conditions in a blast furnace, discusses the technical requirements associated with detecting these conditions, highlights our specific research approach to detecting chilled hearths, and presents the status of our research along with our current conclusions and future work objectives. The second part is a technical summary which goes into detail about the current state our research.}

Back to the Home Page for James Howse

Title: A Learning Algorithm for Applying Synthesized Stable Dynamics to System Identification (PS)

Citation:

This paper has appeared in the journal Neural Networks. The full reference is Neural Networks, vol. 11, no. 1, pp. 81-87, 1998.

Abstract:

In this paper we extend the models discussed by Cohen (1992) by introducing an input term. This allows the resulting models to be utilized for system identification tasks. We prove that this model is stable in the sense that a bounded input leads to a bounded state when a minor restriction is imposed on the Lyapunov function. By employing this stability result, we are able to find a learning algorithm which guarantees convergence to a set of parameters for which the error between the model trajectories and the desired trajectories vanishes.

Back to the Home Page for James Howse

Title: Some Control Theoretic Issues in Neural Networks (PS)

Citation:

This paper was presented in an invited session on ART at the International Conference on Neural Networks in June, 1996. The full reference is Proceedings of the International Conference on Neural Networks, vol. Special Sessions, IEEE Press, pp. 205-210, 1996.

Abstract:

We have observed that many neural network models can be written as a bilinear system with a specific form of nonlinear state-to-input feedback. This framework includes the ART architecture among others. There are two significant results which follow from this observation. First, the parameters of the model determine the controllability of the system. A system is controllable if there exists some input which transfers any initial state to any desired final state in a finite time. If for a given set of these parameters the system is not controllable, then there are regions of the state space which the system can never enter in a finite time for any input. Because of this restriction the learning ability of the system may be severely limited. Second, the multiplicative equation is linear in all of the parameters, and all of the adjustable weights. This means that a provably convergent learning algorithm can be devised for all of these quantities. This does not however circumvent the learning limitation since the learning algorithm is not guaranteed to converge in a finite time. In the paper, we will study these issues as they apply to the ART architecture.

Back to the Home Page for James Howse

Title: Gradient and Hamiltonian Dynamics Applied to Learning in Neural Networks (PS)

Citation:

This paper was presented at the Advances in Neural Information Processing Systems (NIPS*95) conference in November, 1995. The full reference is Advances in Neural Information Processing Systems, vol. 8, The MIT Press, pp. 274-280, 1996.

Abstract:

The process of machine learning can be considered in two stages: model selection and parameter estimation. In this paper a technique is presented for constructing dynamical systems with desired qualitative properties. The approach is based on the fact that an n-dimensional nonlinear dynamical system can be decomposed into one gradient and (n - 1) Hamiltonian systems. Thus, the model selection stage consists of choosing the gradient and Hamiltonian portions appropriately so that a certain behavior is obtainable. To estimate the parameters, a stably convergent learning rule is presented. This algorithm has been proven to converge to the desired system trajectory for all initial conditions and system inputs. This technique can be used to design neural network models which are guaranteed to solve the trajectory learning problem.

Back to the Home Page for James Howse

Title: Gradient and Hamiltonian Dynamics: Some Applications to Neural Network Analysis and System Identification (PS)

Citation:

This is my Ph.D. dissertation, which is also available from University Microfilms Incorporated (UMI)

Abstract:

The work in this dissertation is based on decomposing system dynamics into the sum of dissipative (e.g., convergent) and conservative (e.g., periodic) components. Intuitively, this can be viewed as decomposing the dynamics into a component normal to some surface and components tangent to other surfaces. First, this decomposition was applied to existing neural network architectures to analyze their dynamic behavior. Second, this formalism was employed to create models which learn to emulate the behavior of actual systems. The premise of this approach is that the process of system identification can be considered in two stages: model selection and parameter estimation. In this dissertation a technique is presented for constructing dynamical systems with desired qualitative properties. Thus, the model selection stage consists of choosing the dissipative and conservative portions appropriately so that a certain behavior is obtainable. By choosing the parametrization of the models properly, a learning algorithm has been devised and proven to always converges to a set of parameters for which the error between the output of the actual system and the model vanishes. So these models and the associated learning algorithm are guaranteed to solve certain types of nonlinear identification problems.

Back to the Home Page for James Howse