Stochastic Categorical Sequence Estimation:
        From Speech Recognition to Knowledge Systems Analysis

                             John Hogden

A technique recently developed for speech processing should have
application to a wide variety of problems that require estimating the
probability of sequences of categorical data values.  Such problems
are ubiquitous; they occur in Natural Language Processing (e.g.
analyzing probabilities of word sequences), Anomaly Detection (e.g.
finding unlikely sequences of medical procedures -- possibly
indicating fraud), and are typical of prediction problems (e.g. which
web page will be accessed next).  

The technique, called Maximum Likelihood Continuity Mapping (MALCOM),
makes two basic assumptions: 1) sequences of data values are produced
as an unobservable object moves through space, 2) the data value
produced at time t is a stochastic function of the position of the
object at t. These two assumptions are especially characteristic of
speech: speech sounds are stochastic functions of smoothly varying
articulator positions.  

A major advantage of MALCOM is that the number of parameters required
by MALCOM grows linearly with the vocabulary of a problem, as opposed
to exponentially for high-order Markov models.  Thus, where the MALCOM
assumptions hold, or where the vocabulary is large, MALCOM should
outperform Markov models.  Furthermore, the parameters estimated by
MALCOM may be both meaningful and hard to estimate by any other
technique.  

For example, in a speech recognition task MALCOM's parameters
constitute a mapping between speech sounds and speech articulator
positions.  Finding such a mapping is typically difficult since
articulator data is hard to collect, but, remarkably, MALCOM finds the
mapping without articulator measurements.  For sequences of words,
MALCOM estimates a spatial position for each word -- reminiscent of
latent semantic indexing in Natural Language Processing.

Theoretical and empirical work with MALCOM will be discussed, with an
emphasis on extending MALCOM to new problem such as natural language,
analysis of user traversals, and connectivity in web spaces.