Statistical Class-Based Language Modelling

Scientific Report, University of Cambridge, 1997. — 71 p.In this report, an introduction to natural language modelling is given in the context of speech recognition. Various techniques for formulating stochastic language models are discussed, focusing particularly on N-gram models based on classes of words. A presentation of a number of statistical techniques for the automatic classification of words is given. Results for two automatic clustering techniques are presented along with notes on their implementation in class-level language models. These results are compared with a number of word-level models. Finally, a section on the direction in which subsequent research will develop is included.Introduction
Language Modelling
Automatic Classification Techniques
Results
Plans for Further investigation
A N-gram Statistics for three sizes of Wall-Street Journal Corpora
B Algorithm for word rearrangements
C Update equations implemented for hill-climbing algorithm
D Update equations implemented for multiple word-to-cluster rearrangements
E Experimental Method
F Russian language text sources