ABSTRACT

Algorithmic Complexity Theory and Machine Learning

Allan Erskine, Department of Computer Science, UCL

The theory of algorithmic complexity has an intimate connection to the theory of machine learning, and has done since the very early days of both fields. This talk reviews the prehistory of both theories and in particular notes the genesis of both universal machine inference and algorithmic complexity in a *single* paper by Ray Solomonoff in 1964.

Sophistication, a modern notion of complexity, is then discussed. This measure of "meaningful information", although popularised in science books such as "The Quark and the Jaguar" by Murray Gell-Mann, has only been formalised very recently (see "Meaningful Information" by Paul Vitanyi, http://www.cwi.nl/~paulv). The relationship of sophistication to learning is outlined, first in relation to ideal statistical inference and the minimum description length (MDL) method, then to reinforcement learning and the broader AI effort.


Maintained by rbennett@cs.ucl.ac.uk