We are interested in the question of how to learn rules, when those rules make probabilistic statements about the future. In this paper we discuss issues that arise when attempting to determine what a good prediction function is, when those prediction functions make probabilistic assumptions.
Learning has at least two purposes, 1) to enable the learner to make predictions in the future, and 2) to satisfy intellectual curiosity as to the underlying cause of a process. We will give two results related to these distinct goals. In both cases, the inputs are a countable collection of functions which make probabilistic statements about a sequence of events. One of our results shows how to find one of the functions, which generated the sequence, the other result allows us to do as well in terms of predicting events as the best of the collection. In both cases the results are obtained by evaluating a function based on a trade-off between its simplicity and the accuracy of its predictions.