Information theory is playing a role in the solutions to a growing number of probability and statistics questions. Central to these developments are quantities of information, especially relative entropy and mutual information, and associated tools of Shannon information theory traditionally applied to problems of data compression and communication channel capacity. We will list and briefly discuss a number of the
probability and statistics questions that information theory impacts. With audience help we will pick out two or three of these to discuss at somewhat greater length. Here are three for which there has been substantial recent attention. First, in probability theory, relative entropy captures the exponent of tail probabilities (also called large deviation
probabilities) for empirical averages of Markov chains. This has implications not only for error exponents in communications and hypothesis testing in statistics, but also in determining which Markov chain Monte Carlo samplers (Gibbs, Metropolis, stochastic gradient) rapidly provide accurate path average computations of integrals as arise in statistical mechanics, in optimal Bayes estimates (posterior means), and in machine
learning (e.g. artificial neural nets). Secondly, information quantities recently have been used to give demonstration of the monotonicity of convergence in central limit theorems, for which we now have simple proofs. Thirdly, in the field of statistics, information theory captures large sample efficiency and minimaxity in both parameter estimation and
nonparametric function estimation problems. In essence the minimax rate of function estimation is related to the Shannon capacity of an associated channel.