Monte Carlo, maximum entropy and importance sampling

The probability of an error in a Monte Carlo integration is shown to be exponentially small in the number of points used, with the magnitude of the exponent being determined by a relevant entropy. Implications for importance sampling and for the significance of the maximum entropy formalism are discussed. Specifically it is shown that the optimal sampling distribution is one of maximal entropy. The Monte Carlo method or its variants play an essential role in classical trajectory computations. Practitioners are aware that generating few trajectories is already sufficient for typical quantities such as the mean energy of the products to settle down to the correct value. The present results provide further insight and suggest why a distribution of maximal entropy can provide such useful representation of the results. The discussion is based on the information theoretic bound for the error of transmission and can also be derived from the Chernoff bound in hypothesis testing.