A generalized Cauchy-Schwarz inequality via the Gibbs variational formula

Let be a non-empty finite set. If is a random variable taking values in , the Shannon entropy of is defined as There is a nice variational formula that lets one compute logs of sums of exponentials in terms of this entropy: Lemma 1 (Gibbs variational formula) Let be a function. Then Proof: Note that […]