A generalized Cauchy-Schwarz inequality via the Gibbs variational formula
What's new 2023-12-11
Let be a non-empty finite set. If
is a random variable taking values in
, the Shannon entropy
of
is defined as
Lemma 1 (Gibbs variational formula) Letbe a function. Then
Proof: Note that shifting by a constant affects both sides of (1) the same way, so we may normalize
. Then
is now the probability distribution of some random variable
, and the inequality can be rewritten as
In this note I would like to use this variational formula (which is also known as the Donsker-Varadhan variational formula) to give another proof of the following inequality of Carbery.
Theorem 2 (Generalized Cauchy-Schwarz inequality) Let, let
be finite non-empty sets, and let
be functions for each
. Let
and
be positive functions for each
. Then
where
is the quantity
where
is the set of all tuples
such that
for
.
Thus for instance, the identity is trivial for . When
, the inequality reads
We now prove this inequality. We write and
for some functions
and
. If we take logarithms in the inequality to be proven and apply Lemma 1, the inequality becomes
Lemma 3 (Conditional expectation computation) Letbe an
-valued random variable. Then there exists a
-valued random variable
, where each
has the same distribution as
, and
Proof: We induct on . When
we just take
. Now suppose that
, and the claim has already been proven for
, thus one has already obtained a tuple
with each
having the same distribution as
, and
With a little more effort, one can replace by a more general measure space (and use differential entropy in place of Shannon entropy), to recover Carbery’s inequality in full generality; we leave the details to the interested reader.