A computation-outsourced discussion of zero density theorems for the Riemann zeta function
What's new 2024-07-07
Many modern mathematical proofs are a combination of conceptual arguments and technical calculations. There is something of a tradeoff between the two: one can add more conceptual arguments to try to reduce the technical computations, or vice versa. (Among other things, this leads to a Berkson paradox-like phenomenon in which a negative correlation can be observed between the two aspects of a proof; see this recent Mastodon post of mine for more discussion.)
In a recent article, Heather Macbeth argues that the preferred balance between conceptual and computational arguments is quite different for a computer-assisted proof than it is for a purely human-readable proof. In the latter, there is a strong incentive to minimize the amount of calculation to the point where it can be checked by hand, even if this requires a certain amount of ad hoc rearrangement of cases, unmotivated parameter selection, or otherwise non-conceptual additions to the arguments in order to reduce the calculation. But in the former, once one is willing to outsource any tedious verification or optimization task to a computer, the incentives are reversed: freed from the need to arrange the argument to reduce the amount of calculation, one can now describe an argument by listing the main ingredients and then letting the computer figure out a suitable way to combine them to give the stated result. The two approaches can thus be viewed as complementary ways to describe a result, with neither necessarily being superior to the other.
In this post, I would like to illustrate this computation-outsourced approach with the topic of zero-density theorems for the Riemann zeta function, in which all computer verifiable calculations (as well as other routine but tedious arguments) are performed “off-stage”, with the intent of focusing only on the conceptual inputs to these theorems.
Zero-density theorems concern upper bounds for the quantity for a given and large , which is defined as the number of zeroes of the Riemann zeta function in the rectangle . (There is also an important generalization of this quantity to -functions, but for simplicity we will focus on the classical zeta function case here). Such quantities are important in analytic number theory for many reasons, one of which is through explicit formulae such as the Riemann-von Mangoldt explicit formula
relating the prime numbers to the zeroes of the zeta function (the “music of the primes”). The better bounds one has on , the more control one has on the complicated term on the right-hand side.Clearly is non-increasing in . The Riemann-von Mangoldt formula, together with the functional equation, gives us the asymptotic
in the case, while the prime number theorem tells us that The various zero free regions for the zeta function can be viewed as slight improvements to (2); for instance, the classical zero-free region is equivalent to the assertion that vanishes if for some small absolute constant , and the Riemann hypothesis is equivalent to the assertion that for all .Experience has shown that the most important quantity to control here is the exponent , defined as the least constant for which one has an asymptotic
as . Thus, for instance, , and is a non-decreasing function of , so we obtain the trivial “von Mangoldt” zero density theorem Of particular interest is the supremal value of , which has to be at least thanks to (3). The density hypothesis asserts that the maximum is in fact exactly , or equivalently that for all . This is of course implied by the Riemann hypothesis (which clearly implies that for all ), but is a more tractable hypothesis to work with; for instance, the hypothesis is already known to hold for by the work of Bourgain (building upon many previous authors). The quantity directly impacts our understanding of the prime number theorem in short intervals; indeed, it is not difficult using (1) (as well as the Vinogradov-Korobov zero-free region) to establish a short interval prime number theorem for all if is a fixed exponent, or for almost all if is a fixed exponent. Until recently, the best upper bound for was , thanks to a 1972 result of Huxley; but this was recently lowered to in a breakthrough work of Guth and Maynard.In between the papers of Huxley and Guth-Maynard are dozens of additional improvements on , though it is only the Guth-Maynard paper that actually lowered the supremum norm . A summary of most of the state of the art before Guth-Maynard may be found in Table 2 of this recent paper of Trudgian and Yang; it is complicated, but it is easy enough to get a computer to illustrate it with a plot:
(For an explanation of what is going on under the assumption of the Lindelöf hypothesis, see below the fold.) This plot represents the combined effort of nearly a dozen papers, each one of which claims one or more components of the depicted piecewise smooth curve, and is written in the “human-readable” style mentioned above, where the argument is arranged to reduce the amount of tedious computation to human-verifiable levels, even if this comes the cost of obscuring the conceptual ideas. (For an animation of how this bound improved over time, see here.) Below the fold, I will try to describe (in sketch form) some of the standard ingredients that go into these papers, in particular the routine reduction of deriving zero density estimates from large value theorems for Dirichlet series. We will not attempt to rewrite the entire literature of zero-density estimates in this fashion, but focus on some illustrative special cases.
— 1. Zero detecting polynomials —
As we are willing to lose powers of here, it is convenient to adopt the asymptotic notation (or ) for , and similarly for .
The Riemann-von Mangoldt formula implies that any unit square in the critical strip only contains zeroes, so for the purposes of counting up to errors, one can restrict attention to counting sets of zeroes whose imaginary parts are -separated, and we will do so henceforth. By dyadic decomposition, we can also restrict attention to zeroes with imaginary part comparable to (rather than lying between and .)
The Riemann-Siegel formula, roughly speaking, tells us that for a zero as above, we have
plus terms which are of lower order when . One can decompose the sum here dyadically into pieces that look like for . The component of this sum is basically ; so if there is to be a zero at , we expect one of the other terms to balance it out, and so we should have for at least one value of . In the notation of this subject, the expressions are known as zero detecting (Dirichlet) polynomials; the large values of such polynomials provide a set of candidates where zeroes can occur, and so upper bounding the large values of such polynomials will lead to zero density estimates.Unfortunately, the particular choice of zero detecting polynomials described above, while simple, is not useful for applications, because the polynomials with very small values of , say , will basically obey the largeness condition (6) a positive fraction of the time, leading to no useful estimates. (Note that standard “square root ” heuristics suggest that the left-hand side of (6) should typically be of size about .) However, this can be fixed by the standard device of introducing a “mollifier” to eliminate the role of small primes. There is some flexibility in what mollifier to introduce here, but a simple choice is to multiply (5) by for a small , which morally speaking has the effect of eliminating the contribution of those terms with , at the cost of extending the range of slightly from to , and also introducing some error terms at scales between and . The upshot is that one then gets a slightly different set of zero-detecting polynomials: one family (often called “Type I”) is basically of the form
for , and another family (“Type II”) is of the form for and some coefficients of size ; see Section 10.2 of Iwaniec-Kowalski or these lecture notes of mine, or Appendix 3 of this recent paper of Maynard and Platt for more details. It is also possible to reverse these implications and efficiently derive large values estimates from zero density theorems; see this recent paper of Matomäki and Teräväinen.One can sometimes squeeze a small amount of mileage out of optimizing the parameter, but for the purpose of this blog post we shall just send to zero. One can then reformulate the above observations as follows. For given parameters and , let denote the best non-negative exponent for which the following large values estimate holds: given any sequence of size , and any -separated set of frequencies for which
for some , the number of such frquencies does not exceed . We define similarly, but where the coefficients are also assumed to be identically . Then clearly and the above zero-detecting formalism is (morally, at least) asserting an inequality of the form The converse results of Matomäki and Teräväinen morally assert that this inequality is essentially an equality (there are some asterisks to this assertion which I will gloss over here). Thus, for instance, verifying the density hypothesis (4) for a given is now basically reduced to establishing the “Type I” bound for all , as well as the “Type II” variant as .As we shall see, the Type II task of controlling for small is relatively well understood (in particular, (10) is already known to hold for all , so in some sense the “Type II” half of the density hypothesis is already established); the main difficulty is with the Type I task, with the main difficulty being that the parameter (representing the length of the Dirichlet series) is often in an unfavorable location.
Remark 1 The approximate functional equation for the Riemann zeta function morally tells us that , but we will not have much use for this symmetry since we have in some sense already incorporated it (via the Riemann-Siegel formula) into the condition .
The standard mean value theorem for Dirichlet series tells us that a Dirichlet polynomial with has an mean value of on any interval of length , and similarly if we discretize to a -separated subset of that interval; this is easily established by using the approximate orthogonality properties of the function on such an interval. Since an interval of length can be subdivided into intervals of length , we see from the Chebyshev inequality that such a polynomial can only exceed on a -separated subset of a length interval of size , which we can formalize in terms of the notation as
For instance, this (and (7)) already give the density hypothesis-strength bound (9) – but only at . This initially looks useless, since we are restricting to the range ; but there is a simple trick that allows one to greatly amplify this bound (as well as many other large values bounds). Namely, if one raises a Dirichlet series with to some natural number power , then one obtains another Dirichlet series with , but now at length instead of . This can be encoded in terms of the notation as the inequality for any natural number . It would be very convenient if we could remove the restriction that be a natural number here; there is a conjecture of Montgomery in this regard, but it is out of reach of current methods (it was observed by in this paper of Bourgain that it would imply the Kakeya conjecture!). Nevertheless, the relation (12) is already quite useful. Firstly, it can easily be used to imply the Type II case (10) of the density hypothesis, and also implies the Type I case (9) as long as is of the special form for some natural number . Rather than give a human-readable proof of this routine implication, let me illustrate it instead with a graph of what the best bound one can obtain for becomes for , just using (11) and (12):Here we see that the bound for oscillates between the density hypothesis prediction of (which is attained when ), and a weaker upper bound of , which thanks to (7), (8) gives the upper bound that was first established in 1937 by Ingham (in the style of a human-readable proof without computer assistance, of course). The same argument applies for all , and gives rise to the bound in this interval, beating the trivial von Mangoldt bound of :
The method is flexible, and one can insert further bounds or hypotheses to improve the situation. For instance, the Lindelöf hypothesis asserts that for all , which on dyadic decomposition can be shown to give the bound
for all and all fixed (in fact this hypothesis is basically equivalent to this estimate). In particular, one has for any and . In particular, the Type I estimate (9) now holds for all , and so the Lindeöf hypothesis implies the Density hypothesis.In fact, as observed by Hálasz and Turán in 1969, the Lindelöf hypothesis also gives good Type II control in the regime . The key point here is that the bound (13) basically asserts that the functions behave like orthogonal functions on the range , and this together with a standard duality argument (related to the Bessel inequality, the large sieve, or the method in harmonic analysis) lets one control the large values of Dirichlet series, with the upshot here being that
for all and . This lets one go beyond the density hypothesis for and in fact obtain in this case.While we are not close to proving the full strength of (13), the theory of exponential sums gives us some relatively cood control on the left-hand side in some cases. For instance, by using van der Corput estimates on (13), Montgomery in 1969 was able to obtain an unconditional estimate which in our notation would be
whenever . This is already enough to give some improvements to Ingham’s bound for very large . But one can do better by a simple subdivision observation of Huxley (which was already implicitly used to prove (11)): a large values estimate on an interval of size automatically implies a large values estimate on a longer interval of size , simply by covering the latter interval by intervals. This observation can be formalized as a general inequality whenever and ; that is to say, the quantity is non-decreasing in . This leads to the Huxley large values inequality, which in our notation asserts that for all and , which is superior to (11) when . If one simply adds either Montgomery’s inequality (15), or Huxley’s extension inequality (17), into the previous pool and asks the computer to optimize the bounds on as a consequence, one obtains the following graph:In particular, the density hypothesis is now established for all . But one can do better. Consider for instance the case of . Let us inspect the current best bounds on from the current tools:
Here we immediately see that it is only the case that is preventing us from improving the bound on to below the density hypothesis prediction of . However, it is possible to exclude this case through exponential sum estimates. In particular, the van der Corput inequality can be used to establish the bound for , or equivalently that
for ; this already shows that vanishes unless which improves upon the existing restriction when . If one inserts this new constraint into the pool, we recover the full strength of the Huxley bound valid for all , and which improves upon the Ingham bound for :Remark 2 The above bounds are already sufficient to establish the inequality for all and , which is equivalent to the twelfth moment estimate of Heath-Brown (albeit with a slightly worse type factor). Conversely, adding this twelfth moment to the pool of inequalities does not actually increase the ability to prove additional estimates, although for human written proofs it is a useful tool to shorten the technical verifications.
One can continue importing in additional large values estimates into this framework to obtain new zero density theorems, particularly for large values of , in which variants of the van der Corput estimate, such as bounds coming from other exponent pairs, the Vinogradov mean value theorem or (more recently) the resolution of the Vinogradov main conjecture by Bourgain-Demeter-Guth using decoupling methods, or by Wooley using efficient congruencing methods. We will just give one illustrative example, from the Guth-Maynard paper. Their main technical estimate is to establish a new large values theorem (Proposition 3.1 from their paper), which in our notation asserts that
whenever and . By subdivision (16), one also automatically obtains the same bound for as well. If one drops this estimate into the mix, one obtains the Guth-Maynard addition to the Ingham and Huxley bounds (which are in fact valid for all , but only novel in the interval ):This is not the most difficult (or novel) part of the Guth-Maynard paper – the proof of (20) occupies about 34 of the 48 pages of the paper – but it hopefully illustrates how some of the more routine portions of this type of work can be outsourced to a computer, at least if one is willing to be convinced purely by numerically produced graphs. Also, it is possible to transfer even more of the Guth-Maynard paper to this format, if one introduces an additional quantity that tracks not the number of large values of a Dirichlet series, but rather its energy, and interpreting several of the key sub-propositions of that paper as providing inequalities relating and (this builds upon an earlier paper of Heath-Brown that was the first to introduce non-trivial inequalities of this type).
The above graphs were produced by myself using some quite crude Python code (with a small amount of AI assistance, for instance via Github Copilot); the code does not actually “prove” estimates such as (19) or (21) to infinite accuracy, but rather to any specified finite accuracy, although one can at least make the bounds completely rigorous by discretizing using a mesh of rational numbers (which can be manipulated to infinite precision) and using the monotonicity properties of the various functions involved to control errors. In principle, it should be possible to create software that would work “symbolically” rather than “numerically”, and output (human-readable) proof certificates of bounds such as (21) from prior estimates such as (20) to infinite accuracy, in some formal proof verification language (e.g., Lean). Such a tool could potentially shorten the primary component of papers of this type, which would then focus on the main inputs to a standard inequality-chasing framework, rather than the routine execution of that framework which could then be deferred to an appendix or some computer-produced file. It seems that such a tool is now feasible (particularly with the potential of deploying AI tools to locate proof certificates in some tricky cases), and would be useful for many other analysis arguments involving explicit exponents than the zero-density example presented here (e.g., a version of this could have been useful to optimize constants in the recent resolution of the PFR conjecture), though perhaps the more practical workflow case for now is to use the finite-precision numerics approach to locate the correct conclusions and intermediate inequalities, and then prove those claims rigorously by hand.