June 2017 – Maths and Machine Learning

I had initially planned on starting the blog with a post about my attempts to understand the wonderful (yet probably practically useless) Kolmogorov-Arnold representation theorem. However, I decided that it might be more fun to start with an explanation of the name of the blog as I have been asked about it a few times now. The story of $-\frac{1}{12}\$ is the story of divergent series and complex analysis. It is supposed to be the sum of all natural numbers!

Sum of natural numbers

Consider the sum of all natural numbers:

$S = 1 + 2 + 3 + 4 + \cdots$

It is pretty obvious that this series as written is divergent (i.e. the sum is infinite). Yet there is a famous claim of Ramanujan, that the “constant” of this series is $-\frac{1}{12}$ . In his notebook, he even offered a (rather dodgy) proof. (Side note: This dubious argument was widely popularized in the numberphile video). So as to not throw the baby out with the bathwater, let me note that much more rigorous arguments than the one Ramanujan presented in that notebook have been offered since then (and we will discuss one of them in this post).

A very thorough (and reasonably accessible) discussion of the above series can be found in Terrence Tao’s blog post. In this post, I will not be attempting to either match the rigor or the comprehensiveness of Tao’s treatment because (a) he has already done it so if you are in the market for rigor, click the link above, and (b) I cannot.

What I will offer instead is a more elementary discussion of the concepts behind divergent series. In fact, the only point of this (100% maths, 0% ML) post is to give some justification for the name of the blog which some might have attributed to my unreasonable fondness for negative rational numbers.

So then, let’s dive in to the murk of divergent series by asking some existential questions.

Sum by another name

For a finite set of numbers (or expressions), everyone can agree on what it means to sum them. When one has an infinite number of them, it is much trickier to define the sum. After all, one can’t expect to verify the answer by actually carrying out the computation (on a computer, say). It is clear that to assign a sum to any infinite series one needs to define a sensible procedure. For series where the individual terms decrease sufficiently rapidly, there is a rather obvious choice. One would want the sum to be such that if we included only a finite number of terms we get an approximation of the proposed result. Furthermore, this approximation should get better and better as we include more and more terms. So while, you can still not “prove” this with a computer, you can be pretty confident that you are on the right track. Formally, this is called the method of partial sums and was formally laid out by Cauchy in the 1800’s. We compute successively better finite approximations to the series and then define the sum of the series to be the limit of the finite approximations, if this limit exists.

By definition, for divergent series, the limit of the partial sums does not exist. There are two choices now. One is to say that such series are meaningless. For a long time, this was a popular choice amongst mathematicians. The other, bolder approach is to look for another procedure, which might give meaning to the pathological series while still being consistent with the conventional sum when the latter exists. After all, generalization is at the very heart of mathematics. It is through a process like this that leads to the definition of complex numbers (the corresponding puzzle there was whether one can make sense of square roots of negative numbers). In fact, the great Swiss mathematician Euler was convinced that divergent series can be made sense of. He compared the situation to the problem of defining logarithms for negative numbers. With the usual notion of a logarithm, it is clearly impossible for a negative number to have a valid logarithm. This follows simply from the fact that the log is the inverse of the exponential and the exponential of any number is positive. Hence, there is no number which can be the log of a negative number. Of course, once you admit complex numbers, the logarithm of negative numbers can be defined but it then turns out to be multi-valued (i.e. more than one value for the log of any number!). Through this morass, emerged the beautiful theory of Riemann surfaces. By taking a similar approach, meaning can be assigned to some divergent series if we look for some generalization to the usual notion of summation.

No matter which generalized procedure one comes up with, it is important to realize that it will not, strictly speaking, be the ‘sum’ of the series. Most of our puzzlement at Ramanujan’s claim arise from the abuse of terminology. It would be better to call it a value for the series which coincides with the sum of the series when the sum exists. With this caveat in mind, let us discuss one of the easiest method for assigning value to a series which was advanced by Euler. This will lead us gently into the world of analytic continuation of complex functions.

From Euler to Analytic Continuation

To discuss Euler’s idea, it is useful to consider the following (divergent) series which despite the faster divergence than the sum of natural numbers, is much easier to tackle.

$S_2 = 1 + 2 + 4 + 8 + \cdots$

This series is of the form:

$S_x = 1 + x + x^2 + x^3 + \cdots$

with $x=2$ . If only $x$ was less than 1 in magnitude, we could easily sum this:

$S_x = u(x) = \frac{1}{1-x}$

Euler seemed to view series as arising from Taylor expansions of functions. In this spirit, notice that while the series $S_x$ clearly breaks down if $|x| \geq 1$ , the function $u(x)$ is well defined everywhere except at $x=1$ . Euler proposed that we just think of the function $u(x)$ as the replacement for the series at all points of $x$ where the function is well defined. The divergent series $S_2$ we are trying to sum is what you would get from $S_x$ if you could just substitute $x=2$ . Euler tells us that the value of the series should then be whatever we get if we simplify set $x=2$ in the well defined function $u(x)$ .

$1 + 2 + 4 + 8 + \cdots \sim \frac{1}{1-2} = -1$

This, then, is the sum of the series in this generalized sense. To apply this method more generally is rather tricky. It is not enough to merely know the series you want to sum. You must also have an idea of a function which might formally lead to the series (the analogue of $u(x)$ ). In the above example, we made an obvious choice but there might exist other, not so obvious choices which might produce the above series for some value of their argument. (I can’t think of a general proof that all such functions would lead to the same answer. However, if we only consider functions where the individual terms of the series arise as the Taylor coefficients around some value of $x$ then it is easy to show that all such functions will lead to the same answer.)

Even though it worked for the toy example above, Euler’s method appears a little ad hoc. A better justification for this scheme requires a foray into the complex plane. Functions of a complex variable (specifically, holomophic functions) behave remarkably differently from their real counterparts. In particular, they have the amazing property that if two such functions agree in some region of the complex plane (howsoever small) they must agree everywhere else that they are defined. This is called the Identity Theorem. The idea of analytic continuation is underpinned by this theorem. Suppose you have a function $f(z)$ defined in some subset $F$ of the complex plane and another function $g(z)$ defined in another
subset $G$ which intersects $F$ . Now suppose you also know that on some open subset $D$ of $F\cap G$ , $f$ and $g$ agree with each other, i.e. $f(z) = g(z)$ . From this situation, the identity theorem guarantees that $f$ and $g$ must agree on all of $F\cap G$ . (See my crude diagram below in case it helps.)

Analytic continuation is the idea that we can extend $f$ to all of $G$ if we “define” it to be equal to $g$ on $G$ (and similarly for extension of $g$ to all of $F$ .). This makes sense because the identity theorem guarantees that we can do so uniquely. To illustrate, suppose after we have extended $f$ to all of $G$ by setting it to be $g$ , we discover another function say, $h$ which is also defined on some other intersection of $F$ and $G$ . One might then worry whether this new function $h$ agrees with our previous choice $g$ . Well, the identity theorem assures you that whatever that new value is, it must agree with $g$ and hence our original extension will always be valid. Thus functions can be patched together in a consistent way to expand their domain. Another way to think of this is that holomorphic (more generally, meromorphic but we won’t go there) functions on the complex plane exist independently of their representation. When you write out a specific expression, you are really presenting one specific view of that function which is valid locally. In other regions, there might be other expressions and they will all match up nicely everywhere that they exist at the same time.

To see how analytic continuation can be utilized to sum divergent series, first note that the series $S_x$ is well defined even if we make $x$ a complex number (which we will label $z$ ) so long as $|z| < 1$ (the modulus is now the complex modulus i.e. $|x+Iy| = \sqrt{x^2+y^2}$ ). This function is defined (and is holomorphic) on the set $U = |z| < 1$ . On the other hand, the function $u(z) = \frac{1}{1-z}$ is well defined everywhere on the complex plane except at $z=1$ . Furthermore, it’s value at every point on $U$ matches $S_z$ . Thus, $u(z)$ must be the unique analytic continuation of the series to points outside $U$ . In this sense, the “value” of the divergent series $S_2$ is given by the value of the continued function $u(z)$ evaluated at $z=2$ .
Again, it is important to note that we are not just assigning a value to the series $S_2$ ! Strictly speaking, this procedure can assign a value only to an entire class of series defined by $S_z$ (of which $S_2$ is a member). Hence, we must also have:

$S_3 = 1 + 3 + 9 + 27 + ... = -\frac{1}{2}$

However, there are other methods based on regularization of the series which can assign a value to a divergent series without needing you to guess a function for them. I won’t be discussing them in this post but if you are curious, then you should consult Terrence Tao’s post alluded to earlier. In addition to the dodgy proof that Ramanujan gave, he gave another more robust method to derive the same result (which is now called Ramanujan summation). We can safely conclude that he knew what he was doing.

Euler Zeta function

If we wanted to apply the ideas of the preceeding section to sum the natural numbers, we need to find a complex function which would look like the series $S$ for some value of its argument even if that value is outside its domain of definition. If it is indeed outside the domain, we would also need to find its analytic continuation to that value.

Luckily, Euler solved the first part of this problem when he introduced what we now know as Euler’s zeta function $\zeta(s)$ . The function is defined as:

$\zeta(s) = \sum_{n=1}^{\infty} \frac{1}{n^s}$

It is easy to convince oneself that the function is well defined (i.e. has a finite value) for all $s>1$ , simply because the series converges when $s>1$ . It is more non-trivial but still not too difficult to believe that this representation will also work with a complex value for $s$ if the real part of $s$ is greater than $1$ . It is also easy to see that if we could plug in $s=-1$ , we would get the sum of the natural numbers but sadly, as expected, this value is outside the domain of validity of Euler’s version of the zeta function.

Riemann Zeta function

It was Riemann who figured out how to properly extend the zeta function to the whole of the complex plane. This involves rewriting the summation in terms of some contour integrals. Details of this construction can be found in Riemann’s original paper and I won’t go into it in this post. The resulting complex function is called the Riemann zeta function and is the subject of perhaps, the most important open problem in mathematics, the Riemann hypothesis.

As a byproduct of the analytic continuation, Riemann also found a non-trivial identity for the zeta function.This is the so-called functional equation. It is this equation rather than the details of the analytic continuation which will allow us to compute our series. The functional equation reads:

$\zeta(s) = 2^s \pi^{s-1} \sin\left(\frac{\pi s }{2}\right) \Gamma(1-s) \zeta(1-s)$

This might look long and complicated so let’s swiftly note that it relates the value of the zeta function at $s$ with the value of the zeta function at $1-s$ . Here $\Gamma$ refers to the Gamma function which generalizes the factorial.

The functional equation allows us to move beyond the $Re(s) > 1$ boundary. We want to compute the zeta function at $s=-1$ and the functional equation allows us to relate this value to the value at $1-s=2$ . Specializing to this case, the functional equation becomes (recall that $\sin \pi/2 = 1$ and $\sin(-x) = -\sin(x)$ ):

$\zeta(-1) = -\frac{1}{2\pi^2} \Gamma(2)\zeta(2)$

So, to use this formula, we need to know $\Gamma(2)$ and $\zeta(2)$ . $\Gamma(2)$ is easy because for all positive integers $n$ , we have $\Gamma(n) = (n-1)!$ so $\Gamma(2) = 1$ . The zeta function at $s=2$ is the nicely convergent series:

$\zeta(2) = 1 + \frac{1}{2^2} + \frac{1}{3^2} + \frac{1}{4^2} \cdots$

Finding the sum of this series is known as the Basel problem. Euler had shown that this series is equal to $\frac{\pi^2}{6}$ . Hence we get for $\zeta(-1)$

$\zeta(-1) = -\frac{1}{2\pi^2} \times \frac{\pi^2}{6} = -\frac{1}{12}$

So, indeed, in this very qualified sense, the sum of all natural numbers is what gives this blog its name.

Postscript

As I mentioned above, this is not the only way to arrive at the answer. I had originally thought of writing about the alternative approach based on Ramanujan summation. However, that method involved way too much equation typing given the rather primitive support that wordpress provides for LaTeX.

To conclude this post, I will mention briefly a very quick (but hacky) method to derive this result that I noticed very recently. It involves only two simple ingredients (simple if you are familiar with them) related to Fourier Transforms. The first is the Fourier transform of the absolute value function $g(t) = |t|$ :

$\tilde{g}(f) = \int\limits_{-\infty}^{\infty} dt |t| e^{-2 \pi i f t} = -\frac{1}{2 \pi^2 f^2}$

This is a standard result that you can simply look up (for example, here). Next ingredient is the Poisson summation formula which (roughly) states that a sum over the integers of a function is equal to the sum over the integers of the fourier transform of the function. More precisely, it says:

$\sum\limits_{n=-\infty}^{\infty} g(n) = \sum\limits_{k=-\infty}^{\infty} \tilde{g}(k)$

Applying this formula with the absolute value function on the left hand side and its Fourier transform on the right, we get:

$\sum\limits_{n=-\infty}^{n=\infty} |n| = -\frac{1}{2\pi^2} \sum\limits_{k=-\infty}^{k=\infty} \frac{1}{k^2}$

Rearranging the limits we get the sum of the natural numbers on the left hand side. For the right hand side, we almost get the Basel series except for the divergence at $k=0$ . Calling this divergent term $C_0$ and ploughing on regardless, we can plug in the sum of the inverse square series to arrive at: