Why Twin Primes Keep Appearing Despite Their Rarity

Why Twin Primes Keep Appearing Despite Their Rarity

The twin prime conjecture states there are infinitely many prime pairs separated by two. Despite primes becoming rarer, twin primes keep appearing. In 2013, Yitang Zhang proved there is a finite bound on gaps between primes, sparking progress. Later, James Maynard reduced the bound to 246. The conjecture remains unproven, but advances continue.

If Prime Numbers Become Increasingly Rare, Then Why Do They Keep Showing Up In Pairs?. | Transcript:

- [Derek] On the morning of the 17th of April, 2013, the journal Annals of Mathematics received a curious email. It claimed to contain a 50 page proof relating to one of the oldest unsolved problems in mathematics. A problem that great mathematician Edmund Landau called unattackable. But this proof didn't come from a famous professor, it came from an unknown. (bright music) Someone who had once spent years working at a subway restaurant. - So, they're like, okay, surely this isn't gonna work out, but whatever.

You know, we'll send it to a referee. - [Derek] They expected to find a mistake in an afternoon, but they didn't. So, they went through it again, closely studying the fragile parts where a proof like this normally falls apart, but still nothing. Soon they realized they were witnessing a breakthrough. - Oh, damn, he did it. - So, how did he do It? What did the experts miss and what was the problem he was working on? He was working on a new way to attack one of the hardest unsolved problems in number theory, the twin prime conjecture. The twin primes are prime numbers separated by just one number, like 11 and 13 or 17 and 19.

As you go up, the number line, primes become rarer and twin primes become rarer still. But the twin prime conjecture claims that there are infinitely many of them you never run out. But is it true? Well, one way to approach this problem is to look at the gaps between consecutive primes as you go up the number line. (bright music) Now, at first they seem chaotic, but if you average them out, a clear trend emerges. The average gap between two primes grows roughly as the natural logarithm of the number N. So, for example, the average gap between primes around 100, is approximately 4.6.

The average gap between primes around 1000 is 6.9. Logarithms grow very slowly, but they do keep growing forever. So, as N approaches infinity, the average gap between primes also goes to infinity, which is not particularly encouraging if you expect to always be able to find primes that are just two apart. But if you start checking large numbers, then after a million you quickly find the twin primes, 1,000,037 and 1,000,039. Past a billion, there's 1,000,000,007 and 1,000,000,009. In fact, we have found a twin prime that's as large as 2,996,863,034,895 times two to the power of 1,290,000 plus or minus one. That is a pair of numbers each 388,342 digits long.

If you wanted to print those in a book for some reason, you would need around 260 pages per number. All of this is to say that as far as we've looked, we've kept finding twin primes. But of course, that is not how you solve the twin prime conjecture, because you cannot physically check all the numbers out to infinity. So, we need another way. And around 100 years ago, it felt like mathematicians were getting close with a more sophisticated method. In 1923, English mathematicians, Hardy and Littlewood figured out how you can estimate how many twin primes there should be. To do it, they started with one of the crowning achievements of number theory, the prime number theorem, which tells you that the odds

of a large number near N being prime are roughly one over the natural logarithm of N. So, let's do a quick example to see how they use this. Say you wanna find the odds that 137,037 is prime. Well, you just plug that in and find that it has about an eight and a half percent chance of being prime. Of course, you could use the same trick to find the odds that 137,039 is prime, which is also around 8.5%. So, what are the odds that both of them are prime? Well, you just multiply those odds together, that gives you around a 0.7% chance.

Now we are simplifying a little here because we're assuming that primes are independent, which they're not. But putting that to one side, the odds of both a large number N and N plus two being prime, is one over ln(n) times one over ln(n) plus two, for large N the plus two is insignificant. So, we get the odds of any pair of numbers of size N being a twin prime are roughly one over ln(n) squared. But of course, the problem with this approach is that the odds of a pair being a twin prime decrease as you go to larger numbers. So, to count all twin primes up to N, we add up the odds at every single position. In other words, we integrate this expression from the first prime two up to that number.

Now Hardy and Littlewood refine this argument further to account for the fact that primes aren't really independent. But all that added is just this correction factor upfront. So, the final expression looks like this. (bright music) Now let's plot how many twin primes, this predicts up to 1 trillion. We see that it keeps growing and if we fill in the actual count, you'll notice that it is extremely close to the point where by 1 trillion, our estimate is only off by 0.001%. (bright music) - The only issue though, is that this is just an estimate or what mathematicians call a heuristic. It can tell you roughly how many twin primes to expect, but it can't guarantee that they never stop. As Terry Tao puts it, for all we know,

there could be this fast conspiracy that every time a number N decides to be prime, it has some secret agreement with its neighbor N plus two saying you're not allowed to be prime anymore. So, what we need is a rigorous airtight mathematical proof, but finding such a proof it turns out, is extremely difficult. One of the first mathematicians to really try was a 29-year-old Norwegian Viggo Brun. (bright music) Brun was working during the early years of the first World War, and with travel disrupted and Europe's mathematical centers largely out of reach, much of his work happened in quiet isolation back in Norway.

His goal was simple to try and prove that the number or count of twin primes always increases. And to do this, he took a 2000 year old prime counting tool and tried to adapt it for twin primes. Now the normal version of this tool is called the sieve of Eratosthenes, and it works something like this. (bright music) Say we want to find the primes up to 100. First we remove one because that's not prime. Then we circle the first prime, that is two, and we remove all of its multiples.

We do the same for three. Circle it and remove its multiples, (bright music) and then repeat that for five and seven. And with these four very quick operations, every single number that's left on your 10 by 10 grid is a prime number. Now we stop the sieve at seven, and we could keep going sieve by 11 or 30, but it turns out that we don't have to. So, why is that? Well, the next prime is 11. So, let's check the numbers below 100 that are divisible by 11. 22 is two times 11, but since it's divisible by two, we already caught it.

We also have 33, but that is divisible by three. And a similar logic holds for 55 and 77. But what about 11 times 11? Well notice that is 121, which is not in our range. So, every multiple of 11 below 100 was already caught by a smaller prime. So, sieving by 11 would cross out nothing new. - So, Eratosthenes, has this idea that you only need to sieve out the primes that are at most square root, the size of the number. - But there's also a second wave to visualize this sieve. Take the number line and let each prime send out a wave with the wavelength equal to the size of that number. So, the wave from two lands on every second number

and the wave from three on every third and so on. (bright music) Now, notice that wherever a wave lands, that number has to be composite. But the numbers that escape every preceding wave, well those have to be prime. And so you end up with this beautiful visualization of the sieve. (bright music) And both of these methods make it incredibly easy to identify the primes. - But Brun didn't care about which primes were left, he just needed to know how many were left. And to do this, he counted all the numbers that were crossed out and then subtracted this from the total, since whatever is not crossed out must be prime.

So, let's use this to find the number of primes below 100. We can keep a running count. So, we'll start with 100. We're gonna throw out all the multiples of two. Well, that's 50 numbers left, okay? Then we're gonna throw away multiples of three. Well, how many multiples of three are there less than 100. 100 divided by three is not a whole number. It's like 33 and a third, but we don't throw out fractions. So, okay, forget that stupid little fraction.

We do the same for the multiples of five and seven, but now the count has gone negative. Well, it turns out that some numbers got crossed out twice, like six for example, it's a multiple of two and three. So, we crossed it out once when we removed multiples of two and again when we removed multiples of three. So, we have to add it back once. The same thing happens for multiples of 10, which is two times five, 14 which is two times seven, 15, which is three times five and so on. So, we add these overlaps back and now the count rises to 28, but it's still not quite right.

- Now we've triple added back in the multiples of two times three times five, so that we need to subtract one more time. - The same thing happens for 42, which is two times three times seven, and 70, which is two times five times seven. There's a fourth triple two, three times five times seven, which is 105, but that's already bigger than 100. So, it contributes nothing here. But we'll write it in to keep the pattern complete. After the triples, we add back the quadruple, two times three times five times seven, which is 210. That's also bigger than 100, so it contributes zero as well.

And finally, we add back the four sieving primes themselves and subtract one because it was included in the count. So, the count lands at 25 primes. This process of alternating between subtracting and adding numbers back in is called inclusion exclusion. And it makes it super easy to find out how many primes there are up to some given number. Now this equation still looks a little bit like a mess, but actually we can rearrange it into this form. Now notice that every prime factor we added just adds an additional term in this series.

We've got the two that goes over here, the three goes over here, five goes over here, and seven goes over here. And if we were trying to find primes for some higher number and we wanted to add 11 as a prime factor, it would just add one additional one minus one over 11 term. Now in general, if we keep the sieve going up to some prime P, which is less than the square root of N, then the count becomes approximately this. - But this is still just the count of primes. What Brun really wanted was the count of twin primes. And so this is the part where he adapted his sieve, to find a case where both N and N plus two are prime. For example, when you're sieving by five,

you remove the numbers where N is divisible by five, just as before. But in addition, you need to remove the numbers where N plus two is divisible by five. So, you also remove 23, 28, 33 and so on because in those cases, N plus two is 25, 30, 35 and so on. So, this means that while in the ordinary sieve a prime P removes about one out of every P numbers, the twin prime sieve removes two. And this changes the count to this, where the numerator for all the terms after two, ends up getting a two. Now, if we plot the count of twin primes that this predicts, we see that it grows roughly like N over the natural logarithm of N squared, just as for our heuristic. And so it seems like Brun did it, but unfortunately that is not the case.

- We said that 100 divided by three is 33, and a third, but forget about that little error, those little errors, it's very difficult to control those errors 'cause there are so many of those little errors. Each one of them is at most one it's a rounding error. - Take the traditional sieve of Eratosthenes again, when we're sieving with just one prime, we just have an over two. So, one rounding error, with two primes, two and three, there are three separate rounding errors, but with three primes, two, three, and five, there are already seven separate rounding errors, each for one term in the inclusion exclusion. And so you might start to see the general pattern here.

One prime gives two to the power one minus one terms, two gives two to the power two minus one terms, and three primes gives two to the power three minus one terms. In general, if you sieve by K primes, you get roughly two to the power K minus one error terms, but we can drop the one because it doesn't really make much of a difference. So, that means that if you're sieving for normal primes, the error grows roughly like two to the power K. But if you're saving for twin primes, it gets even worse. And now the error grows roughly like four to the power K. And so this causes a lot of trouble very, very quick. (bright music) And then here for the twin primes, it's even worse.

The main term grows a bit more slowly, but the error term grows much faster. So, it still starts off slow. And so you can quickly see that once these error terms start dominating, you get into trouble. So, the issue he was facing is he had this main term and the main term, you know, it grew in the way he wanted to, but then it just got overtaken and dominated by those error terms. - Exactly. And that's all of analysis, in general analytic number theory in particular is the fight between the main term that you think is the truth, if it actually is the truth, and getting the error term to be provably lower order.

- So, for twin primes, the main term grows roughly like this, something like N over the natural logarithm of N squared. But that's not the true count, because to get the true count, we must add and subtract the error terms, which gives us this upper and lower bound. And now you can see the problem. Because the true count can be anywhere in this range, and as you can see, a big swath of it is negative. So, to prove the conjecture, what we must do is we must show that this lower bound is always positive and growing. But that is where we run into a problem with the sieve we've been using. Because the more primes we sieve by, the more those error terms start to accumulate and we just can show this.

- And so what Brun eventually realized is that if you weaken the sieve, if you don't sieve all the way up to square root X, but to up to something less than that, and you are very careful about your inclusion exclusion, you can actually take care of those error terms. - So, run a sieve by fewer factors, only up to N to the power one over 10. And as a result, he gained enough control over the error term. But this chooses a trade off. Say you wanted to find all the prime numbers up to 10 billion, then using the square root method, you would need to sieve up to the square root of 10 billion, which is 100,000. But that gives you a lot of error terms. But with Brun's method,

you would only need to sieve up to 10 billion to the power one over 10, which is just 10. But there's a catch. Because Brun, didn't sieve by all the primes, there were some survivors that weren't prime at all, but numbers with many prime factors, in Brun's case up to nine. So, what he actually ended up proving is that there are infinitely many pairs of numbers two apart, where each number has at most nine prime factors. - Brun's techniques were improved and improved and improved, and we went from nine prime factors to seven prime factors to three prime factors.

Until in 1973, a Chinese mathematician Chen Jingrun comes along and proved that there are infinitely many prime P, where P plus two has at most two prime factors. - This is as close as you could get to proving the twin prime conjecture without actually getting there. We're stuck like we're like as close as we can get and no further. - Right. So, that's one approximation, one mechanism for approximating twin primes. There's another. So, the other mechanism is what's the smallest gap between two consecutive primes? So, instead of saying they differ by two

and one of them's prime and the other one, well, we're gonna try to get, you know, reduce the number of prime factors that it has. Instead we're gonna say both numbers must be prime, but let's reduce the distance between them. - Now remember, on average consecutive primes sit about the natural logarithm of N apart. And so the question became, can we at least prove that primes come closer than that average gap? For decades, mathematicians kept chipping away at this problem. By 1988, the gap had dropped all the way down to roughly a quarter of the average gap.

This means that say the average gap is 100 at some enormous scale, then primes must sometimes come within about 25 of each other. But then in 2005, Goldston, Pintz and Yildirim, proved a result that shocked the mathematical community. - And they announced the proof of a spectacular result, 0% bounded gaps of 0% of the average. - Wait, what? - They proved that you can make the fraction as small as you want. This means the gap could be one 10th or 100th, or even 1000000th of the average gap. It could be as arbitrarily small as you like, and infinitely often primes get that close.

- When I was young, we didn't know there were gaps of size, less than say one 10th log X or infinitely many primes pairs and Goldston and Yildirim, came up with a method to attack that. - But the method also came close to something even bigger, an absolute bounded gap between two primes. (bright music) - And so that was like a big thing. - What was the big desire to go from 0% or arbitrarily small to a concrete bounded gap? - Well, because we think that the bound between two consecutive primes infinitely often is two, and right now we're showing that it's log X, and X grows.

- If they could get to bounded gaps, then they had another method to attack the conjecture. The only problem was that it seemed like their tool ran into a wall. And so in 2005, the American Institute of Mathematics convened a meeting, gathering all the world's leading experts on this problem. GPY, Andrew Granville, Kannan Soundararajan, all gathered for a week in California with one explicit goal, to prove a bounded gap between primes. - I was a young graduate student, I was very lucky to get to go to this meeting. You're surrounded by the world's top experts on this subject.

We spent an entire week. And the upshot of this week was basically that it's impossible. And Soundararajan shows how it's not possible to do this thing. And so as far as I was concerned, that was that, and I, you know, this wasn't gonna be my thesis problem, I'd have to do something else and I went off and did other things. But there was one person who was not at this meeting, Yitang Zhang. (bright music) - Zhang grew up in China, and around the age of 30 moved to the U.S. to get his PhD in mathematics, but he never got any recommendation letters and so struggled to find a job.

He ended up living in his car for time and ultimately working odd jobs for seven years, including at Subway, where he kept the books and sorted receipts. Yet, while doing all of that in his spare time, he would drive down to the local library to read books and journals on number theory. (bright music) Then in 1999, he got a lucky break. One of his friends helped him get a job as a lecturer at the University of New Hampshire. So, now he could spend all his time doing what he loved most, math.

Zhang said that when he was a kid, he "Imagined there would be a day that I would solve a major math problem." And by 2010, he had identified his major problem to focus on, bounded gaps between primes. But to understand what he was working on, we need to understand what exactly it is that GPY had built. See, they wanted to find two primes within a bounded gap. And to do this, they imagined a stencil of sorts with some holes in it. So, let's do the same, and for the sake of this example, let's say the stencil has a diameter of six and holes at spot zero, two and six. Next we place the stencil anywhere on the number line, say starting at 12. And we ask a simple question, how many primes do we see? In this case, we see 12, 14, and 18,

none of which are prime. So, we write down zero. Next we shift the stencil over one spot and repeat, 13, 15, and 19, two of which are prime. So, we write down two. And then we keep doing this. If you slide it a bit further, we catch two primes again, 23 and 29. Now, if you keep moving the stencil across the number line and it keeps catching two primes, then you have proven that infinitely often pairs of primes exist within a bounded gap. Six in this case, since that's the diameter of our stencil. But there are two problems with this approach. The first is that we don't know exactly where all the primes are. And the second is that of course you can't keep sliding

a stencil down the number line forever because well, it's infinitely long. Fortunately, there is an easy way to get around that first problem. See, while we don't know exactly where all the primes are, we do know how they behave on average. For example, say we take some stretch of the number line from some very large number X to two X, then we can't predict exactly which numbers in this stretch will be prime. But what we can do is put this into the prime number theorem and estimate how many primes to expect on average.

(bright music) And GPY realized they could do a similar thing for their stencil method to build an averaging machine of sorts. All you do is you set up the stencil you want, input your range, and out comes the average number of primes it caught per position. So, let's do a quick example without the machine to see how this helps. Let's use the same stencil from before and use the range 20 to 40. Now, in reality, you would use a much larger range, but for simplicity, let's stick to this. To find the average, you just place the stencil at the start of your range,

and you'll look at which numbers you see, 20, 22 and 26. None of which are prime and so the total prime count is zero. Then we shift the stencil over one spot and see 21, 23 and 27. One of which is prime, so we increase the total by one. Now let's keep shifting the stencil and keep adding all the primes we see. No primes, so add zero, two primes, add two, and so on. (stencil shuffling) Now, if you keep shifting the stencil all the way until the end, you find a total of 14 primes across 21 starting positions.

(bright music) And so the average is 14 over 21, or about 0.67 primes per location. Now in this case where we checked every spot by hand, we could see that the stencil caught several positions with two primes in them. But the averaging machine doesn't tell us this. All that spits out is an average, and that average alone can't guarantee that it caught two primes at once, because you could just as well imagine spreading all of these 14 primes over 14 individual positions. And the same would be true if the average was 0.8, 0.9, or even one. I know it's unlikely, but it could be that every position only caught one prime to bring the average to one. But what if the average was larger than one?

What if we found 22 primes in total? Well, even if every position had caught one prime, you would still have one prime left over and that needs to go in one of those other positions. So, in other words, if the average is above one, then that guarantees that at least one position caught two primes. And so that's the game. Get the average number of primes above one, but that still leaves one question. Even if you could do that, then how does that help you prove that you keep getting bounded gaps all the way up to infinity?

Well, remember you just proved that for any large number X, so there's nothing stopping you from replacing that X with two X, and that two X with four X, and then you could replace the two X with four X and the four X with eight X, and so, on all the way up to infinity. So, if you can prove that the average gap is above one for any large number X, then with this clever setup, it immediately extends all the way out to infinity, and you would've proven your bounded gap. (bright music) - Unfortunately, running the machine with just the current inputs, the stencil visits many positions where it catches zero primes, this drags the average down and means it will never get above one. So, GPY made one final tweak to their machine.

They realize that some starting positions are more likely to catch primes than others. For example, if the stencil only lands on even numbers, it will always give zero primes unless it includes two. So, those positions should not be weighted equally to the others. Similarly, if you can divide one or more of the numbers in the stencil by three, five, seven, or other small prime factors, it's also more unlikely you have primes in your stencil. So, they should also be weighted less. Now, GPY did a few simple checks like this for each position to assign it a weight, and estimate of how likely it is that this position contains primes.

This turned their averaging machine into a weighted averaging machine. And as a result, that weighted average shot up. With their updated machine they proved that if you pick any large number X, then in the range X to two X, you could get the weighted average arbitrarily close to one, but not above one. They were close, but they ran into a wall. A wall that came from their weights, because to figure out how to weigh different positions, they needed to know how primes are distributed in something known as an arithmetic progression.

These are basically just series of numbers where they're all the same step size apart. So, 3, 7, 11, 15, that's an arithmetic progression with a step size of four. Now, GPY needed to know how primes were distributed over many such arithmetic progressions with all kinds of different step sizes. Fortunately, they knew of theorem from the 1960s they could use to do exactly this. - It's the substitute for the Riemann hypothesis on average, and it turns out that's what's actually necessary in a lot of these arithmetic applications.

- But that theorem doesn't work everywhere. It only works for step sizes up to the square root of X or X to the one half. The technical term for this one half is the level of distribution or theta. It tells you how large the step sizes can be while still getting a reliable count of primes in those progressions. And mathematicians believed that the theorem worked beyond X to the half, but this was never proven, and that is where the trouble lies. When GPY ran their calculation with this ceiling, they found the maximum weighted average they could get was two times theta.

In other words, they got really close to one, but they could never actually cross it. - They showed in their method that if you could get beyond a half, you could do 0.5000001. Then instead of just getting 0% of the average gap, they can get bounded gaps. The primes would be some fixed distance apart. - That's what the 2005 meeting was all about. They kept trying to push past that one half barrier, but they ended up concluding that it was impossible. From 2010 on, Zhang spent two years hacking away at it as well, internalizing the GPY argument, working day and night just to try and find a way through. (bright music) But by the summer of 2012, Zhang was exhausted, and had nothing to show for it.

Hoping to clear his head, he visited a friend in Colorado, where one evening while they were waiting to leave for concert, Zhang stepped outside alone into the backyard, looking out for the deer that usually roamed the property, but no deer came. So, he was just walking and thinking when all of a sudden the answer came to him. He hadn't brought any notes or paper, but Zhang believed his idea would work. (bright music) - You see, GPY had to count primes in many different arithmetic progressions with all sorts of step sizes. But Zhang focused on a special class of step sizes, one's built only from small prime factors,

and he realized he could reorganize the error terms into a form where most of them get canceled out. This allowed him to push past the half barrier by a tiny fraction, just one over 584. Zhang spent the next year finalizing his proof, and then on the 17th of April, 2013, he sent off that curious email to the Annals of Mathematics. - Now, the Annals gets a proof of the Riemann Hypothesis every other day, so they're like, okay, surely this isn't gonna work out, but whatever.

You know, we'll send it to a referee. And they're like, well, let's spend a few hours this afternoon. We'll find a mistake. We'll tell the annals, you know, sorry, it didn't work out. Here's where the gap is. And they start reading it, and then they flip back, because they're experts. So, they can just flip through like, yeah, this'll work, this will work, but wait a second, you're gonna get stuck here. And they flip five pages and they're like, oh, that's interesting. That's how you're gonna handle that.

Okay, but then you're gonna have another, it's like trying to lay down a carpet in a room. You're like, okay, I know that corner, that corner's gonna screw you up even if you managed to make it work over there. And then he goes over there, no, he cut it just right. It fits in that corner. Wait a second. And then they flip, you know, five more pages and by the end of the week, they've reconstructed the proof and everything's right. - Wow. (bright music) - Zhang's stencil had 3.5 million slots, spread across a span of 70 million. And by proving two of those slots always catch primes, he proved a bounded gap of 70 million. When the news broke, mathematicians were in disbelief.

- It was basically an unknown in the field. I actually thought when I started reading Yitang Zhang's paper and started realizing it was probably correct, I thought it was probably one of the people I knew under a pseudonym, trying to avoid being embarrassed by being wrong. If they were wrong. - Of course it wasn't, Zhang was real. And just over a year later he was awarded the MacArthur Genius Grant. (bright music) - To me, this is a really interesting aspect of mathematics in particular. I think it's one of the very few fields where we have a truly honest approach to success and what counts as success.

He sent in an argument, people took it seriously. They looked at it. They didn't think that he had a true proof. They sat there, they gave him the time that the argument was due, and he was immediately made a hero as he should have been. To me, it shows that mathematics culture works. We're doing the right thing. I think the cover of like Scientific American or whatever in 2013 is the story Yitang Zhang and how he got this bounded gap between primes toiling in complete isolation. And maybe it's because he was in isolation that he didn't have the group think that the rest of us did. To know that this was not, we were all told it's an impossible problem.

- After realizing a bounded gap was not an impossible dream, people reworked Zhang's method to optimize it. Terence Tao spearheaded an online group called Polymath, and they sharpened the method. So, every month, week or day, that upper bound kept coming down. One of the attendees described being at a conference at the time, as I remember, sort of day by day, everyone was refreshing their screens to see who had the world record now, and ultimately they got the number all the way down to 4,680. - Meanwhile, there's a young postdoc named James Maynard, who just got his PhD at Oxford with Roger Heath-Brown and other one of these analytic number theory world experts. And he's working with Andrew Granville, in Montreal.

And he has a completely orthogonal approach to this problem that he'd been making some very incremental progress on independently. - When Maynard started, his advisor explicitly told him, "I hope you won't work on this problem full time, because I'm really pretty confident you're going to fail." But Maynard ignored the warning, and came up with a different way to attack the problem, and within just a few months, he got it to work. Further bring down the gap to 600. But his method also proved something else.

- He can get three primes in a bounded window. The bound has to change depending on how many primes you wanna put in that window, and that his number is better. And the method has nothing to do with the exponent one half. - Wait, what? - One half was a pure mirage. It was a red herring. - That is crazy. One half was not a fundamental limit at all. See where GPY average was stuck at two times theta, Maynard's average grew roughly like theta over two times the natural logarithm of K, where K is the number of slots in the stencil. So, all Maynard needed was enough slots and it worked.

- You need any number greater than zero. When you go up to one half because you have one half, you'll get better numerics. But the bounded gaps you could do, just going up to, you know, 0.01, not 0.50101. Now, curiously, Terry Tao independently has the same approach. He tells Ben Green about it, Ben Green is meeting with Andrew Granville, and says, "Hey, Terry's got this new idea that he thinks is gonna get even farther." And Granville, says, wait a second. My postdoc, Maynard, is doing exactly the same thing. We gotta get these two guys to talk to one another. Terry's a Fields medalist, he's like a super heavyweight by this point, whereas James is this, you know, fresh PhD,

and Terry says, you take it, I don't need this. You know, you this, it's your idea, you go with it. - By early 2014, Maynard joined forces with Tao's Polymath group. - It was clear that somehow 600 was just a proof of concept, and the same methods would give something smaller, but there were extra ideas that you could maybe use to squeeze everything out. And so the current world record is that there's infinitely many pairs of primes that differ by no more than 246. (bright music) - And that for now is where it stops. In 2022, James Maynard was awarded the Fields Medal, Mathematics, highest honor for his work on prime gaps.

- Yeah, I think that's the basic outline is the kind of two directions of approximation to twin primes Chen's direction, Brun, Chen, whatever, versus Zhang and Maynard, and this story that the Zhang wasn't at this meeting, so he didn't know it was impossible. - It reminded me of the four minute mile where everyone thought it was impossible. No one did it. - And then for the first time ever, Roger Bannister broke it in 1954, and after knowing it was possible, just 46 days later, another runner called John Landy broke it as well. And by the end of 1956, 10 people had broken it, all from knowing that it was possible. So, what about pushing that gap down even lower?

Is that possible? Well, mathematicians have found ways to bring it down even more, but all of these results are conditional. For example, there is the Elliot-Halberstam conjecture, which assumes primes are spread evenly across arithmetic progressions, or that the level of distribution can be taken as large as one. And in 2013, Maynard showed that if you assume this conjecture is true, then the gap plummets to just 12. A year later, the Polymath Group proved that if you assume an even stronger version of this conjecture, the gap drops all the way down to six. But without assuming anything, 246 still stands.

Let me ask you, do you think we're gonna solve the twin Prime conjecture? - I am totally convinced that humanity will eventually solve the twin prime conjecture. Often when I'm asked about some of these big famous problems like Twin Primes or Riemann or something like that, one way of kind of avoiding the question is that if you imagine you are randomly distributed in time, you'd expect to be sort of typically roughly halfway through when the conjectures been open for. So, an actual guess, if you have no other information, is to guess that a problem will be open for as long as it has been open for already.

But obviously this doesn't work for twin primes, 'cause we don't know whether it's- - Right. (all laughing) - 125 years old or whether it's like 2000 years old. So, I think it's a fools game to guess, but it clearly needs a really big idea, but maybe it only needs one big idea. - Although maybe, just maybe, not knowing this with certainty is for the best. Because if we knew for sure that this was impossible, then we would've likely missed out on most of these inventions and new methods over the past century. So, sometimes it pays not to know everything. - You know, before I started this channel, I was a teacher at a tutoring company, and honestly, it was the best job I'd had up until that point.

I could be the person who gave my students that one big idea that would help everything click. You know, the fastest way to learn something is to have someone next to you who already understands it. Unfortunately, many students don't have access to that kind of support. That is, until now, our long-term sponsor Brilliant, has just launched Koji, which is a revolutionary personal tutor that makes one-on-one learning accessible for everyone. Koji can see what you do and answer any questions. He can even draw onscreen to help explain those big ideas, just like a person sitting next to you.

He asks guiding questions, walks you through problems step by step, and adapts to where you're at in real time. You get a world class personal tutor in your pocket. Koji can help you work through math and coding streams from grade five through college and beyond. Each course is designed by experts from places like MIT, Harvard, Stanford, and Caltech. It's a great way to get seriously engaged with math, coding, think hard and have fun, especially for students on summer break to stay sharp or get ahead for next year. So, click the link below to get started with Brilliant's tutor for free,

and you can also upgrade to Premium to get full tutor support. Right now, Brilliant is giving Veritasium viewers a special 20% off an annual premium subscription. Just go to brilliant.org/veritasium. You can scan this QR code or click the link in the description. So, I wanna thank Brilliant for sponsoring this video and I wanna thank you for watching.

More Science Transcript