Knocking on Heaven's Door: How Physics and Scientific Thinking Illuminate the Universe and the Modern World - Lisa Randall (2011)



When I was an MIT professor, the department ran out of office space on the third floor where the particle physicists worked. So I relocated to the open office next door to Alan Guth’s on the floor below, which at the time housed theoretical astronomers and cosmologists. Although Alan started his career as a particle physicist, he is known today as one of the best cosmologists around. At the time of my office move, I had already explored some connections between particle physics and cosmology. But it’s a lot easier to continue such research when your neighbor shares those interests—and is as messy as you are so that in his office you feel right at home.

Many particle physicists have gone further afield than a single floor and crossed over into a wide variety of other research areas. Wally Gilbert, a cofounder of Biogen, started life as a particle physicist but left to do biology and Nobel Prize–winning chemistry research. Many since have followed in his footsteps. On the other hand, many of my graduate student friends left particle physics to be “quants” on Wall Street where they could bet on changes in future markets. They chose just the right time to make such a move since the new financial instruments to hedge such bets were only just being developed at the time. In the crossover to biology, some ways of thinking and organizing problems carried over, whereas in finance some of the methods and equations did.

But the overlap between particle physics and cosmology is of course far deeper and richer than either of the above. Close examination of the universe on different scales has exposed the many connections between elementary particles on the smallest scales and the universe itself at the largest. After all, the universe is by definition unique and encompasses everything within it. Particle physicists, who look inward, ask what type of fundamental matter exists at the core of matter, and cosmologists, who look outward, study how whatever it is that is out there has evolved. The universe’s mysteries—most not ably what it is made of—matter to cosmologists and particle physicists alike.

Both types of researchers investigate basic structure and employ fundamental physical laws. Each needs to take into account the results of the other. The content of the universe that is studied by particle physicists is an important research subject for cosmologists too. Furthermore, the laws of nature that incorporate both general relativity and particle physics describe the universe’s evolution, as they must if both theories are correct and apply to a single cosmos. At the same time, the known evolution of the universe constrains what properties matter can have if it is to avoid disrupting the observed history. The universe was in some respects the first and most powerful particle accelerator. Energies and temperatures were very high in the early stages of its evolution, and the high energies that accelerators currently achieve aim to reproduce some aspects of those conditions today on Earth.

Recent attention to this convergence of interests has led to many fruitful investigations and major insights and will hopefully continue to do so. This chapter considers some of the big open questions in cosmology that particle physicists and cosmologists both explore. The overlapping arenas include cosmological inflation, dark matter, and dark energy. We’ll consider aspects we understand about each of these phenomena and—more important for active research—those that we don’t.


Even though we can’t yet say what happened at the very beginning of the universe, since we would need a comprehensive theory that incorporates both quantum mechanics and gravity, we can assert with reasonable certainty that at some time very early on (perhaps as early as 10−39 seconds into the universe’s evolution), a phenomenon called cosmological inflation occurred.

In 1980, Alan Guth first suggested this scenario, which says that the very early universe essentially exploded outward. Interestingly, he was initially trying to solve a problem for particle physics involving the cosmological consequences of Grand Unified Theories. Coming from a particle background, he used methods rooted in field theory—the theory combining special relativity and quantum mechanics that particle physicists employ for our calculations. But he ended up deriving a theory that revolutionized our thinking about cosmology. How and when inflation occurred is still a matter of speculation. But a universe that underwent this explosive expansion would leave clear evidence, and much of it has now been found.

In the standard Big Bang scenario, the early universe grew calmly and steadily—for example, doubling in size when its age increased by a factor of four. But in an inflationary epoch, a patch of the sky underwent a phase of incredibly rapid expansion, growing exponentially with time. The universe doubled in size in a fixed time and then doubled again in that same time and then kept doubling at least 90 times in a row until the inflationary epoch ended and the universe was as smooth as we see it today. This exponential expansion means, for example, that when the universe’s age had multiplied by 60 times, the size of the universe would have increased by more than a trillion trillion trillions in size. Without inflation, it would have increased by a mere factor of eight. In some sense, inflation was the beginning of our story of evolving from the small to the large—at least the part that we can potentially understand through observations. The initial enormous inflationary expansion would have diluted the matter and radiation content of the universe to practically nothing. Everything we observe today in the universe must therefore have arisen right after inflation, when the energy that drove the inflationary explosion converted into matter and radiation. At this point in time, conventional Big Bang evolution took over—and the universe began its further expansion into the huge structure we see today.

We can think of the inflationary explosion as the “bang” that was the precursor to the universe’s evolving according to the standard Big Bang theory. It’s not truly the beginning—we don’t know what happened when quantum gravity played a role—but it’s when the Big Bang stage of evolution, with matter cooling and eventually aggregating, began.

Inflation also partially answers why there is something rather than nothing. Some of the enormous energy density stored during inflation was converted (consistently with E = mc2) to matter, and that is the matter that evolves to what we see today. As I discuss at the close of this chapter, we physicists still would like to know why there is more matter than antimatter in the universe. But whatever the answer to that question, the matter we know began evolving according to Big Bang theory predictions as soon as cosmological inflation ended.

Inflation was derived as a bottom-up theory. It solved important problems for the conventional Big Bang explosion, but only a few really believed any of the actual models for how it came about. No compelling high-energy theory seemed to obviously imply inflation. Since it was so challenging to make a credible model, many physicists (including those at Harvard when I was a graduate student) doubted the idea could be right. On the other hand, Andrei Linde, a Russian-born physicist now at Stanford, and one of the first to work on inflation, thought it had to be correct simply because no one had found any other solution to the puzzles about the size, shape, and uniformity of the universe that inflation addressed.

Inflation was an interesting example of the truth-beauty connection—or lack thereof. Whereas the exponential expansion of the universe beautifully and succinctly explained many phenomena about how the universe started, the search for a theory that naturally yields the exponential expansion led to many not-so-pretty models.

Recently, however, most physicists—even though not yet satisfied with most models—have become convinced that inflation, or something very similar to inflation, did occur. Observations of the last several years have confirmed the cosmological picture of Big Bang cosmology preceded by inflation. Many physicists now trust that Big Bang evolution and inflation have occurred because predictions based on these theories have been confirmed with impressive precision. The true model underlying inflation is still an open question. But the exponential expansion has a lot of evidence supporting it at this point.

One type of evidence for cosmological inflation has to do with the deviations from perfect uniformity in the cosmic microwave background radiation that the previous chapter introduced. The background radiation tells us much more than just that the Big Bang occurred. The beauty of it is that because it is essentially a snapshot of the universe very early on—before stars had time to form—it lets us look back directly into the beginnings of structure at the time when the universe was still very smooth. Cosmic microwave background (CMB) measurements also revealed tiny departures from perfect homogeneity. Inflation predicts this because quantum mechanical fluctuations caused inflation to end at slightly different times in different regions of the universe, giving rise to tiny deviations from absolute uniformity. The satellite-based Wilkinson Microwave Anisotropy Probe (WMAP), named for the Prince ton physicist David Wilkinson who pioneered the project, made detailed measurements that distinguished inflationary predictions from other possibilities. Despite the fact that inflation happened long ago at incredibly high temperatures, theory based on inflationary cosmology nonetheless predicts the exact statistical properties of the pattern of temperature variations that should be imprinted on the radiation in the sky today. WMAP measured the small inhomogeneities in temperature and energy density with more accuracy and on smaller angular scales than had been done before, and the pattern conformed to inflationary expectations.

The chief confirmation of inflation that WMAP gave us was the measurement of the universe’s extreme flatness. Einstein taught us that space can be curved. (See Figure 73 for examples of curved two-dimensional surfaces.) The curvature depends on the energy density of the universe. At the time when inflation was first proposed, it was known that the universe would suggest, but the measurements were far too imprecise to test the inflationary prediction that the universe would expand so much that any curvature would be stretched away. Microwave background measurements have now demonstrated that the universe is flat at the level of one percent, which would be extremely difficult to understand without some underlying physical explanation.


FIGURE 73 ] Zero, positive, and negative curvature on two-dimensional surfaces. The universe, too, can be curved, but in four-dimensional spacetime that is difficult to draw.

This flatness of the universe was a huge victory for inflationary cosmology. Had it not been true, inflation would have been ruled out. The WMAP measurements were also a victory for science. When theorists first proposed the detailed measurements of the microwave background that would eventually tell us about the geometry of the universe, everyone thought it interesting enough to throw out to the science community, but far too difficult technically to achieve any time soon. Within the decade, confounding all expectation, observational cosmologists made the necessary measurements and gave us amazing insights into how the universe has evolved. WMAP is still providing new results, performing detailed measurements of the variation in temperature across the sky. The Planck satellite in operation today is measuring these fluctuations more precisely still. The CMB measurements have proven to be a prime resource of insight into the early universe and will most likely continue to be so.

Recent detailed studies of the cosmic radiation left throughout the sky have led to other enormous leaps in our quantitative knowledge of the universe and its evolution. The details of the radiation have provided rich information about the matter and energy that surrounds us. In addition to telling us the conditions when the light first started heading toward us, the CMB tells us about the universe through which the light had to travel. If the universe had changed in the last 13.75 billion years, or if its energy were different than expected, relativity tells us that it would have affected the path that the light-ray took and consequently the measured properties of the radiation that was measured. Since it is such a sensitive probe of the energy content of the universe today, the microwave background gives information about what the universe contains. This includes the dark matter and dark energy we will now consider.


In addition to successfully confirming inflationary theory, CMB measurements presented a few major mysteries that cosmologists, astronomers, and particle physicists now want to address. Inflation tells us that the universe should be flat but it doesn’t tell us where the energy required to make it flat now resides. Nonetheless, based on Einstein’s equations of general relativity, we can calculate the energy needed for the universe to be flat today. It turns out that known visible matter alone provides a mere four percent of the energy required.

An additional puzzle that had already indicated the need for something new concerned the tininess of the fluctuations in temperature and density that COBE had measured. With only visible matter and such tiny perturbations, the universe wouldn’t have lasted long enough for the perturbations to have grown large enough for structure to have formed. The existence of galaxies and clusters of galaxies in conjunction with the tininess of the measured fluctuations pointed to the existence of matter that no one had yet directly seen.

In fact, scientists had already known that a new type of matter known as dark matter should exist well before COBE’s microwave radiation results. Other observations that we will get to soon that had already indicated additional unseen matter must exist. This mysterious stuff, which became known as dark matter, exerts gravitational forces, but it doesn’t interact with light. Because it neither emits nor absorbs light, it is invisible—not dark. Dark matter (we’ll keep using the term) has so far provided few tangible identifying features other than its gravitational influence and that it is so feebly interacting.


FIGURE 74 ] Pie chart illustrating the relative amounts of visible matter, dark matter, and dark energy of which the universe is composed.

Furthermore, gravitational influence and measurements indicate the presence of something even more mysterious than dark matter, known as dark energy. This is energy that permeates the universe, but doesn’t clump like ordinary matter or dilute as it expands. It is very much like the energy that precipitated inflation, but its density today is much smaller than it was back then.

Although we now live in a renaissance era of cosmology, in which theories and observations have advanced to the stage where ideas can be precisely tested, we also live in the dark ages. About 23 percent of the universe’s energy is carried by dark matter, and approximately another 73 percent is carried by the mysterious dark energy, as is illustrated in the pie chart. (See Figure 74.)

The last time something was called “dark” in physics was in the mid-1800s, when Urbain Jean Joseph Le Verrier of France proposed an unseen dark planet, which he named Vulcan. Leverrier’s goal was to explain the peculiar trajectory of the planet Mercury. Le Verrier, along with John C. Adams of England, had previously deduced the existence of Neptune based on its effects on the planet Uranus. Yet he was wrong about Mercury. It turned out that the reason for Mercury’s strange orbit was much more dramatic than the existence of another planet. The explanation could be found only with Einstein’s theory of relativity. The first confirmation that his theory of general relativity was correct was that he could use it to accurately predict Mercury’s orbit.

It could turn out that dark matter and dark energy are a consequence of known theories. But it might also be that these missing elements of the universe presage a similar significant change of paradigm. Only time will tell which of these options will resolve the dark matter and dark energy problems.

Even so, I’d say that dark matter is very likely to have a more conventional explanation, consistent with the type of physical laws we now know. After all, even if novel matter acts in accordance with force laws similar to those we know, why should all matter behave exactly like familiar matter? To put it more succinctly, why should all matter interact with light? If the history of science has taught us anything, it should be the shortsightedness of believing that what we see is all there is.

Many people think differently. They find dark matter’s existence very mysterious and ask how it can possibly be that most matter—about six times the amount we see—is something we can’t detect with conventional telescopes. Some are even suspicious that dark matter is really some sort of mistake. Personally, I think quite the opposite (though admittedly not even all physicists see it this way). It would perhaps be even more mysterious if the matter we can see with our eyes is all the matter that exists. Why should we have perfect senses that can directly perceive everything? Again, the lesson of physics over the centuries is how much is hidden from view. From this perspective, it’s mysterious why the stuff we do know should constitute even as much as 1/6 of the energy of all matter, an apparent coincidence that my colleagues and I are currently trying to understand.

We know something with dark matter’s properties has to be there. Although we don’t exactly “see” it, we do detect dark matter’s gravitational influence. We know dark matter exists due to the extensive observational evidence of its gravitational effects in the cosmos. The first clue that it existed came from the speed with which stars rotated in galaxy clusters. In 1933, Fritz Zwicky observed that galaxies in galaxy clusters orbited faster than could be accounted for by the visible mass, and Jan Oort soon after observed a similar phenomenon in the Milky Way. Zwicky was convinced enough by his work to conjecture the existence of dark matter that no one could directly see. But neither of these observations was conclusive. A faulty measurement or some other galaxy dynamics seemed like a far more plausible explanation than some invisible substance invented solely to provide additional gravitational attraction.

At the time Zwicky made his measurements, he didn’t have the resolution to see individual stars. Much more solid evidence for dark matter came from Vera Rubin, an observational astronomer, who much later—in the late 1960s and early 1970s—made detailed quantitative measurements of stars rotating in galaxies. What first seemed to be a “boring” study of stars orbiting in a galaxy—a study Vera turned to since it provided less-well-trodden territory than other astronomical activities at the time—emerged as the first solid evidence of dark matter in the universe. Rubin’s observations with Kent Ford yielded incontrovertible evidence that Zwicky’s conclusion years earlier had been correct.

You might wonder how someone could look through a telescope and see something dark. The answer is that she could see its gravitational consequences. The properties of a galaxy, such as the rate at which its stars orbit around, are influenced by how much matter it contains. With only visible matter present, one would have expected those stars well beyond the galaxy to be rather insensitive to the galaxy’s gravitational influence. Yet stars ten times farther away than the luminous central matter rotated with the same velocity as stars closer to the galaxy’s center. This implied that the mass density did not fall off with distance, at least to distances as far from the galaxy’s center as ten times the distance of the luminous matter. Astronomers concluded that galaxies consisted primarily of unseen dark matter. The luminous matter we see is a significant fraction, but most of the galaxy is invisible, at least in the ordinary sense of the word.

We now have a good deal of other supplementary evidence for dark matter’s existence. Some of the most direct is from lensing, illustrated in Figure 75. Lensing is the phenomenon that occurs when light passes a massive object. Even if that object itself doesn’t emit light, it does exert a gravitational force. And that gravitational force can cause light emitted by a nondark object behind (as seen from our vantage point) to bend. Because the light bends in different directions according to the path it takes around the dark object and because we automatically project straight lines for light, this lensing can produce multiple images of the original bright object in the sky. These multiple images allow us to “see” the dark object—or at least infer its existence and properties by deducing the gravity needed to bend the observed light.


FIGURE 75 ] Light passing a massive object can bend, which from the perspective of the observer appears to create multiple images of the original object.

Perhaps the strongest evidence to date that dark matter, rather than a modified gravitational theory, explains such phenomena comes from the Bullet cluster, which involved two colliding clusters of galaxies. (See Figure 76.) Their collision demonstrated that the clusters contain stars, gas, and dark matter. The hot gas in the cluster interacts strongly—so strongly that the gas remains concentrated in the central collision region. Dark matter, on the other hand, doesn’t interact—at least not very much. So the dark matter just passed through. Lensing measurements showed that the dark matter was indeed separated from the hot gas in just the way implied by a model of very weakly interacting dark and strongly interacting ordinary matter.


FIGURE 76 ] The Bullet cluster indicates that clusters of galaxies contain dark matter, and that their dynamics are unlikely to be explained by modified gravitational laws. That’s because we can see a separation between the more strongly interacting ordinary matter that gets trapped in the middle when two clusters collide and the far more weakly interacting dark matter, which is detected by gravitational lensing, and evidently just passes through.

We have further evidence for dark matter’s existence from the cosmic microwave background discussed earlier. Unlike lensing, the radiation measurements don’t tell us anything about the distribution of dark matter. Instead they tell us the net energy content carried by dark matter—how big a piece of the cosmic pie is constituted by the energy it carries.

CMB measurements tell us a great deal about the early universe and give us detailed information about its properties. These measurements argue not only for dark matter. They also support the existence of dark energy. According to Einstein’s equations of general relativity, the universe could only be flat with just the right amount of energy. Matter, even accounting for dark matter, simply didn’t suffice to account for the flatness measured by WMAP and balloon-based detectors. Other energy had to exist. Dark energy is the only way to account for the universe’s flatness—with no measurable curvature of three-dimensional space and agree with all other measurements to date.

Dark energy, which carries the bulk of the universe’s energy—approximately 70 percent—is even more puzzling than dark matter. The evidence that convinced the physics community of dark energy’s existence was the discovery that the expansion of the universe is currently accelerating—much as it did during inflation earlier on but at a very much slower rate. In the late 1990s, two independent research teams, the Supernova Cosmology Project and the High-z Supernova Team, surprised the physics community when they discovered that the rate of expansion of the universe is no longer slowing down, but is actually increasing.

Before the supernova measurements, a few hints had pointed to the existence of missing energy, but the evidence had been weak. But careful measurements in the 1990s showed that distant supernova were dimmer than expected. Since this particular type of supernova has fairly uniform and predictable emission, this could only be explained by something new. And that something new seems to be an accelerated expansion of the universe—that is, it is expanding at an increasingly faster rate.

This acceleration would not arise from ordinary matter, whose gravitational attraction would slow the universe’s expansion. The only explanation could be a universe that acts like one that is inflating, but with far smaller energy than during the inflationary phase the universe had undergone much earlier on. This acceleration could be due only to something that acted like the cosmological constant that Einstein had introduced, or dark energy, as it has become known.

Unlike matter, dark energy exerts negative pressure on its environment. Ordinary positive pressure favors inward collapse, whereas negative pressure leads to accelerated expansion.68 The most obvious candidate for negative pressure—one that agrees with measurements so far—is Einstein’s cosmological constant, representing an energy and pressure that permeates the universe but is not carried by matter. Dark energy is the more general term we now use to allow for the possibility that the cosmological constant’s assumed relationship between energy and pressure isn’t precisely true but is only approximate.

Today dark energy is the dominant component of the universe’s energy. This is all the more remarkable because the amount of dark energy density turns out to be extraordinarily small. Dark energy has dominated only for the last few billion years. Earlier in the universe’s evolution, first radiation and then matter were dominant. But radiation and matter, which are shared over the volume of an ever-increasing universe, dilute. Dark energy density, on the other hand, remained constant, even when the universe grew. By the time the universe had lasted so long as it has, the energy density in radiation and matter had decreased so enormously that dark energy, which doesn’t dissipate, eventually took over. Despite dark energy’s incredibly tiny size, it was bound to eventually dominate. After 10 billion years of expanding at an increasingly slower rate, the impact of dark energy was finally felt and the universe sped up its expansion. Eventually, the universe will end up with nothing in it but vacuum energy and its expansion will accelerate accordingly. (See Figure 77.) The meek energy might not inherit the Earth, but it is in the process of inheriting the universe.


FIGURE 77 ] The universe has expanded differently over time. During the inflationary phase it quickly expanded exponentially. The conventional Big Bang expansion took over when inflation ended. Dark energy now makes the expansion rate accelerate again.


The necessity for dark energy and dark matter tells us that we can’t be as smug about our understanding of the evolution of the universe as the incredible agreement of cosmological theory with cosmological data might suggest. Most of the universe is stuff whose identity remains a mystery. Twenty years from now, people might smile at our ignorance.

And these are not the only puzzles evoked by the energy of the universe. The value of the dark energy, in particular, is actually the tail end of a much larger mystery: why is the energy that pervades the universe so small? Had the amount of dark energy been greater, it would have dominated matter and radiation much earlier in the evolution of the universe, and structure (and life) would not have had time to form. On top of that, no one knows what was responsible earlier on for the large energy density that triggered and fueled inflation. But the biggest problem with the energy of the universe is the cosmological constant problem.

Based on quantum mechanics, we would have expected a much larger value for dark energy—both during inflation and now. Quantum mechanics tells us that the vacuum—the state with no permanent particles present—is actually filled with ephemeral particles that pop in and out of existence. These short-lived particles can have any energy. They sometimes can have energy so large that gravitational effects can no longer be neglected. These highly energetic particles contribute an extremely large energy to the vacuum—much larger than the long evolution of our universe would allow. In order for the universe to look like the one we see, the value of the vacuum energy has to be an astonishing 120 orders of magnitude smaller than the energy that quantum mechanics would lead us to expect.

And there is yet a further challenge associated with this problem. Why do we happen to live in the time when the energy densities of matter, dark matter, and dark energy are comparable? Certainly dark energy dominates over matter, but it’s by less than a factor of three. Given that these energies in principle have entirely different origins and any one of them could have overwhelmed the others, the fact that their densities are so close seems very mysterious. The peculiarity of this coincidence is especially notable because it is only (roughly speaking) in our time that this coincidence is true. Earlier in the universe, dark energy was a much smaller fraction of the whole. And later on it will be a much greater fraction. Only today are the three components—ordinary matter, dark matter, and dark energy—comparable.

The questions of why the energy density is so extraordinarily tiny and why these different energy sources contribute similar amounts today are entirely unsolved. In fact, some physicists believe that there is no true explanation. They think we live in a universe with such an incredibly unlikely value for the vacuum energy because any larger value would have prevented the formation of galaxies and structure—and us—in the universe. We wouldn’t be here to ask about the value of the energy in any universe with a somewhat larger value of the cosmological constant. Those physicists believe that there are many universes, and each of these universes contains a different value of the dark energy. Out of the many possible universes, only the ones that could give rise to structure could possibly contain us. The value of the energy in this universe is ridiculously small, but we could exist only in a universe with just such a small value. This reasoning is the anthropic principle we considered in Chapter 18. As I said then, I’m not convinced. Nonetheless, neither I nor anyone else has a better answer. The explanation for the value of the dark energy is perhaps the most major mystery particle physicists and cosmologists face today.

In addition to puzzles about energy, we also have a further cosmological mystery about matter: Why is there matter in the universe at all? Our equations treat matter and antimatter on the same footing. They annihilate when they find each other, and both disappear. Neither matter nor antimatter should remain when the universe has cooled.

Whereas dark matter doesn’t interact very much and therefore sticks around, ordinary matter interacts quite a lot through the strong nuclear force. Without an exotic addition to the Standard Model, almost all of our usual matter would have disappeared by the time the universe had cooled to its current temperature. The only reason matter can be left is that there is a predominance of matter over antimatter. This isn’t built into the simplest versions of our theories. We need to find reasons that protons exist but can’t find antiprotons with which they can annihilate. Somewhere a matter-antimatter asymmetry must be built in.

The amount of leftover matter is smaller than the amount of dark matter, but it is still a sizable chunk of the universe—not to mention the source of everything we know and love. How and when this matter-antimatter asymmetry was created is another big question that particle physicists and cosmologists very much want to tackle.

The question of what constitutes the dark matter of course remains critical as well. Perhaps eventually we will find that the underlying model connects the dark matter density to that of matter, as recent research suggests. In any case, we hope to soon learn a lot more about the dark matter question from experiments—a sampling of which we’ll now explore.