Knocking on Heaven's Door: How Physics and Scientific Thinking Illuminate the Universe and the Modern World - Lisa Randall (2011)



Among the many reasons I chose to pursue physics was the desire to do something that would have a permanent impact. If I was going to invest so much time, energy, and commitment, I wanted it to be for something with a claim to longevity and truth. Like most people, I thought of scientific advances as ideas that stand the test of time.

My friend Anna Christina Büchmann studied English in college while I majored in physics. Ironically, she studied literature for the same reason that drew me to math and science. She loved the way an insightful story lasts for centuries. When discussing Henry Fielding’s novel Tom Jones with her many years later, I learned that the edition I had read and thoroughly enjoyed was the one she helped annotate when she was in graduate school.4

Tom Jones was published 250 years ago, yet its themes and wit resonate to this day. During my first visit to Japan, I read the far older Tale of Genji and marveled at its characters’ immediacy too, despite the thousand years that have elapsed since Murasaki Shikibu wrote about them. Homer created the Odyssey roughly 2,000 years earlier. Yet notwithstanding its very different age and context, we continue to relish the tale of Odysseus’s journey and its timeless descriptions of human nature.

Scientists rarely read such old—let alone ancient—scientific texts. We usually leave that to historians and literary critics. We nonetheless apply the knowledge that has been acquired over time, whether from Newton in the seventeenth century or Copernicus more than 100 years earlier still. We might neglect the books themselves, but we are careful to preserve the important ideas they may contain.

Science certainly is not the static statement of universal laws we all hear about in elementary school. Nor is it a set of arbitrary rules. Science is an evolving body of knowledge. Many of the ideas we are currently investigating will prove to be wrong or incomplete. Scientific descriptions certainly change as we cross the boundaries that circumscribe what we know and venture into more remote territory where we can glimpse hints of the deeper truths beyond.

The paradox scientists have to contend with is that while aiming for permanence, we often investigate ideas that experimental data or better understanding will force us to modify or discard. The sound core of knowledge that has been tested and relied on is always surrounded by an amorphous boundary of uncertainties that are the domain of current research. The ideas and suggestions that excite us today will soon be forgotten if they are invalidated by more persuasive or comprehensive experimental work tomorrow.

When the 2008 Republican presidential candidate Mike Huckabee sided with religion over science—in part because scientific “beliefs” change whereas Christians take as their authority an eternal, unchanging God—he was not entirely misguided, at least in his characterization. The universe evolves and so does our scientific knowledge of it. Over time, scientists peel away layers of reality to expose what lies beneath the surface. We broaden and enrich our understanding as we probe increasingly remote scales. Knowledge advances and the unexplored region recedes when we reach these difficult-to-access distances. Scientific “beliefs” then evolve in accordance with our expanded knowledge.

Nonetheless, even when improved technology makes a broader range of observations possible, we don’t necessarily just abandon the theories that made successful predictions for the distances and energies, or speeds and densities, that were accessible in the past. Scientific theories grow and expand to absorb increased knowledge, while retaining the reliable parts of ideas that came before. Science thereby incorporates old established knowledge into the more comprehensive picture that emerges from a broader range of experimental and theoretical observations. Such changes don’t necessarily mean the old rules are wrong, but they can mean, for example, that those rules no longer apply on smaller scales where new components have been revealed. Knowledge can thereby embrace old ideas yet expand over time, even though very likely more will always remain to be explored. Just as travel can be compelling—even if you will never visit every place on the planet (never mind the cosmos)—increasing our understanding of matter and of the universe enriches our existence. The remaining unknowns serve to inspire further investigations.

My own research field of particle physics investigates increasingly smaller distances in order to study successively tinier components of matter. Current experimental and theoretical research attempt to expose what matter conceals—that which is embedded ever deeper inside. But despite the often-heard analogy, matter is not simply like a Russian matryoshka doll, with similar elements replicated at successively smaller scales. What makes investigating increasingly minuscule distances interesting is that the rules can change as we reach new domains. New forces and interactions might appear at those scales whose impact was too tiny to detect at the larger distances previously investigated.

The notion of scale, which tells physicists the range of sizes or energies that are relevant for any particular investigation, is critical to the understanding of scientific progress—as well as to many other aspects of the world around us. By partitioning the universe into different comprehensible sizes, we learn that the laws of physics that work best aren’t necessarily the same for all processes. We have to relate concepts that apply better on one scale to those more useful at another. Categorizing in this way lets us incorporate everything we know into a consistent picture while allowing for radical changes in descriptions at different lengths.

In this chapter, we’ll see how partitioning by scale—whichever scale is relevant—helps clarify our thinking—both scientific and otherwise—and why the subtle properties of the building blocks of matter are so hard to notice at the distances we encounter in our everyday lives. In doing so, this chapter also elaborates on the meaning of “right” and “wrong” in science, and why even apparently radical discoveries don’t necessarily force dramatic changes on the scales with which we are already familiar.


People too often confuse evolving scientific knowledge with no knowledge at all and mistake a situation in which we are discovering new physical laws with a total absence of reliable rules. A conversation with the screenwriter Scott Derrickson during a recent visit to California helped me to crystallize the origin of some of these misunderstandings. At the time, Scott was working on a couple of movie scripts that proposed potential connections between science and phenomena that he suspected scientists would probably dismiss as supernatural. Eager to avoid major solecisms, Scott wanted to do scientific justice to his imaginative story ideas by having them scrutinized by a physicist—namely me. So we met for lunch at an outdoor café in order to share our thoughts along with the pleasures of a sunny Los Angeles afternoon.

Knowing that screenwriters often misrepresent science, Scott wanted his particular ghost and time-travel stories to be written with a reasonable amount of scientific credibility. The particular challenge that he as a screenwriter faced was his need to present his audience not just with interesting new phenomena, but also with ones that would translate effectively to a movie screen. Although not trained in science, Scott was quick and receptive to new ideas. So I explained to him why, despite the ingenuity and entertainment value of some of his story lines, the constraints of physics made them scientifically untenable.

Scott responded that scientists have often thought certain phenomena impossible that later turned out to be true. “Didn’t scientists formerly disbelieve what relativity now tells us?” “Who would have thought randomness played any role in fundamental physical laws?” Despite his great respect for science, Scott still wondered if—given its evolving nature—scientists aren’t sometimes wrong about the implications and limitations of their discoveries.

Some critics go even further, asserting that although scientists can predict a great deal, the reliability of those predictions is invariably suspect. Skeptics insist, notwithstanding scientific evidence, that there could always be a catch or a loophole. Perhaps people could come back from the dead or at the very least enter a portal into the Middle Ages or into Middle-earth. These doubters simply don’t trust the claims of science that a thing is definitively impossible.

However, despite the wisdom of keeping an open mind and recognizing that new discoveries await, a deep fallacy is buried in this logic. The problem becomes clear when we dissect the meaning of such statements as those above and, in particular, apply the notion of scale. These questions ignore the fact that although there will always exist unexplored distance or energy ranges where the laws of physics might change, we know the laws of physics on human scales extremely well. We have had ample opportunity to test these laws over the centuries.

When I met the choreographer Elizabeth Streb at the Whitney Museum, where we both spoke on a panel on the topic of creativity, she too underestimated the robustness of scientific knowledge on human scales. Elizabeth posed a similar question to those Scott had asked: “Could the tiny dimensions proposed by physicists and curled up to an unimaginably small size nonetheless affect the motion of our bodies?”

Her work is wonderful, and her inquiries into the basic assumptions about dance and movement are fascinating. But the reason we cannot determine whether new dimensions exist, or what their role would be even if they did, is that they are too small or too warped for us to be able to detect. By that I mean that we haven’t yet identified their influence on any quantity that we have so far observed, even with extremely detailed measurements. Only if the consequences of extra dimensions for physical phenomena were vastly bigger could they discernibly influence anyone’s motion. And if they did have such a significant impact, we would already have observed their effects. We therefore know that the fundamentals of choreography won’t change even when our understanding of quantum gravity improves. Its effects are far too suppressed relative to anything perceptible on a human scale.

When scientists have turned out to be wrong in the past, it was often because they hadn’t yet explored very tiny or very large distances or extremely high energies or speeds. That didn’t mean that, like Luddites, they had closed their minds to the possibility of progress. It meant only that they trusted their most up-to-date mathematical descriptions of the world and their successful predictions of then-observable objects and behaviors. Phenomena they thought were impossible could and sometimes did occur at distances or speeds these scientists had never before experienced—or tested. But of course they couldn’t yet have known about new ideas and theories that would ultimately prevail in the regimes of those tiny distances or enormous energies with which they were not yet familiar.

When scientists say we know something, we mean only that we have certain ideas and theories whose predictions have been well tested over a certain range of distances or energies. These ideas and theories are not necessarily the eternal laws for the ages or the most fundamental of physical laws. They are rules that apply as well as any experiment could possibly test, over the range of parameters available to current technology. This doesn’t mean that these laws will never be overtaken by new ones. Newton’s laws are instrumental and correct, but they cease to apply at or near the speed of light where Einstein’s theory applies. Newton’s laws are at the same time both correct and incomplete. They apply over a limited domain.

The more advanced knowledge that we gain through better measurements really is an improvement that illuminates new and different underlying concepts. We now know about many phenomena that the ancients could not have derived or discovered with their more limited observational techniques. So Scott was right that sometimes scientists have been wrong—thinking phenomena impossible that in the end turned out to be perfectly true. But this doesn’t mean there are no rules. Ghosts and time-travelers won’t appear in our houses, and alien creatures won’t suddenly emerge from our walls. Extra dimensions of space might exist, but they would have to be tiny or warped or otherwise currently hidden from view in order for us to explain why they have not yet yielded any noticeable evidence of their existence.

Exotic phenomena might indeed occur. But such phenomena will happen only at difficult-to-observe scales that are increasingly far from our intuitive understanding and our usual perceptions. If they will always remain inaccessible, they are not so interesting to scientists. And they are less interesting to fiction writers too if they won’t have any observable impact on our daily lives.

Weird things are possible, but the ones non-physicists are understandably most interested in are the ones we can observe. As Steven Spielberg pointed out in a discussion about a science fiction movie he was considering, a strange world that can’t be presented on a movie screen—and which the characters in a film would never experience—is not so interesting to a viewer. (Figure 1 shows amusing evidence.) Only a new world that we can access and be aware of could be. Even though both require imagination, abstract ideas and fiction are different and have different goals. Scientific ideas might apply to regimes that are too remote to be of interest to a film, or to our daily observations, but they are nonetheless essential to our description of the physical world.


FIGURE 1 ] An XKCD comic that captures the hidden nature of tiny rolled-up dimensions.


Despite this neat separation by distances, people too often take shortcuts when trying to understand difficult science and the world. And that can easily lead to an overzealous application of theories. Such misapplication of science is not a new phenomenon. In the eighteenth century, when scientists were busy studying magnetism in laboratories, others conjured up the notion of “animal magnetism”—a hypothesized magnetic “vital fluid” in animate beings. It took a French royal commission set up by Louis XVI in 1784, which included Benjamin Franklin among others, to formally debunk the hypothesis.

Today such misguided extrapolations are more likely to be made about quantum mechanics—as people try to apply it on macroscopic scales where its consequences usually average away and leave no measurable signatures.5 It’s disturbing how many people trust ideas such as those in Rhonda Byrne’s bestselling book The Secret, about how positive thoughts attract wealth, health, and happiness. Equally disquieting is Byrne’s claim that “I never studied science or physics at school, and yet when I read complex books on quantum physics I understood them perfectly because I wanted to understand them. The study of quantum physics helped me to have a deeper understanding of The Secret, on an energetic level.”

As even the Nobel Prize—winning pioneer of quantum mechanics Niels Bohr noted, “If you are not completely confused by quantum mechanics, you do not understand it.” Here’s another secret (at least as well protected as those in a bestselling book): quantum mechanics is notoriously misunderstood. Our language and intuition derive from classical reasoning, which doesn’t take quantum mechanics into account. But this doesn’t mean that any bizarre phenomenon is possible with quantum logic. Even without a more fundamental, deeper understanding, we know how to use quantum mechanics to make predictions. Quantum mechanics will certainly never account for Byrne’s “secret” about the so-called principle of attraction between people and distant things or phenomena. At those large distances, quantum mechanics doesn’t play this kind of role. Quantum mechanics has nothing to do with many of the tantalizing ideas people often attribute to it. I cannot affect an experiment by staring at it, quantum mechanics does not mean there are no reliable predictions, and most measurements are constrained by practical limitations and not by the uncertainty principle.

Such fallacies were the chief topic in a surprising conversation I had with Mark Vicente, the director of the movie What the Bleep Do We Know!?—a ?lm that is the bane of scientists—in which people claim that human influence matters for experiments. I wasn’t sure where this conversation would lead, but I had time to spare since I was sitting on the tarmac at the Dallas/Fort Worth airport for several hours waiting for mechanics to repair a dent in the wing (which first was described as too small to matter—but then was “measured by technology” before the plane could depart, as one crewmember helpfully informed us).

Even with the delay, I realized if I was going to talk to Mark at any length, I had to know where he stood on his film—which I was familiar with from the numerous people at my lectures who asked me off-the-wall questions based on what they had seen in it. Mark’s answer caught me by surprise. He had made a rather striking about-face. He confided that he had initially approached science with preconceived notions that he didn’t sufficiently question, but that he now viewed his previous thinking as more religious in nature. Mark ultimately concluded that what he had presented in his film was not science. Placing quantum mechanical phenomena at a human level was perhaps superficially satisfying to many of his film’s viewers, but that didn’t make it right.

Even if new theories require radically different assumptions—as was certainly the case with quantum mechanics—valid scientific arguments and experiments ultimately determined that they were true. It wasn’t magic. The scientific method, along with data and searches for economy and consistency, had told scientists how to extend their knowledge beyond what is intuitive at immediately accessible scales to very different ideas that apply to phenomena that are not.

The next section tells more about how the notion of scale systematically bridges different theoretical concepts and allows us to incorporate them into a coherent whole.


Our size happens to fall pretty much randomly close to the middle in terms of powers of ten when placed on a scale between the smallest imaginable length and the enormity of the universe.6 We are very big compared to the internal structure of matter and its minuscule components, while we are extremely tiny compared to stars, galaxies, and the universe’s expanse. The sizes that we most readily comprehend are simply the ones that are most accessible to us—through our five senses and through the most rudimentary measuring tools. We understand more distant scales through observations combined with logical deductions. The range of sizes might seem to involve increasingly abstract and hard-to-keep-track-of quantities as we move further from directly visible and accessible scales. But technology combined with theory allows us to establish the nature of matter over a vast stretch of lengths.

Known scientific theories apply over this huge range—spanning distances as small as the tiny objects explored by the Large Hadron Collider (LHC) up to the enormous length scales of galaxies and the cosmos. And for each possible size of objects or distance between them, different aspects of the laws of physics can become relevant. Physicists have to cope with the abundance of information that applies over this enormous span. Although the most basic laws of physics that apply to tiny lengths are ultimately responsible for those that are relevant to larger scales, they are not necessarily the most efficient means of making a calculation. When the extra substructure or underpinnings are irrelevant to a sufficiently precise answer, we want a more practical way to calculate and efficiently apply simpler rules.

One of the most important features of physics is that it tells us how to identify the range of scales relevant to any measurement or prediction—according to the precision we have at our disposal—and then calculate accordingly. The beauty of this way of looking at the world is that we can focus on the scales that are relevant to whatever we are interested in, identify the elements that operate at those scales, and discover and apply the rules that govern how these components relate. Scientists average over or even ignore (sometimes unwittingly) physical processes that occur on immeasurably small scales when formulating theories or setting up calculations. We select relevant facts and suppress details when we can get away with it and focus on the most useful scales. Doing so is the only way to cope with an impossibly dense set of information.

When appropriate, it makes sense to ignore minutiae in order to focus on the topic of interest and not to obscure it with inessential details. A recent lecture by the Harvard psychology professor Stephen Kosslyn reminded me how scientists—and everyone else—prefer to keep track of information. In a cognitive science experiment that he performed on the audience, he asked us all to keep track of line segments he presented on a screen one after the other. Each of the segments could go “north” or “southeast,” and so on, and together they formed a zigzagging line. (See Figure 2.) We were asked to close our eyes and say what we had seen. We noticed that even though our brains allow us to keep track of only a few individual segments at a time, we could remember longer sequences by grouping them into repeatable shapes. By thinking on the scale of the shape rather than the individual line segment, we could keep the figure in our heads.


FIGURE 2 ] You might choose as your component the individual line segment or a larger unit, such as the group of six segments that appears twice.

For almost anything you see, hear, taste, smell, or touch, you have a choice between examining details by looking very closely or examining the “big picture” with its other priorities. Whether staring at a painting, tasting wine, reading philosophy, or planning your next trip, you automatically parcel your thoughts into the categories of interest—be they sizes or flavor categories, ideas, or distances—and the categories that you don’t find relevant at the time.

The utility of focusing on the pertinent questions and ignoring structures too small to be relevant applies in many contexts. Think about what you do when you use MapQuest or Google maps or look at the small screen on your iPhone. If you were traveling from far away, you would first get some rough idea where your destination is. Subsequently, when you have the big picture, you would zoom into a map with more resolution. You don’t need the additional detailed information in your first pass. You just want to have some sense of location. But as you begin to map out the details of your journey—as your resolution becomes finer in seeking out the exact street you will need—you will care about the details on the finer scale that were inessential to your first exploration.

Of course, the degree of precision you want or need determines the scale you choose. I have friends who don’t pay much attention to hotel location when visiting New York City. For them, the gradations in character of the city’s blocks is irrelevant. But for anyone who knows New York, those details matter. It’s not enough to know you are staying downtown. New Yorkers care if they are above or below Houston Street, or east or west of Washington Square Park, or even whether they are two or five blocks away.

Although the precise choice of scale might differ among individuals, no one would display a map of the United States in order to find a restaurant. The necessary details won’t be resolvable on a computer screen displaying such an overly large scale. On the other hand, you don’t need the details of a floor plan just to know that the restaurant is there in the first place. For any question you ask, you choose the relevant scale. (See Figure 3 for another example.)

The Eiffel Tower


FIGURE 3 ] Different information becomes more obvious when viewed at different scales.

In a similar manner, we categorize by size in physics so we can focus on the questions of interest. Our tabletop looks solid—and for many purposes we can treat it as such—but in reality it is made up of atoms and molecules that collectively act like the hard impenetrable surface we encounter at the scales we experience in our daily lives. Those atoms aren’t indivisible, either. They are composed of nuclei and electrons. And the nuclei are made of protons and neutrons that are in turn bound states of more fundamental objects called quarks. Yet we don’t need to know about quarks to understand the electromagnetic and chemical properties of atoms and elements (the field of science known as atomic physics). People studied atomic physics for years before there was even a clue about the substructure beneath. And when biologists study a cell, they don’t need to know about quarks inside the proton either.

I remember feeling a tad betrayed when my high school teacher, after devoting months to Newton’s Laws, told the class those laws were wrong. But my teacher was not quite right in his statement. Newton’s laws of motion work at the distances and speeds that were observable in his time. Newton thought about physical laws that applied, given the accuracy with which he (or anyone else in his era) could make measurements. He didn’t need the details of general relativity to make successful predictions about what could be measured then. And neither do we when we make the sorts of predictions relevant to large bodies at relatively low speeds and densities that Newton’s Laws apply to. When physicists or engineers today study planetary orbits, they also don’t need to know the detailed composition of the Sun. The laws that govern the behavior of quarks don’t noticeably affect the predictions relevant to celestial bodies either.

Understanding the most basic components is rarely the most efficient way to understand the interactions at larger scales, where tiny substructure generally plays very little role. We would be hard pressed to make progress in atomic physics by studying the even tinier quarks. It is only when we need to know more detailed properties of nuclei that the quark substructure becomes relevant. In the absence of unfathomable precision, we can safely do chemistry and molecular biology while ignoring any internal substructure in a nucleus. Elizabeth Streb’s dance movements won’t change no matter what happens at the quantum gravity scale. Choreography relies only on classical physical laws.

Everyone, including physicists, is happy to use a simpler description when the details are beyond our resolution. Physicists formalize this intuition and organize categories in terms of the distance or energy that is relevant. For any given problem, we use what we call an effective theory. The effective theory concentrates on the particles and forces that have “effects” at the distances in question. Rather than delineating particles and interactions in terms of unmeasurable parameters that describe more fundamental behavior, we formulate our theories, equations, and observations in terms of the things that are actually relevant to the scales we might detect.

The effective theory we apply at larger distances doesn’t go into the details of an underlying physical theory that applies to shorter distance scales. It asks only about things you could hope to measure or see. If something is beyond the resolution of the scales at which you are working, you don’t need its detailed structure. This practice is not scientific fraud. It is a way of disregarding the clutter of superfluous information. It is an “effective” way to obtain accurate answers efficiently and keep track of what is in your system.

The reason effective theories work is that it is safe to ignore the unknown, as long as it won’t make any measurable differences. If the only unknown phenomena occur at scales, distances, or resolutions where the influence is still indiscernible, we don’t need to know about them to make successful predictions. Phenomena beyond our current technical reach, by definition, won’t have any measurable consequences aside from those that are already taken into account.

This is why, even without knowing about phenomena as substantial as the existence of relativistic laws of motion or a quantum mechanical description of atomic and subatomic systems, people could still make accurate predictions. This is fortunate, since we simply can’t think about everything at once. We’d never get anywhere if we couldn’t suppress irrelevant details. When we concentrate on questions we can experimentally test, our finite resolution makes this jumble of information on all scales inessential.

“Impossible” things can happen—but only in environments that we have not yet observed. Their consequences are irrelevant at scales we know—or at least those scales we have so far explored. What is happening at these small distances remains hidden until higher-resolution tools are developed to look directly or until sufficiently precise measurements differentiate and identify the underlying theory through the minuscule distinguishing features it provides at larger distances.

Scientists can legitimately ignore anything too small to be observed when we make predictions. Not only is it impossible to distinguish among the consequences of overly tiny objects and processes, but the physical effects of processes at these scales are interesting only insofar as they determine the physically measurable parameters. Physicists therefore characterize the objects and properties on measurable scales in an effective theory and use these to do science relevant to the scales at hand. When you do know the short-distance details, or the microstructure of a theory, you can derive the quantities in the effective description from more fundamental detailed structure. Otherwise these quantities are just unknowns to be experimentally determined. The observable larger-scale quantities in the effective theory are not giving the fundamental description, but they are a convenient way of organizing observations and predictions.

An effective description can summarize the consequences of any shorter-distance theory that reproduces larger-scale observations but whose direct effects are too tiny to see. This has the advantage of letting us study and evaluate processes using fewer parameters than we would need if we took every detail into account. This smaller set is completely sufficient to characterize the processes that interest us. Furthermore, the set of parameters we use are universal—they are the same independently of the more detailed underlying physical processes. To know their values we just have to measure them in any of the many processes in which they apply.

Over a large range of lengths and energies, a single effective theory applies. After its few parameters have been determined by measurements, everything appropriate to this range of scales can be calculated. It gives a set of elements and rules that can explain a large number of observations. At any given time, the theory we think of as fundamental is likely to turn out to be an effective theory—since we never have infinitely precise resolution. Yet we trust the effective theory because it successfully predicts many phenomena that apply over a range of length and energy scales.

Effective theories in physics not only keep track of short distance information—they can also summarize large distance effects whose consequences might also be too minute to observe. For example, the universe we live in is very slightly curved—in a way that Einstein taught us was possible when he developed his theory of gravity. This curvature applies to larger scales involving the large-scale structure of space. Yet we can systematically understand why such curvature effects are too small to matter for most of the observations and experiments that we perform locally, on much smaller scales. Only when we include gravity in our particle physics description do we need to consider such effects—which are too tiny to matter for much of what I will describe. In that case too, the appropriate effective theory tells us how to summarize gravity’s effects in a few unknown parameters to be experimentally determined.

One of the most important aspects of an effective theory is that while it describes what we can see, it also categorizes what is missing—be it small scale or large. With any effective theory, we can determine how big an effect the unknown (or known) underlying dynamics could possibly have on any particular measurement. Even in advance of new discoveries at different scales, we can mathematically determine the maximal size of the influence any new structure can have on the effective theory at the scale at which we are working. As we will explore further in Chapter 12, it is only when the underlying physics is discovered that anyone fully understands the effective theory’s true limitations.

One familiar example of an effective theory might be thermodynamics, which tells us how refrigerators or engines work and was developed long before atomic or quantum theory. The thermodynamic state of a system is well characterized by its pressure, temperature, and volume. Though we know that fundamentally the system consists of a gas of atoms and molecules—with much more detailed structure than the preceding three quantities can possibly describe—for many purposes we can concentrate on these three quantities to characterize the system’s readily observable behavior.

Temperature, pressure, and volume are real quantities that can be measured. The theory behind their relationships is fully developed and can be used to make successful predictions. The effective theory of a gas makes no mention of the underlying molecular structure. (See Figure 4.) The behavior of those underlying elements determines temperature and pressure, but scientists happily used these quantities to do calculations even before atoms or molecules were discovered.


FIGURE 4 ] Pressure and temperature can be understood at a more fundamental level in terms of the physical properties of individual molecules.

Once the fundamental theory is understood, we can relate temperature and pressure to properties of the underlying atoms and also understand when the thermodynamic description should break down. But we can still use thermodynamics for a wide variety of predictions. In fact, many phenomena are only understood from a thermodynamic point of view, since without huge computing power and memory, well beyond what exists, we can’t track the paths of all the individual atoms. The effective theory is the only way at this point to understand some important physical phenomena that are pertinent to solid and liquid condensed matter.

This example teaches us another critical aspect of effective theories. We sometimes treat “fundamental” as a relative term. From the perspective of thermodynamics, the atomic and molecular description is fundamental. But from a particle physics description that details the quarks and electrons inside the atoms, the atom is composite—made up of smaller elements. Its use from a particle physics perspective is as an effective theory.

This description of the clean developmental progression in science from the well understood to regimes at the frontier of knowledge applies best to fields such as physics and cosmology, where we have a clear understanding of the functional units and their relationships. Effective theories won’t necessarily work for newer fields such as systems biology, where the relationships between activities at the molecular and more macroscopic levels, as well as the relevant feedback mechanisms, are yet to be fully understood.

Nonetheless, the effective theory idea applies in a broad range of scientific contexts. The mathematical equations that govern the evolution of species won’t change in response to new physics results, as I discussed with the mathematical biologist Martin Nowak in response to a question he had asked. He and his colleagues can characterize the parameters independently of any more fundamental description. They might ultimately relate to more basic quantities—physical or otherwise—but that doesn’t change the equations that mathematical biologists use to evolve the behavior of populations over time.

For particle physicists, effective theories are essential. We isolate simple systems at different scales and relate them to each other. In fact, the very invisibility of underlying structure that allows us to focus on observable scales and ignore more fundamental effects keep underlying interactions so well hidden that only with tremendous resources and effort can we ferret them out. The tininess of effects of more fundamental theories on observable scales is the reason that physics today is so challenging. We need to directly explore smaller scales or make increasingly precise measurements if we are to perceive the effects of the more fundamental nature of matter and its interactions. Only with advanced technology can we access very tiny or extremely vast length scales. That is why we need to conduct elaborate experiments—such as those at the Large Hadron Collider—to make advances today.


The story of theories of light nicely exemplifies the ways in which effective theories are used as science evolves, with some ideas being discarded while others are retained as approximations appropriate to their specified domains. From the time of the ancient Greeks, people studied light with geometrical optics. It is one of the topics any aspiring physics graduate student is tested on when taking the physics GRE (the exam that is a prerequisite for graduate school). This theory assumes that light travels in rays or lines and tells you how those rays behave as they travel through different media, as well as how instruments use and detect them.

The strange thing is that virtually no one—at least no one at Harvard where I now teach and was once a student—actually studies classical and geometrical optics. Maybe geometrical optics is taught a little bit in high school, but it is certainly no big part of the curriculum.

Geometrical optics is an old-fashioned subject. It hit its heyday several centuries ago with Newton’s famous Opticks, continuing into the 1800s when William Rowan Hamilton made what is perhaps the first real mathematical prediction of a new phenomenon.

The classical theory of optics still applies to areas such as photography, medicine, engineering, and astronomy, and is used to develop new mirrors, telescopes, and microscopes. Classical optical scientists and engineers work out different examples of various physical phenomena. However, they are simply applying optics—not discovering new laws.

In 2009, I was honored to be asked to give the Hamilton lecture at the University of Dublin—a lecture several of my most respected colleagues had given before me. It is named after Sir William Rowan Hamilton, the remarkable nineteenth-century Irish mathematician and physicist. I confess that the name Hamilton is so universally present in physics that I foolishly didn’t initially make the connection with an actual person who was in fact Irish. But I was fascinated by the many areas of math and physics that Hamilton had revolutionized, including, among them, geometrical optics.

The celebration of Hamilton Day is really quite something. The day’s activities include a procession down the Royal Canal in Dublin where everyone stops at the Broom Bridge to watch the youngest member of the party write down the same equations on the bridge that Hamilton, in the excitement of discovery, had many years past carved into the bridge’s side. I visited the College Observatory of Dunsink where Hamilton lived and got to see the pulleys and wooden structure of a telescope from two centuries ago. Hamilton arrived there after his graduation from Trinity College in 1827, when he was made the chair of astronomy and Astronomer Royal of Ireland. Locals joke that despite Hamilton’s prodigious mathematical talent, he had no real knowledge of or interest in astronomy, and that despite his many theoretical advances, he might have set back observational astronomy in Ireland fifty years.

Hamilton Day nonetheless pays homage to this great theorist’s many accomplishments. These included advances in optics and dynamics, the invention of the mathematical theory of quaternions (a generalization of complex numbers), as well as definitive demonstrations of the predictive power of math and science. The development of quaternions was no small advance. Quaternions are important for vector calculus, which underlies the way we mathematically study all three-dimensional phenomena. They are also now used in computer graphics and hence in the entertainment industry and video games. Anyone with a PlayStation or Xbox can thank Hamilton for some of the fun.

Among his numerous and substantial contributions, Hamilton significantly advanced the field of optics. In 1832, he showed that light falling at a certain angle on a crystal that has two independent axes would be refracted to form a hollow cone of emergent rays. He thereby made predictions about internal and external conical refraction of light through a crystal. In a tremendous—and perhaps the first—triumph of mathematical science, this prediction was verified by Hamilton’s friend and colleague Humphrey Lloyd. It was a very big deal to see verified a mathematical prediction of a never-before-seen phenomenon and Hamilton was knighted for his achievement.

When I visited Dublin, the locals proudly described this mathematical breakthrough—worked out purely on the basis of geometrical optics. Galileo helped pioneer observational science and experiments, and Francis Bacon was an initial advocate of inductive science—where one predicts what will happen based on what came before. But in terms of using math to describe a never-before-seen phenomenon, Hamilton’s prediction of conical refraction was probably the first. For this reason, at the very least, Hamilton’s contribution to the history of science is not to be ignored.

Nonetheless, despite the significance of Hamilton’s discovery, classical geometrical optics is no longer a research subject. All the important phenomena were worked out long ago. Soon after Hamilton’s time, in the 1860s, the Scottish scientist James Clerk Maxwell, among others, developed the electromagnetic description of light. Geometrical optics, though clearly an approximation, is nonetheless a good description for a wave with wavelength small enough for interference effects to be irrelevant, and for the light to be treated as a linear ray. In other words, geometrical optics is an effective theory, valid in a limited regime.

That doesn’t mean we keep every idea that has ever been developed. Sometimes ideas are just proved wrong. Euclid’s initial description of light, resurrected in the Islamic world in the ninth century by Al-Kindi, which claimed that light was emitted by our eyes, was one such example. Although others, such as the Persian mathematician Ibn Sahl, correctly described phenomena such as refraction based on this false premise, Euclid and Al-Kindi’s theory—which predates science and modern scientific methods—was simply incorrect. It wasn’t absorbed into future theories. It was simply abandoned.

Newton didn’t anticipate a different aspect of the theory of light. He had developed a “corpuscular” theory that was inconsistent with the wave theory of light developed by his rival Robert Hooke in 1664 and Christian Huygens in 1690. The debate between them lasted a long time. In the nineteenth century, Thomas Young and Augustin-Jean Fresnel measured light interference, providing a clear verification that light had the properties of a wave.

Later developments in quantum theory demonstrated that Newton was correct in some sense too. Quantum mechanics now tells us that light is indeed composed of individual particles called photons that are responsible for communicating the electromagnetic force. But the modern theory of photons is based on light quanta, the individual particles of which light is made, that have a remarkable property. Even an individual particle of light, a photon, acts like a wave. That wave gives the probability of a single photon being found in any region of space. (See Figure 5.)

Geometrical Optics

Light travels in straight lines.


Wave Optics

Light travels in waves.



Light is transmitted by photons, particles that can act like waves.


FIGURE 5 ] Geometrical optics and waves were precursors to our modern understanding of light, and still apply under appropriate conditions.

Newton’s corpuscular theory reproduces results from optics. Nonetheless, Newton’s corpuscles, which don’t have any wavelike nature, are not the same as photons. So far as we now know, the theory of photons is the most basic and correct description of light, which consists of particles that can also accommodate a wave description. Quantum mechanics gives our currently most fundamental description of what light is and how it behaves. It is fundamentally correct and survives.

Quantum mechanics is now much more of a frontier research area than optics. If people continue to think about new science with optics, they are primarily thinking about new effects possible only with quantum mechanics. Modern science, though no longer advancing the science of classical optics, does therefore include a field of quantum optics, which studies the quantum mechanical properties of light. Lasers rely on quantum mechanics, as do light detectors such as photomultipliers, and photovoltaic cells that convert sunlight into electricity.

Modern particle physics also encompasses the theory of quantum electrodynamics (QED), which Richard Feynman and others developed and which includes not only quantum mechanics but also special relativity. With QED, we study individual particles including photons—particles of light—as well as electrons and other particles that carry electric charge. We can understand the rates at which such particles interact and at which they can be created and destroyed. QED is one of the theories that is heavily used in particle physics. It also has made the most accurately verified predictions in all of science. QED is a far cry from geometrical optics, yet both are true in their appropriate domain of validity.

Every area of physics reveals this effective theory idea at work. Science evolves as old ideas get incorporated into more fundamental theories. The old ideas still apply and can have practical applications. But they aren’t the domain of frontier research. Though the end of this chapter has focused on the particular example of the physical interpretation of light through the ages, all of physics has developed in this manner. Science proceeds with uncertainty at the edges, but it is advancing methodically overall. Effective theories at a given scale legitimately ignore effects that we can prove won’t make a difference for any particular measurement. The wisdom and methods we acquired in the past survive. But theories evolve as we better understand a larger range of distances and energies. Advances give us new insights into what fundamentally accounts for the phenomena we see.

Understanding this progression helps us better interpret the nature of science and appreciate some of the major questions that physicists (and others) are asking today. In the following chapter, we’ll see that in many respects, today’s methodology began in the seventeenth century.