Hello, Lambda - The Face of the Deep - The 4 Percent Universe: Dark Matter, Dark Energy, and the Race to Discover the Rest of Reality - Richard Panek

The 4 Percent Universe: Dark Matter, Dark Energy, and the Race to Discover the Rest of Reality - Richard Panek (2011)

Part III. The Face of the Deep

Chapter 8. Hello, Lambda

ON JANUARY 8, 1998, four astronomers sat at a table at the front of a conference room at the Washington Hilton to deliver the verdict of science. Ruth Daly was there with her radio-galaxy data, and Neta Bahcall was there with her galaxy-cluster data, and representatives from the two supernova teams were there—Peter Garnavich for the High-z collaboration, and Saul Perlmutter for the SCP. The press re-leases from the various institutions had done their job. A couple of dozen journalists filled the seats, including reporters from the New York Times and the Washington Post, and cameras on tripods lined the back of the room, their metal lamps throwing light and heat. The four astronomers represented four independent collaborations, but they spoke with one voice: The universe would expand forever.

One voice, however, was a little stronger than the rest. Perlmutter had flown to D.C. from an observing run in Hawaii. On the plane from Honolulu to San Francisco he had used a seatback phone for the first time, calling his colleagues in Berkeley and dictating the new data he'd collected in recent days at the Keck Telescope, atop Mauna Kea. Then he stopped in Berkeley just long enough to print out a poster incorporating that data. So far the SCP had made seven supernovae public, in a paper that had appeared in Nature a week earlier. But the team had more than forty other supernovae in the pipeline—a quantity that in itself was important. It communicated to the community that the system was working, and that the SCP had mastered it.

But for Perlmutter these results also represented the realization of his dream of using physics to solve the big mysteries. "For the first time," he announced at the AAS press conference, "we're going to actually have data, so that you will go to an experimentalist to find out what the cosmology of the universe is, not to a philosopher." Afterward, he had stayed at a table in the room for an hour, conducting a mini-seminar for the members of the press. They surrounded him, and he held forth. Later, when they played back their cassette tapes, they might think they'd inadvertently hit fast-forward. But no, it was just Saul Perlmutter at regular speed, hyperkinetically trying to convince them that the headline here wasn't just the fate of the universe. It was that we could now know that fate—empirically, scientifically.

The following day Michael Turner paid Perlmutter a visit in the exhibit hall. The SCP team was part of the AAS meeting's poster sessions that day—dozens of presentations tacked to freestanding corkboards in long lanes, hard by the trade-show booths where representatives from weapons manufacturers sat at white-linen-covered tables and explained why the telescopes on their drawing boards were the best. Perlmutter wanted to show Turner something in the data, something he hadn't mentioned at the press conference.

Turner liked Perlmutter, and he liked the project; he didn't need to be convinced that the supernova survey was a worthy effort that deserved support from telescope time allocation committees or the National Science Foundation or the Department of Energy. Turner bent close to the panels—eight in all. The first few explained supernova search methodology to the uninitiated. One showed the logistics of the project: the initial observations at the Cerro Tololo 4-meter telescope, the follow-up observations at Cerro Tololo three weeks later, the spectroscopy at Keck, the photometry with telescopes at Kitt Peak and Isaac Newton and, at the highest redshifts, with the Hubble Space Telescope. The second showed light curves from twenty-one of the team's supernovae, the third showed some spectra, the fourth redshifts. The fifth explained how the SCP had calculated the photometry, made some corrections, and applied the stretch method to convert the Type Ia supernovae into calibrated candles, and how that calibration allowed them to plot the supernovae on a redshift-magnitude Hubble diagram. The sixth panel showed the Hubble diagram from the Nature paper, the basis for the claim that the universe will expand forever. Nothing Turner didn't already know.

But then came the seventh panel. It showed two plots. They were contour plots—plots that take the cumulative statistical effect (rather than individual points) of all the data and plop them on a graph that covers every possible scenario for the life of the universe. If your contour of confidence falls over here, in this region, then you have a universe without a Big Bang. If it falls over there, in that region, then you have a Big Bang universe that expands forever, and if it falls a little bit lower, then you have a Big Bang universe that recollapses eventually.

The plot on the left was, like the diagram in the sixth panel, from the Nature paper. It reflected the statistical effects of six supernovae, including the 1997 supernova that the SCP team had examined with the Hubble Space Telescope. And it did indicate that the addition of the one HST supernova shifted the likelihood up, into the region corresponding to a universe that expands forever.

The plot on the right, however, was new. It reflected the statistical effects of the dozens of other supernovae the team had found as well—forty in all. As you might expect, the addition of all that data had tightened the contours, narrowing the confidence regions. Looking at the graph on the left and then at the graph on the right was like putting on glasses; suddenly the fuzzy outlines of the world—the universe—snapped into focus.

The analysis was preliminary. But the effect so far was arresting. If you knew what you were seeing, you would get it at a glance. Yes, the universe was going to expand forever. But the evidence seemed to be indicating that even in order to exist, the universe couldn't be made up simply of matter, dark or otherwise. It needed something else.

Turner straightened up. "Dave would have liked that," he said.

Turner was at the AAS to lead a memorial service for David Schramm. Already he had attended one in Aspen, and he would be leading another later that month at Rockefeller Chapel, on the University of Chicago campus. Here were the supernovae that Schramm had been pestering Turner about, but if the data held up, here too was a hint of a further tragedy. Schramm had spent decades trying to rethink the universe, only to die, like Hubble at Palomar, in sight of the Promised Land—though perhaps even more poignantly. In the half-century since first light, the telescope at Palomar hadn't delivered what Hubble had hoped it would: the two numbers that had kept his protégé Allan Sandage keening all the way into retirement. But if the SCP data held up, then science was entering an era that Schramm had always envisioned: a new cosmology.

In 1917, in considering the implications of general relativity, Einstein saw that the universe was inherently unstable. Just as Newton had invoked God to keep his version of the universe from collapsing, so Einstein added a symbol to his equations—arbitrarily, the Greek letter lambda, A. Whatever lambda was, it was counteracting gravity, because, in Einstein's idea of a stable universe, something had to be. It was the reason that a universe full of matter attracting other matter through gravity wasn't collapsing. After Hubble's discovery of evidence for the expansion, the universe didn't need lambda, and Einstein abandoned it. Unlike Newton's God, however, you couldn't altogether ignore it. Lambda was, after all, in the equation.

What you could do instead was set lambda to zero. That's what generations of observers and theorists had done. Sometimes they left the assumption implicit, simply failing to mention lambda. Often they stated the assumption explicitly: "Assume A = 0." For most observers and theorists, lambda was there and it wasn't there. It occupied a parallel existence, like a ghost in the attic.

Just because you didn't need it, however, didn't mean you couldn't invoke it, and from time to time theorists had done just that. In 1948, when Hermann Bondi and Thomas Gold and, separately, Fred Hoyle were trying to create a new model of the universe that didn't rely on an initial singularity of infinite density but still seemed to be expanding, they invoked what Bondi and Gold called the "hypothetical and much debated cosmological term." Like Einstein, they didn't know what it was, but they set it to non-zero because something had to be fueling the expansion. But then the validation of the Big Bang theory through the discovery of the cosmic background radiation eliminated the need for what had come to be called "the cosmological constant." Lambda didn't exactly die with the Steady State, but it fled the corpse, like a soul escaping.

It next took up residence in quasars, those mysterious sources of tremendous energy at mystifying distances. In 1967, a trio of Cornell theorists published a paper in the Astrophysical Journal examining, as the title said, "Quasi-Stellar Objects in Universes with Non-Zero Cosmological Constant." They were trying to resolve some possible inconsistencies in the behavior of quasars. But as the understanding of the evolution of quasars became clearer, the need for lambda again receded. Then in 1975 two prominent astronomers argued in Nature that studies of elliptical galaxies as standard candles indicated that "the most plausible cosmological models have a positive cosmological constant." A year later they wrote another paper explaining why elliptical galaxies don't make good standard candles, implicitly undermining their earlier argument.

Four times now, including Einstein, cosmologists had gone up into the attic, and four times they'd returned with the same report: It was just the wind.

Then came inflation. It solved problems, the flatness and horizon problems. It explained improbabilities, the homogeneity and isotropy of the universe on the largest scales. And while the participants of the "Very Early Universe" Nuffield workshop at Cambridge in the summer of 1982 didn't agree on a model for inflation, they did, crucially, agree that a model could exist, and in the weeks and months after the workshop they formed a consensus around one model, giving inflation a solid basis in mathematics. But most important for its long-term survival or eventual obsolescence, inflation came with a prediction: that the universe was flat. That the amount of matter in the universe was equal to the critical amount that would keep it from collapsing. That omega equaled 1.

The problem for the inflation theorists, however, was that the observers were consistently finding evidence that the amount of matter in the universe was only 20 percent of the critical amount—that omega equaled 0.2.

At the final session of the Nuffield workshop, the theoretical physicist Frank Wilczek summarized the conference proceedings, concluding with "A Shopping List of Questions." Among them was whether omega was equal to 1. "If not," he said, "we must give up on inflation." Simple subtraction led you to conclude that for omega to equal 1 while observers were finding evidence that omega equaled 0.2, observers must be missing 0.8, or 80 percent, of the universe.

This discrepancy wasn't as worrisome as one might imagine. Two options immediately presented themselves. Maybe the rest of the matter was in a form that astronomers hadn't yet detected. The community had only recently conceded that the evidence for dark matter was compelling, and theorists were still working through the implications of dark matter for the structure and evolution of the universe. Or maybe the observers were just wrong, and more precise observations with improved instruments would boost omega and resolve the discrepancy.

A third option also existed, and if it, too, beckoned immediately, it did so from a distance, or even a different dimension. In any case, it was easy and probably advisable to ignore. Wilczek ended the Nuffield workshop with the last question of his "shopping list":

What about the cosmological constant?
"Whereof one cannot speak, thereof one must be silent."


Be silent? Be loud! Michael Turner went home from the Nuffield workshop, downed a slice of Primordial Pizza, and, along with fellow theorists Gary Steigman and Lawrence M. Krauss, got to work on a paper that explored the options for making omega equal to 1, entitled "Flatness of the Universe: Reconciling Theoretical Prejudices with Observational Data." Those "theoretical prejudices" referred to inflation's prediction of a flat universe, and the paper explored two ways of reconciling those prejudices with the data. One was a particle of some sort from the era of Big Bang nucleosynthesis—the field that Schramm had pioneered. The other possibility was "a relic cosmological constant."

"The cosmological constant," Turner liked to say, "is the last refuge of scoundrel cosmologists, beginning with Einstein." He himself, in his "heart of hearts," thought the cosmological constant must be zero. But he also knew that the cosmological constant had "every right to be there." And as he and Rocky Kolb often insisted, their generation wasn't going to make the mistake that Einstein and other twentieth-century cosmologists had made by not taking every remotely serious option seriously.

If anything, the self-described scientific conservative Jim Peebles took the idea even more seriously than Turner—but then, Peebles prided himself on trusting observations more than most theorists. "What's best," he would say, shrugging, "is what's true." The truth for him had emerged in a 1983 paper he wrote with Marc Davis, a UC Berkeley astronomer, using the latest and largest survey of galaxies to measure their velocities, infer their masses, and derive the mass density of the universe. Peebles looked at their data and thought, "High mass density is dead in the water." Their conclusion: an omega of 0.2.

The following year, Peebles wrote a paper, "Tests of Cosmological Models Constrained by Inflation," that offered his theoretical interpretation of that data. Maybe omega was indeed 0.2 and lambda equaled 0, he wrote, but in that case "we lose the attractive inflationary explanation for the observed large-scale homogeneity of the universe." He didn't want a cosmological constant. "It's ugly," he often said. "It's an addition." If he were building a universe, he thought, he wouldn't put in a cosmological constant: "No bells and whistles." But perhaps because inflation solved the flatness problem he'd articulated with Dicke, or perhaps because he constitutionally distrusted a simple universe, he accepted the possibility with equanimity. "Considering the observations," he said, "I think the universe might have put in bells and whistles—a cosmological constant."

The paper met with a lot of resistance, which Peebles sort of enjoyed. He found that he could go to conferences and give a talk, and people would rant at him, and then they apparently forgot, because a few months later he would give the same talk and the same people would rant at him. He realized he didn't even have to write a new talk; he could just give the same one over and over. This went on for a dec ade.

Theorists are always saying something. That's their job. They don't need to believe what they're saying. The theorist's goal isn't to be right but to be reasonable—to make an internally consistent argument that observers can then go out and reinforce or disprove. For their part, observers regard theorists with patience and exasperation, like a dog that's always depositing gifts at their feet: a stick, a squeaky toy, a dead bird. Often these offerings just lie there. But once in a while the observers will throw them a bone. Go fetch.

In 1992 observers threw cosmological theorists the biggest bone since the discovery of the cosmic microwave background more than a quarter of a century earlier: the Cosmic Background Explorer results—the ones that said that the universe was flat. The following year, Turner and Kolb added a preface to the paperback edition of The Early Universe reviewing the COBE results and declaring them "a shot in the arm" for a flat universe.

As other observations accumulated that indicated a universe with a low density of matter—especially the kind of studies of galaxies on the largest scale that had first persuaded Jim Peebles a decade earlier—theorists found themselves increasingly less reluctant to suggest, and observers found themselves increasingly less reluctant to consider, the possibility of a cosmological constant. "why a cosmological constant seems inevitable" read a section heading in one influential paper; "The Observational Case for a Low-Density Universe with a Non-Zero Cosmological Constant" was the title of another paper. And then there was Turner again, again with Lawrence Krauss: "The Cosmological Constant Is Back." The cosmological constant was still the last refuge, but it was a refuge nonetheless.

Vera Rubin summarized the situation with a joke. There was a wise rabbi, she said, who was trying to mediate a marital dispute. The husband complained about the wife. "You're right," the rabbi said. The wife complained about the husband. "You're right," the rabbi said. Then the rabbi's own wife emerged from behind a curtain, where she had been eavesdropping. "How can you tell them both they're right?" she said to her husband. To which the wise rabbi replied, "You're right too."

She told this joke at a "Critical Dialogues in Cosmology" conference at Princeton, part of the university's celebration of its 250th anniversary, in the summer of 1996. The purpose of the conference was to bring together the world's leading cosmologists to address the field's greatest challenges. One such event, inevitably, involved the value of omega, and it took the form of a debate. On one side was Avishai Dekel, who had recently measured galaxy motions that were consistent with an omega equal to 1. On the other side was Turner, arguing that the amount of matter in the universe was not enough to nudge omega to 1. But he didn't stop there. Instead he used the forum to argue that omega was indeed equal to 1, because the cosmological constant would close the gap.

The moderator was none other than Bob Kirshner. At one point in the discussion he turned to Saul Perlmutter, who had arrived at Princeton bearing preliminary results from the SCP's first seven supernovae. Kirshner asked what he thought.

Like any cosmologists dealing with omega, the SCP had addressed lambda in paper after paper: "(for ? = 0)," "(for Λ = 0)," "If we assume that the cosmological constant = 0." A year earlier, in 1995, Perlmutter and Ariel Goobar had elevated the cosmological constant from its pro forma purgatory, making its existence the subject of a paper in the Astrophysical Journal. Or, rather, making its nonexistence the subject of the paper, since their assumption while writing it was that matter would indeed account for everything. They figured that they would be explaining how astronomers could use supernovae to show, once and for all, that lambda equals 0.

And that's what Perlmutter had come to Princeton ready to discuss. Yes, he reported. Now the SCP's first seven supernovae were consistent with a universe where omega equals 1 and lambda equals 0.

"This could be a lambda killer," Jim Peebles told a journalist.

Lambda killer. Perlmutter liked the sound of that: Get lambda out of the way so it won't be a spoiler anymore.

Like his mentor David Schramm, Michael Turner didn't like to lose debates. And like Schramm, he wasn't afraid to practice what the Fermilab and Chicago cosmologists called "jugular science." During a break in the Princeton activities, as various astronomers and cosmologists were climbing a flight of stairs to an auditorium, Turner sent Perlmutter a message. Ostensibly talking to the astronomer walking beside him, Turner raised his voice.

"I don't think Saul is that stupid," he said.

Perlmutter didn't appear to hear.

"I said," Turner repeated, raising his voice, "'I don't think Saul is that stupid.'"

Turner was slightly more diplomatic during his own talk, but no less needling. "I am anxiously awaiting the results of the two deep searches for supernovae," he said, referring to the rival teams. "I think they're going to shed some important light on this. To draw any conclusion now would be to take away from their thunder later."

The SCP submitted their data on their first seven supernovae to the Astrophysical Journal that August. If they made the standard assumption regarding lambda—"a A = 0 cosmology"—then omega was 0.88. But given the margin of error, you could reasonably interpret that result as omega equaling 1. If they made the less likely assumption that the universe was flat with a possible component of lambda, then omega, at 0.94, was even closer to 1, while lambda would be 0.06, a negligible amount and, given the margins of error, presumably 0.

The universe was flat. Matter alone was enough to get omega to 1. And we didn't need lambda. Or at least that interpretation was, as the paper said, "consistent" with their results.

Unfortunately for the SCP team, that interpretation wasn't consistent with their own next round of data.

The Astrophysical Journal accepted the paper in February 1997 and published it in the July 10 issue. By then the SCP team was finishing their analysis of the two supernovae they'd examined with HST. Because HST photometry would be so superior to ground-based analyses, the team would place special emphasis on whatever it had to tell them.

Peter Nugent had been hired as a postdoc by Perlmutter a year earlier, part of a campaign to bring astronomers onto the project. Nugent had written his thesis on Type Ia supernovae, and Perlmutter had assigned him to perform photometry. Nugent had a forceful style. He wouldn't have been out of place at the University of Chicago; his bearing and attitude were reminiscent of a David Schramm or a Rocky Kolb: a can-do, answer-any-question, know-the-restaurants-with-the-best-wine-lists spirit. On June 30 he finished the photometry on the two HST supernovae, giving him their magnitudes, the standard measure of luminosity for celestial objects. Spectroscopic analysis had already yielded the redshifts for the two supernovae. Now Nugent plotted the two values against each other, redshift on one axis, magnitude on the other.

You would expect the points on this plot to fall pretty much on the usual 45-degree-angle straight line—the relationship among nearby galaxies that Hubble discovered in 1929. The straight line itself represents a universe that is expanding uniformly, experiencing no effects of gravity—in other words, a universe without mass, a universe with nothing in it. Eventually, at some great distance across space and back in time, the points will have to begin to deviate from the straight line to represent a universe that does have mass. But which kind of universe? The extent to which the most distant points deviate downward from the straight line will be minor, but it will tell you how much brighter the objects are than you would expect them to be at their particular redshifts—the brighter the supernovae, the higher the value of omega. And that value will tell you the weight, shape, and fate of the universe: open, closed, or flat; saddle, globe, or plane; Big Chill, Big Crunch, or Goldilocks.

Nugent began plotting the two HST supernovae. First he looked along the redshift axis—the measurement that corresponded to their distances. Then he moved up the graph until he reached the magnitudes his photometry had given him. He assumed the two points would fall along the deviation—the particular downward curve—consistent with the conclusion that the SCP's latest paper had reached: a flat, all-matter, omega-equals-1 universe. But that's not where these two supernovae fell. They were landing on the other side of the straight 45-degree-angle Hubble relationship, on what would be an upward curve. The difference between what their luminosities should be at their redshifts and what their luminosities were, was approximately half a magnitude, meaning that the two supernovae were 1.6 times fainter than he expected.

"There goes the universe," he wrote in the e-mail to his team. Not that he was ready to draw any conclusions about cosmology. After all, as he wrote, "it's only two data points." And he wasn't the team member responsible for determining the omega and lambda measurements. But the discrepancy between the magnitudes he expected and the magnitudes he measured was unequivocally jarring. "Hopefully this will be enough from me to get the paper out this week," he added. "I do think it has to go out now since the other group is most likely going to submit something soon (very soon) [about their own HST results]. It's good with the data—as-is. Lets get the damn thing out there!"

But they didn't. The team quickly realized that they needed to decide not only whether to publish, but what.

In the jargon of science, the two HST supernovae were "fighting" the earlier, all-matter, omega-equals-1 result. That summer, the team threw out two of the first seven supernovae—one that further analysis determined to be a core-collapse supernova rather than a Type Ia, and another one that was an obvious outlier. They also eliminated the 1996 HST supernova because they felt that, while the individual measurements were probably accurate, they didn't have enough observations—enough points on the light curve—to subject the supernova to peer review. But the similarity in results between the 1996 and 1997 HST supernovae did reinforce the team's confidence in the 1997. By September they had settled on six supernovae in total as well as a conclusion—albeit one inconsistent with a paper they had published only two months earlier.

One member of the collaboration wrote to Nugent that he "must realize that we will look very bad if we change our limits every time we add *one* SN to the total sample without a discussion [in the paper]. How can anybody trust what we say if they know we are going to say something else in a few months time without any explanation?" After all, the two papers weren't dependent on two separate samples or a significantly larger set of data. The new paper had two fewer supernovae than the previous paper. The only addition to the data was a single supernova from HST.

The point of the paper, Nugent argued back, would be to demonstrate what HST could do for a distant supernova search— "NOT" to declare that the universe has certain values of omega and lambda, "says God." The number of supernovae didn't matter. "I've never given a rat's ass about one data point (or even a number under 10 for that matter) in my life when the error bars are so large." The method was what was worth reporting.

Still, writing a paper that reverses a result even implicitly was going to require some finesse. Not until September 1997 did the team have a draft they would submit to Nature, and by then they had larded the prose with enough qualifiers to choke even a Kirshner: "we use the words 'preliminary', 'initial' and 'if...' all over this paper," Nugent reassured a colleague in a September 27 e-mail. And when the paper got to the omega and lambda part, it delivered a double qualifier: "these new measurements suggest that we may live in a low mass-density universe" (emphases added).

The team submitted its paper on the HST supernova to Nature the first week of October. Sure enough, just as Nugent had fretted at the end of June, the High-z team followed with its own HST supernova paper, with Garnavich as the lead author, posting it on the Internet on October 13. The High-z paper reported that its sample, too, "suggests that matter alone is insufficient to produce a flat Universe." Clearly the two groups were converging on the result that had motivated their supernova searches: the fate of the universe.

If there was no cosmological constant, then omega was low and the universe was open—destined to keep expanding for all time. Even if there was a cosmological constant, then omega was still low and the universe was flat—slowing to a virtual halt, but not collapsing. Either way, the expansion of the universe would continue forever. That fall the American Astronomical Society invited both teams to participate in a press conference at the AAS meeting in January 1998. The press department at the AAS usually organized four or five press conferences during the course of the semiannual five-day meetings, and a discussion of the fate of the universe seemed like the kind of topic that would draw a crowd. Sure, the two teams told the AAS, we'd be glad to send representatives to a press conference.

But a subtler, and certainly more esoteric, question remained: Was there a cosmological constant?

Gerson Goldhaber, anyway, thought there was. On September 24 he showed the group the histograms compiling all the supernovae, one for a no-tambda universe, and one for an omega-plus-tambda-equals-1 flat universe. For a measurement as delicate as the one the team was trying to make, binning supernovae into broad categories wasn't going to be as persuasive as plotting individual points. But a trend was clearly developing. The more supernovae the team analyzed, the lower the value of omega seemed to be heading. Two weeks later, the minutes from another team meeting reflected the trend: "Perhaps the most disturbing thing is that the first 7"—the bunch on which the team had based their previous paper—"were consistent within themselves but the next 31 Sne give what seems to be a consistent answer that is lower."

In the 1930s Fritz Zwicky had discovered a set of supernovae that he assumed were examples of the implosion process that he and Walter Baade had predicted; in retrospect, those supernovae all turned out to be examples of an explosion process that hadn't yet been discovered. Now the SCP team was realizing that they, too, had defied the odds. Even after eliminating the obvious outlier and the Type II from the original set of seven, those five initial supernovae still appeared to be on the bright side. As a result, the addition of the dozens of fainter supernovae was driving the value of omega down. In the histogram analysis of the data, a sharp peak was developing around an omega of 0.2.

On December 14, 1997, Goldhaber presented his findings at a seminar at the Institute for Theoretical Physics at UC Santa Barbara. Kirshner was in residence at the institute that fall, on sabbatical from Harvard, and as usual Goldhaber found him to be "antagonistic." Kirshner interrupted the presentation: An omega of 0.2; so what else is new? But Goldhaber thought he was making an argument that omega could be 0.2 only if accompanied by lambda. At least the director of the institute, David Gross, seemed to understand, though when he asked Goldhaber why he believed the results, all Goldhaber could offer was that he had a long history of interpreting histograms. "I'm convinced," he said.

Perlmutter, too, was presenting preliminary results in public that fall, carrying his transparencies of low-omega scatter plots from colloquium to colloquium—the first on October 23 at the Physics Department at UC San Diego, the second on December 1 at the Physics Department at UC Berkeley, and a third on December 11 at the Physics Department at UC Santa Cruz. As in the Nature paper, he was careful to qualify his comments, but he also made sure to let his audiences know that the data contained the possibility of "some rather striking consequences for physics," as he said at the Berkeley colloquium. "In particular, if you consider the flat-universe case—the case of the inflationary universe that's favored—a mass of this sort, a mass density of this sort, means that the cosmological constant has to be contributing a cosmological constant's energy density of about 0.7." In case the non-cosmologists in the audience were missing the point, the astrophysicist Joel Primack stood up at the end of Perlmutter's talk at Santa Cruz to say the results were "earthshaking." Then he added the crucial caveat: "If true."

For the High-z team, Adam Riess was now "it." Riess knew that his team was at a disadvantage concerning the quantity of supernovae, if only because Peter Nugent kept reminding him. The two of them were in a group that got together on weekends in a city park to play a variation on football called, for obvious reasons, mudball. Sometimes the trash talk took the form of my-distant-supernova-search-is-better-than-yours. One day Riess decided he was tired of hearing how many supernovae the SCP was raking in and how far behind the High-z search was. If you couldn't beat the SCP on quantity, he figured, you could beat them on quality.

For his master's thesis Riess had tackled the problem of dimness. His light-curve shape method proposed a mathematical solution to deriving luminosity from the rise and fall of light-curve shapes. For his PhD thesis Riess had approached the problem of dust. If you're trying to determine the distance of a supernova by measuring its redshift, then you need to know to what extent dust is contributing to the reddening of the light (just as dust in the atmosphere reddens a sunset). In Riess's multicolor light-curve shape method, or MLCS, the observations of light in several color filters would provide a cumulative measure of the effect of dust, allowing you to derive a more accurate determination of distance.

As his team's resident expert on correcting for intergalactic dust between the supernova and the observer, he might be able to clean up the supernovae in such a way that they provided a tighter margin of error than the SCP's. He wouldn't even need a greater number of distant supernovae, though they were always welcome. Even nearby supernovae would do the trick. If he could anchor the lower end of the Hubble diagram with sufficiently reliable data, then the higher-redshift supernovae—while fewer in number than the SCP's—would be more reliable as well. And he knew where he could get nearby supernovae: observations he had already made, as part of his thesis research, at the 1.2-meter telescope on Mount Hopkins, in Arizona. Twenty-two supernovae in all. None of them yet published.

The addition of those supernovae, however, created a new problem. Never mind a universe with no matter. His calculations were producing a universe with negative matter.

"I'm only a postdoc," Riess told himself. "I'm sure I've screwed up in ten different ways." Computers, he thought, don't know physics. They know only what we program them to know. Clearly he had programmed his computer with impossible physics. So Riess checked his math, and he checked the computer code he'd written, and he couldn't find any mistakes. Of course, Einstein's equations allowed for another option—a universe with a positive lambda. Plugging his data into that universe brought the amount of matter up, into the positive range. But that option, he knew, wasn't palatable to most astronomers—for instance, the team leader, Brian Schmidt, who liked to say that astronomers who talk about the cosmological constant are astronomers without many friends.

Riess sent his results to Schmidt.

"Adam is sloppy," Schmidt reminded himself. Brilliant, but prone to mathematical errors. Schmidt agreed to double-check the results. As a rule, mathematicians check each other's work not by looking back over the same calculations but by performing the calculations independently, so as not to be lulled into making the same mistakes. Schmidt and Riess soon developed a routine. Riess would e-mail a problem, and a day later Schmidt would respond. I started with this image, and my analysis said the supernova was this bright—how about you? Or We observed in this filter, and I found that the redshift was equivalent to this number—how about you? They signed their e-mails Pons and Fleischmann, after the two physicists who, in 1989, had "discovered" cold fusion, and who, after a long period of infamy, had fallen into obscurity. If you're Stephen Hawking and you make a major mistake, you're still Stephen Hawking. If you're a postdoc under thirty and you make a major mistake, you're history. Sometimes when Riess couldn't wait for an answer, the phone would ring in the Schmidt household. Schmidt's wife, sleepless from caring for a six-month-old, would say, "If that's Adam, tell him to—"

Schmidt: "Hello, Adam."

Riess: "Oh." A pause. "Is it early there?"

Schmidt: "It's four in the morning."

Riess: "Oh." A pause. "So, what do you know?"

What Schmidt knew, night after night, was the same thing: So far, so good.

Riess remembered now that one day when he was a graduate student at Harvard, Kirshner had brought Mike Turner and Alan Guth by his office and encouraged Riess to show them what he was working on. Riess had just taken the team's first Type Ia supernova, 1995K, and plotted it on the Hubble diagram. The supernova fell on the "bright" side of the 45-degree straight line, but its location didn't matter; it was only one point. What mattered was that the team actually had a point to plot. Still, Turner couldn't help mentioning that it was in the "wrong" part of the diagram.

"How embarrassing," Riess had thought. "There's never been so much brainpower in P-306"—his office at the time—"and we're probably showing them that we're not even doing the experiment right."

But now, a couple of years later, he thought maybe the location of that point had mattered more than anyone knew. Maybe the answer to the fate of the universe had been right in front of them from the very first supernova.

Riess was getting married in January 1998—the weekend at the end of the AAS meeting, in fact. When his future wife flew home from Berkeley to her family in Connecticut a few weeks early to take care of the final preparations, Riess sequestered himself in his office in Campbell Hall, on the Berkeley campus, and began writing a paper that would report the results—if they held up.

The campus was empty for holiday break. The heating was off, and Riess had to bundle up; even in California, December can get chilly. But every day, walking past the locked office doors and under the unlit hallway lights, he went to work. On December 22, he wrote an outline and started a draft. Garnavich's HST paper, the first from the group, had been a short letter. Riess figured the next paper would have to be the War and Peace version, as scientists like to say; if you're claiming something surprising, you have to show all the work. In the coming days he also contacted Nick Suntzeff, down in Chile, and asked him to double-check some photometry, though he didn't say why so as not to prejudice the result. At one point he beckoned a colleague into his office.

Alex Filippenko, who also had been taking advantage of the semester break to catch up on work, greeted Riess with his usual wide and deep smile, rectangular and cavernous. Nobody could be that happy all the time, and Filippenko wasn't. He had once been a member of the SCP, and as an astronomer on a team with a particle physics mentality, he had experienced the clash of cultures probably more acutely than anyone else on either team. He disliked the hierarchical structure that awarded Perlmutter lead authorship on the important papers; Filippenko would go to astronomy conferences and hear about "this supernova survey" that Saul had organized, and he'd have to inform his peers that he was actually part of that collaboration. He watched as his friends in the supernova game—Kirshner, Riess, Schmidt, Suntzeff—coalesced into a collaboration of their own. He complained to them that he had been warning his SCP colleagues about the possible non-standardness of Type Ia, about dust, about the difficulty of photometry and spectroscopy—all the concerns that Kirshner had been raising for years as a member of the External Advisory Board. He said he felt that the Berkeley Lab physicists regarded these concerns as if they were "irritations" and "annoyances" rather than supernova astronomy's swords of Damocles. He felt marginalized and ignored on the SCP collaboration, and he suspected that they kept him around only as the "token astronomer" who could get them time on telescopes.

But all his friends could do was shrug and say that yeah, they would love him to be on their team, but he was part of the other team.

In early 1996, Filippenko defected. A few months later he was able to exact a revenge of sorts. As Riess was finishing his PhD work at Harvard, Perlmutter approached him with the offer of a position at LBL and, by extension, on the SCP team. Down the hill at UC Berkeley, Filippenko countered with a Miller Fellowship—the same honor that Filippenko himself had once held as a young postdoc.

Riess didn't have to think too hard. He'd be doing astronomy with a friend. He'd be doing astronomy on a team to which he already belonged. He'd be doing astronomy.

Exactly. For Filippenko, Riess's MLCS method was precisely the kind of tool that an astronomer would know was necessary—would feel a need to invent—before proceeding to the next step. And now Riess apparently wanted to show him where that next step had led.

Riess pointed to the notebook on his desk. He walked Filippenko through the calculations he'd made, he described the back-and-forth that he and Schmidt had been having, and he said that the result didn't seem to be going away. Filippenko studied the notebook for a few moments, then straightened up. He was shaking his head.

"Man," Filippenko said, "be sure the measurements are done right."

Well, yes. By January 4, Riess had taken the paper as far as he could. He sent Schmidt the material for the final round of cross-checking. Then he waited.

"Well Hello Lambda!" Schmidt e-mailed him on January 8, the day of the AAS press conference. Schmidt had finished his spot checks and found nothing wrong. His statistical level of confidence was the same as Riess's: 99.7 percent. It was time to let the rest of the team know.

When Pete Garnavich got to the AAS meeting, he had already studied the SCP's paper in Nature, and as the lead author on the High-z team's HST paper—due to appear in the Astrophysical Journal on February 1—he certainly would have sought out the SCP's even more recent data on his own. But he knew, too, that his team was getting a bizarre result; Riess and Filippenko had confided it to him before the press conference. They had also instructed him to keep the result quiet. On January 9, Garnavich visited the AAS poster session meeting to see for himself how close the SCP was to getting that same bizarre result.


Clearly the SCP supernovae were falling in the low-omega range. The fate of the universe was to expand forever. Ho hum.

But what about a cosmological constant? Could the SCP claim evidence for a non-zero lambda? Could they say with some conviction that the universe would have negative mass—essentially, that it wouldn't exist—without the addition of a positive lambda to the equations?

Not quite, as far as Garnavich could see. The error bars above and below the points on the graph representing the supernovae could certainly accommodate such an argument. Some of the upper limits, and some of the supernovae themselves, fell within the range of the upward curve designating a universe with non-zero lambda. But some didn't. Garnavich concluded that the SCP wasn't ready to claim anything explicit, anything definitive. He reported back to his colleagues that High-z was still in the lambda game.

Riess flew east for his wedding on January 10. He returned to Berkeley two days later, for a one-night stopover on his way to his honeymoon. That evening he checked his e-mail. The string of boldface unread messages stretched down the screen. Riess scrolled. Still it stretched. When he got to the bottom of the list, he checked the time stamp to see how long the conversation had been going on without him. Forty-eight hours—"an eternity."

He started at the bottom—a question from Schmidt: "how confident are we in this result?"

"In your heart," Kirshner wrote back, "you know this is wrong, though your head tells you that you don't care and you're just reporting the observations."

"I don't know about anyone else," another team member responded later that day, "but MY heart tells me nothing about the cosmological constant." They had a result; they had a confidence level. Which led to the second question: Did they need to believe it in order to publish it?

Kirshner didn't want to risk reporting evidence of a cosmological constant that they would have to retract later. "That would be like saying 'Omega must be 1' based on 4 supernovae and then saying 'Omega must be Zero' when you get one more. Perlmutter has already done that. He's a year ahead of us, but I don't think we want to duplicate that path!"

Bruno Leibundgut, writing back from Germany that same day, agreed. "There is no point in writing an article if we are not very sure we are getting the right answer."

Mark Phillips, in Chile, concurred. "Press releases and a barrage of ApJ Letter/Nature articles may impress the public or scientists who have only a casual interest in the subject, but the hard-core cosmology community is not going to accept these results unless, as Bruno says, we can truly defend them."

Schmidt, however, disagreed. "As uncomfortable as I am with a Cosmological constant," he wrote, "I do not believe we should sit on our results until we can find a reason for them being wrong (that too is not a correct way to do science)."

Correct way or not, there was a further concern: priority.

"Of course we want to remain true to our scientific ideals," wrote another member of the collaboration. "But this has to be balanced with realpolitik."

"Who knows?" Filippenko wrote. "This might be the right answer. And I would hate to see the other group publish it first." And if it's the wrong answer? Another team member argued that there was no downside: "If it turns out in the fullness of time that a cosmological constant exists they"—the SCP—"can claim to have found it. If it does not, their claim will be forgotten and no one will attach much blame to them for being wrong." Reporting that same conclusion before SCP did was was a no-lose proposition. If the High-z team was right, they would get priority; if they were wrong, they'd get a free pass. High-z had a reasonable argument, so why not make it?

Why not? Riess saw no reason. He leaned into his keyboard and started to compose a response that he could send to the whole group. When he looked up from the screen, he found his bride staring back at him.

"I cannot believe," she said, "you are working on an e-mail when we are on our way from our wedding to our honeymoon."

"Well, this is a really important one."

"Oh, I think I'm going to be hearing this all the time."

"No, no," he said. "This—you're not. This really is—this is the one."

She shook her head and left the room. Riess bent back to the keyboard, composing an e-mail that would answer all the questions.

Heart or head?

"The data require a nonzero cosmological constant!" he typed. "Approach these results not with your heart or head but with your eyes. We are observers after all!"

Publicity? Priority? Realpolitik?

"You see, I feel like the tortoise racing the hare. Everyday I see the LBL guys running around but I think if I keep quiet I can sneak up....shhhh..."

Finally, quick letter or War and Peace?

"I think I can answer the group's dilemma about a quick kill paper vs a detailed explanation .... you all said you wanted a detailed exposition of the data so that's is what I have been working on. Brian said that adding the data stuff to the paper should only take a week, well I did it already before the wedding."

He hit send, and the next morning he left for his honeymoon in Hawaii. (Also, an observing run at Keck.)

For Perlmutter, the extra effort he'd put into the preparations for the AAS meeting had paid off. The media coverage focused primarily, and rightly, on the consensus that the participants in the press conference had reached—the fate of the universe. The New York Times ran it on the front page, under the headline "New Data Suggest Universe Will Expand Forever." The San Francisco Chronicle, the hometown paper for the SCP team, had also put the news on [>]. The local paper for the AAS meeting, the Washington Post, ran its story on page A3: "Universe Will Keep Expanding Forever, Research Teams Say." But it was the SCP that the Post singled out for a rave. "Perlmutter bowled over the audience with an unexpectedly large sample," the article said. "Garnavich's team presented three." And then, of even greater significance to the astronomy community, came a news article in the journal Science three weeks later.

The author of the article, James Glanz, a PhD in astrophysical sciences, covered his beat as if it were City Hall. He had written about the supernova searches over the past several years, but his most recent reporting mentioned a possibly imminent discovery. In the October 31 issue he wrote that both teams had submitted papers supporting the conclusion that the universe would expand forever—a scenario that held whether the universe was open or flat, whether omega was less than 1 or exactly 1. But then he added that such a never-ending expansion would be "perhaps boosted by large-scale repulsive forces."

"The results," he continued, a few paragraphs into the article, "still leave an opening for some theories in which matter plus its equivalent in energy, supplied by the cosmological constant, add up to a flat universe."

The article in late January also included a reference to "a quantum-mechanical shimmer in empty space, called the cosmological constant," but this time Glanz focused on the SCP's contribution to the AAS meeting:

Not only did the results support the earlier evidence that the expansion rate has slowed too little for gravity ever to bring it to a stop; they also hinted that something is nudging the expansion along. If they hold up, says Perlmutter, "that would introduce important evidence that there is a cosmological constant."

"It would be a magical discovery," adds Michael Turner....

Since the frantic exchange of e-mails early in the month, the High-z collaboration had been trading drafts, exploring the math, debugging the code (Schmidt and Riess had missed a few glitches, but nothing important), examining the photometry and the spectroscopy and the charts and graphs and tables—all in the cause of making their case scientifically responsible. Now they had a further concern. Glanz's article, complete with a reproduction of a contour graph showing SCP's preliminary analysis of forty supernovae, seemed to be suggesting that the SCP was beating them at beating the SCP at beating them at their own game.

Alex Filippenko would be speaking at the upcoming UCLA Third International Symposium on Sources and Detection of Dark Matter in the Universe, in Marina del Rey. The High-z team would be submitting their paper only a couple of weeks after that. Why wait? he asked. Filippenko suggested he could announce the team's findings at the UCLA meeting. "This is our chance to make a big splash," he said.

But why not wait? argued other members of the team. The paper will be out soon enough. Let the science speak for itself.

Filippenko, however, argued back that if the SCP was as close to claiming a discovery as their AAS presentations suggested, then those two weeks might make a crucial difference in terms of establishing priority. "You can check and recheck your results forever," he said, "but at some point you've got to have the balls, basically, you've got to have the courage, to announce your result and to say, 'Okay, here is an accounting of our uncertainties. This is where we stand.'"

The discussion stalled there. But then, just days before the UCLA meeting, Jim Glanz called Filippenko. He didn't know whether Glanz was using the old reporter's trick of pretending to know more than he did, but for Filippenko the conversation was all he needed to convince a majority of the High-z team. "Glanz is going to be breaking this story whether we're in it or not," Filippenko said. "So why not be in it?" Give him the evidence. Give him the quotes. And give him the news peg.

Make the announcement, and tell him about it first.

On February 22, Filippenko took his seat at the UCLA conference and listened as Gerson Goldhaber gave a presentation on the SCP team's latest results. Then he listened as Saul Perlmutter gave a presentation on the SCP team's latest results. As far as Filippenko could tell, nobody was claiming a discovery; all he heard was that the SCP had "evidence" for lambda.

He took a deep breath. It was now his turn to present. Filippenko stood up, paused, and then said either you had a result or you didn't. And the High-z team did.

The ghost was real, and it was most of the universe.