The 4 Percent Universe: Dark Matter, Dark Energy, and the Race to Discover the Rest of Reality - Richard Panek (2011)
Part II. Lo and Behold
Chapter 6. The Game
POP! SN 1994F went off.
Pop! SN 1994G went off.
Pop! SN 1994H went off.
The Berkeley team had hung numbered tags around the necks of the champagne bottles, "1" through "6," one for each supernova they had discovered during their most recent run at the Isaac Newton Telescope, December 1993 through February 1994, plus a "0" for their 1992 supernova. The members of the Supernova Cosmology Project—as they were now calling themselves—had gathered at Gerson Goldhaber's house in the Berkeley hills. They laughed at themselves for being "lightweights"; they probably couldn't finish even two bottles. But the champagne inside the bottles wasn't the point, of course; the number of bottles was. It had taken the team four years to get their first supernova. Now, it had taken them three months to get their next six.
But they weren't celebrating only the supernovae. They were toasting their survival. Carl Pennypacker was no longer part of the team. Pushed? Jumped? Who knew? At least they still had a team. In the fall of 1994 the Center for Particle Astrophysics and the LBL Physics Division had convened a Project Review Committee to determine whether the supernova search should continue. Even when the committee decided favorably, Bernard Sadoulet cut the CfPA's contribution to the supernova budget by half and gave it to his own project; Robert Cahn, the head of the LBL Physics Division, then informed Sadoulet that he was cutting LBL's contribution to Sadoulet's dark-matter experiment by half and giving it to the Supernova Cosmology Project. Not only was the SCP solvent for a change, but they had a guardian angel at LBL, a division director who understood why a physics lab might want to do a distant supernova search. And now they'd gone and gotten six more supernovae. They weren't just in the distant supernova game, they were the game.
Pop! SN 1994al. Pop! SN 1994am. Pop! SN 1994an.
The party didn't last long.
For years Bob Kirshner, as a member of the supernova project's External Advisory Board, had been saying that the LBL collaboration didn't know what it was doing—that team members weren't taking dust into account or paying sufficient attention to photometry or concerning themselves with whether Type Ia supernovae were standard candles. He didn't seem to understand that for LBL such considerations were beside the point—or weren't yet the point. The team was just hoping to prove it could do what it was trying to do: detect supernovae distant enough to be useful for doing cosmology. Their early efforts were what team members called, on various occasions, "demonstration runs" that were part of a "pilot search" in an "exploratory program." Then when they did find the 1992 supernova, Kirshner's objections as referee on that paper held up publication until 1995, when a more sympathetic referee, Allan Sandage, approved it. Breezing into Berkeley from Harvard, Kirshner seemed oblivious to the growing consternation, frustration, and anger not only at his objections but at him. A colleague of his on the External Advisory Board, speaking at a cosmology conference, characterized Kirshner's contribution to the discussion of the LBL approach: "No! This could not work! It couldn't possibly discover these high-redshift supernovae!"
And now here Kirshner was, saying, Well, maybe.
At least the LBL team had a six-year head start—surely that counted for something. Besides, their faith in Type Ia as standard candles had been rewarded. First Mark Phillips had demonstrated that the light curve for an inherently dimmer supernova falls off sooner—its descent is steeper—than the curve for an inherently brighter one. Then the Berkeley team had arrived at their own variation on his technique. They made Type Ia light curves uniform by treating them like images in a funhouse mirror—stretching them "fatter" or compressing them "thinner" until they fit an idealized Type Ia template. (The team often took advantage of an LBL photocopier that could distort images in precisely that manner.) If Type Ia supernovae were less a type than a family, then each member of that family was less a standard candle than a calibrated candle.
And now, three years after proving to themselves that they could find a distant supernova, they had gone ahead and figured out how to find supernovae on a regular basis. After discovering the three in early 1994 on the Isaac Newton Telescope, they found three more with the Kitt Peak 4-meter telescope, in the mountains southwest of Tucson, Arizona. By June 1995 they had accumulated eleven distant Type Ia in total, and they were ready to make their first major statement to the community in the form of four papers at the NATO Advanced Study Institute Thermonuclear Supernovae Conference in Aiguablava, on the Mediterranean coast of Spain: They had figured out a way to discover Type Ia supernovae whenever they wanted.
They called it the "batch" method. Just after a new moon they would make as many as a hundred observations, each image containing hundreds of galaxies as well as, if possible, clusters of galaxies. In one several-day run they could gather tens of thousands of galaxies. Two and a half to three weeks later, just before the next new moon, they would return to those same tens of thousands of galaxies. In an update of the old galaxy-by-galaxy blinking technique, computer software would subtract the earlier reference image from each new image, searching the hundreds of galaxies for the new dot of light that might signal the emergence of a supernova. Then, again by using new software, the astronomers could determine that same night whether that dot actually was a supernova. They could then relay those coordinates to team members waiting at other telescopes, who would perform the necessary spectroscopy and photometry using telescopes on which they'd reserved time months earlier. (Having a successful track record worked wonders with time allocation committees.) You might not know in advance exactly where a supernova was going to go off, but you knew that one or more would. In effect, they had figured out how to "schedule supernova explosions"*—just like that.
Pop, pop, pop.
So Berkeley had a six-year head start. So what? Schmidt and Suntzeffs team had astronomers—professionals who didn't need to learn how to do photometry and spectroscopy, who needed only to do them well and then to make improvements where necessary.
A lot of Schmidt and Suntzeffs team had been at the NATO meeting, too. (The conference was organized by a former postdoc of Kirshner's.) The Harvard and Chile guys regarded the Berkeley team with some incredulity. "'I just heard about it, and I just thought about it, so—this is my subject!'" is how Kirshner characterized the SCP's attitude toward supernovae. SCP team members were talking about timing their observations to the new moon—as if astronomers hadn't been doing just that for thousands of years. The "batch" method? Radio astronomers were taking that approach in the 1960s. Supernovae on demand? José Maza was delivering that in the 1970s.
Naturally, with all these scientists pursuing the same goal, meeting in the same place, there had been talk of a collaboration. But some of the members of Schmidt and Suntzeffs team left Spain with the impression that, as Kirshner said, "working together meant working for them." Why would one of the world's most knowledgeable supernova specialists want to be subordinate to Saul Perlmutter, Type Ia neophyte and ten years his junior? For that matter, why would any of these purebred astronomers put themselves in a position where they might be reporting to purebred physicists? Perlmutter was talking about how "rare," "rapid," and "random" supernovae were. And they were! But Schmidt and Suntzeffs team preferred to put the emphasis on "dimness," "distance," and "dust"—how to tell whether a supernova is intrinsically dim, dim because it's distant, or dim because of dust. While the physicists were worrying about how to find distant supernovae, the astronomers were worrying about what to do with the distant supernovae once they found them.
Which they had (or, at least, a distant supernova). But they still couldn't be sure what type it was. The problem had been there from the start. In the same April 6 e-mail that Leibundgut sent to Schmidt celebrating the difference that one supernova can make, he also mentioned, almost as an aside, that "the 'supernova' spectrum still has a lot of galaxy in it"—that the light from the apparent supernova was difficult to separate from the light of the host galaxy. The spectrum could tell you the redshift of the galaxy, and therefore the redshift of the supernova residing in it. But in order to see the spectrum of the supernova itself, you were going to have to isolate its light.
First Mark Phillips tried. One week after notifying the team of Hamuy's calculation that the supernova was the most distant ever, he was ready to give up. "I've spent too much time the last few days looking at this," he wrote the team. "The conclusion I've reached is that the SN spectrum is of such low S/N"—signal-to-noise, useful supernova light versus the optical equivalent of static from the galaxy—"that it is impossible to tell what type it is."
Leibundgut tried next. And tried. "And no to the spectrum," he wrote in an e-mail at the end of May. "I have tried several ways of extraction but without any improvements." When he got to the Aiguablava conference, he told his collaborators that he, too, was ready to give up. "I don't know what to do anymore," he said. "I'm not sure I can confirm it's Type Ia."
"Crap!" Suntzeff said. "Saul's pulling in supernovae by the handful, and we only have one, and we can't even tell if it's a Ia!"
At one point Leibundgut was discussing the problem with Phillips in the lobby of the hotel. The sea was outside. They were inside. Phillips turned to Leibundgut and said, "Why don't you subtract the galaxy?"
"Subtracting the galaxy" is just about the first thing you do if you're trying to get the spectrum of a supernova. If you want to isolate the supernova light, you take a spectrum from the part of the galaxy containing the supernova, which is flooded with light from the galaxy, and then you take a spectrum from a different part of the galaxy, away from the supernova, and then you subtract the second reading from the first. Ideally, the spectrum of the supernova itself pops out.
This supernova, however, had been so overwhelmed by galaxy light that Leibundgut hadn't tried the obvious. Nobody had. From Aiguablava he flew to Hawaii for another conference, and then home to Munich. He fiddled a little with the overall galaxy light, dividing its intensity by ten. Why ten? No reason. The spectrum from the galaxy would still be the same; he wasn't changing the quality of the data. He was just changing its intensity. He subtracted this spectrum from the supernova spectrum (which also contained the galaxy spectrum), and out popped a beautiful supernova spectrum.
"The spectrum of 95K looks great!" Phillips wrote him on August 1. "I'm now very convinced that this was a genuine type Ia."
They were back in the game. Now what they needed was to formalize their existence as a team.
From the start—during their first discussions in La Serena, in early 1994—Schmidt and Suntzeff knew what kind of team they would want. Theirs wouldn't resemble a particle physics collaboration. It wouldn't have the same rigid top-down hierarchy, the same plodding bureaucracy, the same assembly-line mentality. Instead, their collaboration would follow a traditional astronomy aesthetic. It would be as nimble, as independent, as Hubble on Mount Wilson or Sandage on Mount Palomar.
Already that approach had paid off. Like the professional astronomers they were, they had asked what they considered the key question first: Are Type Ia supernovae really standard candles? Only when they knew that Type Ia could be calibrated did they actually go looking for a distant one. And they almost hadn't found it. But in the end they did find their high-redshift supernova, and it was indeed a Type Ia. They'd salvaged their collaboration, and maybe their credibility, by making the discovery "on the smell of an oily rag in a quasi-chaotic fashion," as Schmidt liked to say. The process hadn't been pretty—it was, Suntzeff thought, more like "anarchy"—but it was astronomy.
And yet, astronomy itself was changing. The traditional go-it-alone aesthetic was disappearing. The diversity of the science and the complications of technology were forcing the field into greater and greater specialization. You couldn't just study the heavens anymore; you studied planets, or stars, or galaxies, or the Sun. But you didn't study just stars anymore, either; you studied only the stars that explode. And you didn't study just supernovae; you studied only one type. And you didn't study just Type Ia; you specialized in the mechanism leading to the thermonuclear explosion, or you specialized in what metals the explosion creates, or you specialized in how to use the light from the explosion to measure the deceleration of the expansion of the universe—how to perform the photometry or do the spectroscopy or write the code. A collaboration could easily become unwieldy.
Suntzeff and Schmidt recognized that their team would have to reflect the reality of increasing specialization. As the project evolved from the back of a sheet of computer paper in the spring of 1994 to actual astronomers chewing antacid tablets at observatories, they had to consider not only who would work hard but who possessed what areas of expertise and who had access to the right telescopes. The team's first foray into legitimacy, the September 1994 proposal for observing time at Cerro Tololo the following spring, cited twelve collaborators at five institutions on three continents. After the team confirmed that they'd found their first distant supernova during that run, in early April 1995, Schmidt sent around a reminder of who they were: fourteen astronomers at six institutions. The paper announcing the discovery in the ESO Messenger that fall carried seventeen authors at seven institutions.
Yet even as the collaboration grew, Schmidt and Suntzeff wanted to preserve the dexterity afforded by old-fashioned astronomy—and to turn their familiarity with that tradition to their advantage. They were, after all, playing catch-up.
"We can only do it if we're fast," Suntzeff said. "The only way we're going to get this done is if we recruit as many young people as possible." Young astronomers. Postdocs. Graduate students.
They also wanted the collaboration to be fair. "I'm tired of seeing people get screwed by the system," Suntzeff said—the system where the postdoc did the work and the senior astronomer who had tenure would be first author, getting the credit and going to conferences, while the postdoc wound up without a job.
By the time Schmidt and Suntzeff gathered their collaborators at Harvard in the late summer of 1995, they had formulated a strategy for delegating responsibilities in a way that would move the project forward quickly and fairly. Each semester, one of the sponsoring institutions—Harvard, or Cerro Tololo, or the European Southern Observatory, or the University of Washington—would be in charge of gathering the data from all the collaborators, reducing it, and preparing a paper for publication. And whoever did the most work on the paper would be the first author.
Unlike an anarchy, a democracy—even a revolutionary one—needs a leader. In one sense, Kirshner was the obvious choice. But he was also the embodiment of what Schmidt and Suntzeff wanted to avoid; in addition to Go fast and Be fair, they had framed a corollary: No big guns. Back in La Serena in the spring of 1994, when they had made the initial list of potential participants in the distant supernova search, they hadn't even included Kirshner. In his doctoral thesis, Schmidt had used Type II supernovae to derive a Hubble constant of 60, then watched Kirshner crow about it at conferences. Although they'd eventually recognized that sidelining one of the world's most prominent supernova authorities, and a mentor to many in the group, probably wouldn't be wise either scientifically or politically, Schmidt and Suntzeff remained wary of big-gun syndrome.
And with good reason. At the January 1994 AAS meeting in Washington, Mario Hamuy had given a talk about the Calán/Tololo survey; he was working on an elaboration of Mark Phillips's discovery of the relationship between light curves and absolute luminosity. Afterward, at Kirshner's invitation, Hamuy continued on to Cambridge to give a colloquium on the subject at the Center for Astrophysics. After the talk, a graduate student had invited Hamuy to his office. Adam Riess, not yet twenty-five, radiated the kind of confidence that comes from being the younger brother of two adoring sisters. When he got into the supernova game, he saw no reason why he shouldn't solve the biggest problem out there—how to standardize Type Ia's. Now he wanted to show Hamuy a technique he was developing. Like Phillips's method, Riess's light-curve shape (LCS) allowed you to determine the intrinsic brightness of a supernova; unlike Phillips's method, LCS also provided a statistical measure—a way to refine the margin of error. It quantified the quality of the result.
Hamuy examined it and told Riess he thought it was, as scientists say by way of praise, "robust."
Riess, however, said he had a problem. So far he hadn't been able to test the LCS method on real data. Could he see Hamuy's?
Hamuy hesitated. Your data was your data. Until you published it, it was yours and yours alone. But Riess was persistent, and Hamuy was a guest (at Harvard, of Bob Kirshner), and he relented. Hamuy agreed to show Riess his first thirteen light curves, though not before exacting a promise: Riess could use them only to test his technique, not as part of a paper about the technique.
A few weeks later, Hamuy got an e-mail from Riess. The technique was working. Riess was excited. Could he publish the results after all?
That, Hamuy reminded Riess, wasn't part of the deal. But again he relented, though not before exacting another promise: that Riess wouldn't publish his paper using Hamuy's data before Hamuy published his own paper on the thirteen supernovae. Riess would have to wait until Hamuy's paper had cleared the referee stage at the Astronomical Journal. When it did, in early September 1994, Hamuy let Harvard—meaning Riess and Kirshner, as well as William Press, who provided mathematical guidance—know that they were free to submit their own paper.
They did. But they submitted it to Astrophysical Journal Letters, a publication that, as its name suggests, traffics in shorter papers—and, therefore, briefer lead times.
Hamuy had to work hard to convince the Astronomical Journal to rush his own paper into print. In the end both papers appeared in January 1995. Both papers used the data to derive a value for the Hubble constant. And both arrived at a Hubble constant in the 60s—Hamuy's "62–67," and Riess's "67 ±7." Forevermore, Hamuy understood, the two papers would be cited side by side, as simultaneous publications.
"How could I be stupid enough to say okay?" Hamuy moaned to his colleagues in Chile. "'Mario! Mario! Mario!'" he wailed, mocking himself as much as Riess's entreaties.
Nick Suntzeff could see Kirshner's clumsy thumbprints all over the handling of the timing of the Riess et al. paper. Besides, he was already suffering from his own brush-with-astronomy-greatness fascination. Allan Sandage had encouraged Suntzeff to use CCD technology on Type Ia supernovae to find the Hubble parameter, and Suntzeff had helped his team do so, but the value the Calán/Tololo collaboration derived was on the "wrong" side of 60. Astronomers had estimated that the oldest stars in globular clusters were around sixteen to eighteen billion years old. A Hubble constant of 50 would correspond to a universe that was maybe twenty billion years old; a Hubble constant over 60 corresponded to a universe that was maybe ten billion years old—a universe younger than its oldest stars.
Suntzeff knew Sandage's reputation even as he was befriending him in the early 1980s. Everyone in astronomy knew Sandage's reputation. Even Sandage knew it. But he couldn't help himself; he took the Hubble parameter personally. He had inherited the program from Hubble himself, he had pursued it for four decades, he had wrestled the value down from the ridiculous mid-three-digits to the realistic mid-two. In the 1970s Gérard de Vaucouleurs had taken it upon himself to challenge Sandage's methodology and assumptions, and he'd arrived at a Hubble constant of 100. Other astronomers had begun finding values that roughly split the difference between 50 and 100. Sandage wouldn't budge. The Hubble constant had to be less than 60, he insisted; the age of the universe demanded it. "The answer will come," Sandage once sneered, "when responsible people go to the telescope."
And now Suntzeff had joined the ranks of the irresponsible. He had received a note from Sandage, accusing him of having fallen prey to unsavory influences. Suntzeff tried to contact Sandage. Then Phillips tried. But Sandage was done with them.
Suntzeff, however, wasn't done with Sandage. Having helped derive a value for one of the two numbers in cosmology, he was now mounting an assault on the other. Suntzeff could tell himself that the "competition" with Sandage was in Sandage's head; it was just Uncle Allan being avuncular with a vengeance. He had always known that Sandage might one day turn on him, just as Sandage had turned on other acolytes and colleagues once he thought they'd turned on him. But this business with Hamuy and Kirshner was something else. It wasn't just personally disappointing. It was professionally dangerous.
Actually, the battle line, as Suntzeff saw it, wasn't Hamuy versus Kirshner. It was Calán/Tololo versus Kirshner. You could hardly blame Riess, an overall affable guy, a graduate student presumably wilting under the will of a powerful mentor. But Kirshner should have known better. Did know better. And didn't care. José Maza, the University of Chile astronomer who had served as Hamuy's mentor, resigned from the collaboration even before the initial observing run in February 1995. Hamuy himself, disgusted and disillusioned, decided that now would be a good time to go back to school for his doctorate; he would be heading for the University of Arizona in the fall of 1995. Suntzeffs colleague at Cerro Tololo, Mark Phillips, adopted a "We have to get past this" attitude; Kirshner had served on the advisory board at Cerro Tololo, and it was Kirshner who told Phillips about 1986g, the supernova that had launched Phillips's and Suntzeffs careers in the supernova game. Yet even Phillips readily acknowledged feeling that what Kirshner had done was "improper."
And then it got worse, at least from the point of view of the Chilean part of the collaboration. Even before the Hamuy et al. and Riess et al. papers made their simultaneous appearances in January 1995, Riess and Kirshner had submitted another paper using Hamuy's data, this time to study the local motions of galaxies. The Calán/Tololo collaboration felt, as Suntzeff said, "as if blood was shooting out of our eyes." Shouldn't the guys at Harvard have known that it was a subject Hamuy was likely to pursue? Shouldn't they at least have contacted him and offered to collaborate?
And now, only a month after that paper appeared in the Astrophysics Journal, Suntzeff had to help decide whether Kirshner should lead the team he and Schmidt had created.
Suntzeff wouldn't be team leader; he had known that from the start. While he wanted to be sure that the Chilean contribution was recognized, he also understood the reality of his situation.
"I'm a staff astronomer in Chile," he told Schmidt. This kind of project would take a 100 percent commitment, and he already had a full-time job—and that job was in a place that left him "really isolated." But there was an even more important consideration, he argued: The post would need someone who could bridge both worlds—or both hemispheres, anyway. It needed Schmidt.
In terms of leading the team, Schmidt was equal to Kirshner in all ways except seniority. He'd helped found the group. He'd led the charge in Chile. Perhaps most important, he was no longer at Harvard; he'd moved to Australia earlier in 1995 (a fourth continent!). And over the past several years, both as a postdoc and now on his own as an astronomer, he'd been in Chile often enough to know everyone there well and for everyone there to trust him.
Schmidt was reluctant. But he was also the guy who thought he could write code in two months.
"Yeah," he finally told Suntzeff, "I can do it."
Suntzeff campaigned quietly on Schmidt's behalf. Brian, he argued, had the personality to hold the group together, and he had the drive to get the job done. Eventually Suntzeff talked to just about everyone in the collaboration. Everyone except Kirshner.
Kirshner campaigned on his own behalf. His argument was that he knew the supernova game better than anyone. To a large extent, over the past quarter of a century he had made the supernova game what it was. He had a long history of writing proposals, securing support, keeping collaborators together. He reminded the team members that having all this young talent together in one place—the Harvard Center for Astrophysics—was "not an accident." He was the one who identified the promising graduate students; he was the one who hired the postdocs. "That's something," he told them, "that has to do with making a place where this subject is being done at the highest level."
The more he talked, the more he sounded like a big gun.
The team met in a seminar room in the basement of the Center for Astrophysics. Kirshner and Schmidt waited outside. After a short while, the door opened.
The Big Gun was out. The Young Turk was in.
Schmidt had learned his lesson: This time he went to Chile.
And not only did he go to Chile for the fall 1995 observing season, he got there nearly a week early to test the new code he'd written. He immediately discovered that it didn't work. He was still at the mercy of the observatory's computers; if aspects of the computers had changed since he wrote his code back home in Australia, then he'd have to rewrite his code. The first day in Chile he worked ten hours trying to fix it. The second day he worked twelve or fourteen hours. Third day, sixteen or eighteen. Fourth day, twenty, and then twenty hours the next day, and twenty the day after that. When he started running a fever and having heart palpitations, Schmidt figured it was time to sleep.
Given the standard scheduling logistics at telescopes, his team had had to apply for time at Cerro Tololo the previous spring, even before they'd found their first distant supernova. If they hadn't discovered 1995K, who knows if they would have received the time? If they hadn't satisfied themselves that it was a Type Ia, who knows if they would be using the time, or at least using it to search for distant supernovae? But they were a true team now; they'd even put the idea of distant supernovae in their name: the High-z team (z being the symbol for redshift). It had all worked out, though when Schmidt had told the team via e-mail that they'd gotten time on the telescope, he'd added, "The bad news is that Perlmutter has more nights."
The two teams had applied for time during the same observing season, and the Time Allocation Committee at Cerro Tololo had taken the Solomonic approach of assigning the teams alternating nights. To make the situation even more awkward, one of Nick Suntzeffs duties at the observatory was to provide technical assistance to visiting observers. On nights that he wasn't participating in his own team's search for supernovae, he was, grimly, watching over Saul's. The objective astronomer in him found the Berkeley team's work "quite impressive." Personally, though, he could only shake his head and deliver his verdict to his collaborators: "They're well ahead of us."
The corks were still popping, one for each supernova, but now most of the champagne was going down the drain.
During that observing run at Cerro Tololo in the fall of 1995, the SCP team discovered eleven more supernovae, in one run doubling the number they had gathered over the preceding three years. They had mastered the technique. Astronomers gathered data in Chile, forwarded it to their colleagues in Berkeley, who passed along the information to colleagues at the new 10-meter telescope at the W. M. Keck Observatory in Hawaii, where the team had already secured time because they knew, months in advance, that they would have supernovae to observe on that date. For the astronomers at the telescopes, the observations still contained drama: corrections to code, struggles with weather, decisions on what to target, bouts of diarrhea, and, in Chile, the occasional earthquake. But back in Berkeley, the overnight delivery of data was becoming routine. After all, in a universe full of billions of galaxies, stars were exploding all the time. Supernovae were out there by the thousands, by the millions, every night, waiting to be harvested. The Berkeley team had refined their collaboration, turning it into the kind of assembly-line operation that Alvarez and Muller had foreseen nearly two decades earlier. They were producing the intuitively paradoxical and once unthinkable: "supernovae on demand."
Somewhere in the universe, a civilization died. In Berkeley, they yawned.
In January 1996, at the AAS meeting in San Antonio, Saul Perlmutter sought out Robert Williams, the director of the Space Telescope Science Institute—the scheduling headquarters for the Hubble Space Telescope. Perlmutter wanted to talk about the "batch" method.
"I think with this technique," he said, "we now have the possibility, for the first time ever, of applying for HST time to follow up these very high-redshift supernovae." He explained that by now the SCP team had discovered twenty-two distant supernovae—mostly Type Ia—through the batch method. They had proven that they could predict the date they would find supernovae: whenever they got telescope time. And they could predict where: among whichever thousands of galaxies they chose to scour. They could guarantee the discovery of supernovae. The choice of when and where was now theirs, not the night sky's.
HST required just that level of certainty. It wasn't like earthbound telescopes. You couldn't just submit a proposal and six months later show up with a finding chart. The instrument required extremely complicated programming; you had to have your metaphorical finding chart in hand months in advance, with very little leeway for last-minute (actually, last-week) adjustments. Perlmutter's argument was that this kind of preparation was what the batch method allowed. The combination of confidence and specificity could meet the intricate dance of demands that came with booking time on the Space Telescope.
The logistical details would still be daunting, but they'd be worth it. The Hubble Space Telescope didn't see a lot; its field of view was minuscule compared with the old 200-inch or new 10-meter behemoths on terra firma. But what it saw, it saw with a clarity that no other telescope could approach. Through a CCD camera on an earthbound telescope, a very distant galaxy appeared as a smudge of pixels. Subtracting the light of the galaxy to isolate the light of the supernova was difficult work; witness the four months Leibundgut needed to figure out that the "very faint" 1995K was a Type Ia. The high resolution of HST, however, would make a supernova pop out of its host galaxy. Subtracting the light of the galaxy would be not only easier but far more precise.
That increase in photometric precision was crucial. At the time, it was perhaps the only justification for using HST on a supernova search. As everyone in astronomy knew, the purpose of HST was to perform science you could do only from space. The two distant supernova teams had proven that you could do their science from the ground—not as well as you could do it with HST, but you could do it nonetheless. What Perlmutter would need from Williams, then, was a slice of Director's Discretionary Time, a perk that routinely comes with the title of observatory director.
Williams said the idea sounded promising. He suggested that Perlmutter submit a proposal. A month later, Perlmutter did.
Williams, however, was no expert on the subject of distant supernova searches. Few people were, and with the exception of some Danes, nearly all of them were on one of the two competing teams. Three months later, at the annual May symposium at the institute, the SCP proposal came up during an open discussion. Bob Kirshner was in the audience. He had served on an HST Time Allocation Committee that had considered a similar proposal from SCP, and he had advised against it if only because the mission of HST was to do astronomy you could do only in space. He began to object, but Williams said they would continue the discussion in private. At the next break, Williams ushered Kirshner as well as Mark Phillips and Nick Suntzeff into his office.
"Is this a good idea?" he asked.
Kirshner immediately spoke up.
"No," he snapped. "It's the wrong idea." The point of a space telescope, he reminded Williams, was to do observations that you couldn't do from the ground. That was what the Observing Proposal paperwork said. That was what the previous STScI director had always insisted.
Williams listened. Then he said, "Yes, but I'm the director, and I can do what I want. This is really good science, and I think the Space Telescope ought to do anything that's really good."
Kirshner disagreed, and he and Williams went back and forth like this for a while. Occasionally Phillips and Suntzeff spoke up, echoing Kirshner's arguments. Still, the three High-z members knew what was at stake: If Saul got HST time, that could well be the game. And clearly Williams wanted to give HST time to Saul. He didn't want to hear the argument that nobody should be using HST to do follow-up photometry on distant supernovae. He wanted the best science to come out of HST. He wanted the best science to come out of his telescope.
Maybe they all realized it at once. Maybe they realized it one at a time. But at some point each of the three High-z members at the meeting understood what Williams was really saying. If they asked for HST time, right then and there, they'd get it too.
"My God, what an idiot!" Suntzeff thought as he left Williams's office. "Instead of pushing for the science that I want, I'm trying to argue, for moral reasons, about why we shouldn't be getting the data that we want! How stupid can you get?"
From the West Coast, the SCP watched agog. The dots weren't difficult to connect. Bob Williams had been director of Cerro Tololo from the mid-1980s to 1993. Mark Phillips and Nick Suntzeff had worked at Cerro Tololo under Williams. Bob Kirshner had served as an advisor to Cerro Tololo during this period. If you wanted to see evidence of an old boys' network, you didn't have to look very hard.
Bob Cahn, the director of the Physics Division at LBL, got on the phone with Kirshner and yelled for a while. He got on the phone with Williams and yelled for a while.
Williams responded calmly, trying to explain his reasoning. HST was an important resource, and the search for high-redshift supernovae was a new field, and HST would surely get better results if both groups used it for their nearly identical experiments.
Cahn replied that he was familiar with important resources. He explained that high-energy physics, too, uses an important resource. He reminded Williams that this important resource was one that LBL had helped invent: the gigantic particle accelerator. But when a group applied for time on a gigantic particle accelerator, the proposal was confidential. Wasn't that how astronomy worked?
Williams conceded that that was indeed how astronomy worked, usually. He suggested a compromise. Both teams would receive Director's Discretionary Time, and the SCP would get to go first.
Cahn and Perlmutter had no choice but to accept. Afterward, though, whenever SCP team members talked among themselves about their rivals, they did so with a new appreciation of just how formidable they were. In the culture of high-energy physics, scientists have to work in large collaborations, and those collaborations have to endure for a long time. You can't afford to alienate your competitors, if only because they'll soon be your collaborators. Astronomers, however, still roamed some Wild West of the mind, where resources were scarce, competition was fierce, and survival depended on small alliances of convenience, often enduring just long enough to publish a paper. Astronomers could afford to play for keeps.
Not that high-energy particle physicists weren't competitive. But in the end they had to get along if they wanted to get work done. They could be tough. But next to astronomers, they were, said one SCP partisan, "pussycats."
By the autumn of 1997, the two teams had enough data to try to find at least a preliminary answer to how much the rate of expansion of the universe was slowing down, and therefore whether the universe was heading toward a Big Crunch or a Big Chill.
As part of the High-z team's fast-and-fair philosophy, Schmidt had divvied up the responsibilities not only institution by institution but junior astronomer by junior astronomer. In this game of tag, the Australian National University Mount Stromlo and Siding Spring Observatories' Schmidt was "it" first. He would write up the paper broadly introducing the collaboration's methods and goals. Then the team tagged Harvard and Peter Garnavich; he would take the three Type Ia supernovae that the team had measured photometrically with HST in the spring of 1997, add 1995K, and try to figure out a value for the Hubble constant. Just as important, the paper would help justify requests for more HST time.
On the SCP side, no one person was working exclusively on the problem. In keeping with the particle physics culture, the team was moving forward collectively. In fact, they'd already moved forward; a year earlier they announced the results from the first seven Type Ia, which suggested that the universe was flat—neither expanding forever nor eventually contracting. But the margins of error and the size of the sample were such that the result was preliminary at best.
Or wrong, as the team was beginning to suspect, on the basis of the one supernova on which they had managed to conduct reliable HST photometry. That one "guy," as astronomers like to call a piece of evidence, was indicating a possible shift in another direction, toward an open universe.
One approach astronomers could use in trying to make this determination was a histogram. On the morning of September 24, Gerson Goldhaber sat at his desk at LBL to prepare for the weekly team meeting. Unlike a graph, which plots each individual point of data, a histogram gathers several pieces of data at a time and "bins" them in categories. That morning, Goldhaber took each of the thirty-eight SCP supernovae so far and, based on its brightness and redshift, put it in a column corresponding to the amount of matter that this one supernova suggested the universe needed in order to slow the expansion to a halt: 0 to 20 percent of the necessary mass density, 20 to 40 percent, and so on, up to 100 percent. When he was done, the two tallest columns by far, one of them bulging with ten supernovae and the other with nine—half the total sample—told him that not only did the universe not have enough matter to slow the expansion to a halt, it had 0 to negative 40 percent.
"Lo and behold," he said to himself.
For the High-z team, Adam Riess was working on a statistical approach to the problem. His task was to take all the supernova data collected so far—all the pixels of spectroscopy and photometry, all the galaxy subtractions, all the light curves, all the margins of error—and develop software that would compare it with millions of different models of the universe. Some of those models would be absurd: relationships between magnitudes and redshifts that, on a sheet of graph paper, would fall far from the straight, 45-degree line, off in remote corners where the punch holes were. Other models, however, would match slight deviations from that seemingly straight line. Within this subset, some models would match even slighter deviations, even subtler departures from the "norm." One of those universes would match his data.
And one did. It was a universe that not only didn't have enough matter to slow the expansion but had a mass density of negative 36 percent. It was a universe without matter. It was a universe that didn't exist.
"Lo and behold," Riess told himself.
Both teams had been operating under the assumption that the universe was full of matter and only matter. They knew some of it was dark, of course, but what was missing was still fundamentally matter. They had therefore assumed that only matter would be influencing the expansion of the universe.
Abandon those assumptions, however, and these seemingly nonsensical results might make sense after all. If the two teams considered a universe in which something else was affecting the expansion—a universe that consisted of something other than matter—then the universe would have matter in it again. They looked at the error bars and figured that the matter, dark or otherwise, was maybe 20 or 30 or 40 percent. Which left 60 or 70 or 80 percent ... something else.
As for the fate of the universe: They had their answer. Maybe even the answer—one they could quantify: It would expand forever.
What they didn't have—between the dark matter they couldn't see and this new force they couldn't imagine—was any idea what the universe was.