Quantum: Einstein, Bohr and the Great Debate About the Nature of Reality - Manjit Kumar (2009)
Part I. THE QUANTUM
'Briefly summarized, what I did can be described as simply an act of desperation.'
—MAX PLANCK
'It was as if the ground had been pulled out from under one, with no firm foundation to be seen anywhere, upon which one could have built.'
— ALBERT EINSTEIN
'For those who are not shocked when they first come across quantum theory cannot possibly have understood it.'
— NIELS BOHR
Chapter 1. THE RELUCTANT REVOLUTIONARY
'A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it', wrote Max Planck towards the end of his long life.1 Bordering on cliché, it could easily have served as his own scientific obituary had he not as an 'act of desperation' abandoned ideas that he had long held dear.2 Wearing his dark suit, starched white shirt and black bow tie, Planck looked the archetypal late nineteenth-century Prussian civil servant but 'for the penetrating eyes under the huge dome of his bald head'.3 In characteristic mandarin fashion he exercised extreme caution before committing himself on matters of science or anything else. 'My maxim is always this,' he once told a student, 'consider every step carefully in advance, but then, if you believe you can take responsibility for it, let nothing stop you.'4Planck was not a man to change his mind easily.
His manner and appearance had hardly changed when to students in the 1920s, as one recalled later, 'it seemed inconceivable that this was the man who had ushered in the revolution'.5 The reluctant revolutionary could scarcely believe it himself. By his own admission he was 'peacefully inclined' and avoided 'all doubtful adventures'.6 He confessed that he lacked 'the capacity to react quickly to intellectual stimulation'.7 It often took him years to reconcile new ideas with his deep-rooted conservatism. Yet at the age of 42, it was Planck who unwittingly started the quantum revolution in December 1900 when he discovered the equation for the distribution of radiation emitted by a blackbody.
All objects, if hot enough, radiate a mixture of heat and light, with the intensity and colour changing with the temperature. The tip of an iron poker left in a fire will start to glow a faint dull red; as its temperature rises it becomes a cherry red, then a bright yellowish-orange, and finally a bluish-white. Once taken out of the fire the poker cools down, running through this spectrum of colours backwards until it is no longer hot enough to emit any visible light. Even then it still gives off an invisible glow of heat radiation. After a time this too stops as the poker continues to cool and finally becomes cold enough to touch.
It was the 23-year-old Isaac Newton who, in 1666, showed that a beam of white light was woven from different threads of coloured light and that passing it through a prism simply unpicked the seven separate strands: red, orange, yellow, green, blue, indigo, and violet.8 Whether red and violet represented the limits of the light spectrum or just those of the human eye was answered in 1800. It was only then, with the advent of sufficiently sensitive and accurate mercury thermometers, that the astronomer William Herschel placed one in front of a spectrum of light and found that as he moved it across the bands of different colours from violet to red, the temperature rose. To his surprise it continued to rise when he accidentally left the thermometer up to an inch past the region of red light. Herschel had detected what was later called infrared radiation, light that was invisible to human eyes from the heat that it generated.9 In 1801, using the fact that silver nitrate darkens when exposed to light, Johann Ritter discovered invisible light at the other end of the spectrum beyond the violet: ultraviolet radiation.
The fact that all heated objects emit light of the same colour at the same temperature was well known to potters long before 1859, the year that Gustav Kirchhoff, a 34-year-old German physicist at Heidelberg University, started his theoretical investigations into the nature of this correlation. To help simplify his analysis, Kirchhoff developed the concept of a perfect absorber and emitter of radiation called a blackbody. His choice of name was apt. A body that was a perfect absorber would reflect no radiation and therefore appear black. However, as a perfect emitter its appearance would be anything but black if its temperature was high enough for it to radiate at wavelengths from the visible part of the spectrum.
Kirchhoff envisaged his imaginary blackbody as a simple hollow container with a tiny hole in one of its walls. Since any radiation, visible or invisible light, entering the container does so through the hole, it is actually the hole that mimics a perfect absorber and acts like a blackbody. Once inside, the radiation is reflected back and forth between the walls of the cavity until it is completely absorbed. Imagining the outside of his blackbody to be insulated, Kirchhoff knew that if heated, only the interior surface of the walls would emit radiation that filled the cavity.
At first the walls, just like a hot iron poker, glow a deep cherry-red even though they still radiate predominantly in the infrared. Then, as the temperature climbs ever higher, the walls glow a bluish-white as they radiate at wavelengths from across the spectrum from the far infrared to the ultraviolet. The hole acts as a perfect emitter since any radiation that escapes through it will be a sample of all the wavelengths present inside the cavity at that temperature.
Kirchhoff proved mathematically what potters had long observed in their kilns. Kirchhoff's law said that the range and intensity of the radiation inside the cavity did not depend on the material that a real blackbody could be made of, or on its shape and size, but only on its temperature. Kirchhoff had ingeniously reduced the problem of the hot iron poker: what was the exact relationship between the range and intensity of the colours it emitted at a certain temperature to how much energy is radiated by a blackbody at that temperature? The task that Kirchhoff set himself and his colleagues became known as the blackbody problem: measure the spectral energy distribution of blackbody radiation, the amount of energy at each wavelength from the infrared to the ultraviolet, at a given temperature and derive a formula to reproduce the distribution at any temperature.
Unable to go further theoretically without experiments with a real blackbody to guide him, Kirchhoff nevertheless pointed physicists in the right direction. He told them that the distribution being independent of the material from which a blackbody was made meant that the formula should contain only two variables: the temperature of the blackbody and the wavelength of the emitted radiation. Since light was thought to be a wave, any particular colour and hue was distinguished from every other by its defining characteristic: its wavelength, the distance between two successive peaks or troughs of the wave. Inversely proportional to the wavelength is the frequency of the wave - the number of peaks, or troughs, that pass a fixed point in one second. The longer the wavelength, the lower the frequency and vice versa. But there was also a different but equivalent way of measuring the frequency of a wave: the number of times it jiggled up and down, 'waved', per second.10
Figure 1: The characteristics of a wave
The technical obstacles in constructing a real blackbody and the precision instruments needed to detect and measure the radiation ensured that no significant progress was made for almost 40 years. It was in the 1880s, when German companies tried to develop more efficient light bulbs and lamps than their American and British rivals, that measuring the blackbody spectrum and finding Kirchhoff's fabled equation became a priority.
The incandescent light bulb was the latest in a series of inventions, including the arc lamp, dynamo, electric motor, and telegraphy, fuelling the rapid expansion of the electrical industry. With each innovation the need for a globally agreed set of units and standards of electrical measurement became increasingly urgent.
Two hundred and fifty delegates from 22 countries gathered in Paris, in 1881, for the first International Conference for the Determination of Electrical Units. Although the volt, amp and other units were defined and named, no agreement was reached on a standard for luminosity and it began to hamper the development of the most energy-efficient means of producing artificial light. As a perfect emitter at any given temperature, a blackbody emits the maximum amount of heat, infrared radiation. The blackbody spectrum would serve as a benchmark in calibrating and producing a bulb that emitted as much light as possible while keeping the heat it generated to a minimum.
'In the competition between nations, presently waged so actively, the country that first sets foot on new paths and first develops them into established branches of industry has a decisive upper hand', wrote the industrialist and inventor of the electrical dynamo, Werner von Siemens.11 Determined to be first, in 1887 the German government founded the Physikalisch-Technische Reichsanstalt (PTR), the Imperial Institute of Physics and Technology. Located on the outskirts of Berlin in Charlottenburg, on land donated by Siemens, the PTR was conceived as an institute fit for an empire determined to challenge Britain and America. The construction of the entire complex lasted more than a decade, as the PTR became the best-equipped and most expensive research facility in the world. Its mission was to give Germany the edge in the appliance of science by developing standards and testing new products. Among its list of priorities was to devise an internationally recognised unit of luminosity. The need to make a better light bulb was the driving force behind the PTR blackbody research programme in the 1890s. It would lead to the accidental discovery of the quantum as Planck turned out to be the right man, in the right place, at the right time.
Max Karl Ernst Ludwig Planck was born in Kiel, then a part of Danish Holstein, on 23 April 1858 into a family devoted to the service of Church and State. Excellence in scholarship was almost his birthright. Both his paternal great-grandfather and grandfather had been distinguished theologians, while his father became professor of constitutional law at Munich University. Venerating the laws of God and Man, these duty-bound men of probity were also steadfast and patriotic. Max was to be no exception.
Planck attended the most renowned secondary school in Munich, the Maximilian Gymnasium. Always near the top of his class, but never first, he excelled through hard work and self-discipline. These were just the qualities demanded by an educational system with a curriculum founded on the retention of enormous quantities of factual knowledge through rote learning. A school report noted that 'despite all his childishness' Planck at ten already possessed 'a very clear, logical mind' and promised 'to be something right'.12 By the time he was sixteen it was not Munich's famous taverns but its opera houses and concert halls that attracted the young Planck. A talented pianist, he toyed with the idea of pursuing a career as a professional musician. Unsure, he sought advice and was bluntly told: 'If you have to ask, you'd better study something else!'13
In October 1874, aged sixteen, Planck enrolled at Munich University and opted to study physics because of a burgeoning desire to understand the workings of nature. In contrast to the near-militaristic regime of the Gymnasiums, German universities allowed their students almost total freedom. With hardly any academic supervision and no fixed requirements, it was a system that enabled students to move from one university to another, taking courses as they pleased. Sooner or later those wishing to pursue an academic career took the courses by the pre-eminent professors at the most prestigious universities. After three years at Munich, where he was told 'it is hardly worth entering physics anymore' because there was nothing important left to discover, Planck moved to the leading university in the German-speaking world, Berlin.14
With the creation of a unified Germany in the wake of the Prussian-led victory over France in the war of 1870-71, Berlin became the capital of a mighty new European nation. Situated at the confluence of the Havel and the Spree rivers, French war reparations allowed its rapid redevelopment as it sought to make itself the equal of London and Paris. A population of 865,000 in 1871 swelled to nearly 2 million by 1900, making Berlin the third-largest city in Europe.15 Among the new arrivals were Jews fleeing persecution in Eastern Europe, especially the pogroms in Tsarist Russia. Inevitably the cost of housing and living soared, leaving many homeless and destitute. Manufacturers of cardboard boxes advertised 'good and cheap boxes for habitation' as shanty towns sprung up in parts of the city.16
Despite the bleak reality that many found on arriving in Berlin, Germany was entering a period of unprecedented industrial growth, technological progress, and economic prosperity. Driven largely by the abolition of internal tariffs after unification and French war compensation, by the outbreak of the First World War Germany's industrial output and economic power would be second only to the United States. By then it was producing over two-thirds of continental Europe's steel, half its coal, and was generating more electricity than Britain, France and Italy combined. Even the recession and anxiety that affected Europe after the stock market crash of 1873 only slowed the pace of German development for a few years.
With unification came the desire to ensure that Berlin, the epitome of the new Reich, had a university second to none. Germany's most renowned physicist, Herman von Helmholtz, was enticed from Heidelberg. A trained surgeon, Helmholtz was also a celebrated physiologist who had made fundamental contributions to understanding the workings of the human eye after his invention of the ophthalmoscope. The 50-year-old polymath knew his worth. Apart from a salary several times the norm, Helmholtz demanded a magnificent new physics institute. It was still being built in 1877 when Planck arrived in Berlin and began attending lectures in the university's main building, a former palace on Unter den Linden opposite the Opera House.
As a teacher, Helmholtz was a severe disappointment. 'It was obvious,' Planck said later, 'that Helmholtz never prepared his lectures properly.'17 Gustav Kirchhoff, who had also transferred from Heidelberg to become the professor of theoretical physics, was so well prepared that he delivered his lectures 'like a memorized text, dry and monotonous'.18 Expecting to be inspired, Planck admitted 'that the lectures of these men netted me no perceptible gain'.19Seeking to quench his 'thirst for advanced scientific knowledge', he stumbled across the work of Rudolf Clausius, a 56-year-old German physicist at Bonn University.20
In stark contrast to the lacklustre teaching of his two esteemed professors, Planck was immediately enthralled by Clausius' 'lucid style and enlightening clarity of reasoning'.21 His enthusiasm for physics returned as he read Clausius' papers on thermodynamics. Dealing with heat and its relationship to different forms of energy, the fundamentals of thermodynamics were at the time encapsulated in just two laws.22 The first was a rigorous formulation of the fact that energy, in whatever guise, possessed the special property of being conserved. Energy could neither be created nor destroyed but only converted from one form to another. An apple hanging from a tree possesses potential energy by virtue of its position in the earth's gravitational field, its height above the ground. When it falls, the apple's potential energy is converted into kinetic energy, the energy of motion.
Planck was a schoolboy when he first encountered the law of the conservation of energy. It struck him 'like a revelation' he said later, because it possessed 'absolute, universal validity, independently from all human agency'.23 It was the moment he caught a glimpse of the eternal, and from then on he considered the search for absolute or fundamental laws of nature 'as the most sublime scientific pursuit in life'.24 Now Planck was just as spellbound reading Clausius' formulation of the second law of thermodynamics: 'Heat will not pass spontaneously from a colder to a hotter body.'25 The later invention of the refrigerator illustrated what Clausius meant by 'spontaneously'. A refrigerator needed to be plugged into an external supply of energy, in this case electrical, so that heat could be made to flow from a colder to a hotter body.
Planck understood that Clausius was not simply stating the obvious, but something of deep significance. Heat, the transfer of energy from A to B due to a temperature difference, explained such everyday occurrences as a hot cup of coffee getting cold and an ice cube in a glass of water melting. But left undisturbed, the reverse never happened. Why not? The law of conservation of energy did not forbid a cup of coffee from getting hotter and the surrounding air colder, or the glass of water becoming warmer and the ice cooler. It did not outlaw heat flowing from a cold to a hot body spontaneously. Yet something was preventing this from happening. Clausius discovered that something and called it entropy. It lay at the heart of why some processes occur in nature and others do not.
When a hot cup of coffee cools down, the surrounding air gets warmer as energy is dissipated and irretrievably lost, ensuring that the reverse cannot happen. If the conservation of energy was nature's way of balancing the books in any possible physical transaction, then nature also demanded a price for every transaction that actually occurred. According to Clausius, entropy was the price for whether something happened or not. In any isolated system only those processes, transactions, in which entropy either stayed the same or increased were allowed. Any that led to a decrease of entropy were strictly forbidden.
Clausius defined entropy as the amount of heat in or out of a body or a system divided by the temperature at which it takes place. If a hot body at 500 degrees loses 1000 units of energy to a colder body at 250 degrees, then its entropy has decreased by -1000/500 = -2. The colder body at 250 degrees has gained 1000 units of energy, +1000/250, and its entropy has increased by 4. The overall entropy of the system, the hot and cold bodies combined, has increased by 2 units of energy per degree. All real, actual processes are irreversible because they result in an increase in entropy. It is nature's way of stopping heat from passing spontaneously, of its own accord, from something cold to something hot. Only ideal processes in which entropy remains unchanged can be reversed. They, however, never occur in practice, only in the mind of the physicist. The entropy of the universe tends towards a maximum.
Alongside energy, Planck believed that entropy was 'the most important property of physical systems'.26 Returning to Munich University after his year-long sojourn in Berlin, he devoted his doctoral thesis to an explora-tion of the concept of irreversibility. It would be his calling card. To his dismay, he 'found no interest, let alone approval, even among the very physicists who were closely concerned with the topic'.27 Helmholtz did not read it; Kirchhoff did, but disagreed with it. Clausius, who had such a profound influence on him, did not even answer his letter. 'The effect of my dissertation on the physicists of those days was nil', Planck recalled with some bitterness even 70 years later. But driven by 'an inner compulsion', he was undeterred.28 Thermodynamics, particularly the second law, became the focus of Planck's research as he began his academic career.29
German universities were state institutions. Extraordinary (assistant) and ordinary (full) professors were civil servants appointed and employed by the ministry of education. In 1880 Planck became a privatdozent, an unpaid lecturer, at Munich University. Employed neither by the state nor the university, he had simply gained the right to teach in exchange for fees paid by students attending his courses. Five years passed as he waited in vain for an appointment as an extraordinary professor. As a theorist uninterested in conducting experiments, Planck's chances for promotion were slim, as theoretical physics was not yet a firmly established distinct discipline. Even in 1900 there were only sixteen professors of theoretical physics in Germany.
If his career was to progress, Planck knew that he had 'to win, somehow, a reputation in the field of science'.30 His chance came when Göttingen University announced that the subject for its prestigious essay competition was 'The Nature of Energy'. As he worked on his paper, in May 1885, 'a message of deliverance' arrived.31 Planck, aged 27, was offered an extraordinary professorship at the University of Kiel. He suspected it was his father's friendship with Kiel's head of physics that had led to the offer. Planck knew there were others, more established than he, who would have expected advancement. Nevertheless, he accepted and finished his entry for the Göttingen competition shortly after arriving in the city of his birth.
Even though only three papers were submitted in search of the prize, an astonishing two years passed before it was announced that there would be no winner. Planck was awarded second place and denied the prize by the judges because of his support for Helmholtz in a scientific dispute with a member of the Göttingen faculty. The behaviour of the judges drew the attention of Helmholtz to Planck and his work. After a little more than three years at Kiel, in November 1888, Planck received an unexpected honour. He had not been first, or even second choice. But after others had turned it down, Planck, with Helmholtz's backing, was asked to succeed Gustav Kirchhoff at Berlin University as professor of theoretical physics.
In the spring of 1889, the capital was not the city Planck had left eleven years earlier. The stench that always shocked visitors had disappeared as a new sewer system replaced the old open drains, and at night the main streets were lit by modern electric lamps. Helmholtz was no longer head of the university's physics institute but running the PTR, the majestic new research facility three miles away. August Kundt, his successor, had played no part in Planck's appointment, but welcomed him as 'an excellent acquisition' and 'a splendid man'.32
In 1894 Helmholtz, aged 73, and Kundt, only 55, both died within months of each other. Planck, only two years after finally being promoted to the rank of ordinary professor, found himself as the senior physicist at Germany's foremost university at just 36. He had no choice but to bear the weight of added responsibilities, including that of adviser on theoretical physics for Annalen der Physik. It was a position of immense influence that gave him the right of veto on all theoretical papers submitted to the premier German physics journal. Feeling the pressure of his newly elevated position and a deep sense of loss at the deaths of his two colleagues, Planck sought solace in his work.
As a leading member of Berlin's close-knit community of physicists, he was well aware of the ongoing, industry-driven blackbody research programme of the PTR. Although thermodynamics was central to a theoretical analysis of the light and heat radiated by a blackbody, the lack of reliable experimental data had stopped Planck from trying to derive the exact form of Kirchhoff's unknown equation. Then a breakthrough by an old friend at PTR meant that he could no longer avoid the blackbody problem.
In February 1893, 29-year-old Wilhelm Wien discovered a simple mathematical relationship that described the effect of a change in temperature on the distribution of blackbody radiation.33 Wien found that as the temperature of a blackbody increases, the wavelength at which it emits radiation with the greatest intensity becomes ever shorter.34 It was already known that the rise in temperature would result in an increase in the total amount of energy radiated, but Wien's 'displacement law' revealed something very precise: the wavelength at which the maximum amount of radiation is emitted multiplied by the temperature of a blackbody is always a constant. If the temperature is doubled, then the 'peak' wavelength will be half the previous length.
Figure 2: Distribution of blackbody radiation which shows Wien's displacement law
Wien's discovery meant that once the numerical constant was calculated by measuring the peak wavelength - the wavelength that radiates most strongly, at a certain temperature - then the peak wavelength could be calculated for any temperature.35 It also explained the changing colours of a hot iron poker. Starting at low temperatures, the poker emits predominantly long-wavelength radiation from the infrared part of the spectrum. As the temperature increases, more energy is radiated in each region and the peak wavelength decreases. It is 'displaced' towards the shorter wavelengths. Consequently the colour of the emitted light changes from red to orange, then yellow and finally a bluish-white as the quantity of radiation from the ultraviolet end of the spectrum increases.
Wien had quickly established himself as a member of that endangered breed of physicist, one who was both an accomplished theorist and a skilled experimenter. He found the displacement law in his spare time and was forced to publish it as a 'private communication' without the imprimatur of the PTR. At the time he was working as an assistant in the PTR's optics laboratory under the leadership of Otto Lummer. Wien's day job was the practical work that was a prerequisite for an experimental investigation of blackbody radiation.
Their first task was to construct a better photometer, an instrument capable of comparing the intensity of light - the amount of energy in a given wavelength range - from different sources such as gas lamps and electric bulbs. It was the autumn of 1895 before Lummer and Wien devised a new and improved hollow blackbody capable of being heated to a uniform temperature.
While he and Lummer developed their new blackbody during the day, Wien continued to spend his evenings searching for Kirchhoff's equation for distribution of blackbody radiation. In 1896, Wien found a formula that Friedrich Paschen, at the University of Hanover, quickly confirmed agreed with the data he had collected on the allocation of energy among the short wavelengths of blackbody radiation.
In June that year, the very month the 'distribution law' appeared in print, Wien left the PTR for an extraordinary professorship at the Technische Hochschule in Aachen. He would win the Nobel Prize for physics in 1911 for his work on blackbody radiation, but left Lummer to put his distribution law through a rigorous test. To do so required measurements over a greater range and at higher temperatures than ever before. Working with Ferdinand Kurlbaum and then Ernst Pringsheim, it took Lummer two long years of refinements and modifications but in 1898 he had a state-of-the-art electrically heated blackbody. Capable of reaching temperatures as high as 1500°C, it was the culmination of more than a decade of painstaking work at the PTR.
Plotting the intensity of radiation along the vertical axis of a graph against the wavelength of the radiation along the horizontal axis, Lummer and Pringsheim found that the intensity rose as the wavelength of radiation increased until it peaked and then began to drop. The spectral energy distribution of blackbody radiation was almost a bell-shaped curve, resembling a shark's dorsal fin. The higher the temperature, the more pronounced the shape as the intensity of radiation emitted increased. Taking readings and plotting curves with the blackbody heated to different temperatures showed that the peak wavelength that radiated with maximum intensity was displaced towards the ultraviolet end of the spectrum with increasing temperature.
Lummer and Pringsheim reported their results at a meeting of the German Physical Society held in Berlin on 3 February 1899.36 Lummer told the assembled physicists, among them Planck, that their findings confirmed Wien's displacement law. However, the situation regarding the distribution law was unclear. Although the data was in broad agreement with Wien's theoretical predictions, there were some discrepancies in the infrared region of the spectrum.37 In all likelihood these were due to experimental errors, but it was an issue, they argued, that could be settled only once 'other experiments spread over a greater range of wavelengths and over a greater interval of temperature can be arranged'.38
Within three months Friedrich Paschen announced that his measurements, though conducted at a lower temperature than those of Lummer and Pringsheim, were in complete harmony with the predictions of Wien's distribution law. Planck breathed a sigh of relief and read out Paschen's paper at a session of the Prussian Academy of Sciences. Such a law appealed deeply to him. For Planck the theoretical quest for the spectral energy distribution of blackbody radiation was nothing less than the search for the absolute, and 'since I had always regarded the search for the absolute as the loftiest goal of all scientific activity, I eagerly set to work'.39
Soon after Wien published his distribution law, in 1896, Planck set about trying to place the law on rock-solid foundations by deriving it from first principles. Three years later, in May 1899, he thought he had succeeded by using the power and authority of the second law of thermodynamics. Others agreed and started calling Wien's law by a new name, Wien-Planck, despite the claims and counter-claims of the experimentalists. Planck remained confident enough to assert that 'the limits of validity of this law, in case there are any at all, coincide with those of the second fundamental law of the theory of heat'.40 He advocated further testing of the distribution law as a matter of urgency, since for him it would be a simultaneous examination of the second law. He got his wish.
At the beginning of November 1899, after spending nine months extending the range of their measurements as they eliminated possible sources of experimental error, Lummer and Pringsheim reported that they had found 'discrepancies of a systematic nature between theory and experiment'.41 Although in perfect agreement for short wavelengths, they discovered that Wien's law consistently overestimated the intensity of radiation at long wavelengths. However, within weeks Paschen contradicted Lummer and Pringsheim. He presented another set of new data and claimed that the distribution law 'appears to be a rigorously valid law of nature'.42
With most of the leading experts living and working in Berlin, the meetings of the German Physical Society held in the capital became the main forum for discussions concerning blackbody radiation and the status of Wien's law. It was the subject that again dominated the proceedings of the society at its fortnightly meeting on 2 February 1900 when Lummer and Pringsheim disclosed their latest measurements. They had found systematic discrepancies between their measurements and the predictions of Wien's law in the infrared region that could not be the result of experimental error.
This breakdown of Wien's law led to a scramble to find a replacement. But these makeshift alternatives proved unsatisfactory, prompting calls for further testing at even longer wavelengths to unequivocally establish the extent of any failure of Wien's law. It did, after all, agree with the available data covering the shorter wavelengths, and all other experiments bar those of Lummer-Pringsheim had found in its favour.
As Planck was only too well aware, any theory is at the mercy of hard experimental facts, but he strongly believed that 'a conflict between observation and theory can only be confirmed as valid beyond all doubt if the figures of various observers substantially agree with each other'.43 Nevertheless, the disagreement between the experimentalists forced him to reconsider the soundness of his ideas. In late September 1900, as he continued to review his derivation, the failure of Wien's law in the deep infrared was confirmed.
The question was finally settled by Heinrich Rubens, a close friend of Planck's, and Ferdinand Kurlbaum. Based at the Technische Hochschule on Berlinerstrasse, where at the age of 35 he had recently been promoted to ordinary professor, Rubens spent most of his time as a guest worker at the nearby PTR. It was there, with Kurlbaum, that he built a blackbody that allowed measurements of the uncharted territory deep within the infrared region of the spectrum. During the summer they tested Wien's law between wavelengths of 0.03mm and 0.06mm at temperatures ranging from 200 to 1500°C. At these longer wavelengths, they found the difference between theory and observation was so marked that it could be evidence of only one thing, the breakdown of Wien's law.
Rubens and Kurlbaum wanted to announce their results in a paper to the German Physical Society. The next meeting was on Friday, 5 October. With little time to write a paper, they decided to wait until the following meeting two weeks later. In the meantime, Rubens knew that Planck would be eager to hear the latest results.
It was among the elegant villas of bankers, lawyers, and other professors in the affluent suburb of Grunewald in west Berlin that Planck lived for 50years in a large house with an enormous garden. On Sunday, 7 October, Rubens and his wife came for lunch. Inevitably the talk between the two friends soon turned to physics and the blackbody problem. Rubens explained that his latest measurements left no room for doubt: Wien's law failed at long wavelengths and high temperatures. Those measurements, Planck learnt, revealed that at such wavelengths the intensity of blackbody radiation was proportional to the temperature.
That evening Planck decided to have a go at constructing the formula that would reproduce the energy spectrum of blackbody radiation. He now had three crucial pieces of information to help him. First, Wien's law accounted for the intensity of radiation at short wavelengths. Second, it failed in the infrared where Rubens and Kurlbaum had found that intensity was proportional to the temperature. Third, Wien's displacement law was correct. Planck had to find a way to assemble these three pieces of the blackbody jigsaw together to build the formula. His years of hard-won experience were quickly put into practice as he set about manipulating the various mathematical symbols of the equations at his disposal.
After a few unsuccessful attempts, through a combination of inspired scientific guesswork and intuition, Planck had a formula. It looked promising. But was it Kirchhoff's long-sought-after equation? Was it valid at any given temperature for the entire spectrum? Planck hurriedly penned a note to Rubens and went out in the middle of the night to post it. After a couple of days, Rubens arrived at Planck's home with the answer. He had checked Planck's formula against the data and found an almost perfect match.
On Friday, 19 October at the meeting of the German Physical Society, with Rubens and Planck sitting among the audience, it was Ferdinand Kurlbaum who made the formal announcement that Wien's law was valid only at short wavelengths and failed at the longer wavelengths of the infrared. After Kurlbaum sat down, Planck rose to deliver a short 'comment' billed as 'An Improvement of Wien's Equation for the Spectrum'. He began by admitting that he had believed 'Wien's law must necessarily be true', and had said so at a previous meeting.44 As he continued, it quickly became clear that Planck was not simply proposing 'an improvement', some minor tinkering with Wien's law, but a completely new law of his own.
After speaking for less than ten minutes, Planck wrote his equation for the blackbody spectrum on the blackboard. Turning around to look at the familiar faces of his colleagues, he told them that this equation 'as far as I can see at the moment, fits the observational data, published up to now'.45 As he sat down, Planck received polite nods of approval. The muted response was understandable. After all, what Planck had just proposed was another ad hoc formula manufactured to explain the experimental results. There were others who had already put forward equations of their own in the hope of filling the void, should the suspected failure of Wien's law at long wavelengths be confirmed.
The next day Rubens visited Planck to reassure him. 'He came to tell me that after the conclusion of the meeting he had that very night checked my formula against the results of his measurements,' Planck remembered, 'and found satisfactory concordance at every point.'46 Less than a week later, Rubens and Kurlbaum announced that they had compared their measurements with the predictions of five different formulae and found Planck's to be much more accurate than any of the others. Paschen too confirmed that Planck's equation matched his data. Yet despite this rapid corroboration by the experimentalists of the superiority of his formula, Planck was troubled.
He had his formula, but what did it mean? What was the underlying physics? Without an answer, Planck knew that it would, at best, be just an 'improvement' on Wien's law and have 'merely the standing of a law disclosed by a lucky intuition' that possessed no more 'than a formal significance'.47 'For this reason, on the very first day when I formulated this law,' Planck said later, 'I began to devote myself to the task of investing it with true physical meaning.'48 He could achieve this only by deriving his equation step by step using the principles of physics. Planck knew his destination, but he had to find a way of getting there. He possessed a priceless guide, the equation itself. But what price was he prepared to pay for such a journey?
The next six weeks were, Planck recalled, 'the most strenuous work of my life', after which 'the darkness lifted and an unexpected vista began to appear'.49 On 13 November he wrote to Wien: 'My new formula is well satisfied; I now have also obtained a theory for it, which I shall present in four weeks at the Physical Society here [in Berlin].'50 Planck said nothing to Wien either of the intense intellectual struggle that had led to his theory or the theory itself. He had strived long and hard during those weeks to reconcile his equation with the two grand theories of nineteenth-century physics: thermodynamics and electromagnetism. He failed.
'A theoretical interpretation therefore had to be found at any cost,' he accepted, 'no matter how high.'51 He 'was ready to sacrifice every one of my previous convictions about physical laws'.52 Planck no longer cared what it cost him, as long as he could 'bring about a positive result'.53 For such an emotionally restrained man, who only truly expressed himself freely at the piano, this was highly charged language. Pushed to the limit in the struggle to understand his new formula, Planck was forced into 'an act of desperation' that led to the discovery of the quantum.54
As the walls of a blackbody are heated they emit infrared, visible, and ultraviolet radiation into the heart of the cavity. In his search for a theoretically consistent derivation of his law, Planck had to come up with a physical model that reproduced the spectral energy distribution of blackbody radiation. He had already been toying with an idea. It did not matter if the model failed to capture what was really going on; all Planck needed was a way of getting the right mix of frequencies, and therefore wavelengths, of the radiation present inside the cavity. He used the fact that this distribution depends only on the temperature of the blackbody and not on the material from which it is made to conjure up the simplest model he could.
'Despite the great success that the atomic theory has so far enjoyed,' Planck wrote in 1882, 'ultimately it will have to be abandoned in favour of the assumption of continuous matter.'55 Eighteen years later, in the absence of indisputable proof of their existence, he still did not believe in atoms. Planck knew from the theory of electromagnetism that an electric charge oscillating at a certain frequency emits and absorbs radiation only of that frequency. He therefore chose to represent the walls of the blackbody as an enormous array of oscillators. Although each oscillator emits only a single frequency, collectively they emit the entire range of frequencies found within the blackbody.
A pendulum is an oscillator and its frequency is the number of swings per second, a single oscillation being one complete to and fro swing that returns the pendulum to its starting point. Another oscillator is a weight hanging from a spring. Its frequency is the number of times per second the weight bounces up and down after being pulled from its stationary position and released. The physics of such oscillations had long been understood and given the name 'simple harmonic motion' by the time Planck used oscillators, as he called them, in his theoretical model.
Planck envisaged his collection of oscillators as massless springs of varying stiffness, so as to reproduce the different frequencies, each with an electric charge attached. Heating the walls of the blackbody provided the energy needed to set the oscillators in motion. Whether an oscillator was active or not would depend only upon the temperature. If it were, then it would emit radiation into, and absorb radiation from, the cavity. In time, if the temperature is held constant, this dynamic give and take of radiation energy between the oscillators and the radiation in the cavity comes into balance and a state of thermal equilibrium is achieved.
Since the spectral energy distribution of blackbody radiation represents how the total energy is shared among the different frequencies, Planck assumed that it was the number of oscillators at each given frequency that determined the allocation. After setting up his hypothetical model, he had to devise a way to share out the available energy among the oscillators. In the weeks following its announcement, Planck discovered the hard way that he could not derive his formula using physics that he had long accepted as dogma. In desperation he turned to the ideas of an Austrian physicist, Ludwig Boltzmann, who was the foremost advocate of the atom. On the road to his blackbody formula, Planck became a convert as he accepted that atoms were more than just a convenient fiction, after years of being openly 'hostile to the atomic theory'.56
The son of a tax collector, Ludwig Boltzmann was short and stout with an impressive late nineteenth-century beard. Born in Vienna on 20February 1844, he was, for a while, taught the piano by the composer Anton Bruckner. A better physicist than a pianist, Boltzmann obtained his doctorate from the University of Vienna in 1866. He quickly made his reputation with fundamental contributions to the kinetic theory of gases, so called because its proponents believed that gases were made up of atoms or molecules in a state of continual motion. Later, in 1884, Boltzmann provided the theoretical justification for the discovery by Josef Stefan, his former mentor, that the total energy radiated by a blackbody is proportional to the temperature raised to the fourth power, T4 or T×T×T×T. It meant that doubling the temperature of a blackbody increased the energy it radiated by a factor of sixteen.
Boltzmann was a renowned teacher and, although a theorist, a very capable experimentalist despite being severely shortsighted. Whenever a vacancy arose at one of Europe's leading universities his name was usually on the list of potential candidates. It was only after he turned down the professorship at Berlin University left vacant by the death of Gustav Kirchhoff that a downgraded version was offered to Planck. By 1900 a much-travelled Boltzmann was at Leipzig University and universally acknowledged as one the great theoreticians. Yet there were many, like Planck, who found his approach to thermodynamics unacceptable.
Boltzmann believed that properties of gases, such as pressure, were the macroscopic manifestations of microscopic phenomena regulated by the laws of mechanics and probability. For those whose believed in atoms, the classical physics of Newton governed the movement of each gas molecule, but using Newtonian laws of motion to determine that of each of the countless molecules of a gas was for all practical purposes impossible. It was the 28-year-old Scottish physicist James Clerk Maxwell who, in 1860, captured the motion of gas molecules without measuring the velocity of a single one. Using statistics and probability, Maxwell worked out the most likely distribution of velocities as the gas molecules underwent incessant collisions with each other and the walls of a container. The introduction of statistics and probability was bold and innovative; it allowed Maxwell to explain many of the observed properties of gases. Thirteen years younger, Boltzmann followed in Maxwell's footsteps to help shore up the kinetic theory of gases. In the 1870s he went one step further and developed a statistical interpretation of the second law of thermodynamics by linking entropy with disorder.
According to what became known as Boltzmann's principle, entropy is a measure of the probability of finding a system in a particular state. A well-shuffled pack of playing cards, for example, is a disordered system with high entropy. However, a brand-new deck with cards arranged according to suit and from ace to king is a highly ordered system with low entropy. For Boltzmann the second law of thermodynamics concerns the evolution of a system with a low probability, and therefore low entropy, into a state of higher probability and high entropy. The second law is not an absolute law. It is possible for a system to go from a disordered state to a more ordered one, just as a shuffled pack of cards may, if shuffled again, become ordered. However, the odds against that happening are so astronomical that it would require many times the age of the universe to pass for it to occur.
Planck believed that the second law of thermodynamics was absolute - entropy always increases. In Boltzmann's statistical interpretation, entropy nearly always increases. There was a world of difference between these two views as far as Planck was concerned. For him to turn to Boltzmann was a renunciation of everything that he held dear as a physicist, but he had no choice in his quest to derive his blackbody formula. 'Until then I had paid no attention to the relationship between entropy and probability, in which I had little interest since every probability law permits exceptions; and at that time I assumed that the second law of thermodynamics was valid without exceptions.'57
A state of maximum entropy, maximum disorder, is the most probable state for a system. For a blackbody that state is thermal equilibrium - just the situation that Planck faced as he tried to find the most probable dis-tribution of energy among his oscillators. If there are 1000 oscillators in total and ten have a frequency v, it is these oscillators that determine the intensity of radiation emitted at that frequency. While the frequency of any one of Planck's electric oscillators is fixed, the amount of energy it emits and absorbs depends solely upon its amplitude, the size of its oscillation. A pendulum completing five swings in five seconds has a frequency of one oscillation per second. However, if it swings through a wide arc the pendulum has more energy than if it traces out a smaller one. The frequency remains unchanged because the length of the pendulum fixes it, but the extra energy allows it to travel faster through a wide arc. The pendulum therefore completes the same number of oscillations in the same time as an identical pendulum swinging through a narrower arc.
Applying Boltzmann's techniques, Planck discovered that he could derive his formula for the distribution of blackbody radiation only if the oscillators absorbed and emitted packets of energy that were proportional to their frequency of oscillation. It was the 'most essential point of the whole calculation', said Planck, to consider the energy at each frequency as being composed of a number of equal, indivisible 'energy elements' that he later called quanta.58
Guided by his formula, Planck had been forced into slicing up energy (E) into hv-sized chunks, where v is the frequency of the oscillator and h is a constant. E=hv would become one of the most famous equations in the whole of science. If, for example, the frequency was 20 and h was 2, then each quantum of energy would have a magnitude of 20×2=40. If the total energy available at this frequency were 3600, then there would be 3600/40=90 quanta to be distributed among the ten oscillators of that frequency. Planck learnt from Boltzmann how to determine the most probable distribution of these quanta among the oscillators.
He found that his oscillators could only have energies: o, hv, 2hv, 3hv, 4hv … all the way up to nhv, where n is a whole number. This corresponded to either absorbing or emitting a whole number of 'energy elements' or 'quanta' of size hv. It was like a bank cashier able to receive and dispense money only in denominations of £1, £2, £5, £10, £20 and £50. Since Planck's oscillators cannot have any other energy, the amplitude of their oscillations is constrained. The strange implications of this are manifest if scaled up to the everyday world of a spring with a weight attached.
If the weight oscillates with an amplitude of 1cm, then it has an energy of 1 (ignoring the units of measuring energy). If the weight is pulled down to 2cm and allowed to oscillate, its frequency remains the same as before. However its energy, which is proportional to the square of the amplitude, is now 4. If the restriction on Planck's oscillators applied to the weight, then between 1cm and 2cm it can oscillate only with amplitudes of 1.42cm and 1.73cm, because they have energies of 2 and 3.59 It cannot, for example, oscillate with an amplitude of 1.5cm because the associated energy would be 2.25. A quantum of energy is indivisible. An oscillator cannot receive a fraction of a quantum of energy; it must be all or nothing. This ran counter to the physics of the day. It placed no restrictions on the size of oscillation and therefore on how much energy an oscillator can emit or absorb in a single transaction - it could have any amount.
In his desperation Planck had discovered something so remarkable and unexpected that he failed to grasp its significance. It is not possible for his oscillators to absorb or emit energy continuously like water from a tap. Instead they can only gain and lose energy discontinuously, in small, indivisible units of E=hv, where v is the frequency with which the oscillator vibrates that exactly matches the frequency of the radiation it can absorb or emit.
The reason why large-scale oscillators are not seen to behave like Planck's atomic-sized ones is because h is equal to 0.000000000000000000000000006626 erg seconds or 6.626 divided by one thousand trillion trillion. According to Planck's formula, there could be no smaller step than h in the increase or decrease of energy, but the infinitesimal size of h makes quantum effects invisible in the world of the everyday when it comes to pendulums, children's swings and vibrating weights.
Planck's oscillators forced him to slice and dice radiation energy so as to feed them the correct bite-sized chunks of hv. He did not believe that the energy of radiation was really chopped up into quanta. It was just the way his oscillators could receive and emit energy. The problem for Planck was that Boltzmann's procedure for slicing energy required that at the end the slices be made ever thinner until mathematically their thickness was zero and they vanished, with the whole being restored. To reunite a sliced-up quantity in such a fashion was a mathematical technique at the very heart of calculus. Unfortunately for Planck, if he did the same his formula vanished too. He was stuck with quanta, but was unconcerned. He had his formula; the rest could be sorted out later.
'Gentlemen!' said Planck as he faced the members of the German Physical Society seated in the room at Berlin University's Physics Institute. He could see Rubens, Lummer and Pringsheim among them as he began his lecture, 'Zur Theorie des Gesetzes der Energieverteilung im Normalspektrum', On the Theory of the Energy Distribution Law of the Normal Spectrum. It was just after 5pm on Friday, 14 December 1900. 'Several weeks ago I had the honour of directing your attention to a new equation that seemed suitable to me for expressing the law of the distribution of radiating energy over all areas of the normal spectrum.'60 Planck now presented the physics behind that new equation as he derived it.
At the end of the meeting his colleagues roundly congratulated him. Just as Planck regarded the introduction of the quantum, a packet of energy, as a 'purely formal assumption' to which he 'really did not give much thought', so did everyone else that day. What was important to them was that Planck had succeeded in providing a physical justification for the formula he had presented in October. To be sure, his idea of chopping up energy into quanta for the oscillators was rather strange, but it would be ironed out in time. All believed that it was nothing more than the usual theorist's sleight of hand, a neat mathematical trick on the path to getting the right answer. It had no true physical significance. What continued to impress his colleagues was the accuracy of his new radiation law. Nobody really took much notice of the quantum of energy, including Planck himself.
Early one morning Planck left home with his seven-year-old son, Erwin. Father and son were headed to nearby Grunewald Forest. Walking there was a favourite pastime of Planck's and he enjoyed taking his son along. Erwin later recalled that as the pair walked and talked, his father told him: 'Today I have made a discovery as important as that of Newton.'61 When he recounted the tale years later, Erwin could not remember exactly when the walk took place. It was probably some time before the December lecture. Was it possible that Planck understood the full implications of the quantum after all? Or was he just trying to convey to his young son something of the importance of his new radiation law? Neither. He was simply expressing his joy at discovering not one but two new fundamental constants: k, which he called Boltzmann's constant, and h, which he called the quantum of action but which physicists would call Planck's constant. They were fixed and eternal, two of nature's absolutes.62
Planck acknowledged his debt to Boltzmann. Having named k after the Austrian, a constant that he had discovered in his research leading up to the blackbody formula, Planck also nominated Boltzmann for the Nobel Prize in 1905 and 1906. By then it was too late. Boltzmann had long been plagued by ill health - asthma, migraines, poor eyesight and angina. Yet none of these were as debilitating as the bouts of severe manic depression he suffered. In September 1906, while on holiday in Duino near Trieste, he hanged himself. He was 62, and though some of his friends had long feared the worst, news of his death came as a terrible shock. Boltzmann had felt increasingly isolated and unappreciated. It was untrue. He was among the most widely honoured and admired physicists of the age. But continuing disputes over the existence of atoms had left him vulnerable during periods of despair to believing that his life's work was being undermined. Boltzmann had returned to Vienna University for the third and last time in 1902. Planck was asked to succeed him. Describing Boltzmann's work as 'one of the most beautiful triumphs of theoretical research', Planck was tempted by the Viennese offer but declined.63
h was the axe that chopped up energy into quanta, and Planck had been the first to wield it. But what he quantised was the way his imaginary oscillator could receive and emit energy. Planck did not quantise, chopinto hv-sized chunks, energy itself. There is a difference between making a discovery and fully understanding it, especially in a time of transition. There was much that Planck did that was only implicit in his derivation, and not even clear to him. He never explicitly quantised individual oscillators, as he should have done, but only groups of them.
Part of the problem was that Planck thought he could get rid of the quantum. He only realised the far-reaching consequences of what he had done much later. His deep conservative instincts compelled him to try for the best part of a decade to incorporate the quantum into the existing framework of physics. He knew that some of his colleagues saw this as bordering on a tragedy. 'But I feel differently about it', Planck wrote.64 'I now know for a fact that the elementary quantum of action [h] played a far more significant part in physics that I had originally been inclined to suspect.'
Years after Planck's death in 1947, at the age of 89, his former student and colleague James Franck recalled watching his hopeless struggle 'to avoid quantum theory, [to see] whether he could not at least make the influence of quantum theory as little as it could possibly be'.65 It was clear to Franck that Planck 'was a revolutionary against his own will' who 'finally came to the conclusion, "It doesn't help. We have to live with quantum theory. And believe me, it will expand."'66 It was a fitting epitaph for a reluctant revolutionary.
Physicists did have to learn to 'live with' the quantum. The first to do so was not one of Planck's distinguished peers, but a young man living in Bern, Switzerland. He alone realised the radical nature of the quantum. He was not a professional physicist, but a junior civil servant whom Planck credited with the discovery that energy itself is quantised. His name was Albert Einstein.