Colossus, codebreaking, and the digital age. Stephen Budiansky Stephen Budiansky

Transcription

1 1 Colossus, codebreaking, and the digital age Stephen Budiansky 2005 Stephen Budiansky The paths that took ordinary men and women from their ordinary lives and deposited them on the doorstep of the odd profession of cryptanalysis were always tortuous and accidental and unpredictable. The full story of the Colossus, the pioneering electronic device developed by the British Government Code & Cypher School (GC&CS) to break German teleprinter ciphers in World War II, is fundamentally a story of several of these accidental paths converging at a remarkable moment in the history of electronics and of the wartime urgency that set these men and women on these odd paths. Were it not for the wartime necessity of codebreaking, and were it not for particular statistical and logical properties of the teleprinter ciphers that were so eminently suited to electronic analysis, the history of computing might have taken a very different course. The fact that Britain s codebreakers cracked the high-level teleprinter ciphers of the German Army and Luftwaffe high command during World War II has been public knowledge since the 1970s. 1, 2, 3 But the recent declassification of new documents about Colossus and the teleprinter ciphers, and the willingness of key participants to discuss their roles more fully, has laid bare as never before the technical challenges they

2 2 faced not to mention the intense pressures, the false steps, and the extraordinary risks and leaps of faith along the way. It has also clarified the true role that the Colossus machines played in the advent of the digital age. Though not actual computers themselves, and programmable only in the very remotest sense of that term (recent British claims to the contrary notwithstanding), the Colossi sparked the imaginations of many scientists, among them Alan Turing and Max Newman, who would go on to help launch the computer revolution. But the story of Colossus really begins not with electronics at all, but with codebreaking; and to understand how and why the Colossi were developed and to properly place their capabilities in historical context, it is necessary to understand the problem they were built to solve, and the people who were given the job of solving it. It is a story that really begins with a man who knew nothing about electronics, and almost nothing about statistics or Boolean logic. John Tiltman was the accidental cryptanalyst par excellence. His clipped mustache and regimental bearing made him look like a British Army colonel from central casting every inch a soldier, friends said. He had been severely wounded on the Somme in the First World War, winning the Military Cross for his troubles. Given a desk job to recuperate, he landed totally by fluke in the small postwar bureau of codebreakers that would soon be known as GC&CS. 4 Tiltman years later said he had had no knowledge of higher mathematics, not even statistics or probability. But he did have that indefinable knack of sensing patterns, combined with that indispensable tenacity that is the hallmark of all great codebreakers. Within a year he was on his way to Simla, in the Himalayan foothills of British India, with the assignment of breaking coded Soviet diplomatic traffic passing between Moscow and Kabul and Tashkent. His success fully justified the effort, for it revealed

3 3 that the Great Game had hardly ended with the overthrow of the Czar and the coming of Bolshevism: messages Tiltman broke throughout the 1920s revealed a concerted effort by Moscow to foment anti-british feeling and subversion along the frontier of the Raj. Tiltman was definitely of the old school of codebreakers, the kind who could have walked off the pages of Edgar Alan Poe s or Arthur Conan Doyle s stories. He always worked by himself, standing at a specially made desk, following leads and hunches for days and weeks on end more like a detective than a modern cryptanalyst, said one of his more modern, and mathematically inclined, colleagues. He had a knack for seat-of-thepants cryptography, and an even greater knack for languages, acquiring Russian, German, and Japanese. Beneath the Colonel Blimp exterior lay an extraordinarily supple mind, and a certain humorous impatience with conventional authority as well. A private assigned to the military codebreaking section at Bletchley Park the not-so-stately mansion sixty miles northwest of London that was GC&CS s secret headquarters during the Second World War never forgot his introduction to Tiltman as he reported for duty. Approaching the colonel s desk, he stamped the floor in proper drill fashion, saluted, ending with another thunderous crash of his army boots on the wooden floor. Tiltman looked up wordlessly, looked down at the private s feet, and then back up at the young soldier s face. I say, old boy, he finally said after a long pause, must you wear those damned boots? 5 It wasn t usual for privates to wear battle dress and white running shoes, but it wasn t usual for colonels to address privates old boy either, and both, to the delight of the private and the disgust of the regular army adjutant assigned to Bletchley Park, became the norm in Tiltman s section.

4 4 Tiltman s follow-the-hunches approach to sniffing out the weak point of an enemy codes was actually, in many cases, the best way to tackle the coded signals of the Japanese and German forces that British and American listening posts began intercepting in the 1930s. Many of the enemy codes were hand systems that used code books or key squares in which words or letters or pairs of letters were substituted according to a prescribed pattern, and a trained human mind s ability to spot such patterns in coded messages just as in chess games or crossword puzzles was still the best tool for tackling them. In July 1941, as German panzers thundered into Russia, Tiltman s group broke the hand cipher used by the German Police in the East and began to read the first hints of unimaginable horrors to come: tallies of numbers of Jews shot in cleansing operations by the advancing German troops who had been told by Hitler that this was to be no contest in the knightly fashion. But the Germans themselves were fully aware of the weakness of hand ciphers and, on a scale unlike any other nation, had built and deployed in the field thousands of coding machines. These electrical machines, using wired rotors that would turn to generate a new scrambling pattern as each letter of a message was typed in A standing for E in one spot, A standing for X in another were capable of millions upon millions of permutations that defied hand methods of decipherment. The most famous of these machines was the Enigma, which the German armed forces had been using since the mid- 1930s. But following the invasion of Russia in June 1941, another sort of signal started appearing in the operational radio traffic of the German Army, first on a link between Vienna and Athens. 6 Clearly, this was the product of a machine cipher of even greater complexity. The signals were being sent not in Morse code, but in a five-bit binary code; that immediately suggested to Bletchley s experts that the signals were being generated by a teleprinter machine, as the standard teleprinter alphabet was based on five-bit

5 5 patterns (in which, for example, stands for A, for B). It was also at once apparent, however, that the standard teleprinter code was being scrambled probably, the codebreakers surmised, by coding wheels attached to the teleprinter that automatically masked the standard teleprinter code as the text was transmitted. These signals would subsequently be known by the British codename Tunny, and it would eventually be learned they were the product of an enciphering teleprinter machine made by the Lorenz firm, known as the Schlusselsatz 40, or SZ 40. (A later model also appeared, the SZ 42.) Another teleprinter machine codenamed Sturgeon (the Siemens T52) would come into use by the Luftwaffe over the next year. 7 By the end of the war, Bletchley Park was routinely cracking hundreds of these high-level Fish signals a month in an operation that involved hundreds of men and women, including some of the finest mathematical minds in the country, aided by a battery of ten specially built digital electronic machines of unprecedented size and complexity. The historian of cryptography Ralph Erskine has called the breaking of Tunny the greatest cryptanalytical feat of World War II. 8 Because Tunny circuits linked Berlin directly to the headquarters of theater and army group commanders, the messages that Bletchley s codebreakers broke revealed German intentions and strategy at the highest levels. In the weeks leading up to D-Day, decoded Tunny traffic provided a series of strategic appreciations by the German Commander in Chief West, Field Marshal Gerd von Runstedt, of the Allied invasion threat. These provided important reassurance to Allied commanders that their audacious deception plan was succeeding. 9 Breaking Tunny was also the feat that irrevocably hurled codebreaking headlong into the modern age of computing and mathematical analysis. It is no exaggeration to say that it helped to launch the entire modern digital revolution by providing an urgent

6 6 impetus to the development of some of the key foundations of the modern computer that might otherwise have taken years more to mature. The mathematical intensity of the work was unlike anything that had come before. By the end of the war the theorems, reasoning, statistics, and Boolean logic that underlay the attack against Tunny had filled thousands of pages of notebooks. Many, indeed perhaps all, of the notebooks may have been lost by GC&CS s successor agency, but a detailed technical history of the work the General Report on Tunny does survive, and was declassified and released to the British Public Record Office in fall Yet it was Lieutenant Colonel John Tiltman, a relic of the old pre-mathematical, precomputing days of codes and codebreaking, who ever so fitting to the quirky ways of cryptography lit the spark of revolution. On August 30, 1941, British radio operators picked up two Tunny signals both bearing the same twelve-letter starting code: HQIBPEXEZMUG. The Bletchley codebreakers already had established that these starting groups were indicators ; that is, they told the recipient of the message how to set the starting position of the wheels on the coding attachment. They had also figured out that these wheels generated a series of quasi-random five-bit numbers that would be added bit by bit to each letter; the recipient at the other end of the link would turn his wheels to the same setting to automatically strip off this added key and reveal the original text of the incoming signal. For example, if the letter B, 10011, was typed in and the coding wheels at that particular point in the message generated the key 11001, the resulting sum adding bit by bit, without carrying would be = In the case of the August 30 messages, however, the German teletype operator had committed a fabulous breach of security procedures. He had resent the same 3,976- character-long message at the same setting of the coding machine. But the second time he had begun by abbreviating the word spruchnummer ( message number ) which he had

7 7 spelled out in full the first time round. He had also in the course of retyping the message made other slip-ups, misspellings, changes in spacing. By the 3,976th letter the two messages were off by about 100 characters. This was a gift from heaven. The machine s code wheels had of course generated exactly the same sequence of key in both messages. So simply subtracting the first message from the second zeroed the key out of the equation altogether; what was left was a stream of five-bit binary numbers that represented only the plain text of the two messages subtracted from one another. Tiltman was given the job of trying to figure out the actual text of both messages. This was a crossword-solver s delight. By guessing at likely words in one of the messages and seeing what letter that implied for the second, Tiltman in short order had teased out the full text. Even more important, by subtracting the plain text from the original coded signal, the Bletchley codebreakers now had an actual string of 3,976-character long stretch of key that just might reveal how the Tunny machine was constructed how many wheels it had, how they turned, how they were wired. The Bletchley codebreakers would never see an actual Tunny machine until the war was over. Their job was thus, sight unseen, to imagine how it was constructed, with nothing but this external evidence to go on. This was clearly a job for a mathematician. GC&CS had begun the war with an extreme reluctance to hire anyone with a scientific or technical background at all. Recruiting was done through an old-boy network with a vengeance. The principle contacts that GC&CS had with the academic world were through men who had worked in the Admiralty s famous Room 40 codebreaking operation during the First World War; and these were mainly linguists who were now Oxford and Cambridge dons in such fields as classics, history, and modern languages. Alastair Denniston, GC&CS s operational head, explained that these men knew the type

8 8 required, professors and bright undergraduates alike, and discreet inquiries were accordingly made. Of the first twenty-one men of the professor type (as Denniston called them) who were hired by GC&CS immediately after the outbreak of war in September 1939, eighteen were humanists historians of art, professors of medieval German, lecturers in ancient Greek. 11 Partly this was just an inevitable reflection of the contacts Denniston had. But partly it was a manifestation of the enduring British publicschool prejudice against anything even remotely associated with trade ; properly educated boys studied Latin and Greek, not science or engineering. Peter Twinn, one of the very few early recruits who did have a technical background later recalled that there had been grave doubts about hiring him because mathematicians were regarded as strange fellows ; if hiring someone with scientific training were regretfully to be accepted as an unavoidable necessity, as Twinn facetiously put it, the prevailing view was that it might be better to look for a physicist on the grounds that they might be expected to have at least some appreciation of the real world. 12 Alan Turing, the brilliant theoretical mathematician whose prophetic 1936 paper on computability had laid the foundation of the stored-program digital computer, was at once the most notable confirmation of, and exception, to the rule. Brought in to consult on the Enigma problem in 1939, he was almost completely unworldly in some ways, eccentric, given to scribbling out his analyses in such atrociously sloppy handwriting that, one of GC&CS s senior officials fumed, he often made mistakes because he couldn t even read his own writing. 13 Yet his astonishing group-theoretical attack on the seemingly unbreakable Enigma in autumn of 1939, and in particular his success against the extremely difficult coding variations used by the German Navy for its version of the Enigma (Turing s key to breaking the naval Enigma involved a brilliant and involved use of statistics), made it clear to a number of people in authority at GC&CS that

9 9 mathematically adept recruits would be needed to deal with this new world of cryptanalysis. Many bright mathematicians began swelling the ranks. 14 Thus by late autumn of 1941, when three months of frustrating work with Tiltman s recovered key sequence had gone nowhere, the GC&CS authorities were accustomed enough to the idea that mathematicians might actually be of some use that they were willing to risk turning the job over to one of these strange fellows. What was more, they actually had some of them around to turn to. The job of trying to reconstruct the key wheels of Tunny thus fell to William Tutte, a young Cambridge mathematician. Tutte tried writing out the key sequence for each of the five bits in grids with varying numbers of columns, looking for any recurring patterns. When he tried a grid with forty-one columns for the first bit, a pattern suddenly popped out. Eventually Tutte was able to determine that each of the five bits generated for each letter was encoded by two separate wheels, a total of ten wheels in all. The first set of wheels were dubbed the χ ( chi ) wheels, and these turned one notch with each letter typed in, generating a sequence of 1s and 0s that would be added to each bit of the standard teletype code impulse of each letter in turn. The second set of wheels, the ψ ( psi ) wheels, moved in an irregular pattern, sometimes advancing with each letter but sometimes staying put. The 1s or 0s from the ψ wheels would be added in as well. A final pair of wheels, dubbed the motor wheels, sent out 1s or 0s that would determine whether the ψ wheels turned or not. 15 At first all of the work was done by hand. The trick was to find two messages that had been sent with very similar indicators it was clear that the twelve letters of the indicator represented the start positions of the twelve coding wheels, and so two messages that differed by only one indicator letter would likely share a long sequence of overlapping key. Messages sent with the same key were said to be in depth in the cryptanalysts jargon, and these near-depths proved vital especially for figuring out

10 10 new wheel pattern sequences, which were changed every month. (The codebreakers called this process wheel-breaking. ) The indicators carried on each message then told the codebreakers how to set up those recovered wheel patterns to decode each subsequent message received for that month (this was the problem of wheel-setting ). By July 1942 the first actual German messages were read currently; from July to October 1942 virtually every message was read. 16 But that sudden success hit a just as sudden catastrophe in October, when the Germans, recognizing the inherent insecurity of the twelve-letter indicators, abruptly changed the indicator system. In place of the twelve letters that specified directly how to set the twelve wheels, they began using a simple number. The codebreakers surmised that the number was keyed to a daily list of settings provided to all users; in any case, it was no longer possible simply by looking at the message indicators to tell whether two messages were near-depths. The only hope now for wheel-breaking was to find two messages sent on the same day with identical indicators, a much longer shot. Even worse, wheel setting of subsequent messages was likewise only possible when two messages were sent the same day with an identical indicator. Depths were now required for every message that was to be read, an almost impossible limitation for practical intelligence production. It was then that Tutte made one of the most crucial mathematical discoveries in the project. Building on an initial insight by Turing, 17 he realized that adding a string of enciphered letters to itself, but shifted one letter over, could yield information not apparent in the original string. Because the ψ wheels often did not advance from one letter to the next, that meant that when a psi wheel generated a 1, more often than not it would generate a 1 again for the next letter. Likewise, a 0 would more often than not be followed by another 0. That meant that when a whole string of ψ-wheel key was added to

11 11 itself with a shift of one, the resulting string would have a predominance of 0s, since in noncarrying binary addition = 0 but also = 0. In fact, such a shift-and-add treatment would produce a string containing about 70 percent 0s, Tutte subsequently calculated. Likewise, he discovered that because of the appearance of repeated letters, diphthongs such as ei, and other patterns in the German language and just by the chance way that letters were rendered in the standard teletype code the first two bits of the plain text when shifted over one and added to itself also had a preponderance of 0s, about 60 percent on average. And that meant that if the first two bits of a string of coded traffic were subjected to this same shift-and-add treatment, the effect of both plain text and the ψ wheels would more often than not zero out. What would be left was a pattern of 1s and 0s that tended to track the contribution of the χ- wheels only. This break-in procedure depended heavily on statistical calculations, and the trick was clearly going to be devising a way to slide the χ wheels in all possible positions against a string of intercepted cipher text, hunting for the one that gave the best statistical fit. Tutte s idea was tried by hand with stencils and paper to test its logical feasibility, and it worked. But the experiment also confirmed, if confirmation were even necessary, that operationally it was simply impossible to think of doing such a task manually. No one could possibly try thousands upon thousands of possibilities, calculate the statistics for each, and find the best fit for every single message by hand. The Germans were by now transmitting hundreds of Tunny messages a day. It was intelligence of the greatest importance, clearly and appropriately of the greatest corresponding technical difficulty to break. What would be required was mechanization on a scale that no one had ever dared dreamed of before.

12 12 The job of putting Tutte s idea into practical effect fell to three of the oddest of Bletchley Park s odd assortment of men and women. Not odd in the sense of eccentric; but certainly odd in the paths that brought them to this task, for none seemed on the face to be well suited to organizing what would become a huge engineering and management task. Max Newman, who headed the effort, was a theoretical mathematician of the purest kind. His research at Cambridge before the war had focused on the foundations of mathematics and issues related to Gödel s famous proof of the impossibility of a mathematical system being both complete and consistent. I never saw him do anything except write on paper. He was not really of a mechanical bent, recalls his son William. Max Newman at first assumed he would be ineligible for war work because of his German ancestry. 18 But a close friend introduced him to someone at Bletchley, and by 1942 he was brought in, and assigned to work on the Tunny under Tiltman. Newman found the heavily linguistic sort of pencil-and-paper work frustrating and almost considered returning to Cambridge. But the October 1942 setback changed everything. In December, Newman was given the green light to explore the idea of mechanizing a purely statistical attack on Tunny. Newman s first assistant was an even unlikelier candidate to work on a pioneering project in automated computing. In the spring of 1943 Donald Michie was all of nineteen years old. He had won a scholarship to study classics at Balliol College, Oxford, and had no experience with either machines or mathematics. But hoping to do something for the war effort and assured that he was automatically guaranteed a place at university once the war ended if he postponed his enrollment to take up war service he decided to enroll in a crash Japanese language course that he had learned about through a family friend who was an official of the War Office.

13 13 It was in fact none other than the ever resourceful John Tiltman who had launched this six-month training course for intelligence officers, brashly ignoring the sober advice of the experts at London s School of Oriental and African Studies who insisted that no one could possibly master even the rudiments of such a difficult language as Japanese in less than two years. The course took place in a nondescript room above the Gas Company s offices in the city of Bedford, fifteen miles from Bletchley. Michie went at once and presented himself for an interview, and was crestfallen to be told that he had been misinformed; the course was already full and the next enrollment was not until autumn. What happened next changed his life. Cheer up, the officer said, seeing how disappointed he was, and went on to explain that there was a cryptography course starting the next Monday that might interest him as a way of filling in the time. The officer told Michie if he went home for his belongings and papers and came right back and showed up at 9 a.m. Monday morning, we can probably get you in. Michie and his teachers immediately discovered that he had an amazing natural talent for the subject. He shot ahead of the rest of the class and just six weeks later when an officer from Bletchley arrived to recruit someone to work on Tunny, my selection was pretty much a foregone conclusion, Michie recalls. He was assigned to a section at Bletchley Park that had been established in July 1942 under Major Ralph Tester the Testery 19 that was tackling Tunny using hand methods. In no time he was in charge of the mathematically intensive, and extremely tedious and laborious, task of determining the start settings of individual Tunny messages. When it came time for Newman to poach an assistant, he wanted someone who was intimately familiar with the ins and outs of the hand methods, and Michie was it. 20

14 14 Michie s discovery that he had an aptitude for probability, statistics, and Boolean logic and was even able to hold his own in the coffee-break and lunchtime competitive exchanges of brain-teasers and recreational math with his far more mathematically trained colleagues had an extraordinary impact on a subsequently extraordinary career. After the war he did take up his classics scholarship but soon found it boring and abruptly switched to human anatomy and physiology, did important work in mammalian genetics, and then in the early 1960s shifted again, to computer science, and became a leading researcher in artificial intelligence. The third member of Newman s core group was, like Newman, another extremely theoretical mathematician. But I. J. Jack Good was recruited to Bletchley in a more conventional way in one sense, for he was an extremely good chess player, and Bletchley was teeming with extremely good chess players. The two men who would eventually head the naval Enigma and army air Enigma sections for most of the war Hugh Alexander and Stuart Milner-Barry were both members of the British national chess team and were convinced that good chess players made good cryptanalysts. Good had won the Cambridge chess championship one year ( but what I was very good at, he says, was five-minute chess ) and knew both Alexander and Milner-Barry from the chess world. He also knew that they were working on some secret project. A Cambridge colleague, Good recalls, was fairly sure it was codebreaking. 21 In general it was extraordinary how tight the security surrounding Bletchley s work was; the archival records of Bletchley at the Public Record Office note only two instances in which anyone shot off his or her mouth even to friends, family, and trusted colleagues about what was going on there. It was a testimony to the absolute code of secrecy which prevailed that when, just two weeks before Good went there, he ran into Milner-Barry at a chess match and tactlessly asked him if he was at Bletchley working on codes, Milner-

15 15 Barry, not batting an eye, replied, No, my address is Room 47, Foreign Office. Two weeks later, Good arrived at Bletchley and discovered Milner-Barry there (and learned that Room 47, Foreign Office, was the cover address assigned to everyone at Bletchley.) Good, now 84 years old and a professor emeritus of statistics at Virginia Tech in the mountains of southwestern Virginia, spent a decade after the war working for GC&CS s successor agency on cryptanalytic studies; among other things he carried out some crucial statistical analyses for Project VENONA the ultrasecret US British effort that successfully broke Soviet espionage codes despite the fact they were enciphered in theoretically unbreakable one-time pads. (Good s car bears the facetiously selfdramatizing license plate 007IJG. ) It was odd that it was Newman, the theoretician s theoretician, who first proposed mechanizing the Tunny problem. Yet the idea of using a machine to do what humans could not was in the air, and in Newman s consciousness, for several reasons. Newman had taught Turing at Cambridge and was well familiar with Turing s famous paper on computability in which he had advanced what became known as the Turing Machine, a conceptual device for representing the solution of a formal problem in mathematics in terms of a sequence of fundamentally mechanical steps. Turing s brilliant success in developing a method to break Enigma in 1939 had relied fundamentally on a mechanized approach the so called bombes, which were large electromechanical analogues of a series of Enigma machines. Turned by a high-speed motor through all possible wheel starting positions, the machines would electrically test for certain consistent patterns that revealed the daily settings the Germans were using on their various Enigma networks. In addition, the American Army and Navy codebreakers were both developing a number of Rapid Analytical Machines using paper tape or film to record data and to search for instances in which the same code group appeared in the same position in two

16 16 different messages. The work had begun in the mid-1930s with a contract to Vannevar Bush of M.I.T. from the U.S. Navy to build an optical comparator that had potential application to cryptanalytical problems in a broad range of enciphered codes and machines ciphers. 22 Some of these optical RAM devices used simple but ingenious complementary systems for encoding the cipher text on two films which would be superimposed and fed, in various relatives positions, through a scanning head; when the same cipher group appeared simultaneously in the same position the beam of light transiting the scanning head would be completely masked, an event that would be detected by a photocell. 23 Some of the devices also used digital vacuum tube circuits to count the number of coincidences. (In 1945, just weeks before the European war ended, the U.S. Army sent to GC&CS an optical comparator machine called 5202, built by Eastman Kodak, which was designed specifically to tackle Fish problems. It was put to operational use on April 19, 1945.) 24 In Britain, developing the bombes had involved the British General Post Office s research establishment the Post Office was also responsible for telephones, and thus had considerable expertise with high-speed electrical relays and similar circuitry. Newman accordingly asked the GPO s research group if they could build a machine that would rapidly compare two punched paper tapes, one containing cipher text and the other the χ-wheel key sequence. Electronic circuits would perform the add-and-shift rule (known as delta-ing ) to each stream of data from each tape before making the comparisons. An electronic counter would then tally up the number of coincidences and display the results at the end of each test. The tapes could be made in continuous loops and of different lengths, so that when each test was completed a second test would begin automatically with the χ-wheel key shifted over to a new relative position. The test that

17 17 gave the highest score would thus be the best bet for the χ-wheel start position which had been used for that particular message. C. E. Wynn-Williams, a physicist who had joined the British radar research program when the war began, also had been involved in trying to develop an improved bombe, and more than that he had, back in 1931, invented a pioneering electronic binary counter, using vacuum tube circuits. In 1935 he had developed a decimal counter. The work on building the actual Tunny-breaking machine, dubbed Heath Robinson (the American equivalent of the name is Rube Goldberg) began in early 1943, and within a few months Max Newman was able to report to Edward Travis (who had replaced Denniston as head of Bletchley Park in February 1942) that the scanning apparatus from the Post Office, together with the counting circuits from Wynn-Williams, had arrived on June Two days later the first message and key tapes were fed through the machine; as a test a message that had already been broken by hand in the Testery was used, and Heath Robinson came through with the correct χ-wheel setting. It was a brief-lived triumph, though. The shortcomings with both Heath Robinson and the whole approach were embarassingly manifest. As Good recalls, It was built just well enough to get a few successes. One fault could readily be diagnosed because it would cause the mechanism feeding the paper tape to jam and try to set the machine on fire, giving off a distinctive aroma (many anecdotal accounts to the contrary, however, it never actually caught fire this way). The tapes tended to stretch and get out of synchronization, rendering the results meaningless. Even tiny errors in transcribing the intercepted message would also throw everything off. The pressure was intense to produce some results in order to justify Newman s quite risky attempt at mechanization of the problem in the face of Major Tester s resentment and philosophical opposition to the entire mechanized approach. 26 (The General Report

18 18 on Tunny somewhat drily observed that the initial efforts of the Newmanry were regarded by members of Major Tester s section with some amusement. ) 27 The pressure was intense, too, to get results period: the Testery s hand methods, which still relied on obtaining depths for every breakable message, were solving only about a fifth of the messages on some Tunny networks, almost none at all on others. Newman accordingly decreed that Heath Robinson was to be used for operational traffic only. That is, it had to be run not as a research tool or toy for his bright young assistants to play around with, but run on actual, current German messages that urgently needed to be broken for their intelligence value. 28 Michie recalls that he and Good tried to argue that that was putting the cart before the horse: It was apparent that far more than any mechanical glitches, the biggest obstacle they faced was the need for much finer statistics and logical analysis of delta-ed German plain text. It wasn t enough to look for rough matches between text and χ patterns; what was needed was an exhaustive study of the actual Tunny messages that had been broken with the aim of finding additional, precise, but often subtle rules governing how a 1 or 0 in one spot of delta-ed text related to a 1 or 0 elsewhere. The only way to get those statistics was to do exactly what Newman had forbade them to: use Heath Robinson for research. Newman would not budge. Good and Michie resorted to subterfuge. By day they went through the futile motions of tackling operational traffic; by night they crept back with an electronics engineer they had dragooned in tow and pushed ahead with our survey of plain text. By October 1943 the ruse had paid off, and Heath Robinson was at last producing real results. Michie says that Newman was a terrific manager, with a will of steel but flexible steel. Obstinate as a mule, but bearing no grudge if obliged to yield. William Newman believes his father was ahead of his time in practicing an open style of

19 19 management; partly it was simply that he was awkward with people who were not his intellectual equals so he was sure to recruit people who were on his level, and whom he could therefore trust. The Newmanry kept a series of log books in which anyone could write an organizational suggestion or technical idea, and every few weeks a 4 p.m. tea party was held at which ideas were put on the board and debated openly. ( It was a democratic assembly with legislative powers, noted an official glossary of terms produced by the Newmanry.) 29 Back in February, electronics engineer Tommy Flowers at the Post Office had suggested a rather daring improvement to the design of Heath Robinson, and had been greeted with considerable skepticism, in his words. Before the war, Flowers had become intrigued by the idea of using vacuum tube circuits in place of relays in telephone switching devices, and had convinced himself that an entire telephone exchange could be built with electronic circuits. So Flowers suggested that instead of running two tapes, the key sequence could be stored electronically, with vacuum tube circuits. Flowers was told so long as he pressed on with Heath Robinson as top priority, he could try out his electronic idea as well. 30 By February 1944 the first Colossus was delivered to the Newmanry and put to work on operational traffic. 31 With only one tape instead of two, it was no longer necessary to feed the tapes by sprockets; instead they were just shot over pulleys, and the photocell scanner that read each five-bit code provided the synchronization pulse. Speeds of 5,000 characters per second were achieved. Good says he still regards it one of the best kept secrets of the war that ordinary teleprinter tape could be run over pulleys at 30 miles per hour without tearing. 32 Vacuum tube circuits not only served as the internal memory for the designated χ- wheel key sequence that was to be tested but also performed the Boolean operations

20 20 specified for a given test. Patch cords were used to set up the desired operations. The printing typewriter rapped out the results of each count, a vast improvement over Heath Robinson, in which the operator had to watch the display and write it down quickly before it was blanked out in preparation for the next loop of the tape. The typewriter did however have a tendency to walk across the room unless lashed into position. 33 Official photographs of Colossus omit the jury-rigged rope that kept the typewriter on station. The Post Office researchers had discovered from their earlier experiments with communications electronics that one of the major faults of vacuum tubes their tendency to blow out could be almost eliminated through the simple expedient of keeping the tubes turned on all the time. The engineers and Wrens members of the Women s Royal Naval Service who were assigned to actually run the Colossi remember fondly the warm glow that would emanate from thousands of tubes on a cold winter morning. Though in other seasons the heat could be so intense that Good and a colleague once proposed that operators work topless, a suggestion that apparently was not acted upon. The success of Colossus I came just in the nick of time. In December 1943 the Germans had added a new feature to their coding machines. Now even when two messages were sent using precisely the same wheel setting, the key sequence would not be the same, for a special gizmo that Bletchley called the P5 limitation was modifying the output of the coding wheels according to whether bit 5 of the plain text letter that had just been typed in was a 1 or a 0. In other words, the content of the message text was used to modify the key itself. That meant there were no depths to be had, period and Tester s hand methods were completely impotent. With D-Day looming, an order came from the highest levels of the British government: the Post Office engineers were told they had to have a dozen Colossi ready by June 1. Flowers was flabbergasted; his engineers had built the first Colossus in 11

21 It didn t work. Midnight came and went, and the team installing the machine went 21 months start to finish, and had extra parts on hand to make one or two more machines, but a dozen in four months was simply impossible. Nonetheless, working around the clock with no time off but a half day a week for the necessities of life such as talking to their wives and getting their hair cut, 34 Flowers s staff delivered Colossus II on May home dispirited. They returned the next morning at 8:30 to find the machine running like a top: one of the engineers on the night shift had discovered the trouble to be a parasitic oscillation in the power supply, which he cured by soldering in a few resistors. 36 Five days later, Allied forces landed at Normandy. The invasion was followed by another change in German code procedures that also was providentially anticipated by another one of the Newmanry s innovations. The Germans began changing the wheel patterns every day instead of every month. The work of wheel breaking had continued to be a manual job for the Testery. But Michie several months before had had the idea of using Colossus for wheel breaking as well as wheel setting. A crash project to incorporate a special front-panel attachment for wheel breaking was launched and was incorporated into Colossus II and subsequent models. Colossus II had other improvements made on the fly as well. In addition to patch cords used to define logical operations, a Q panel of switches to select Boolean operations was provided. An additional thousand vacuum tubes were added so that each five-bit character read in from the tape could be simultaneously tested against five different χ-wheel settings, with the results fed to five different counters. 37 This parallel processing allowed an effective processing rate of 25,000 bits per second, a speed not matched by many postwar computers for many years to come. Good invented a process called spanning that allowed only a portion of the tape to be read and tallied at a time;

22 22 though originally developed to deal with the P5 problem, it turned out to be invaluable in working around the problem of the invariable errors in transcribing intercepted messages onto the tapes. 38 By the time of the German surrender, Colossus X was in operation. Most of the ten Colossi were promptly dismantled after the war, but their influence on the history of computing was inextinguishable. It is often claimed by British boosters in the who invented the first computer sweepstakes that Colossus was the world s first digital electronic programmable computer, but Michie and Good carefully qualify such claims. It was a large scale electronic machine is as far as Good goes when asked, Was it a computer? The Colossi counted, and performed Boolean operations, he says that s all. They were programmable only by plugging cords and flipping switches, and a great deal of the programming work that Good and Michie did was in fact in plotting out decision trees that the operators would follow in deciding what tests to run next depending on the counts they obtained from the previous test. Nor were the Colossi by any stretch of the imagination general purpose machines. Geoffrey Timms, one of the Newmanry mathematicians, after the war worked out an elaborate system of plugging up the machine so that it could in theory perform base-ten multiplication but it was such a stretch and so complicated a process for the Colossus s circuits that the machine could not in fact complete the operation before the next clock pulse turned over. Michie says he finds it improbable that any of the proto-programming tricks they developed for Colossus had any impact on the development of postwar general purpose computers. William Newman likewise doubts that any of the circuitry of Colossus found its way into postwar computers. But Flowers pointed out that Colossus did have one unmistakable impact for the future of computing: Showing Turing, Newman, and others what electronics could

23 23 do. 39 Newman and Turing and Good all worked at the University of Manchester s computing project after the war, developing a truly programmable electronic computer. And without the urgency of wartime necessity, the experience and confidence they gained from seeing what electronics could do might simply never have happened.

24 24 References 1B. Randell, The Colossus, in A History of Computing in the Twentieth Century, edited by N. Metropolis et al. (New York: Academic Press, 1980), pp Thomas H. Flowers, The Design of Colossus, Annals of the History of Computing 5 (1983): Allen W. M. Coombs, The Making of Colossus, Annals of the History of Computing 5 (1983): Dictionary of National Biography, s.v. Tiltman, John Hessell. P. William Filby, Bletchley Park and Berkeley Street, Intelligence and National Security 10(3): (1983). p General Report on Tunny, Vol. II (HW 25/5, Public Record Office, Kew, U.K.), 7 F. H. Hinsley et al., eds., British Intelligence in the Second World War(London: HMSO, 1984), vol. III, pt. 1, pp Author s interview with Ralph Erskine, December Hinsley et al., British Intelligence, vol. III, pt. 2, pp. 53, 799. General Report on Tunny, Vols. I and II (HW 25/4 and HW 25/5, Public Record Office, Kew, U.K.),. 11 Ralph Erskine, GC and CS Mobilizes Men of the Professor Type, Cryptologia 10 (1986): Christopher Andrew, Her Majesty s Secret Service: The Making of the British Intelligence Community (New York: Penguin) 1987, p. 453.

25 25 13 History of Hut Eight (NR 4685, Historic Cryptographic Collection, RG 457, National Archives at College Park, College Park, Md.), pp Memorandum to Commander Denniston, November 18, 1939 (HW 14/2, PRO); Gordon Welchman, The Hut Six Story (Revised Ed., Cleobury Mortimer, U.K.: M & M Baldwin, 1997), p W. T. Tutte, Fish and I, lecture, June 19, 1998, Centre for Applied Cryptographic Research, University of Waterloo. 16 General Report on Tunny, Vol. I, p General Report on Tunny, vol. II, pp Author s interview with William Newman, December General Report on Tunny, Vol. I, p Author s interview with Donald Michie, December Author s interview with Jack Good, December Naval Security Group History to World War II, Part I (SRH-355, RG 457, NACP), pp Stephen Budiansky, The Code War, American Heritage of Invention & Technology, Summer 2000: The 5202 (NR 2748, Historic Cryptographic Collection, RG 457, NACP). 25 For Commander Travis from newman, 18 June 1943 (HW 14/79, PRO). 26 Author s interview with Donald Michie, December General Report on Tunny, Vol. I, p Author s interview with Donald Michie, December General Report on Tunny, Vol. II, p Flowers, Design of Colossus, p. 244.

26 26 31 General Report on Tunny, Vol. I, p Jack Good, Enigma and Fish, in F. H. Hinsley and Alan Stripp, eds., Codebreakers (Oxford: Oxford University Press, 1994), p Gil Hayward, Operation Tunny, in Hinsley and Stripp, eds., Codebreakers, p Flowers, Design of Colossus, p Flowers, Design of Colossus, p Flowers, Design of Colossus, p General Report on Tunny, Vol. II, pp. 329, Author s interview with Jack Good, December Flowers, Design of Colossus, p. 252.