This was the US unofficial designation for part of the ‘Magic’ intelligence programme to derive and disseminate information from the cryptanalysis of messages enciphered using the Japanese foreign office’s ‘Purple’ machine (1939/45).
In 1923, a US Navy officer acquired a stolen copy of the Secret Operating Code codebook which had been used by the Imperial Japanese navy during World War I. Photographs of the codebook were given to the cryptanalysts at the US Navy’s Research Desk and the processed code was kept in red-coloured folders (to indicate its Top Secret classification), and the code thus came to be called 'Red'.
In 1930, the Japanese government created a more complex code which the USA codenamed 'Blue', although 'Red' was still being used for low-level communications. The 'Blue' code had been broken by the Research Desk no later than 1932, and US communications intelligence signals intercept stations began to monitor command-to-fleet, ship-to-ship and land-based communications.
After Germany, which was allied to Japan in the Tripartite Pact, triggered World War II in September 1939, the German government began to send technical assistance to Japan for the upgrade of Japan’s communications and cryptography capabilities. One part of this effort was the despatch to the Japanese of modified Enigma machines to secure Japan’s high-level communications with Germany. The new code, codenamed 'Purple' (the colour obtained by mixing red and blue) was baffling. The system began its communications with the same line of code but then became an unfathomable jumble. US codebreakers tried to break 'Purple' by hand, but found the task impossible and only then realised that 'Purple' was not a manual additive or substitution code like 'Red' and 'Blue', but a machine-generated code similar to Germany’s Enigma cipher. Decoding was slow and much of the traffic was still hard to break. By the time the traffic was decoded and translated, the contents were often out of date.
A reverse-engineered machine created in 1939 by a team of technicians led by William Friedman and Frank Rowlett could decrypt part of the 'Purple' code by replicating some of the settings of the Japanese Enigma machines. This allow the decryption process to be speeded, and in 1942 the addition of more translators to the staff made it easier and quicker to decipher the traffic intercepted.
The ‘Purple’ machine (97-shiki obun inji-ki, or System 97 Printing Machine for European Characters, otherwise Angoki B-kata, or Type B Cipher Machine) was itself first used by the Japanese for diplomatic traffic in 1940, and US and British cryptographers had broken some of its messages well before the Japanese attack on Pearl Harbor, the translations being stored in purple binders.
US cryptographers decrypted and translated the 14-part Japanese diplomatic message breaking off relations with the USA at 13.00 Washington time on 7 December 1941, some time before the Japanese embassy in Washington was able to achieve the same task. The USA discovered no hint of the attack on Pearl Harbor in the ‘Purple’ traffic at the time, nor could it have done as the Japanese were very careful to not discuss the planned attack in foreign office communications. In fact, no detailed information about the planned attack was even available to the Japanese foreign office as the matter was regarded by the military, particularly the more nationalistic elements of the military, as wholly military and therefore outside the purview of the foreign office, which was seen as insufficiently ‘reliable’.
US access to Japanese diplomatic communications (even the most secret ones) was less useful than it might otherwise have been because Japanese policy, during the pre-war period, was controlled largely by military groups (especially in China and Manchuria) and not the foreign office. This latter deliberately kept from its embassies and consulates much of the information it did have, so the Allies’ ability to read ‘Purple’ traffic was less than definitive regarding Japanese tactical or strategic military intentions. Even so, the diplomatic information discovered was of even more limited value to the US because of its dissemination pattern within the US government.
‘Magic’ information from ‘The Bureau’, as the combined US Army’s Signals Intelligence Section and US Navy’s Communication Special Unit was known, was distributed in a manner which meant that many policy makers who should have access to it in fact knew nothing of it, and those to whom it actually was distributed (at least in the period before the attack on Pearl Harbor) saw each message only briefly as the courier waited to take it back, in isolation from all others, and without the ability to take a copy or notes. Before Pearl Harbor, in any event, those on the distribution list saw only the decrypts thought ‘important enough’ by the distributing officers of the US Army and US Navy.
The ability to read ‘Purple’ messages nonetheless offered the Allies a great advantage during the war. The Japanese ambassador to Germany, for instance, produced long reports which were encrypted with the ‘Purple’ machine and included accounts of personal discussions with Adolf Hitler and of a tour of the invasion defences in northern France, including those assaulted in ‘Overlord’.
When ‘Purple’ was broken by the Signals Intelligence Service, this raised several problems for the Americans: who would get the decrypts, and which decrypts were to be disseminated and, in the case of the latter how often, under what circumstances, and crucially (given inter-service rivalries) by whom? The US Navy and the US Army each insisted that it alone should handle the delivery of all decrypted traffic, especially to highly placed policy makers in the USA.
After a highly contentious dispute, a compromise was reached whereby each service was responsible on alternate days. The distribution list eventually included some but not all military intelligence leaders in Washington and elsewhere, and some but again not all civilian policy makers in Washington.
The eventual routine for distribution included the following steps: the relevant officer decided which decrypts were significant or interesting enough for distribution; the selected items were then collected, locked into a briefcase, and turned over to a relatively junior officer (not always cleared to see the decrypts) to ‘make the rounds’ to the appropriate offices; no copies of any decrypts were left with anyone on the list; the recipient was allowed to read the translated decrypt, in the presence of the distributing officer, and was required to return it immediately upon finishing. Before Pearl Harbor, that was the first and also the last time anyone on the list saw any particular decrypt.
There were several steps in the process before any decrypt was ready for distribution. First, the message had to be intercepted. The Japanese foreign office used both radio and cable transmission to communicate with its offshore departments and embassies. Radio transmissions were intercepted wherever this was possible, and any of several listening stations (Hawaii, Guam in the Marianas islands, Bainbridge island in Washington state, etc.) and the raw cipher groups were forwarded to Washington. Eventually, there were also decryption stations (including a copy of the US Army’s ‘Purple’ machine) in the Philippines islands.
Up to a time late in 1941, cable traffic was collected at cable company offices by a military officer who made copies and forwarded them to Washington. Cable traffic in Hawaii was not intercepted until David Sarnoff of RCA agreed to allow it during a visit to Hawaii during the first week of December 1941. At one point, intercepts were being mailed to US Army or US Navy intelligence from the field. Then the raw intercept had to be deciphered. This was done by either the US Army or the US Navy depending on the day and, because of the nature of the cipher and the break into it, was usually successful. Having obtained the plain text, in Latin letters, the message was next translated. Because the US Navy had more officers able to speak Japanese, much of the translating burden fell onto this service. And because Japanese is a difficult language, in which much meaning is dependent on context, effective translation required not only a fluent command of Japanese, but considerable knowledge of the context within which the message had been sent. The translated decrypt was then evaluated for its intelligence content. For example, was the ostensible content of the message meaningful? If it was, for instance, part of a power contest within the foreign office or some other part of the Japanese government, its meaning and implications would be quite different than if it was a simple informational or instructional message to an embassy. Or might it be another message in a series whose meaning, taken together, was more than the meaning of any individual message.
Thus the fourteenth message to an embassy instructing that embassy to order Japanese merchant ships calling at a port in that country to return to home waters before, say, the end of November was more significant than a single such message meant for a single ship or port.
Only after having evaluated a translated decrypt for its intelligence value could anyone decide whether it deserved to be distributed. In the period before the 'Ai' attack on Pearl Harbor, the material was handled both awkwardly and inefficiently, and was distributed still more awkwardly. Nevertheless, the extraordinary experience of reading a foreign government’s most closely held communications, sometimes even before the intended recipient, was astonishing. It was so astonishing that someone (possibly President Franklin D. Roosevelt) called it magic, and the name stuck.
An aspect of ‘Magic’ which remains controversial is the extent to which the intercepts were involved in the issue of US Executive Order 9066 leading to the internment of Japanese Americans living in the west coast states of the continental USA. Those defending the internment point to ‘Magic’ intercepts as being justification for the internment. The rationale for this is that several ‘Magic’ intercepts discussed the development of a spy ring among Japanese Americans by Japanese consulates, showing that the Japanese American community was an espionage risk. Those in opposition point out that the commanding officer who executed the internment, Lieutenant General J. L. DeWitt, was not on the ‘Magic’ intercept list, but that his superior, Secretary of War Henry L. Stimson, was on the list and requested justification for the internment from DeWitt (if ‘Magic’ intercepts provided justification, why ask DeWitt for further justification?).
‘Purple’ provided an enticing but, in tactical terms, quite limited window into Japanese planning and policy because of the peculiar nature of Japanese policy making before the attack on Pearl Harbor.
Early on, a better tactical window was provided by the Japanese fleet code (actually an encoded cipher) called JN-25 by US Navy cryptographers. Breaking into the version used in the months after the attack on Pearl Harbor provided enough information to create the US Navy’s strategic victories in the Battle of the Coral Sea and the Battle of Midway, which checked the Japanese advance to the south with a tactical success in the former and a strategic success in the latter by eliminating most of Japan’s carrierborne air power.
Broken JN-25 traffic also provided, at a later date, the schedule and routing of the aeroplane in which Admiral Isoroku Yamamoto would be flying for an inspection tour in the South-West Pacific, giving the USAAF the chance to launch the ‘Vengeance’ undertaking for long-range fighters to intercept and shoot down the aeroplane, so killing the officer who had devised ‘Ai’.
Still later, access to Japanese army messages from decrypts of army communications traffic assisted in planning the US ‘island hopping’ campaign to the Philippines and beyond.
Public notice had actually been served that Japanese cryptography was dangerously inadequate by the Chicago Tribune, which published a series of stories just after Midway in 1942 directly claiming that the victory was due in large part to US breaks into Japanese cryptography systems (in this case, the JN-25 cipher, though which system[s] had been broken was not mentioned in the newspaper stories). Fortunately, neither the Japanese nor anyone who might have told them seem to have noticed either the newspaper’s coverage, or the stories based on the Chicago Tribune’s account republished in other US papers. Nor did they notice announcements made on the floor of the US Congress to the same effect.
Thus there were no changes in Japanese cryptography which could, then or now, be connected with those newspaper accounts or Congressional disclosures.