America's "Big Brother": A Century of U.S. Domestic Surveillance

The potential of the National Security Agency (NSA) “to violate the privacy of American citizens is unmatched by any other intelligence agency,” especially as telecommunication technology advances.

Perhaps surprisingly, that warning was not issued in the wake of the cache of documents leaked by former civilian security contractor Edward Snowden to the Washington Post and the Guardian this past summer. It was made instead in the 1970s by the Senate’s Church Committee when it held hearings on the activities of the CIA, the FBI, and the NSA.

Domestic surveillance intended to protect American citizens has been a part of the fabric of American life for more than a hundred years. Yet, the massive NSA intelligence-gathering programs revealed by Snowden are unprecedented in their sheer scale, their advanced technologies, and in the legal foundation that the Foreign Intelligence Surveillance Court offered them.

One NSA program, unofficially called 215, collected metadata from U.S. telecommunications companies, revealing the date, time, location, and recipient of phone calls across the United States. The agency also monitored foreign emails through a program called Prism, part of a larger internet collection effort called Upstream; while targeted abroad, the NSA has admitted that U.S. emails were gathered up as part of the program. One of the documents leaked to the Guardian claimed that the NSA’s tools allowed them to see “nearly everything a typical user does on the internet.”

President Barack Obama condemned Snowden for leaking, claiming there were other, official channels through which Snowden could have made known his concerns with the program. Obama presented the activities as part of a necessary activity to protect the United States from terrorism, while asserting that laws prevented the “willy-nilly” taking of private information. He promised that reforms would be made to foster greater trust in the system.

While the Snowden revelations have created tremendous controversy in the United States and around the world (as Germany’s Angela Merkel will attest), some of the questions the documents raise have been with us for at least a century.

Can an intelligence agency successfully operate within the bounds of the law and under the supervision of a democratic government? Can the United States spy on foreign targets without creating an organization able to spy domestically? Can a line be successfully drawn between domestic and foreign intelligence? Can we ensure that innocent citizens do not get caught up in the hunt for internal enemies? Who should be considered an internal enemy?

Regardless of what happens in the wake of the Snowden controversy, these tensions are not likely to evaporate as long as the United States continues to engage in any form of spying and to search domestically for enemies, both real and perceived.

Keeping Tabs on Anarchists and Bolsheviks

Before the United States worried about the threat of jihadi terrorists, the nation was gripped by a fear of anarchists.

Political terrorists had killed scores of European officials in the last decades of the nineteenth century. Even Tsar Alexander II of Russia, liberator of the serfs, had been blown up on the streets of St. Petersburg.

And the U.S. was not immune to this violence. In September 1901, U.S. President William McKinley was assassinated by anarchist Leon Czolgosz in Buffalo, New York.

Robert Pinkerton, of the famous private Pinkerton detective agency that had supplied the Union with intelligence during the Civil War and spent subsequent decades helping break labor movements, advised that radicals “should be marked and kept under constant surveillance.”

Theodore Roosevelt, who succeeded McKinley, did not immediately follow Pinkerton’s advice. In 1908, though, he directed the Attorney General, Charles Bonaparte (the great-nephew of Napoleon) to set up an investigative service within the Department of Justice. Congress, however, turned him down.

The Congressmen did not want to create a secret, internal police similar to those employed by European governments, like the Russian Okhrana. Not discouraged, Bonaparte waited until Congress adjourned and, with the approval of Roosevelt, established the Bureau of Investigation in December 1908.

Shortly thereafter, as the United States was drawn into World War I, the modest bureau expanded to meet the challenge of ensuring internal security, especially the perceived threat of Germans living in the United States. As President Woodrow Wilson began to crack down on domestic dissent with the Espionage Act, the bureau began its long career of hunting down spies and radicals.

The bureau was joined in these efforts by the Office of Naval Intelligence. The ONI had its own programs against potential domestic threats, which soon expanded beyond looking for potential foreign spies to investigating “subversive elements.”

After the war, Bolsheviks replaced Germans as the chief focus of the nascent intelligence community. A hunt for communists ensued, driven on by fear of the new Soviet regime, which emerged in November 1917.

In 1919, a series of bomb attacks on government targets convinced many Americans that threats from anarchists and Bolsheviks were clear and present. In the wake of the bombings, Attorney General A. Mitchell Palmer launched a series of raids on radicals in cities around the country, though many had no connection to any violent organization.

The First Red Scare, as it came to be known, resulted in deportations including well-known figures like Emma Goldman, and it broke the back of the Industrial Workers of the World, a prominent union.

To lead the charge against subversives, Palmer created the Radical Division of the Bureau of Investigation, and selected 24-year-old J. Edgar Hoover to lead it. Hoover assumed his new position August 1, 1919, and would spend the rest of his life, more than 50 years, directing law enforcement against communists.

When the FBI was organized from the Bureau of Investigation in 1935, Hoover became its first director. He remained a committed anticommunist. To Hoover, however, “communist” became synonymous with almost anyone on the political left. Under Hoover, various leftist organizations would be infiltrated by federal informants.

In addition to the use of informants to gather human intelligence, early electronic surveillance was used in the tapping of phones. Such tapping was deemed legal by the Supreme Court in the 1927 case Olmstead v. United States, which found that the wiretapping of phones did not violate the Fourth or Fifth Amendments. That decision would not be reversed until 1967.

Domestic surveillance continued in the 1930s, much of it legal although sometimes exceeding legal limits. At the request of Franklin Roosevelt, Hoover begrudgingly began investigations of the American fascist movement. He also monitored civil rights groups, suspecting them of communist influence.

Captain Hayne Ellis, the director of the ONI in 1931, expanded beyond communist groups to investigate pacifist organizations like the National Council for the Prevention of War and the Women’s League for Peace.

As war loomed in Europe, FDR in 1939 directed the FBI, ONI, and the Military Intelligence Division to take charge of tracking subversives in the United States.

The Cold War: The Modern Architecture of Surveillance Takes Shape

World War II witnessed, amongst other things, the birth of the first centralized U.S. intelligence organization, the Office of Strategic Services. Included in its mandate was counter-intelligence, a duty it shared with the FBI. Like the FBI, the OSS would occasionally ignore legal requirements in its pursuit of subjects, most notably when pursuing State Department leaks to the magazine Amerasia.

By the end of World War II, domestic intelligence responsibilities in the United States had grown—in fits and starts—and were now included in the purview of several government agencies under the big umbrella of “national security.” How that term was defined, and who, therefore, constituted a threat to it, was left to particular organizations and their leaders. Thus groups such as the Women’s League for Peace and the National Association for the Advancement of Colored People found themselves at various times considered national security threats and placed under surveillance.

As the nation moved from World War II to Cold War, domestic security became even more a priority. The tension between the United States and Soviet Union led to a variety of activities against communist organizations in America, and organizations perceived to be affiliated with communism. This environment produced many of the secretive and shrouded security organizations we know today in the United States.

In 1947, the National Security Act established the Central Intelligence Agency. Similar to concerns over the original Bureau of Investigation, opponents warned the CIA could become a secret police force, an American Gestapo. To forestall any abuse of the intelligence system, Congress forbade the CIA from operating domestically, leaving domestic security the responsibility of the FBI.

Five years later and with considerably less publicity, the National Security Agency was founded to take responsibility for U.S. signals intelligence, guarding U.S. electronic communications and breaking the encryptions of the enemy.

Like the CIA, the NSA was directed not to spy domestically. Unlike the CIA, the NSA remained under the radar, to such an extent that its staff jokingly claimed its initials stood for “No Such Agency.”

By 1963, the NSA operated from a massive facility at Fort Meade, Maryland, and was busily at work intercepting and decrypting communications. By 1980, the NSA was classifying fifty to one hundred million reports each year, which, in an era of paper and magnetic tape records, led to a serious problem with materials storage.

The growth of the U.S. national security apparatus was matched by a growing fear within the United States of communist infiltration and subversion. However, it would be one of the older organizations, the FBI, which proved most aggressive against a perceived communist threat.

Through several turns of good fortune, the FBI was able to uncover several networks of Soviet spies in the United States. Hoover, in addition to directing the FBI’s more legitimate counter-intelligence activities, cooperated with hardcore anticommunist politicians such as Richard Nixon and Joseph McCarthy.

The Fine Line between Domestic and Foreign

In 1956, under the orders of Hoover and the direct supervision of the chief of research and analysis in the FBI’s Intelligence Division, William Sullivan, the first FBI Counter Intelligence Program (COINTELPRO) was launched. COINTELPRO was the next stage of evolution in the FBI’s anticommunist agenda.

While including domestic surveillance, COINTELPRO went far beyond mere monitoring. COINTELRPO sought actively to disrupt targets by spreading false information within the ranks of targeted groups—a practice that at times led some groups to retaliate violently against, and even murder, those set up to appear to be informers. A variety of COINTELPROs, each directed at a specific target, would be established by the 1970s.

The CIA did not begin domestic spying programs in the 1950s. It did, however, begin some long-lasting domestic programs at that time. Under the direction of intelligence veteran Thomas Braden, the CIA began to wage a “cultural Cold War,” funding domestic and international leftist anticommunist organizations in the hope of forming a reliable non-communist left. At home, this consisted of a variety of activities, such as supporting the work of Jackson Pollack and funding the National Student Association.

These domestic activities violated the CIA’s mandate not to operate in the United States, but they had the general approval of the president. Many of those the CIA supported did not know where their money came from.

For example, most members of the National Student Association had no idea the CIA provided them with funds. In 1964 the president of the group explained that the mysterious funds it had received came from a wealthy Greek rug merchant whom he had met on a train. The rug merchant had apparently been so moved by the organization’s activities that he generously provided money to help them meet operating costs.

The precedent for their domestic activities established, the CIA expanded those activities to include surveillance during the tumultuous 1960s. As opposition to the Vietnam War increased at home, President Johnson grew convinced that part of the problem had to be that foreign agents were aiding the antiwar effort, and also, he feared, stoking the fires of the race riots that destabilized U.S. cities during the long hot summers of the 1960s. In 1967, he directed then-CIA director Richard Helms to investigate these possibilities.

Under the codename CHAOS, the CIA infiltrated the antiwar movement. The program expanded under Nixon, who like Johnson insisted foreign elements were behind the movement, leaving Helms straining to prove a negative by widening the surveillance net on the antiwar movement.

Eventually, the CIA recruited 4,067 informers in the antiwar movement. When the program was revealed by Seymour Hersh of the New York Times in 1974, he reported that CHAOS kept 10,000 CIA files on American citizens. That, it turned out, was a low estimate. Subsequent disclosures revealed that CHAOS held an index of 300,000 names, with especially extensive files on 7,200 U.S. citizens.

The NSA also came to monitor domestic targets. Its watch list included Jane Fonda, Joan Baez, Benjamin Spock, and Martin Luther King, Jr., among others. In 1969, the NSA formalized its activities into Operation Minaret, the full details of which have only recently been revealed. The NSA spied on figures such as New York Times reporter Tom Wicker, Washington Post columnist Art Buchwald, and two antiwar U.S. senators, Frank Church and Howard Baker.

The most aggressive domestic operations remained the FBI’s COINTELPROs, which, like the CIA, expanded in the 1960s. They investigated the New Left, the civil rights movement, Black Power advocates, and the American Indian Movement.

Hoover had long been suspicious of African Americans, having investigated W.E.B. DuBois in his early days as well as the National Association for the Advancement of Colored People (NAACP). Convinced that King must be a communist, Hoover began monitoring him heavily, eventually running eight wiretaps and 16 bugs on King.

Hoover discovered that King engaged in occasional extramarital affairs. Sullivan, still running the broad range of COINTELRPO operations, had tapes of the affairs sent to King along with a letter urging him to commit suicide.

Ironically, documents leaked from the Soviet KGB after the Cold War revealed that the KGB was engaged in a similar operation. Convinced that America’s racial tension would ultimately lead to a U.S. collapse, the KGB were disheartened that King’s message for social justice did not advocate communism.

The KGB thus began “active measures” against King, seeking to discredit him by portraying him as an Uncle Tom figure beholden to Lyndon Johnson. As intelligence historian Christopher Andrew points out, King was therefore probably the only major American political leader to be spied on by both the FBI and the KGB.

COINTELPRO continued into the late 1960s and early 1970s, seeking to disrupt radical movements and at times endorsing police violence against its targets. Such was the case with Fred Hampton, a young and charismatic Black Panther whose organization was infiltrated by a COINTELPRO informant.

The informant reportedly provided a layout of Hampton’s headquarters in Chicago to the Chicago police department and drugged Hampton’s drink to ensure he would not wake up when the CPD came in through the front door. While the official record of the COINTELPRO-assisted police raid isn’t entirely clear on the events, we do know the CPD shot Hampton while still in bed.

COINTELPROs were occasionally targeted at organizations on the far right as well as the left. At Johnson’s urging, Hoover launched a COINTELPRO operation against the Ku Klux Klan in 1965, following the disappearance of three civil rights workers in Mississippi in the summer of 1965. The infiltration of the white power movement was reportedly quite successful. According to FBI agent Joseph Rucci, “There would be a Klan meeting with ten people there, and six of them would be reporting back [to the FBI] the next day.”

Nixon and Watergate

The legal framework surrounding intelligence gathering that evolved from Theodore Roosevelt through Lyndon Johnson (under the shaping hand of J. Edgar Hoover), foundered following its encounter with Richard Nixon. In Nixon, the surveillance community finally found an executive who both went too far with internal surveillance and was foolish enough to entrust his own intelligence gathering operation to incompetent men while leaving reams of incriminating evidence in his wake.

Nixon’s relationship with Hoover began the way it had with other presidents: Hoover fed Nixon useful political information. However, following a break-in at an FBI office in Media, Pennsylvania in 1971, which threatened to expose the full scope of the FBI’s COINTELPRO activities, Hoover shut the program down to protect the FBI.

Their relationship soon frayed; at the time of Hoover’s death in May 1972, Nixon was looking to replace him. Without Hoover’s political intelligence, Nixon set up his own inside-the-White House intelligence unit headed by former FBI agent G. Gordon Liddy and former CIA agent E. Howard Hunt. The “plumbers,” as they were called, broke into the Democratic Party campaign headquarters in 1972 and caused the Watergate Scandal.

Nixon’s relationship with the CIA also deteriorated. Helms, who had enjoyed a long career in the CIA prior to becoming its director, wanted to refocus the agency away from covert operations and towards more pedestrian intelligence gathering and analysis. Ironically, Helms would become the first DCI to be indicted for lying to Congress after being ordered to facilitate Nixon and Kissinger’s efforts against Salvador Allende in Chile and denying the activity in testimony on Capitol Hill.

However, Helms refused to play ball with the Nixon administration during the investigation into Watergate. Nixon wanted Helms and his deputy, Vernon Walters, to tell the FBI not to investigate the break-in due to a fictional connection between the break-in and the CIA, which both men refused to do. Nixon forced Helms out of the CIA.

The prolonged agony of Watergate revealed something of the shadowy world that had grown up to fight the Cold War. More was revealed when Hersh released, in December 1974, his story on Operation CHAOS. Further controversy came when Gerald Ford, in some spectacularly ill-chosen comments, argued that the CIA should not be investigated to avoid, among other things, details of assassinations.

The Church Committee: A Watershed in Law and Technology

In 1975, the Senate Select Committee on Intelligence, called the Church Committee after its chairman Frank Church (himself a subject of NSA surveillance), revealed the “family jewels” of the CIA. These included its domestic activities, some of its efforts to overthrow foreign governments, and several of its attempts to assassinate world leaders like Cuba’s Fidel Castro and Patrice Lumumba in the Congo.

Nothing like the investigative scale of the Church Committee had occurred before. Unlike previous slaps on the wrists, the CIA after the Church Committee would find itself under continual Congressional observation.

The Church Committee curbed the wildest excesses of the FBI, CIA, and NSA, such as the break-ins, unofficial infiltrations, disregard for Fourth and Fifth Amendment protections, and the disruption efforts against domestic political groups. For the FBI, legal avenues for the use of informants and undercover agents were expected to be the norm.

Methods of gathering intelligence were already changing dramatically by the time of the Church Committee, however. In the course of investigating the actions of the NSA, the Committee became concerned that electronic intelligence collection, by forgoing the need for physical penetration, could more easily circumvent the law and violate Americans’ right to privacy.

The Church Committee sought to establish some protections of civil liberties through the passage of the Foreign Intelligence Surveillance Act in 1978, which set up the FISA Court to provide a check on the wiretapping activities of the intelligence community. The FISA Court made keeping watch lists of specific individuals illegal; warrants issued by the Court would be required.

Located on the sixth floor of the Robert F. Kennedy Justice Department building until 2009, the court’s goal was to clarify the divisions between foreign and domestic intelligence. It was meant to ensure that various agencies of the intelligence and law enforcement communities could cooperate in monitoring potential threats to the United States while protecting civil liberties of Americans.

The FISA Court is only concerned with foreign intelligence gathering where it might intersect with domestic activities. The court was to demand “minimization” procedures from the general counsel seeking a surveillance warrant, meaning procedures to reduce the possible interception of the communications of U.S. citizens. Once someone leaves the United States, they lose FISA protections.

FISA represents a “compromise” between Congress and the intelligence community; although, with the secretive procedures of the FISA Court, it is not one that has much balance. Since 1978, according to the Electronic Privacy Information Center, FISA has heard over 17,000 requests and denied a total of 11. Such lopsided numbers, however, do not include times when the Court has demanded revision of a request before granting a warrant.

From September 11 to Snowden

In the 1980s and 1990s, the FISA Court, and the issue of domestic surveillance more generally, did not command much attention. That changed after the attacks of September 11, 2001. Just as the threat of anarchists and communists had inspired earlier regimes of domestic surveillance, so too did the terrorist attacks on the Twin Towers and the Pentagon.

Passed shortly after the attacks, the USA PATRIOT Act granted wide legal authority to the intelligence community. Among those new powers, included in Article 215, is the right to obtain business records, while imposing a gag order on the party holding those records. This law remains the bedrock behind the current NSA phone metadata collection campaign.

In addition to these lawful actions, the NSA also began a campaign of warrantless wiretaps, eventually revealed by the New York Times in 2005. Attorney General John Ashcroft was skeptical about this program. When Ashcroft was hospitalized, White House Counsel Alberto Gonzales went to his bedside and attempted to get a weakened Ashcroft to sign off on the program. Ashcroft refused, though the warrantless wiretapping campaign did not end until 2007. Gonzales became attorney general himself during George W. Bush’s second term.

What grew to replace the warrantless wiretaps after 2007 was the series of programs revealed this summer by Edward Snowden. The NSA began to run some of its operations through the FISA Court.

Declassifications after Snowden’s revelations show that in 2011, the FISA Court found the NSA had acted illegally in intercepting U.S. digital traffic through the Prism program in addition to foreign activity. Some 56,000 domestic communications were collected each year in the three years Prism operated before 2011.

In its collection of metadata, however, declassified records have shown that not only has the Court generally supported the NSA, no telecommunications company has challenged the order to hand over their records. As telephone service has transformed and the internet has grown, the NSA has apparently proven adept at using those telecommunications as an unprecedented intelligence gathering tool.

Additionally, more recent revelations indicate that at times the NSA circumvented the FISA Court entirely. In addition to its court orders for telecommunications records, for example, the NSA, in conjunction with British intelligence, also broke into the main communication links between Yahoo and Google data centers. Since many of these centers are overseas, the NSA could potentially access the data of American citizens through foreign surveillance, a consequence of the international nature of telecommunications infrastructure.

Aside from questions of their legality, however, the ultimate utility of these programs is unclear. The NSA claims its programs have disrupted more than 50 terror attacks. However, the FBI reportedly calls the leads generated by the NSA “ghost-chasing.” The FBI’s current director, Robert Mueller, apparently complained to NSA director Keith Alexander that the NSA’s leads were “a time suck” for his agency. This observation would suggest that, unlike previous domestic intelligence efforts, the FBI is not cooperating with the NSA in an extralegal domestic campaign.

The fact that this information is known through the FISA Court, and that the FISA Court rejected as illegal some of the NSA’s activities, is proof for some that the FISA system works. Benjamin Wittes, recently writing in The New Republic, argues that, as all U.S. courts are not investigative agencies, the fact that it ruled on NSA activities at all demonstrates that the NSA has been keeping it informed.

The Future of Domestic Surveillance

That the NSA was found to have violated the law at times is not surprising in the context of previous domestic surveillance programs. Under pressure from existential threats both real and perceived, and under the control of the executive, intelligence and law enforcement agencies have pushed to the very limits of the law and beyond in pursuit of their goals.

The NSA’s wire-tapping program before 2007, and its continued collection of domestic U.S. intelligence, represents a more technologically advanced form of the kind of wire-tapping carried out as far back as World War I.

The NSA’s current activities also dwarf previous domestic surveillance programs. Rather than focus on specific individuals or groups, the NSA has taken a mass approach. Every day, the NSA intercepts some 1.7 billion various communications, ranging from telephone calls to emails to tweets. Some of the programs revealed this summer were not actually conducting surveillance, but rather were designed to search through the oceans of data collected.

Beyond its scale, the current NSA program is particularly noteworthy within the history of domestic surveillance because it has developed under a specific legal regime. That regime has been criticized as being weak and lacking oversight; even under its generous provisions, the NSA has occasionally blatantly violated the law since September 11.

Yet, this more legal footing is still a considerable change from earlier campaigns, and stems from the exposure of many of those earlier activities in the 1970s. Though at times frustratingly opaque, the FISA Court provides at least some level of control previously lacking in domestic intelligence affairs. Whether Congress, in its current state, could organize something akin to the Church Committee seems unlikely.

The NSA programs of today do not appear to be matched domestically by the kind of action seen by the FBI and the CIA in the 1960s, such as the infiltration of suspect groups that were perceived to be political threats to the United States. There is apparently nothing of the violence and illegality of COINTELPRO.

Regardless of the ultimate fallout of these recent revelations, however, something like the NSA’s activities will almost certainly continue so long as there is the perception of an existential threat against America—a threat that the government believes intelligence agencies must be directed against.