Fifty years ago last year, the US government first proposed a “data center”. Not only was this surprisingly close to the data centers of today, it caused an outcry over possible privacy breaches, which will still has strong resonances today.

In 1965, the government proposed creating a National Data Center, which would centralize statistical data from Federal bodies. The data would be kept on magnetic tapes so it would be easy to refer to.

Almost forgotten now, the proposal caused a huge storm in Congress and amongst the public, who felt such a thing would infringe their privacy, and was eventually abandoned in 1968.

Is privacy a trade-off?

This sounds a little like the arguments over the NSA’s ability to snoop on electronic communications, but it was actually quite a different debate. I first heard about it in an article from EMC, by marketing SVP Chris Ratcliffe, which says: “The original proposal inspired a great deal of Big Brother-esque debate, and accelerated our collective awareness of matters of privacy and control,” before concluding that today’s consumers are sophisticated enough to accept a privacy trade-off, and now “expect that the organizations they interact with will use data to understand them better and serve-up more accurate and efficient user-experiences.”

Well, that’s exactly the message you’d expect from today’s computer industry. But the full story of the Federal Data Center is well worth looking into.

The proposal emerged from requests by social scientists for better government data to work with, and the Census Department led the debate (albeit reluctantly), according to Statistical Déjà Vu: The National Data Center Proposal of 1965 and Its Descendants, a paper by Dr Rebecca Kraus of the Census Department’s history staff, published in the Journal of Privacy and Confidentiality in 2011.

Kraus says the Census Department was handed the job of developing the data center proposals, and stumbled unaware into a massive privacy row. The proposal progressed through various levels of committee reports, after Yale economist Richard Ruggles’ report in April 1965 suggested the data center should “have the authority to obtain computer tapes produced by other agencies,” and aggregate that data for research purposes.

The idea caught the attention of Lyndon Johnson, who commissioned the committee that proposed the data center.

As years roll by, pressure will mount to program into the computers more and more information on individuals

Cornelius Gallagher, 1968

Lie detectors

The committees working on the idea knew that data was given to the government under a pledge of confidentiality, so it should be “disguised” (anonymized) and aggregated so confidential data wouldn’t be disclosed.

But this was the 1960s, a time of “lie detectors, psychological testing [and] background investigations of job applicants”, says Kraus. The academics proposing the idea got lambasted in Congressional debates by representatives who saw this as a “total information system” which would compile dossiers on individuals.

Even if that wasn’t in the initial proposal, Cornelius Gallagher, Democratic representative for New Jersey, warned: “our concern is what an innocent statistical center could turn into as the years roll by and pressure mounts to program into the computers more and more information on individuals.”

Congress feared that even an innocent data center would be plagued with errors and distortions, and open to misuse, accidental breaches through remote access and outright violations of privacy rules.

The public debate got even more heated, It was denounced by the Daughters of the American Revolution in 1967, and other bodies weighed in.

In a sermon in Washington Cathedral, Canon Michael Hamilton supported the idea of creating the data center, “for not to do so would be to lose faith in ourselves as a nation and in the flexibility of our political system to adapt creatively to change.”

Meanwhile, Rabbi Norman Lamm said “the whole sense of Jewish law and universal morality must reject such a plan as abhorrent.”

In August 1968, the House Special Subcommittee on Invasion of Privacy strongly recommended that the creation of a national data center be postponed until the technical requirements for protecting privacy could be fully explored.

EMC clearly feels we’ve worked it all out, but it seems to me we have just come to accept that big data sets will be collected and used and there’s not much we can do about it.

It also seem to me, as I try not to look at the US election roadshow, that the quality of political debate hasn’t progressed much either.