The government’s Investigatory Powers Bill, or “The Snoopers’ Charter” as it’s been dubbed in the press, has sparked debate over the balance between privacy concerns and national security. The controversy surrounding encryption, bulk data collection and hacking have all been drawing the big headlines. But last week Theresa May outlined the revised bill, which many were surprised to find had been toughened up despite the criticism it received.
Part of the newly outlined bill requires service providers to keep “internet connection records” detailing the browser history of every citizen for 12 months. Putting the privacy and security debate to one side, the impact this bill will have on providers is massive. The sheer volume of this data will prove incredibly testing for service providers to manage – putting their current data storage capabilities to the test.
Where have you browsed?
In practice the information would take the form of an itemised list of each citizen’s browsing history. This would not be a list of the specific web pages, but the main domain name which a basic online footprint can then be drawn up from. This information could then be requested by officials for use in court and investigations.
There will be a further challenge to ISPs to manage the security of this very private data. Whilst there are clear rules proposed for official access, keeping this data securely for 12 months will require extensive cyber and physical security measures. Citizens’ browsing histories would be a rich prize indeed for hackers.
This new legislation will force service providers to re-evaluate their storage and data centre requirements in order to manage the volume of information the bill will create. The storage costs involved will be high – and it’s still unclear who will be footing the bill. Will it be the government or the hosts? And what will happen if the service providers cannot physically store data on this scale; will government agencies take it on?
Still, it’s an opportunity…
Regardless, data storage on this scale is an opportunity for co-location providers and could prove to be an entry point for the adoption of virtualized data centres. Software defined networking (SDN) brings the flexibility and scale needed to manage this size of data flow more efficiently and cost-effectively.
Traditional data centre networks direct traffic based on a limited view of the data carried. The software and hardware are coupled to manage the traffic flows – and are inherently ineffective at meeting the demands of “Big Data”. Expensive to maintain, expensive to scale, traditional facilities leave networks with vast areas in need of improvement but unable to do so through vendor lock-in. To properly manage their data, data centres need to be optimised to handle compute, storage and networking needs – and the Snoopers’ Charter could become the pressure point to force operators to virtualize.
With a central SDN solution, the network routing can be customised more easily, allowing providers to shape it to the specific interests and needs of that datacentre. In the case of managing data in relation to the Snoopers’ Charter, like internet connection records, data centres need to be able to hold the vast quantity of data generated and pull the relevant records needed in relation to an individual quickly at the request of police or government.
The Snoopers’ Charter has sparked debate, that much is clear, but it has also highlighted how many organizations just cannot manage vast amounts of data effectively. Looking to the future the amount of data individuals, businesses and society generate is going to continue growing at an exponential rate. Data centres, as the backbone of internet, need to modernise the management of this data now – and perhaps a controversial bill from Teresa May will get service providers acting.