The Big Data Security Transformation

For those of us in the security profession it an extremely exciting but also daunting time.  The number and aggressiveness of threats are increasing while, at the same time, governmental bodies are requiring more and more for compliance. This growing challenge is being accompanied by the “Big Data” movement.

For those of us in the security profession it an extremely exciting but also daunting time.  The number and aggressiveness of threats are increasing while, at the same time, governmental bodies are requiring more and more for compliance. This growing challenge is being accompanied by the “Big Data” movement.

Big Data is truly innovating the security profession like never before. In security terms, Big Data doesn’t simply mean lots of data; it demands significant intelligence to analytics to spot threats early on with infrastructure to collect and process data at scale. Today’s security systems still have long ways to go before being fully integrated in true big data sense. Security professionals need to be able to get increasing value from the data they already collect and analyze on top of the data they still are not getting.

Data from both IT and business is at the point where old school ad hoc processing simply will not work anymore, but much of the security industry is still doing things this way and it’s giving cyber attackers the upper hand. For example, according to the Verizon Breach Investigation Report, 91% of breaches led to compromise within days or less, but 79% of these took weeks or more to discover! Obviously this is a huge issue and shows that our defenses are falling behind attackers. The reasons for this are numerous, but I see three keys as to why:

1. Attackers are getting more organized and better funded – attacks are dynamic but defenses are still very much static in nature.
2. IT has becoming more and more complex – organizations are now more open and agile resulting in new opportunities for communication, collaboration but also increases vulnerabilities.
3. Compliance has grown much more far reaching and business are having a harder time keeping up with keeping controls in place to ensure proper management of them.

Implementation of Big Data in security is no longer a want, it’s become a necessity. Implementation of the big data methodology into security has three foundational elements: Threat Intelligence, Analytics & Visualization and Scaled Out Infrastructure.

Threat Intelligence

Threat Intelligence encompasses two major views to complete a holistic knowledge of what is occurring at all times. This means that not only do organizations need to fully understand their organization internally, but they must also have plentiful information on the currently external threat environment. Only then can security teams have a full view to correlate risks and events with clarity. Big data allows organizations to not only gain internal insight but also the major external data points for this correlation, a state that far too many security teams still lack.

Analytics & Visualization

The setup of analytics and visualization tools need to support the variety of security analysts and their specialties. For example, managers will most likely only need high-level visualizations and trending, while network forensics need to fully reconstruct all log and network information about specific sessions to determine exactly what happened.

Scaling Infrastructure

Internal infrastructures need to be able scale with agility to responding the ever changing IT environment, supporting new applications and methods of delivery like virtualization cloud computing and outsourcing. The security management infrastructure needs to have access to collect and manage data from all these at an enterprise scale.

Big Data Drives Efficient Security

One of the largest areas lacking in today’s security environment is efficiency but big data can provide dramatic advances in this in a number of ways:

1. Eliminate manual tasks – Systems need to reduce the amount of manual repetitive tasks in investigations, like toggling between consoles. While it’s not possible to do this overnight, steady movement away from manual tasks is a key.
2. Use context to highlight largest issues – Understanding the underlying business context is a key to prioritization of issues. A map between applications and the business process they support is highly important and Big Data provides this.
3. Present only the most relevant info – Big Data enables the elimination of noise to allow a focus on high impact issues along with supporting data to highlight what the likely problems are.
4. Include human comprehension – This also enables the reduction in analysis of the wrong items. Providing a built-in ability to identify issues using a level of human like intelligence allows security analysts to analyze only the most crucial issues.
5. Predict future threats – Not only does the system need to defend against modern security risks but also include a predictive model that takes external threat data and internal situational awareness which moves a security group from passive to active.

(image: big data security / shutterstock)