Fluency Blog Capturing All the Data

12 March, 2019 by Al Wissinger

Capturing All the Data

IT environments grow more and more complex each day. As they do, so does the depth, breadth and footprint of the resulting data. All too often I hear InfoSec folks explain that their current legacy SIEM tools are simply unable to capture all the data they need to realize “ground truth” in their threat analyses. The causes are clear: systemically, the costs of both storing this data and analyzing the increasing number of events per second (EPS) are escalating and often prohibitive. In addition to the escalating data size and costs, many systems lack the ability to correlate this data to active users.

At Fluency Security, we architected our advanced SIEM around the cloud using our own patented database. It allows for petabytes of data to be ingested daily (see graphic above), and current clients are already ingesting terabytes daily and working with billions of events. We believe that real-time situational awareness based on “ground truth” will help our client analysts identify and remediate threats more quickly, thus protecting their data and clients’ privacy.

Our design model seeks out all input sources like syslogs, active directories, Microsoft Office365, network traffic and more. We help all our clients write parsers and set up agents to ensure this ever-increasing amount of data can be captured in order to realize “ground truth” analysis. Built into the solution is multi-tenant capability, which enables different divisions to be separated. An MSSP can even use this capability for multiple clients feeding into Fluency’s data hungry solution.

Many sources of data like CrowdStrike don’t or can’t provide the active IP of the originating source, causing further analysis time and slowing the meantime to detection. With Fluency’s patented correlation methodology, these results can be aligned back to individual users allowing for security teams to take quicker remediation action. Additionally, Fluency uses a machine learning model similar to Twitter, which seeks anomalous behavior across a rolling 30-day period.

Fluency has incorporated GDPR privacy standards into our solution which ensure all data coming into our system is privatized. Using the pseudonym approach defined in the GDPR articles we work with our clients to define all known privacy fields. Then, we insert pseudonyms in their place. We retain a master key file which only privileged access can reveal. This approach will help our clients meet the log management privacy requirements for GDPR, CCPA, PIPEDA and others. By contrast, certain legacy SIEM tools use a single hash token for privatization. With this approach, when faced with a request to be forgotten, the token can be regenerated. But with Fluency, the pseudonym is erased; the log file is retained, and all is well with regulations.

Handling massive volumes of data is troubling at best. However, when using Fluency’s solution, you’ll be assured all relevant data can be ingested, privatized, correlated and risk scored so security teams have “ground truth” leading to real-time situational awareness.