The Splunk Gap
The cost of buying and operating Splunk for security information and event management (SIEM) creates a security gap. Splunk is a good tool, it has some key limitations. The main limitation is cost - Splunk charges for data ingestion and retention, so organizations often have to limit what data they send to Splunk to control costs. This creates a 'Splunk gap' where relevant security data is not ingested and retained. When a breach occurs, the lack of full historical data makes investigation and root cause analysis difficult. The article advocates using a solution like Fluency, which can ingest and retain all security data at a lower cost. With full data retention, organizations can detect threats early and perform robust investigations when breaches occur. Key points are that comprehensive data collection is critical for security, but Splunk's pricing model forces organizations to selectively collect data. Fluency can augment Splunk by ingesting and retaining all data at an affordable price, closing the Splunk gap.
Splunk is an Application Performance Management (APM) tool that has been extended to handle security information event management (SIEM) processes. In particular, the edition of Enterprise Security allows for the discovery of breach events in the audit logs. With Splunk’s ability to store data from a large set of applications, and its ability to index and display data, the tool is well suited for security data analysis. Three areas are commonly known as gaps in Splunk’s operational use are:
High Operations Staff
How Does Fluency SIEM/Platform fills these gaps.
Choosing where to store data
Fluency Platform is the underlying backend to our SIEM. It performs streaming processing of messages prior to the data storage. Message processing is critical to the use and capabilities of SIEM. Fluency Platform performs:
When it comes to storage costs routing creates flexibility. Fluency Platform can choose to where to route the processed message.
Send those messages that support "Splunk Notables" to Splunk.
Those elements that support investigations or validation to Fluency's SIEM
Noise data that needs to be still stored for compliance to inexpensive storage.
Useless data can be discarded.
The results is a data storage approach that supports operational processes with the greatest reduction in storage costs.
Note, you can learn more about Fluency Platform here.
Detailed data is often recorded for the sole purpose of creating and alerting on metrics. Since Fluency platform can create complicated metrics, the detailed data can be routed to the noise category or just discarded. This results in cost saving for computing metrics while greatly reduces the storage cost of messages used for metrics. Lastly. metrics data has advantages over raw data being stored then computed. First, the amount of stored data is smaller, and the alerts are immediate. Secondly, trend analysis overtime is easier with metrics than by searching raw data.
Performing searches of large amounts of data causes problems for most SIEMs, and that is true for Splunk. Fluency's processing language allows complicated searches to span message data, entity data, and API calls form a single program. This program can map-reduce results over large spans of time and volume, avoiding the limitations that are normally placed on searches. Fluency was an RSA sandbox finalist for our database technology. In short, historical searching has always been stronger in Fluency than any other solution.
Fluency is a pure SaaS solution. There is no need to hire a database expert or system administrator to maintain the system or perform upgrades. Many of the functions that require a Splunk engineer to code solutions is not needed, as they are built into the system. In general, our clinets find they can put their best engineers into other security needs.
Splunk is a good solution when it is already in place and supporting defined processes. It comes at a cost to operate. The need to address cost often puts focus on maintaining staff needed to operate Splunk and paying for many of Splunk's add-ons. This results in organizations lowering the cost of Splunk by lowering the data analyzed and retained. Without data to analyze, the reduced data entry creates security gaps. Fluency Platform and Fluency SIEM can reduce the overall cost of operating Splunk to fill in these gaps, while at the same time reducing the cost to operate.