Fluency log4j Response

Fluency launched an all-in effort to support its clients with the recent Apache log4j vulnerability issue that led to CVE-2021-44228 and CVE-2021-45046. In doing so, we were able to provide tools to identify indicators of compromise (iocs) and assist our clients in identifying possibly exploited systems.

Background

Researchers identified a vulnerability that could be remotely exploited in a commonly used java library by the Apache Foundation called log4j. This library is used extensively in applications and games for its logging capability. The root of the vulnerability resided in the ability to make jndi (Java Naming and Directory Interface) calls arbitrarily. An attacker could craft a string that could invoke jndi to download and execute arbitrary code from the internet.

The vulnerability was so widespread that even large services provided by Facebook, Google, and Apple were affected. The mitigations required users to upgrade their vulnerable software (where updates were available) or to block affected services from the internet (not always possible). Since most users did not have an immediate update path, they needed to rely on monitoring to identify possibly exploited services and web applications.

Initial Notification

As soon as Fluency became aware of this vulnerability, we began notifying our clients of the issue and giving initial advice on how to detect it. Fluency communicates with its clients in several ways, including a dedicated Slack made available to clients, e-mail, Microsoft Teams, and other methods as appropriate. This allows Fluency to provide real-time intelligence that it feels is important enough to be broadcast.

Initially, as researchers were still learning the scope of the vulnerability, Fluency provided a background to the issue as well as multiple sample searches that clients could use. Fluency allows Lucene and Regular Expression searches through its interface, and furthermore allows searching of not only flow data but also a collection of event data across clients various services. These real-time communications channels were used throughout the initial period to update clients on the issue.

Custom log4j Facet and Behavior

Due to the severity of this vulnerability and the widespread use of the log4j library, Fluency created a custom facet as well as a behavior within its tool for all of its clients to use. This custom facet setup a regular expression search through client events looking for indicators of compromise (or attempts at compromise). Clients could immediately use this, which the public knowledge at the time of the vulnerability, to identify problematic events.

The behavior was deployed to client sites in order to automate and alert upon detection of attempted exploits within client event logs. It can be found in Event Watch with the name Log4j_RCE_Exploit. It also makes use of Fluency's Regular Expression Engine.

Fluency Analysis

Not being content with giving our clients the tools to search for indicators themselves, Fluency co-founder Kun Luo, himself, conducted a search across all of the Fluency clients, identifying multiple indicators of compromise (both successful and attempted). He immediately notified the affected clients so that they could investigate and/or remediate.

Aftermath

Since the initial effort, Fluency has continuously updated its support as new information has come to light regarding the vulnerability. Several clients that were affected were able to remediate and begin their update process (or block process) where feasible. This is just one example of where Fluency provides real-time support to its clients when the proverbial ‘stuff’ hits the fan.