A Green Database

by Christopher Jordan

Oct 21, 2019

Datacenters are basically toxic computer equipment in a constantly cooled warehouse. Their footprint is growing across the globe in places like Loudoun County, Virginia. A place not known for its cold weather. Much of this growth is based on companies not knowing how to be efficient in the cloud. The scaling of inefficient code and processes means significant amounts of computer resources are needed. To address this environmental impact, cloud code needs to be more efficient and smarter in how it scales. This is the aim of a green database.

Green Database Benefits:

  • Less Negative Environmental Impact (less systems, physical waste and electricity)
  • Less Cost
  • Faster Searches

  • Fluency has already invested six years in developing LavaDB to be a green database. Fluency’s objective is to advance technology and change the economics of the log analytics industry. Fluency does not simply seek to lower the cost of log management and data retention; instead it aims to lower the cost until it is practical to ingest and analyze everything. A green database is the cornerstone of this mission.

    Why the Cloud needs to be Green

    When we think of saving the planet, we think of climate change, carbon release and pollution. We blame things like our cars, drinking bottles and plastic straws, for what needs to be changed. But when we look at the Internet industry and how data is doubling in volume every two years, the processing, storing and searching of this database has a significant impact on the environment. Today’s inefficiency is often seen by companies in the high cost of a cloud project, as there is a direct relationship between cloud cost and the physical cloud presence.

    Cloud systems scale painlessly. You do not have to wait for a machine to be delivered, installed and configured. With a click of a button or a call of a process, a new system or drive is requested and put into use. The ease of this process allows companies to quickly scale a process to hundreds of machines. IT departments can allocate new storage without facility impacts or delays. The image of the cloud being somewhere else makes the physical issues of infrastructure disappear to the decision maker.

    Scaling is a two-edged sword. There is unlimited power, but it comes at a price. Bad code, or the use plain average code, scales. And a small inefficiency scales to a rather large one quickly. This shows up in the bill. But it also physically shows up in the growth of datacenters.

    If you fly out of Dulles Virginia, you will see a growing landscape of datacenters. Datacenters house massive amounts of servers. The most notable part of a datacenter is an equal amount of air conditioning units. These oversized units compose entire walls, hidden behind vents. There is nothing cost-effective about placing datacenters in a Southern state.

    Datacenters are called that, for their primary purpose is the storing of data, which doubles every two years. With the storing of data, the processes that use datacenters also move into the cloud. In order to reduce the environmental impact of datacenters, we need to reduce the cost of storing and analyzing data. The cost is environmental, but in reducing environmental impacts (number of processes and amount of storage), we also reduce the overall cost. Green databases are good for business.

    What is Green?

    What defines a green database? A green database needs to do two things. First, the compression of raw data to the stored data needs to be below 20%. This is consistent with the general rule of 1:8 compression. The difference between these two numbers is the database needs to provide high-availability/high-durability. Second, the speed of the database needs to be within the N Log N search. Besides storing data, the processing power to search and maintain the database impacts both why people use it and the electricity it needs.

    Fluency’s green database, LavaDB, aims at using one-eighth (12%) of the electricity and physical infrastructure derived from the raw data it receives. That is four times better than any other commercial or open source database being used today. This relates to a reduction of the overall datacenter environmental impact by 75%.

    Fluency aims to make audit storage cost effective to the point that a company can save all their logs at a lower price than the equipment generating those logs. If you have ever paid a bill to a cloud SIEM vendor, you know this cost.