About Hunters – SOC platform

Hunters SOC platform is a purpose-built, turn-key security data and analytics platform. It’s a modern solution that provides cloud-scale access to telemetry data across the entire attack surface coupled with an intelligent automated event correlation, investigation and prioritization.

Hunters׳ is a group of cyber and tech experts looking to revolutionize security operations by combining data engineering. The solution combines security expertise, data engineering and layers of automation to accelerate decision making.

The Challenge

One of Hunters’ core backnes is a component based on a large scale number of Apache Flink applications running simultaneously while new jobs keep coming. Hunters were out to look for an alternative and perhaps better way of deploying and running flink applications. In that time, Hunters’ team incharge was under a lot of work preparing for a major flink version upgrade and needed a hand offloading the benchmarking and analysis of an alternative. The requirements were mainly benchmarking reasonably similar workloads in terms of data volume, velocity and stateful stream processing characteristics such as windowing and watermarks. 

The Solution

CloudZone’s team is heavily invested in open source as a mantra. Aside from cloud vendor services and 3rd party solutions which we deliver, we also work with many open source tools, especially in the area of data processing & analytics. The Data team has flink programmers within and could immediately understand the customer needs. Upon agreeing on the best alternatives to benchmark and defining clear KPIs for the benchmark, CloudZone developed a MVP Java Flink application to imitate the workload in a sufficient way. Since the focus of the benchmark were the deployment & orchestration, imitating data volume and velocity and stateful aggregations was sufficient enough for the tests.

The results

During the tests which were according to customer scenarios defined in the design phase, all relevant metrics were recorded with proper monitoring we setup and were wrapped in a well organized benchmark summary document delivered to the customer.