SIEM Log Aggregation and Parsing

Omer Gunal
November 29, 2020
Detection Engineer
SIEM Log Aggregation and Parsing

The first place where the generated logs are sent is the log aggregator. We can edit the logs coming here before sending them to the destination. For example, if we want to get only status codes from a web server logs, we can filter among the incoming logs and send only the desired parts to the target.

If you have no lab environment, you can use our blue team lab

log management

Aggregator EPS

What is EPS?

EPS is an event per seconds. The formula is Events/Time period of seconds.

As the EPS value increases, the aggregator and storage area that should be used also increases.

Scaling the Aggregator

More than one aggregator can be added so that the incoming logs do not load the same aggregator each time. And sequential or random selection can be provided.

log flow

Log Aggregator Process

The log coming to the Aggregator is processed and then directed to the target. This process can be parsing, filtering, enrichment.

log aggregator

Log Modification

In some cases, you need to edit the incoming log. For example, while the date information of most logs you collect comes in the format dd-mm-yyyy, if it comes from a single source as mm-dd-yyyy, you would want to convert that log. Another example, you may need to convert UTC + 2 incoming time information to UTC + 1.

Log Enrichment

Enrichment can be done to increase the efficiency of the collected logs and to save time.

Example enrichments:

  • Geolocation
  • DNS
  • Add/Remove


The geolocation of the specified IP address can be found and added to the log. Thus, the person viewing the log saves time. It also allows you to analyze location-based behavior.


With DNS queries, the IP address of the domain can be found or the IP address can be found by doing reverse DNS.

letsdefend description card

You might also be interested in ...

Start learning cybersecurity today