I built goxe, a tool designed to tackle one of the most annoying (and expensive) problems in infrastructure: log spam.
When a service starts failing, it often floods your pipeline with thousands of identical log lines. This doesn't just create noise; it increases bandwidth usage and inflates storage costs in platforms like Datadog or Splunk.
Goxe sits in your pipeline (currently via Syslog/UDP), normalizes incoming logs by stripping timestamps and variable noise, and aggregates them in real-time. Instead of 10,000 lines of the same error, you get one line with an occurrence count: error: connection failed (x10000).
Key Features:
Performance: Written in Go with a worker pool architecture and streaming design for low memory overhead.
Normalization: Strips dates, converts to lowercase, and removes whitespace to ensure near-identical logs are grouped correctly.
Clustering: It handles similarity clustering to catch messages that are almost the same.
Transparency: Apache 2.0 license.
Repo: https://github.com/DumbNoxx/Goxe
Thanks for checking it out!