Naim Kabir
Nov 19, 2020

--

This is interesting, but then you run into issues when you have dynamic fields--traceIDs, customerIDs, custom params, etc. etc.

What you're describing functionally kinda reduces to gzipping your logs before sending them over the wire, though, no? Or some similar compression.

That wouldn't help as much with the anomaly detection piece, which you describe--but that CNN method wouldn't work super duper well with dynamic fields either unless you were sure every field had a strict set of values they could be. If you're working with real values like times, dollars, etc. how would we get it to work?

--

--

Naim Kabir
Naim Kabir

Written by Naim Kabir

Engineer. Focused on experimentation, causal inference, and good software design. naimkabir.com

Responses (1)