Reducing log volume not only has the benefit of reducing your cost, but it also reduces log noise. Below are tips we provide to customers wondering how to reduce their log volume. They're ordered by their effort to return ratio.
When it comes to log filtering, you have two choices: filter application side or filter Timber side. We highly recommend starting with the application side.
Filtering your logs on the application is much more efficient because it reduces the log burden on your app, avoiding the overhead of sending the log data to Timber entirely. There are a few ways you can do this:
Simply remove the log statement. If you don't find it helpful then it makes sense to remove it entirely.
Move the log statement to a lower level, such as
debug, and then configure your app to log at a higher level, such as
info. This is a great strategy because you don't lose the ability to generate that log if and when you need it. You can simply drop your log level down to
If you find that you cannot filter your log on the application side then we recommend filtering it on the Timber side. Timber offers a log filtering feature that allows you to drop logs containing specific key words. More information on that here:
As an alternative to filtering logs entirely, you can sample them at a rate. This simply reduces the volume, and at high numbers a small subset of logs usually provide enough visibility as to not reduce utility. Depending on your language, your sampling implementation will will differ. To keep things simple, we recommend starting with random sampling. For example, in Ruby you could do:
if Random.rand(3) == 0logger.info("Log message")end
This effectively reduces the log volume to 30% of the original. More advanced sampling strategies will rate limit logs based on time, or bucket logs based on a context factor.
If 1 and 2 above do not solve your problem, then you'll need to look at ways to trim your log size. Because Timber is focused on structured logging, there is possibility that your log metadata could be larger than expected. In this case, we recommend trimming metadata to only the attributes necessary.