Log retention on a startup budget
Every early-stage team eventually hits the same moment: the logging bill grows faster than the product does. The instinct is usually to just retain less — cut everything down to seven days and hope nothing important happened eight days ago. That works until it doesn't, usually right when you need last month's data for a security review or a customer escalation.
The better fix isn't retaining less. It's retaining differently, based on how likely you actually are to query something again.
How we think about tiers
- Hot (7–14 days): Fully indexed, fast search, used for active debugging and on-call. This is the expensive tier, so keep it short.
- Warm (30–90 days): Indexed but on cheaper storage, slightly slower to query. Good for "what happened last sprint" questions.
- Cold (90+ days): Compressed archives, not indexed, restored on demand. Cheap enough to keep for a year or more, rarely touched.
The trick is deciding what goes in each tier by source, not blanket policy. Application error logs and security audit events earn a longer hot window. High-volume debug logs from a single noisy service usually don't need more than a few days anywhere.
A rule of thumb that's served us well
If a log source hasn't been queried manually in 30 days across your whole team, it's a strong candidate for a shorter hot tier. We review this quarterly rather than guessing once and never revisiting it — usage patterns shift as the product changes.
What this actually saves
For a team our size, moving from "index everything for 30 days" to a tiered approach cut our storage spend by roughly half, without losing the ability to answer questions about anything in the last year. The data didn't disappear — it just got cheaper to keep around.