Skip to content

Feature Request: Time-Bounded Log Slicing for /var/log/messages and journald (--since / --until) #308

Description

@dmivor

Problem Statement

In customer environments where /var/log/messages is unrotated or when -l (additional rotated logs) is enabled, supportconfig can collect gigabytes of irrelevant historical logs for incidents that occurred within a specific time window (e.g., a single day or specific outage timeframe). Current line-count (-I, VAR_OPTION_MSG_LINE_COUNT) and size limits (-X) rely on tail, which only captures the most recent events and risks missing older incidents on busy hosts.

Proposed Solution (Inspired by hb_report)

Introduce time-window log filtering inspired by hb_report's -f/-t mechanism, built with standard core utilities (date, awk, journalctl):

  1. CLI / Config Options: Add --since <time> and --until <time> options (avoiding collisions with existing -f / -t directory flags), backed by VAR_OPTION_LOG_FROM_TIME and VAR_OPTION_LOG_TO_TIME in supportconfig.conf.
  2. Zero-Dependency Normalization: Use GNU date -d to parse human relative dates ("yesterday", "2 hours ago") and absolute ISO timestamps into Epoch seconds and ISO-8601 strings.
  3. High-Performance Syslog Engine: A single-pass awk stream filter that processes both RFC 5424 ISO and legacy BSD timestamps while preserving multi-line stack traces without high memory overhead.
  4. Rotated Logs Optimization: Filter messages-*.xz archives by file timestamp before streaming via xzcat | awk, avoiding local disk decompression bloat.
  5. Full SLES 12 LTSS to SLES 16 Compatibility: Strictly uses Bash 4.3+, GNU Coreutils, and Awk available across all SLES releases (including SLES 12 SP5 LTSS).

Full Architecture & Implementation Plan

A detailed architectural feasibility study, component breakdown, and phased implementation roadmap is documented here:
https://docs.google.com/document/d/1gPazSN8t-SgPQo41dJqDkFYPev2haJrGd1egyuLiPnI/edit

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions