sanitizing /var/log

I don’t know about you, but I like to have my system logs split day by day, particularly on busy machines. I also like to have a full timestamp (including year and timezone thank you) in system log files.

The venerable syslog package can not do these things – but syslog-ng can.

On Debian/Ubuntu, the default syslog-ng config file is set up to mimic the old way of logging – dump everything in various files under /var/log/, and let logrotate rotate the logs. That’s understandable from an upgrade path perspective but it does not exploit syslog-ng’s strengths at all.

Here’s how I configure syslog-ng for a machine that logs locally as well as to a remote syslog-ng server:

options {

source inputs {

destination logpile {
        template("$ISODATE $MSG\n") template_escape(no)
        owner(root) group(root) perm(0600)
        create_dirs(yes) dir_perm(0700));

destination remote { tcp(""); };
log { source(inputs); destination(remote); };
log { source(inputs); destination(logpile); };

This config puts logfiles in /var/log but splits them up per day with a filename like syslog.YYYYMMDD. It also makes sure that the first element on each line is a fully qualified timestamp.

I also have a small script that runs from cron


# This script moves log files older than 3 days from /var/log/ to
# /var/log/archive/, in bzip2 format. It assumes you've set up syslog-ng to
# split logs per day and put them in /var/log. The script also maintains a
# symlink to the most recent logfile in /var/log/.
# Run this script from cron, ideally as close to midnight as possible so that the
# symlink points to the current logfile, always.
# Ward Vandewege, 2008-02-12

THREEDAYSAGO=`date --date='3 days ago' +%Y%m%d`
YEARTHREEDAYSAGO=`date --date='3 days ago' +%Y`
MONTHTHREEDAYSAGO=`date --date='3 days ago' +%m`
TODAY=`date +%Y%m%d`

if [ ! -d /var/log/archive ]; then
  mkdir /var/log/archive

if [ ! -d /var/log/archive/$YEARTHREEDAYSAGO/$MONTHTHREEDAYSAGO ]; then

for LOG in syslog mail cron daemon auth authpriv; do
  if [ -f /var/log/$LOG.$THREEDAYSAGO ]; then
    bzip2 /var/log/$LOG.$THREEDAYSAGO
  if [ -h /var/log/$LOG ]; then
    rm -f /var/log/$LOG
  ln -s /var/log/$LOG.$TODAY /var/log/$LOG

This script keeps 3 days worth of logs in /var/log. Older logs are compressed and archived under /var/log/archive/YYYY/MM/. The script also maintains a symlink in /var/log for each log file syslog-ng generates, pointing to the current log file. I run this script from cron a few minutes after midnight.

You’ll also want to remove /etc/logrotate.d/syslog-ng, since that file serves no purpose anymore.
The combination of syslog-ng and this maintenance script keeps /var/log in order with log files split up per day. It also keeps the most recent log files available right in /var/log, and the rest only a few keystrokes away.

This entry was posted in Sysadmin. Bookmark the permalink.

Leave a Reply