More nxlog logging tricks

In a previous post I talked about "Sending Windows logs to Papertrail with nxlog". In the mean time I had to work through a couple of nxlog issues that weren't quite obvious to solve -- hence this quick post.

Scenario 1: You don't want to send a given log file to Papertrail

My solution:

In this section:

# Monitor MyApp1 log files 
START_ANGLE_BRACKET Input MyApp1 END_ANGLE_BRACKET
 Module im_file
 File 'C:\\MyApp1\\logs\\*.log' 
 Exec $Message = $raw_event; 
 Exec if $Message =~ /GET \/ping/ drop(); 
 Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1; 
 SavePos TRUE 
 Recursive TRUE 
START_ANGLE_BRACKET /Input END_ANGLE_BRACKET

add a line which drops the current log line if the file name contains the pattern you are looking to skip. For example, for a file name called skip_this_one.log (from the same log directory), the new stanza would be:

# Monitor MyApp1 log files 
START_ANGLE_BRACKET Input MyApp1 END_ANGLE_BRACKET
 Module im_file
 File 'C:\\MyApp1\\logs\\*.log' 
 Exec $Message = $raw_event; 
 Exec if $Message =~ /GET \/ping/ drop(); 

 Exec if file_name() =~ /skip_this_one.log/ drop();
 Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1; 
 SavePos TRUE 
 Recursive TRUE 
START_ANGLE_BRACKET /Input END_ANGLE_BRACKET

Scenario 2: You want to prefix certain log lines depending on their directory of origin

Assume you have a test app and a dev app running on the same box, with the same exact log format, but with logs saved in different directories, so that in the Input sections you would have 

File 'C:\\MyTestApp\\logs\\*.log' for the test app and 

File 'C:\\MyDevApp\\logs\\*.log' for the dev app.

The only solution I found so far was to declare a filewatcher_transformer Processor section for each app. The default filewatcher_transformer section I had before looked like this:

START_ANGLE_BRACKET  Processor filewatcher_transformer END_ANGLE_BRACKET

  Module pm_transformer

  

  # Uncomment to override the program name

  # Exec $SourceName = 'PROGRAM NAME';

  Exec $Hostname = hostname();

  OutputFormat syslog_rfc5424

START_ANGLE_BRACKET/Processor END_ANGLE_BRACKET

I created instead these 2 sections:

START_ANGLE_BRACKET Processor filewatcher_transformer_test END_ANGLE_BRACKET

  Module pm_transformer

  

  # Uncomment to override the program name

  # Exec $SourceName = 'PROGRAM NAME';

  Exec $SourceName = "TEST_" + $SourceName;

  Exec $Hostname = hostname();

  OutputFormat syslog_rfc5424

START_ANGLE_BRACKET/Processor END_ANGLE_BRACKET

START_ANGLE_BRACKET Processor filewatcher_transformer_dev END_ANGLE_BRACKET

  Module pm_transformer

  

  # Uncomment to override the program name

  # Exec $SourceName = 'PROGRAM NAME';

  Exec $SourceName = "DEV_" + $SourceName;

  Exec $Hostname = hostname();

  OutputFormat syslog_rfc5424

START_ANGLE_BRACKET/Processor END_ANGLE_BRACKET

As you can see, I chose to prefix $SourceName, which is the name of the log file in this case, with either TEST_ or DEV_ depending on the app.

There is one thing remaining, which is to define a specific route for each app. Before, I had a common route for both apps:

START_ANGLE_BRACKET  Route 2 END_ANGLE_BRACKET
Path MyAppTest, MyAppDev=> filewatcher_transformer => syslogout

START_ANGLE_BRACKET /Route END_ANGLE_BRACKET

I replaced the common route with the following 2 routes, each connecting an app with its respective Processor section.

START_ANGLE_BRACKET  Route 2 END_ANGLE_BRACKET
Path MyAppTest=> filewatcher_transformer_test => syslogout

START_ANGLE_BRACKET /Route END_ANGLE_BRACKET

START_ANGLE_BRACKET  Route 3 END_ANGLE_BRACKET
Path MyAppDev=> filewatcher_transformer_dev => syslogout

START_ANGLE_BRACKET /Route END_ANGLE_BRACKET

At this point, I restarted the nxlog service and I started to see log filenames in Papertrail of the form DEV_errors.log and TEST_errors.log.