Introduction


This manual describes the Nagios add-on script pnp4n_send_service_mail.pl. Its purpose is to handle the Nagios service e-mail notifications. After receiving the Nagios service state information, it formats the e-mail according to various flags and sends the notification through an SMTP gateway. As a unique feature, it implements the inclusion of service-specific performance graphs from the Nagios graphing package PNP4Nagios, enhancing the information value of an Nagios e-mail alert.

Dependencies


Nagios: Nagios-based monitoring system with Nagios environment macros enabled. This is usually done in [nagios_home]/etc/nagios.cfg.

enable_environment_macros=1

Perl: The script is written in Perl, and depends on the following Perl packages:

Usage


pnp4n_send_service_mail.pl [-v] [-V] [-h] [-t] [-H <SMTP host>] [-p <company-name>] [-r <to_recipients> or -g <to_group>] [-c <cc_recipients>] [-b <bcc_recipients>] [-f <text|html|multi|graph>] [-u] [-l <en|jp|fr|de> (or other languages if added)]

Options


-v, --verbose
      print extra debugging information

-V, --version
      prints version number

-h, --help
      shows the programs help and options

-H, --smtphost=HOST
      name or IP address of SMTP gateway

-t, --test
      sends a test e-mail to confirm mail functions

-p, --customer
      adds a company name header to the ouput

-r, --to-recipients
      with this option, the Nagios-provided $CONTACTEMAIL$ list of to: recipients is overridden. It is good to use this arg together with -t/--test option

-g, --to_group
      Use $CONTACTGROUPMEMBERS$ instead of -r (--to_recipient). This will create a list of contactgroup members. Their complete the mail address with the hard defined $domain in this script. This is only possible when the contact name "abcd" works under the address "abcd@domain".

-c, --cc-recipients
      to add cc: recipients, add this argument and set it to receive the Nagios-provided $CONTACTADDRESS1$ list

-b, --bcc-recipients
      the Nagios-provided $CONTACTADDRESS2$ list of bcc: recipients

-f, --format=text | html | multi | graph
      the email format to generate: either plain text, HTML, multipart S/Mime ouput with a inline logo or multipart S/Mime adding the PNP4Nagios performance image

-u, --addurl
      this adds URL's to the Nagios web GUI for check status, host and hostgroup views into the html mail, requires -f html, -f multi or -f graph

-l, --language=en | jp | fr | de
      the prefered e-mail language. The content-type header has been hard-coded to UTF-8. This might need to be changed if recipients require a different characterset encoding. It is easy to define additional languages inside the script. Currently the script supports the following languages: English, Japanese, French and German.

Setup and Test


First, copy the downloaded script into a suitable Nagios location. Good places are the Nagios program directory nagios/bin, or the Nagios plugin directory nagios/lib. The scripts need to be executable by Nagios.

susie:~ # mv /tmp/pnp4n_send_service_mail.pl /srv/app/nagios/lib/pnp4n_send_service_mail.pl
susie:~ # chown nagios:nagios /srv/app/nagios/lib/pnp4n_send_service_mail.pl

Next, adjust the parameters on the top part of the script. Please see below a extract of parameters:

susie:~ # vi /srv/app/nagios/lib/pnp4n_send_service_mail.pl
# ########################################################################
# For tests using the -t/--test option, if we want to see PNP4Nagios
# graphs we need to set a valid host name and service name below.
# ########################################################################
my $test_host = "susie";       # existing host in PNP4Nagios
my $test_service = "os_cpu_load"; # existing services in PNP4nagios

# ########################################################################
# Here we set the URL to pick up the RRD data files for the optional graph
# image generation. Modified by Robert Becht for use with PNP4Nagios.
# The PNP4Nagios URL : if not used we can set $pnp4nagios_url = undef;
# ########################################################################
my $pnp4nagios_url     = "http://nagios.frank4dd.com/pnp4nagios";
my $graph_history      = 48; # in hours, a good range is between 12...48

# ########################################################################
# If web authentication is needed, configure the access parameters below:
# ########################################################################
my $pnp4nagios_auth    = undef; # $pnp4nagios_auth    = "true";
my $server_port        = undef; # $server_port        = "nagios.frank4dd.com:80";
my $auth_name          = undef; # $auth_name          = "PNP4Nagios Authentication";
my $web_user           = undef; # $web_user           = "pnp4nget";
my $web_pass           = undef; # $web_pass           = "mypass";

# ########################################################################
# SMTP related data: If the commandline argument -H/--smtphost was not
# given, we use the provided value in $o_smtphost below as the default.
# If the mailserver requires auth, an example is further down the code.
# ########################################################################
my $o_smtphost         = "192.168.1.64";
my $domain             = "\@frank4dd.com";
my @listaddress        = ();

# ########################################################################
# This is the logo image file, the path must point to a valid JPG, GIF or
# PNG file, i.e. the nagios logo. Best size is rectangular up to 160x80px.
# example: [nagioshome]/share/images/NagiosEnterprises-whitebg-112x46.png
# ########################################################################
my $logofile = "/srv/www/std-root/nagios.frank4dd.com/images/nagios-logo.gif";
...

In most cases, the web access to PNP4Nagios will be protected through user authentication. Either we configure a web user and password within the notification script, or we disable the webserver authentication to PNP4Nagios for our script access. This is fairly safe, since the script will always connect from our trusted localhost (127.0.0.1).

Exemplary authentication exception for localhost in Apache:

susie:~ # vi /etc/apache2/vhosts/nagios.frank4dd.com
...
<Directory /srv/www/std-root/nagios.frank4dd.com/pnp4nagios/>
        Order deny,allow
        Deny all
        AuthUserFile /etc/apache2/basic.pwd 
        AuthName "PNP4Nagios Authentication" 
        AuthType Basic 
        Require user valid-user 
        Allow from 127.0.0.1
        Satisfy any
</Directory>
...

Now we can try and run the script from hand. For easy troubleshooting, the script provides a test option -t, and in combination with the verbose -v option, many issues can be identified right away.

Failure example - script authentication error:

susie:~ # /srv/app/nagios/lib/pnp4n_send_service_mail.pl -v -t -r
 fm@frank4dd.com -f graph -u -l en
main: trying to create the a PNP4Nagios graph image.
create_graph_img: Cannot download PNP4Nagios image file. Server response:
 401 Authorization Required
create_graph_img: Returning empty image file, format: gif

main: Sending HTML email (language: en) with inline logo.
...

The example output above shows what happens if the web authentication prevents our script from getting the graph image. Here we receive a HTTP error code "401 Authorization Required", and instead of the beautiful graph, an empty 1x1 pixel image is used. If you see this error, it typically is a problem with the web username, password, or the webserver auth domain name. Either correct the web authentication settings in the scripts top section as shown below, or create the mentioned webserver authentication exception for localhost as shown further up on this page.

pnp4n_send_service_mail.pl authentication setup

Now let's have a look at a good example - If all goes well, the test output should look like this:

susie:~ # /srv/app/nagios/lib/pnp4n_send_service_mail.pl -v -t -r
 support@frank4dd.com -f graph -u -l en
main: trying to create the a PNP4Nagios graph image.
create_graph_img: Downloaded PNP4Nagios image file. Server response: 200 OK

b64encode_img: completed conversion of source image file: /tmp/Iit99Q0th_ -
create_graph_img: Encoded PNP4Nagios image file, format: png

main: Sending HTML email (language: en) with inline logo.
b64encode_img: completed conversion of source image file:
 /srv/www/std-root/nagios.frank4dd.com/images/nagios-logo.gif -
main: Converted inline logo to base64 and set type to gif.
Sendmail Log says:
Mail::Sendmail v. 0.79 - Mon May 28 15:08:26 2012
Date: Mon, 28 May 2012 15:08:26 +0900
Server: localhost Port: 25
From: nagiosadmin@yourdomain
Subject: Nagios: TEST service os_cpu_load on susie (Linux Servers) is UNKNOWN
To: support@frank4dd.com

Result: 250 2.0.0 Ok: queued as 37A8A378D1

After checking the mailbox to verify the e-mail arrived and is complete, we can proceed by configuring Nagios to use the new notification script.

Nagios Notification Setup


The data handover between Nagios and the pnp4n notification scripts works largely through Nagios environment macros. To ensure the scripts can pick up Nagios data, check the following setting in nagios.cfg:

susie:~ # grep enable_environment_macros /srv/app/nagios/etc/nagios.cfg
enable_environment_macros=1

Next we define how Nagios calls the notification scripts. Below is an example of the notification definition in the Nagios command.cfg file. I typically move all notification commands into its own configuration file notifications.cfg (example). Depending on the type of e-mail you want to generate, you might want to select different script options.

susie:~ # vi /srv/app/nagios/etc/objects/notification.cfg

###########################################################
# 'service-email-pnp4n-int-en' command definition, sends
# multipart HTML e-mails, English with Nagios URL's + graph
###########################################################
define command{
        command_name    service-email-pnp4n-int-en
        command_line    /srv/app/nagios/lib/pnp4n_send_service_mail.pl \
-p "ACME Corporation, London Branch" \
-c "$CONTACTADDRESS1$" \
-f graph -u -l en
}

Next, we link up the name of the new notification method with a Nagios contact person:

susie:~ # vi /srv/app/nagios/etc/objects/contacts.cfg
...
define contact {
  contact_name                  frank4dd-support           ; Short name of user
  use                           generic-contact            ; Inherit defaults
  alias                         Frank4DD support team      ; Full name of user
  email                         support@frank4dd.com       ; e-mail address
  host_notification_commands    host-email-pnp4n-int-en    ; host alert type
  service_notification_commands service-email-pnp4n-int-en ; service alert type
}
...

Reloading Nagios should enable the new notifications. Now we can manually generate a test message from Nagios, using the "Send custom service notification" command. By including the keyword email-debug into the comment field, the script will create additional debug output into the e-mail itself.

Nagios send test notification example

Another way to troubleshoot potential issues is to call the script from Nagios with the verbose -v option, and redirect the output into a log file. This way, we can capture possible problems with variable content coming from the Nagios environment.

Nagios configuration example for troubleshooting:

susie:~ # vi /srv/app/nagios/etc/objects/notification.cfg

###########################################################
# 'service-email-pnp4n-int-en' command definition, sends
# multipart HTML e-mails, English with Nagios URL's + graph
###########################################################
define command{
command_name    service-email-pnp4n-int-en
command_line    /srv/app/nagios/lib/pnp4n_send_service_mail.pl \
-p "ACME Corporation, London Branch" \
-c "$CONTACTADDRESS1$" \
-f graph -u -l en -v > /tmp/pnp4n_send_service_mail.log
}

If all is well, we should be able to get Notifications as seen in the following example screenshots.

Notification Example Screenshots


In the following example screenshots, I used the operating systems memory check to demonstrate how a message would look in various configurations.

PNP4Nagios Service Notification Status Line

First, the notification script with its most basic plaintext notification. The language has been set to French.

PNP4Nagios notification example using text: service unknown

Second is an example notification using the -f html and -u options, formatting the content as HTML with links to Nagios. The language has been set to Japanese.

PNP4Nagios notification example using html: service OK

Finally, making full use of all functions, we are giving the -f graph -p company-name and -u options, formatting the content as S/Mime multipart HTML with a logo image, links to Nagios, including the matching PNP4Nagios graph image: (script version 1.8.0 example mail)

PNP4Nagios notification example using graph: service critical

Comments


Please edit the head of the scripts to set the mail return address and Nagios base URL. Particularly the scripts mail sender address. I get occasional notification e-mails from various companies for system or service problems, because the scripts still have my sender mail address.

pnp4n_send_service_mail.pl is able to pick up the on-the-fly generated graph image generated by PNP4Nagios. This works with PNP4Nagios version 0.6.17 and possibly also with other versions. If no graphing package is available, then it is best to only use this script's notification formats 'text', 'html' and 'multi'.

The PNP4Nagios notification consists of a pair of scripts. This script is dedicated for Nagios service notifications. For host notifications, a separate script, pnp4n_send_host_mail.pl has been created. It can be downloaded here.

The original idea for the scripts, and their first implementation was done during the New Years holidays 2010/2011. These original scripts [1] [2] were developed for the Nagiosgraph package. Unlike those, the pnp4n notification scripts do not generate the graph image from the RRD database, but fetch them through the web interface. The unnecessary overhead of making a web request is not ideal, but unfortunately PNP4Nagios uses a special framework (Kohana), which does not expose any functions to commandline access. Additional reverse-engineering would be necessary to re-create the graph images in the same beautiful way of PNP4Nagios. Besides overhead, another drawback of the current method is the adjustment of the image size between the PNP4Nagios web interface and the typically smaller size preferred for e-mails.

Icinga: Although this manual only mentions Nagios, the scripts are reported to work with Icinga after minor changes. Replace all instances of 'nagios' in the script, make sure to take care of the case. Run a find and replace on 'nagios', 'Nagios' and 'NAGIOS' and replace with 'icinga', 'Icinga' and 'ICINGA'. Give it a try.

OMD the turnkey Open Monitoring Distribution comes with Mathias Kettners MK Livestatus. The stability of this module requires to disable Nagios environment macros, and the OMD distribution does it by default, setting enable_environment_macros=0 in nagios.cfg. Unfortunately, the pnp4n notification scripts rely on the Nagios environment variables to pick up the notification data, and to make them work this value must be set to 1. The unfortunate choice is between using MK Livestatus, using the pnp4n notification scripts, or spending considerable time and effort to change either one or Nagios itself, where the design choices came from...

Bugs:


The scripts reported to fail when they are used with the Nagios embedded Perl interpreter. The error message is similar to this:

    **ePN failed to compile /etc/nagios3/plugins/nagios_send_host_mail.pl:
    "Variable "$Version" will not stay shared at (eval 1) line 156," at /usr/
    lib/nagios3/p1.pl line 250.

I added a '# nagios: -epn' directive on the scripts line 3 to explicitly disable the use of the embedded Perl interpreter. Using ther embedded Perl interpreter is dangoerous because of the easy introduction of memory leaks.

Nagios v4.0.0-4.0.2 with Nagios BugID 0000498: In these affected versions, the Nagios environment variables are not exported, and as a result the data handover to the scripts does not work. No notifications go out since no destination e-mail address comes from Nagios. This problem is fixed from Nagios 4.0.3 and up.

Download:


nagios_send_host_mail.pl (Latest version 1.8.0, 42333 Bytes)

nagios_send_service_mail.pl (Latest version 1.8.0, 43369 Bytes)

For an alternate download location, see also the corresponding github repository.

Older Versions:

pnp4n_send_host_mail-v173.pl Obsolete version 1.7.3, 40778 Bytes

pnp4n_send_service_mail-v173.pl Obsolete version 1.7.3, 41850 Bytes

pnp4n_send_host_mail-v15.pl Obsolete version 1.5, 38019 Bytes

pnp4n_send_service_mail-v15.pl Obsolete version 1.5, 41533 Bytes

pnp4n_send_host_mail-v14.pl Obsolete version 1.4, 37926 Bytes

pnp4n_send_service_mail-v14.pl Obsolete version 1.4, 40968 Bytes

Thanks and Credits



Implementation Guide - Further information and implementation notes can be found under Flexible Notifications for Nagios.

nagios_send_host_mail.pl - The Nagios monitoring script for hosts and Nagiosgraph can be found here: nagios_send_host_mail manual.

nagios_send_service_mail.pl - The Nagios monitoring script for services and Nagiosgraph can be found here: nagios_send_service_mail manual.

Topics:

More Information: