Introduction
This manual describes the Nagios add-on script pnp4n_send_service_mail.pl. Its purpose is to handle the Nagios service e-mail notifications. After receiving the Nagios service state information, it formats the e-mail according to various flags and sends the notification through an SMTP gateway. As a unique feature, it implements the inclusion of service-specific performance graphs from the Nagios graphing package PNP4Nagios, enhancing the information value of an Nagios e-mail alert.
Dependencies
Nagios: Nagios-based monitoring system with Nagios environment macros enabled. This is usually done in [nagios_home]/etc/nagios.cfg.
enable_environment_macros=1
Perl: The script is written in Perl, and depends on the following Perl packages:
- Perl Mail::Sendmail package, i.e. perl-Mail-Sendmail-0.79-102.1.i586
- Perl Getopt::Long
- Perl Digest::MD5
- Perl LWP, i.e. perl-libwww-perl-6.03-2.1.2.noarch
- Perl MIME::Base64, i.e. perl-MIME-tools-5.502-2.1.2.i586
- Perl File::Temp and Filehandle
- For the optional PNG-to-JPG conversion, the libnetpbm and netpbm packages
Usage
pnp4n_send_service_mail.pl [-v] [-V] [-h] [-t] [-H <SMTP host>] [-p <company-name>] [-r <to_recipients> or -g <to_group>] [-c <cc_recipients>] [-b <bcc_recipients>] [-f <text|html|multi|graph>] [-u] [-l <en|jp|fr|de> (or other languages if added)]
Options
-v, --verbose
print extra debugging information
-V, --version
prints version number
-h, --help
shows the programs help and options
-H, --smtphost=HOST
name or IP address of SMTP gateway
-t, --test
sends a test e-mail to confirm mail functions
-p, --customer
adds a company name header to the ouput
-r, --to-recipients
with this option, the Nagios-provided $CONTACTEMAIL$ list of to: recipients is overridden. It is good to use this arg together with -t/--test option
-g, --to_group
Use $CONTACTGROUPMEMBERS$ instead of -r (--to_recipient). This will create a list of contactgroup members. Their complete the mail address with the hard defined $domain in this script. This is only possible when the contact name "abcd" works under the address "abcd@domain".
-c, --cc-recipients
to add cc: recipients, add this argument and set it to receive the Nagios-provided $CONTACTADDRESS1$ list
-b, --bcc-recipients
the Nagios-provided $CONTACTADDRESS2$ list of bcc: recipients
-f, --format=text | html | multi | graph
the email format to generate: either plain text, HTML, multipart S/Mime ouput with a inline logo or multipart S/Mime adding the PNP4Nagios performance image
-u, --addurl
this adds URL's to the Nagios web GUI for check status, host and hostgroup views into the html mail, requires -f html, -f multi or -f graph
-l, --language=en | jp | fr | de
the prefered e-mail language. The content-type header has been hard-coded to UTF-8. This might need to be changed if recipients require a different characterset encoding. It is easy to define additional languages inside the script. Currently the script supports the following languages: English, Japanese, French and German.
Setup and Test
First, copy the downloaded script into a suitable Nagios location. Good places are the Nagios program directory nagios/bin, or the Nagios plugin directory nagios/lib. The scripts need to be executable by Nagios.
susie:~ # mv /tmp/pnp4n_send_service_mail.pl /srv/app/nagios/lib/pnp4n_send_service_mail.pl susie:~ # chown nagios:nagios /srv/app/nagios/lib/pnp4n_send_service_mail.pl
Next, adjust the parameters on the top part of the script. Please see below a extract of parameters:
susie:~ # vi /srv/app/nagios/lib/pnp4n_send_service_mail.pl
# ########################################################################
# For tests using the -t/--test option, if we want to see PNP4Nagios
# graphs we need to set a valid host name and service name below.
# ########################################################################
my $test_host = "susie"; # existing host in PNP4Nagios
my $test_service = "os_cpu_load"; # existing services in PNP4nagios
# ########################################################################
# Here we set the URL to pick up the RRD data files for the optional graph
# image generation. Modified by Robert Becht for use with PNP4Nagios.
# The PNP4Nagios URL : if not used we can set $pnp4nagios_url = undef;
# ########################################################################
my $pnp4nagios_url = "http://nagios.frank4dd.com/pnp4nagios";
my $graph_history = 48; # in hours, a good range is between 12...48
# ########################################################################
# If web authentication is needed, configure the access parameters below:
# ########################################################################
my $pnp4nagios_auth = undef; # $pnp4nagios_auth = "true";
my $server_port = undef; # $server_port = "nagios.frank4dd.com:80";
my $auth_name = undef; # $auth_name = "PNP4Nagios Authentication";
my $web_user = undef; # $web_user = "pnp4nget";
my $web_pass = undef; # $web_pass = "mypass";
# ########################################################################
# SMTP related data: If the commandline argument -H/--smtphost was not
# given, we use the provided value in $o_smtphost below as the default.
# If the mailserver requires auth, an example is further down the code.
# ########################################################################
my $o_smtphost = "192.168.1.64";
my $domain = "\@frank4dd.com";
my @listaddress = ();
# ########################################################################
# This is the logo image file, the path must point to a valid JPG, GIF or
# PNG file, i.e. the nagios logo. Best size is rectangular up to 160x80px.
# example: [nagioshome]/share/images/NagiosEnterprises-whitebg-112x46.png
# ########################################################################
my $logofile = "/srv/www/std-root/nagios.frank4dd.com/images/nagios-logo.gif";
...
In most cases, the web access to PNP4Nagios will be protected through user authentication. Either we configure a web user and password within the notification script, or we disable the webserver authentication to PNP4Nagios for our script access. This is fairly safe, since the script will always connect from our trusted localhost (127.0.0.1).
Exemplary authentication exception for localhost in Apache:
susie:~ # vi /etc/apache2/vhosts/nagios.frank4dd.com
...
<Directory /srv/www/std-root/nagios.frank4dd.com/pnp4nagios/>
Order deny,allow
Deny all
AuthUserFile /etc/apache2/basic.pwd
AuthName "PNP4Nagios Authentication"
AuthType Basic
Require user valid-user
Allow from 127.0.0.1
Satisfy any
</Directory>
...Now we can try and run the script from hand. For easy troubleshooting, the script provides a test option -t, and in combination with the verbose -v option, many issues can be identified right away.
Failure example - script authentication error:
susie:~ # /srv/app/nagios/lib/pnp4n_send_service_mail.pl -v -t -r fm@frank4dd.com -f graph -u -l en main: trying to create the a PNP4Nagios graph image. create_graph_img: Cannot download PNP4Nagios image file. Server response: 401 Authorization Required create_graph_img: Returning empty image file, format: gif main: Sending HTML email (language: en) with inline logo. ...
The example output above shows what happens if the web authentication prevents our script from getting the graph image. Here we receive a HTTP error code "401 Authorization Required", and instead of the beautiful graph, an empty 1x1 pixel image is used. If you see this error, it typically is a problem with the web username, password, or the webserver auth domain name. Either correct the web authentication settings in the scripts top section as shown below, or create the mentioned webserver authentication exception for localhost as shown further up on this page.
Now let's have a look at a good example - If all goes well, the test output should look like this:
susie:~ # /srv/app/nagios/lib/pnp4n_send_service_mail.pl -v -t -r support@frank4dd.com -f graph -u -l en main: trying to create the a PNP4Nagios graph image. create_graph_img: Downloaded PNP4Nagios image file. Server response: 200 OK b64encode_img: completed conversion of source image file: /tmp/Iit99Q0th_ - create_graph_img: Encoded PNP4Nagios image file, format: png main: Sending HTML email (language: en) with inline logo. b64encode_img: completed conversion of source image file: /srv/www/std-root/nagios.frank4dd.com/images/nagios-logo.gif - main: Converted inline logo to base64 and set type to gif. Sendmail Log says: Mail::Sendmail v. 0.79 - Mon May 28 15:08:26 2012 Date: Mon, 28 May 2012 15:08:26 +0900 Server: localhost Port: 25 From: nagiosadmin@yourdomain Subject: Nagios: TEST service os_cpu_load on susie (Linux Servers) is UNKNOWN To: support@frank4dd.com Result: 250 2.0.0 Ok: queued as 37A8A378D1
After checking the mailbox to verify the e-mail arrived and is complete, we can proceed by configuring Nagios to use the new notification script.
Nagios Notification Setup
The data handover between Nagios and the pnp4n notification scripts works largely through Nagios environment macros. To ensure the scripts can pick up Nagios data, check the following setting in nagios.cfg:
susie:~ # grep enable_environment_macros /srv/app/nagios/etc/nagios.cfg enable_environment_macros=1
Next we define how Nagios calls the notification scripts. Below is an example of the notification definition in the Nagios command.cfg file. I typically move all notification commands into its own configuration file notifications.cfg (example). Depending on the type of e-mail you want to generate, you might want to select different script options.
susie:~ # vi /srv/app/nagios/etc/objects/notification.cfg
###########################################################
# 'service-email-pnp4n-int-en' command definition, sends
# multipart HTML e-mails, English with Nagios URL's + graph
###########################################################
define command{
command_name service-email-pnp4n-int-en
command_line /srv/app/nagios/lib/pnp4n_send_service_mail.pl \
-p "ACME Corporation, London Branch" \
-c "$CONTACTADDRESS1$" \
-f graph -u -l en
}Next, we link up the name of the new notification method with a Nagios contact person:
susie:~ # vi /srv/app/nagios/etc/objects/contacts.cfg
...
define contact {
contact_name frank4dd-support ; Short name of user
use generic-contact ; Inherit defaults
alias Frank4DD support team ; Full name of user
email support@frank4dd.com ; e-mail address
host_notification_commands host-email-pnp4n-int-en ; host alert type
service_notification_commands service-email-pnp4n-int-en ; service alert type
}
...Reloading Nagios should enable the new notifications. Now we can manually generate a test message from Nagios, using the "Send custom service notification" command. By including the keyword email-debug into the comment field, the script will create additional debug output into the e-mail itself.
Another way to troubleshoot potential issues is to call the script from Nagios with the verbose -v option, and redirect the output into a log file. This way, we can capture possible problems with variable content coming from the Nagios environment.
Nagios configuration example for troubleshooting:
susie:~ # vi /srv/app/nagios/etc/objects/notification.cfg
###########################################################
# 'service-email-pnp4n-int-en' command definition, sends
# multipart HTML e-mails, English with Nagios URL's + graph
###########################################################
define command{
command_name service-email-pnp4n-int-en
command_line /srv/app/nagios/lib/pnp4n_send_service_mail.pl \
-p "ACME Corporation, London Branch" \
-c "$CONTACTADDRESS1$" \
-f graph -u -l en -v > /tmp/pnp4n_send_service_mail.log
}If all is well, we should be able to get Notifications as seen in the following example screenshots.
Notification Example Screenshots
In the following example screenshots, I used the operating systems memory check to demonstrate how a message would look in various configurations.
First, the notification script with its most basic plaintext notification. The language has been set to French.
Second is an example notification using the -f html and -u options, formatting the content as HTML with links to Nagios. The language has been set to Japanese.
Finally, making full use of all functions, we are giving the -f graph -p company-name and -u options, formatting the content as S/Mime multipart HTML with a logo image, links to Nagios, including the matching PNP4Nagios graph image: (script version 1.8.0 example mail)
Comments
Please edit the head of the scripts to set the mail return address and Nagios base URL. Particularly the scripts mail sender address. I get occasional notification e-mails from various companies for system or service problems, because the scripts still have my sender mail address.
pnp4n_send_service_mail.pl is able to pick up the on-the-fly generated graph image generated by PNP4Nagios. This works with PNP4Nagios version 0.6.17 and possibly also with other versions. If no graphing package is available, then it is best to only use this script's notification formats 'text', 'html' and 'multi'.
The PNP4Nagios notification consists of a pair of scripts. This script is dedicated for Nagios service notifications. For host notifications, a separate script, pnp4n_send_host_mail.pl has been created. It can be downloaded here.
The original idea for the scripts, and their first implementation was done during the New Years holidays 2010/2011. These original scripts [1] [2] were developed for the Nagiosgraph package. Unlike those, the pnp4n notification scripts do not generate the graph image from the RRD database, but fetch them through the web interface. The unnecessary overhead of making a web request is not ideal, but unfortunately PNP4Nagios uses a special framework (Kohana), which does not expose any functions to commandline access. Additional reverse-engineering would be necessary to re-create the graph images in the same beautiful way of PNP4Nagios. Besides overhead, another drawback of the current method is the adjustment of the image size between the PNP4Nagios web interface and the typically smaller size preferred for e-mails.
Icinga: Although this manual only mentions Nagios, the scripts are reported to work with Icinga after minor changes. Replace all instances of 'nagios' in the script, make sure to take care of the case. Run a find and replace on 'nagios', 'Nagios' and 'NAGIOS' and replace with 'icinga', 'Icinga' and 'ICINGA'. Give it a try.
OMD the turnkey Open Monitoring Distribution comes with Mathias Kettners MK Livestatus. The stability of this module requires to disable Nagios environment macros, and the OMD distribution does it by default, setting enable_environment_macros=0 in nagios.cfg. Unfortunately, the pnp4n notification scripts rely on the Nagios environment variables to pick up the notification data, and to make them work this value must be set to 1. The unfortunate choice is between using MK Livestatus, using the pnp4n notification scripts, or spending considerable time and effort to change either one or Nagios itself, where the design choices came from...
Bugs:
The scripts reported to fail when they are used with the Nagios embedded Perl interpreter. The error message is similar to this:
**ePN failed to compile /etc/nagios3/plugins/nagios_send_host_mail.pl:
"Variable "$Version" will not stay shared at (eval 1) line 156," at /usr/
lib/nagios3/p1.pl line 250.
I added a '# nagios: -epn' directive on the scripts line 3 to explicitly disable the use of the embedded Perl interpreter. Using ther embedded Perl interpreter is dangoerous because of the easy introduction of memory leaks.
Nagios v4.0.0-4.0.2 with Nagios BugID 0000498: In these affected versions, the Nagios environment variables are not exported, and as a result the data handover to the scripts does not work. No notifications go out since no destination e-mail address comes from Nagios. This problem is fixed from Nagios 4.0.3 and up.Download:
nagios_send_host_mail.pl (Latest version 1.8.1, 44172 Bytes)
nagios_send_service_mail.pl (Latest version 1.8.1, 45474 Bytes)
For an alternate download location, see also the corresponding github repository.
Older Versions:
pnp4n_send_host_mail-v180.pl Obsolete version 1.8.0, 42333 Bytes
pnp4n_send_service_mail-v180.pl Obsolete version 1.8.0, 43369 Bytes
pnp4n_send_host_mail-v173.pl Obsolete version 1.7.3, 40778 Bytes
pnp4n_send_service_mail-v173.pl Obsolete version 1.7.3, 41850 Bytes
pnp4n_send_host_mail-v15.pl Obsolete version 1.5, 38019 Bytes
pnp4n_send_service_mail-v15.pl Obsolete version 1.5, 41533 Bytes
pnp4n_send_host_mail-v14.pl Obsolete version 1.4, 37926 Bytes
pnp4n_send_service_mail-v14.pl Obsolete version 1.4, 40968 Bytes
Thanks and Credits
- This script would not exist but for the considerable contribution of Robert Becht. Introducing a number of improvements, such as extensible multi-language support, contact group mailing, raw code for PNP4Nagios image fetching, and and others. His dedication to programming, testing and answering questions made it largely possible.
- Also thanks to the community, the demand for a solution covering PNP4Nagios was constantly there, and the appreciation and encouragement kept the efforts going.
See also:
Implementation Guide - Further information and implementation notes can be found under Flexible Notifications for Nagios.
nagios_send_host_mail.pl - The Nagios monitoring script for hosts and Nagiosgraph can be found here: nagios_send_host_mail manual.
nagios_send_service_mail.pl - The Nagios monitoring script for services and Nagiosgraph can be found here: nagios_send_service_mail manual.