Windows 2008 R2 Cluster Report with PowerShell

You know how it is, manager love reports and really we geeks like some reports as well. Recently I was tasked to generate a health check report for some of the Windows 2008 R2 Active / Passive clusters using PowerShell.

With Windows 2008 R2 cluster Microsoft has provided FailoverClusters modules as well as the deprecated cluster command. There is no per say report to check for a status on clusters. Using the Failover Cluster PowerShell modules it is possible to create a report. There is a good reference for the cluster modules here .

For this post I was able to create a scheduled PowerShell script to gather Cluster VIP, Quorum, node status, groups, resources and resource dependencies and put in a formatted text file and emailed.

This can be done using Get-Cluster, Get-ClusterQuorum, Get-ClusterNode, Get-ClusterGroup, Get-ClusterResource and Get-ClusterResourceDependency cmdlets from FailoverCluster PowerShell module.

The Report is populated using Add-Content and Out-File and setting the encoding to ASCII and finally using Format-Table to make it look nice. Also, the inclusion of $LogFileName variable to call out the filename in one place and reference the variable. The script is pretty easy to figure out how it works and doesn’t need much explanation, so here it is in all it’s glory.

# Name: cluster_ps.ps1 # Revision: 1 # Can only be run on Windows 2008 clusters or higher # # Get the date and set the variable $Now = Get-Date # Import the cmdlets Import-Module failoverclusters

# Get the cluster vip and set to variable
 $clustervip = Get-Cluster | foreach-object {$_.name}

# delete previous days check
$LogFileName = "c:\logs\clustercheck.txt"
del $LogFileName

Out-File -FilePath $LogFileName -Encoding ASCII
 Add-Content -Path $LogFileName -Value "Cluster Healthcheck for $clustervip" -Encoding ASCII
 Add-Content -Path $LogFileName -Value "`n$Now" -Encoding ASCII
 Add-Content -Path $LogFileName -Value "`n" -Encoding ASCII

# get the vip
 $clustervip = Get-Cluster | foreach-object {$_.name}
 Add-Content -Path $LogFileName -Value "`nCluster VIP" -Encoding ASCII
 Add-Content -Path $LogFileName -Value "`n$clustervip" -Encoding ASCII
 Add-Content -Path $LogFileName -Value "`n" -Encoding ASCII

# Get Quorum Owner
 Add-Content -Path $LogFileName -Value "`nCluster Quorum" -Encoding ASCII
 Get-ClusterQuorum | Format-Table -AutoSize -Wrap Cluster, QuorumResource, QuorumType | Out-File -append  -Encoding ASCII -FilePath $LogFileName

# Get Cluster Node status
 Add-Content -Path $LogFileName -Value "`nListing Cluster Nodes Status" -Encoding ASCII
 Get-ClusterNode | Format-Table -AutoSize | Out-File -append  -Encoding ASCII -FilePath $LogFileName

# Get Cluster Groups
 Add-Content -Path $LogFileName -Value "`nListing Cluster Groups" -Encoding ASCII
 Get-ClusterGroup | Format-Table -AutoSize | Out-File -append  -Encoding ASCII -FilePath $LogFileName

# See which resources are in my group
 Add-Content -Path $LogFileName -Value "`nListing Cluster Resources" -Encoding ASCII
 Get-ClusterGroup | Get-ClusterResource | Format-Table -AutoSize -Wrap | Out-File -append  -Encoding ASCII -FilePath $LogFileName

# Get resource dependency
 Add-Content -Path $LogFileName -Value "`nListing Cluster Resource Dependencies" -Encoding ASCII
 Get-ClusterGroup | Get-ClusterResource | Get-ClusterResourceDependency | Format-Table -AutoSize -Wrap | Out-File -append  -Encoding ASCII -FilePath $LogFileName

Send-MailMessage -To me@myserver.com -From maint@myserver.com -subject "Cluster Health Report" -SmtpServer my.smtp.server -Attachments $LogFileName

Hope this is of to you as it is for me and getting a daily report of what is online and offline. Look for future posts for how to take this report and use the output to create monitors using PowerShell. I have attached the script cluster_ps.ps1, just rename from txt to ps1.

Installing Nagios Part 2 – The Agents

In following with a previous post on installing Nagios we are ready to move to installing the agents. In particular Linux, Solaris 10 and Windows agents to allow Nagios to do what it does best and monitor.

Linux

For my installation I am using CentOS 6.2 for which the EPEL repository has all the packages I need for NRPE and Nagios Plugins.

# yum search nagios

nagios.x86_64 : Nagios monitors hosts and services and yells if somethings breaks
nagios-common.x86_64 : Provides common directories, uid and gid among nagios-related packages
nagios-devel.i686 : Provides include files that Nagios-related applications may compile against
nagios-devel.x86_64 : Provides include files that Nagios-related applications may compile against
nagios-plugins.x86_64 : Host/service/network monitoring program plugins for Nagios
nagios-plugins-all.x86_64 : Nagios Plugins – All plugins
nagios-plugins-apt.x86_64 : Nagios Plugin – check_apt
nagios-plugins-bdii.x86_64 : Nagios Plugin – check_bdii_entries
nagios-plugins-breeze.x86_64 : Nagios Plugin – check_breeze
nagios-plugins-by_ssh.x86_64 : Nagios Plugin – check_by_ssh
nagios-plugins-check-updates.x86_64 : A Nagios plugin to check if Red Hat or Fedora system is up-to-date
nagios-plugins-check_sip.x86_64 : A Nagios plugin to check SIP servers and devices
nagios-plugins-cluster.x86_64 : Nagios Plugin – check_cluster
nagios-plugins-dhcp.x86_64 : Nagios Plugin – check_dhcp
nagios-plugins-dig.x86_64 : Nagios Plugin – check_dig
nagios-plugins-disk.x86_64 : Nagios Plugin – check_disk
nagios-plugins-disk_smb.x86_64 : Nagios Plugin – check_disk_smb
nagios-plugins-dns.x86_64 : Nagios Plugin – check_dns
nagios-plugins-dpm-disk.x86_64 : Nagios probes to be run in the DPM disk nodes
nagios-plugins-dpm-head.x86_64 : Nagios probes to be run in the DPM head node
nagios-plugins-dummy.x86_64 : Nagios Plugin – check_dummy
nagios-plugins-file_age.x86_64 : Nagios Plugin – check_file_age
nagios-plugins-flexlm.x86_64 : Nagios Plugin – check_flexlm
nagios-plugins-fping.x86_64 : Nagios Plugin – check_fping
nagios-plugins-game.x86_64 : Nagios Plugin – check_game
nagios-plugins-hpjd.x86_64 : Nagios Plugin – check_hpjd
nagios-plugins-http.x86_64 : Nagios Plugin – check_http
nagios-plugins-icmp.x86_64 : Nagios Plugin – check_icmp
nagios-plugins-ide_smart.x86_64 : Nagios Plugin – check_ide_smart
nagios-plugins-ifoperstatus.x86_64 : Nagios Plugin – check_ifoperstatus
nagios-plugins-ifstatus.x86_64 : Nagios Plugin – check_ifstatus
nagios-plugins-ircd.x86_64 : Nagios Plugin – check_ircd
nagios-plugins-lcgdm.x86_64 : Nagios probes to be run remotely against DPM / LFC nodes
nagios-plugins-lcgdm-common.x86_64 : Common libraries and files to all LCGDM nagios packages
nagios-plugins-ldap.x86_64 : Nagios Plugin – check_ldap
nagios-plugins-lfc.x86_64 : Nagios probes to be run in the LFC node
nagios-plugins-linux_raid.x86_64 : Nagios Plugin – check_linux_raid
nagios-plugins-load.x86_64 : Nagios Plugin – check_load
nagios-plugins-log.x86_64 : Nagios Plugin – check_log
nagios-plugins-mailq.x86_64 : Nagios Plugin – check_mailq
nagios-plugins-mrtg.x86_64 : Nagios Plugin – check_mrtg
nagios-plugins-mrtgtraf.x86_64 : Nagios Plugin – check_mrtgtraf
nagios-plugins-mysql.x86_64 : Nagios Plugin – check_mysql
nagios-plugins-nagios.x86_64 : Nagios Plugin – check_nagios
nagios-plugins-nrpe.x86_64 : Provides nrpe plugin for Nagios
nagios-plugins-nt.x86_64 : Nagios Plugin – check_nt
nagios-plugins-ntp.x86_64 : Nagios Plugin – check_ntp
nagios-plugins-nwstat.x86_64 : Nagios Plugin – check_nwstat
nagios-plugins-openmanage.x86_64 : Nagios plugin to monitor hardware health on Dell servers
nagios-plugins-oracle.x86_64 : Nagios Plugin – check_oracle
nagios-plugins-overcr.x86_64 : Nagios Plugin – check_overcr
nagios-plugins-perl.x86_64 : Nagios plugins perl dep.
nagios-plugins-pgsql.x86_64 : Nagios Plugin – check_pgsql
nagios-plugins-ping.x86_64 : Nagios Plugin – check_ping
nagios-plugins-procs.x86_64 : Nagios Plugin – check_procs
nagios-plugins-radius.x86_64 : Nagios Plugin – check_radius
nagios-plugins-real.x86_64 : Nagios Plugin – check_real
nagios-plugins-rhev.noarch : Nagios Plugin – check_rhev
nagios-plugins-rpc.x86_64 : Nagios Plugin – check_rpc
nagios-plugins-sensors.x86_64 : Nagios Plugin – check_sensors
nagios-plugins-smtp.x86_64 : Nagios Plugin – check_smtp
nagios-plugins-snmp.x86_64 : Nagios Plugin – check_snmp
nagios-plugins-ssh.x86_64 : Nagios Plugin – check_ssh
nagios-plugins-swap.x86_64 : Nagios Plugin – check_swap
nagios-plugins-tcp.x86_64 : Nagios Plugin – check_tcp
nagios-plugins-time.x86_64 : Nagios Plugin – check_time
nagios-plugins-ups.x86_64 : Nagios Plugin – check_ups
nagios-plugins-users.x86_64 : Nagios Plugin – check_users
nagios-plugins-wave.x86_64 : Nagios Plugin – check_wave
perl-Nagios-NSCA.noarch : Nagios::NSCA Perl module
perl-Nagios-Plugin.noarch : Family of perl modules to streamline writing Nagios plugins
perl-Nagios-Plugin-WWW-Mechanize.noarch : Login to a web page as a user and get data as a Nagios plugin
pnp4nagios.x86_64 : Nagios performance data analysis tool
ndoutils.x86_64 : Stores all configuration and event data from Nagios in a database
nrpe.x86_64 : Host/service/network monitoring agent for Nagios
nsca.x86_64 : Nagios Service Check Acceptor

From the output we see there is alot to choose from. For the basic monitoring we will want nrpe, nagios-plugins-disk,

nagios-plugins

# yum install nrpe nagios-plugins nagios-plugins-load nagios-plugins-disk nagios-plugins-procs nagios-plugins-users

==================================================================================================
Package                    Arch               Version                     Repository        Size
==================================================================================================
Installing:
nagios-plugins             x86_64             1.4.15-2.el6                epel             186 k
nrpe                       x86_64             2.12-16.el6                 epel             219 k
nagios-plugins-disk        x86_64             1.4.15-2.el6                epel             67 k
nagios-plugins-load        x86_64             1.4.15-2.el6                epel             29 k
nagios-plugins-procs       x86_64             1.4.15-2.el6                epel             66 k
nagios-plugins-users       x86_64             1.4.15-2.el6                epel             28 k

Installing for dependencies:
nagios-common x86_64 3.3.1-3.el6 epel 16 k

Plugins will be located here
# ls /usr/lib64/nagios/plugins/
check_disk check_load check_procs check_users negate urlize utils.sh

Edit nrpe config at /etc/nagios/nrpe.cfg
Change the allowed hosts to your server.

Add port 5666 to etc/services
# vi /etc/services
nrpe 5666/tcp # NRPE

Restart nrpe to accept the changes

If you want to compile and download them and not use epel you can download and compile

Create the Nagios user and set the password.
# useradd nagios
# passwd nagios

nagios-plugins-1.4.15.tar.gz
# tar xzf nagios-plugins-1.4.15.tar.gz
# ./configure –with-nagios-user=nagios –with-nagios-group=nagios
# make
# make install

tar -xzvf nrpe-2.12.tar.gz
./configure
make
make install

Add port 5666 to etc/services
# vi /etc/services
nrpe 5666/tcp # NRPE

Test NRPE Daemon Install

Check NRPE daemon is running and listening on port 5666:
# netstat -at |grep nrpe

Output should be NRPE version:
NRPE v2.12

Windows

There are several clients available out there for Windows systems to talk to Nagios servers. The most popular is NSclient++. This client is a favorite because it can do alot and talk to the many different interfaces of Nagios. NSClient++ can be found at http://www.nsclient.org/.

Download NSClient++ 0.3.9 (stable) from http://www.nsclient.org/nscp/downloads. Download the 32 or 64 bit msi package
At this point you can double-click and run the installation or use the unattended install which is my preference, especially for large and silent deployments, the syntax is:

msiexec /i <MSI FILE> /quiet

ex: msiexec /i NSClient++-0.3.9-x64.msi /quiet

This command will install all the features and reboot the server. If this is ok for your situation then you are on your way.

Quiet install without restart and with all features but the firewall exception:

msiexec /i <<<MSI FILE>>> /quiet /norestart ADDLOCAL=ALL REMOVE=”FireWallException”

Passive install (displays progress bar) without restart and with several feature exceptions:

msiexec /i <<<MSI FILE>>> /passive /norestart ADDLOCAL=”ALL” REMOVE=”Documentation,NSCPlugins,NSCAPlugin,SampleScripts”

Solaris 10

Add “nagios” user as “/usr/local/nagios” as home directory.
# useradd -c “Nagios User” -d /usr/local/nagios -m nagios

Change ownership of directory to nagios
# chown nagios:nagios /usr/local/nagios/

download nagios-plugins and nrpe from net.
# mkdir /nagios; cd /nagios
# wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz

wget http://prdownloads.sourceforge.net/sourceforge/nagios/nrpe-2.13.tar.gz

Now extract them

# gunzip nagios-plugins-1.4.15.tar.gz; gunzip nrpe-2.13.tar.gz

# tar xvf nagios-plugins-1.4.15.tar.gz; tar xvf nrpe-2.13.tar.gz

Before compiling I had to set PATH to find gcc binary,

# export PATH=$PATH:/usr/sfw/sbin:/usr/sfw/bin:/usr/ccs/bin

# cd nagios-plugins-1.4.15;

# ./configure –without-mysql (I did want to install with mysql support)
# make; make install
# chown -R nagios:nagios /usr/local/nagios/libexec

Install NRPE with SSL library support otherwise you will get error while compilation like this,

“checking for SSL headers… configure: error: Cannot find ssl headers”

# cd nrpe-2.13; ./configure –with-ssl=/usr/sfw/ –with-ssl-lib=/usr/sfw/lib/ –with-ssl-inc=/usr/sfw/include

Still if you compilation fails please apply these faqs/solutions given in nagios faqs.

http://www.nagios.org/faqs/index.php?section_id=4&expand=false&showdesc=true

In my case I had to make changes in src/nrpe.c for encryption. Do make all, make install to create respective binaries.

# make all; make install; make install-daemon-config;

Add following line at the end of /etc/services

nrpe 5666/tcp # NRPE

Also add this line to /etc/inetd.conf and convert it into SMF and enable service with -e option. Also checkout whether it went online.

nrpe stream tcp nowait nagios /usr/sfw/sbin/tcpd /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -i

# inetconv; inetconv -e

# svcs | grep nrpe

Make sure that your /etc/hosts.allow and /etc/hosts.deny does not block your nagios server. Here are the entries

hosts.allow: nrpe: 127.0.0.1, 192.168.1.85

hosts.deny: nrpe: ALL

Final command, make sure that nrpe returns correct output.

# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.13

Add “nagios” user as “/usr/local/nagios” as home directory.

# useradd -c “Nagios User” -d /usr/local/nagios -m nagios

Change ownership of directory to nagios

# chown nagios:nagios /usr/local/nagios/

download nagios-plugins and nrpe from net. I have download them from sourceforge.

# mkdir /nagios; cd /nagios

wget http://sourceforge.net/project/downloading.php?group_id=29880&filename=nagios-plugins-1.4.15.tar.gz

wget http://sourceforge.net/project/downloading.php?group_id=26589&filename=nrpe-2.13.tar.gz

Now extract them

# gunzip nagios-plugins-1.4.15.tar.gz; gunzip nrpe-2.13.tar.gz

# tar xvf nagios-plugins-1.4.15.tar.gz; tar xvf nrpe-2.13.tar.gz

Before compiling I had to set PATH to find gcc binary,

# export PATH=$PATH:/usr/sfw/sbin:/usr/sfw/bin:/usr/ccs/bin

# cd nagios-plugins-1.4.15;

# ./configure –without-mysql (I did want to install with mysql support)
# make; make install
# chown -R nagios:nagios /usr/local/nagios/libexec

Install NRPE with SSL library support otherwise you will get error while compilation like this,

“checking for SSL headers… configure: error: Cannot find ssl headers”

If you run “dmesg” or if you check system messages you can see this error.

feb 26 17:08:23 solaris10.remotehost.com inetd[24233]: [ID 702911 daemon.error] Failed to set credentials for the inetd_start method of instance svc:/network/nrpe/tcp:default (chdir: No such file or directory)
Feb 26 17:15:24 solaris10.remotehost.com inetd[24241]: [ID 702911 daemon.error] Failed to set credentials for the inetd_start method of instance svc:/network/nrpe/tcp:default (chdir: No such file or directory)

# cd nrpe-2.13; ./configure –with-ssl=/usr/sfw/ –with-ssl-lib=/usr/sfw/lib/ –with-ssl-inc=/usr/sfw/include

Still if you compilation fails please apply these faqs/solutions given in nagios faqs.

http://www.nagios.org/faqs/index.php?section_id=4&expand=false&showdesc=true

In my case I had to make changes in src/nrpe.c for encryption. Do make all, make install to create respective binaries.

# make all; make install; make install-daemon-config;

Add following line at the end of /etc/services

nrpe 5666/tcp # NRPE

Also add this line to /etc/inetd.conf and convert it into SMF and enable service with -e option. Also checkout whether it went online.

nrpe stream tcp nowait nagios /usr/sfw/sbin/tcpd /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -i

# inetconv; inetconv -e

# svcs | grep nrpe

Make sure that your /etc/hosts.allow and /etc/hosts.deny does not block your nagios server. Here are the entries

hosts.allow: nrpe: 127.0.0.1, 192.168.1.85

hosts.deny: nrpe: ALL

Final command, make sure that nrpe returns correct output.

# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.12

Now that we have running agents you will need to create monitors, host and hostgroups in Nagios to start monitoring the servers. Look for Part 3 of the series coming soon.

Multiple Nagios Instances on a Single Server

In this post I will cover how to run Multiple Nagios Instances on a Single Server. This will take a single installation of Nagios and allow you to run separate instances as you would to separate out different environments, such as production, development, certification, etc….

It allows for flexibility of configurations and allows Nagios to run faster for checks and execution. This installation scenario is based on RedHat 5.x installation with Nagios 3.2.0, Nagios Plugins 1.4.13, Nagiosgraph, rrdtool and LDAP integration.

Installation

Download and Install Boutell’s GD library

wget http://www.libgd.org/releases/gd-2.0.35.tar.gz
./configure
make install

Compile and install the Nagios and Plugins

wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.0.tar.gz
wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.13.tar.gz
tar -xzf nagios-3.2.0.tar.gz
tar -xzf nagios-plugins-1.4.13.tar.gz

export NAGIOSHOME=/usr/local/nagios
cd nagios-3.2.0
./configure --enable-embedded-perl --with-command-group=nagios --with-gd-lib=/usr/local/lib --with-gd-inc=/usr/local/include; make  all; make install install-init;make-install-config;make install-commandmode

Compile and install the Nagios plugins

cd nagios-plugins-1.4.13
./configure --prefix=$NAGIOSHOME/core/nagios-plugins --with-nagios-user=nagios --with-nagios-group=nagios; make ; make install

File Copy and Directory Creation
Create directories which hold your instances and their subdirectories

NAGIOSHOME=/usr/local/nagios

mkdir $NAGIOSHOME/instances
mkdir $NAGIOSHOME/instances/nagios_prod
mkdir $NAGIOSHOME/instances/nagios_prod/etc
mkdir $NAGIOSHOME/instances/nagios_prod/etc/monitors
mkdir $NAGIOSHOME/instances/nagios_prod/var
mkdir $NAGIOSHOME/instances/nagios_prod/var/rw
mkdir $NAGIOSHOME/instances/nagios_prod/var/spool
mkdir $NAGIOSHOME/instances/nagios_prod/var/spool/checkresults
mkdir $NAGIOSHOME/instances/nagios_prod/share
mkdir $NAGIOSHOME/instances/nagios_prod/sbin

Create Symbolic Links

ln -s $NAGIOSHOME/nagios/bin $NAGIOSHOME/instances/nagios_prod/bin
ln -s $NAGIOSHOME/nagios-plugins/libexec $NAGIOSHOME/instances/nagios_prod/libexec

Copy files

cp -r $NAGIOSHOME/sbin/*  $NAGIOSHOME/instances/nagios_prod/sbin/
cp –r $NAGIOSHOME/nagios/share/* $NAGIOSHOME/instances/nagios_prod/share/
cp $NAGIOSHOME/etc/cgi.cfg  $NAGIOSHOME/instances/nagios_prod/etc/
cp $NAGIOSHOME/etc/resource.cfg  $NAGIOSHOME/instances/nagios_prod/etc/
cp $NAGIOSHOME/etc/nagios.cfg  $NAGIOSHOME/instances/nagios_prod/etc/

Do the same with all the other instances you are going to deploy.

Configuration
In the editing of the Nagios config files there will be instances that point to a central shared location and to individual settings. The configurations that are shared are checkcommands.cfg, misccommands.cfg, contacts.cfg, contactgroups.cfg, timeperiods.cfg, templates.cfg, dependencies.cfg & escalations.cfg.
The individual ones are the specific monitors and hosts, which in the nagios.cfg we will state a directory instead of the individual configuration files.

nagios.cfg modifications

Example:

log_file=/usr/local/nagios/instances/nagios_prod/var/nagios.log

# OBJECT CONFIGURATION FILE(S)
cfg_file=/usr/local/nagios/etc/objects/checkcommands.cfg
cfg_file=/usr/local/nagios/etc/objects/misccommands.cfg
cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
cfg_file=/usr/local/nagios/etc/objects/contactgroups.cfg
cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg
cfg_file=/usr/local/nagios/etc/objects/templates.cfg
cfg_file=/usr/local/nagios/etc/objects/dependencies.cfg
cfg_file=/usr/local/nagios/etc/objects/escalations.cfg

# Definitions for monitoring
cfg_dir=/usr/local/nagios/instances/nagios_prod/etc/monitors

object_cache_file=/usr/local/nagios/instances/nagios_prod/var/objects.cache
precached_object_file=/usr/local/nagios/instances/nagios_prod/var/objects.precache
resource_file=/usr/local/nagios/instances/nagios_prod/etc/resource.cfg
status_file=/usr/local/nagios/instances/nagios_prod/var/status.dat
command_file=/usr/local/nagios/instances/nagios_prod/var/rw/nagios.cmd
lock_file=/usr/local/nagios/instances/nagios_prod/var/nagios.lock
temp_file=/usr/local/nagios/instances/nagios_prod/var/nagios.tmp
log_archive_path=/usr/local/nagios/instances/nagios_prod/var/archives
state_retention_file=/usr/local/nagios/instances/nagios_prod/var/retention.dat
debug_file=/usr/local/nagios/instances/nagios_prod/var/nagios.debug

CGI.CFG File Edits

The cgi.cfg file must be edit for Apache to be able to run the correct CGI scripts for each instance you plan to run. To follow the examples in previous sections we use nagios_prod as our instance.

main_config_file=/usr/local/nagios/instances/nagios_prod/etc/nagios.cfg
physical_html_path=/usr/local/nagios/instances/nagios_prod/share
url_html_path=/nagios_prod/

Resource.cfg file edits

$USER1$=/etc/nagios/instances/nagios_prod/libexec

Create nagios configuration for Apache
Create a nagios_prod.conf file in the /etc/httpd/conf.d directory with the following contents.
Note: this example uses ldap and the string will vary for your instance.

ScriptAlias /nagios_prod/cgi-bin/ "/usr/local/nagios/instances/nagios_prod/sbin/"

   SetEnv NAGIOS_CGI_CONFIG /usr/local/nagios/instances/nagios_prod/etc/cgi.cfg
   Options ExecCGI
   Options FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthType Basic
   AuthName "Nagios Area51"
   AuthBasicProvider ldap
   AuthzLDAPAuthoritative off
   AuthLDAPURL ldaps://myldaprod:636/ou=IMEROOT,o=corp?uid

Alias /nagios_prod/ "/usr/local/nagios/instances/nagios_prod/share/"

   Options None
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthType Basic
   AuthName "Nagios Area51"
   AuthBasicProvider ldap
   AuthzLDAPAuthoritative off
   AuthLDAPURL ldaps://myldaprod:636/ou=IMEROOT,o=corp?uid

Edit config.inc.php

The config.inc.php file is located in the /usr/local/nagios/instances/nagios_prod/share/ directory. Make the path changes for the instance.

Example:

Correct Directory and File PermissionsAt this point it is a good time to correct the file and directory permissions to the correct owner. Nagios is very particular on the correct settings. This is a painful process and is a good candidate to dump in a script and ran.

# chown –R nagios.nagios /usr/local/nagios/
# chown –R root.root /usr/local/nagios/instances/nagios_prod/share/locale
# chmod 775 /usr/local/nagios/instances
# chmod 775 /usr/local/nagios/instances/nagios_prod
# chmod 775 /usr/local/nagios/instances/ nagios_prod /etc
# chmod 775 /usr/local/nagios/instances/ nagios_prod /etc/monitors
# chmod 775 /usr/local/nagios/instances/ nagios_prod/sbin
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/docs
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/contexthelp
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/images
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/images/logos
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/includes
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/media
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/ssi
# chmod 775 /usr/local/nagios/instances/ nagios_prod/share/stylesheets
# chmod 775 /usr/local/nagios/instances/ nagios_prod/var
# chmod 775 /usr/local/nagios/instances/ nagios_prod/var/rw
# chmod 775 /usr/local/nagios/instances/ nagios_prod/var/spool
# chmod 775 /usr/local/nagios/instances/ nagios_prod/var/spool/checkresults

Now the files

# chmod 664 /usr/local/nagios/instances/nagios_prod/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod /etc/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod /etc/monitors/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/sbin/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/docs/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/contexthelp/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/images/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/images/logos/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/includes/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/media/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/ssi/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/share/stylesheets/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/var/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/var/rw/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/var/spool/*
# chmod 664 /usr/local/nagios/instances/ nagios_prod/var/spool/checkresults/*

Create Monitors
At this point you can start the creation of the monitors, hosts, hostgroups, contacts. Contactgroups and all the other dependent settings. Refer to Nagios site for syntax on each of the different objects. Foe the monitors you can use the default monitors created at install in /usr/local/nagios/etc/objects and copy the configs for localhost.cfg and windows.cfg as a starting basis.
Once you have your settings completed you need to run a check on the configs running the command

# /usr/local/nagios/instances/nagios_prod/bin/nagios -v /usr/local/nagios/instances/nagios_prod/etc/nagios.cfg

Fix any errors at this point and recheck.

Startup Scripts (init.d)
Add Nagios to the list of system services and have it automatically start when the system boots.
Create init.d scripts for each instance you want to create. Copy the Nagios installed script, and name it nagios_prod. Open in your favorite editor and find the line prefix= and change the path.

Example:

prefix=/usr/local/nagios/instances/nagios_prod

Save the changes and change the permissions to 755 on the script.

Use chkconfig to add it to the startup and enable it.

# chkconfig --add nagios_prod
# chkconfig nagios_prod on

At this point you can already start your instances:

# service nagios_prod start

Now restart your Apache and open your browser

# service httpd restart

NagiosGraph (Performance Monitoring)
This section explains how to configure Nagiosgraph for multiple instances to gathering and graphing performance data created by Nagios. Each of the steps will have to be done for each individual instance you want to collect and graph performance data with the exception of rrdtool and intltool installs, those are only required installed once.

Requirements
intltool.x86_64 to 0.35 or higher
rrdtool
nagiosgraph

Update intltool.x86_64 to 0.35 or higher

# yum -y install  intltool.x86_64

Install rrdtool

# wget 'http://dag.wieers.com/rpm/packages/rrdtool/rrdtool-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/perl-rrdtool-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/rrdtool-devel-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/ruby-rrdtool-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/python-rrdtool-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/php-rrdtool-1.0.50-3.el5.rf.x86_64.rpm'
# wget 'http://dag.wieers.com/rpm/packages/rrdtool/rrdtool-1.2.23-1.el5.rf.x86_64.rpm'
# wget 'http://mesh.dl.sourceforge.net/sourceforge/nagiosgraph/nagiosgraph-1.3.1.tar.gz'
# yum install -y libart_lgpl
# rpm -hiv *rrdtool*.rpm

Install NagiosGraph
Download NagiosGraph 1.3.1 from http://nagiosgraph.sourceforge.net/

# cd /usr/local/src
# tar xvf nagiosgraph-1.3.1.tar.gz
# cd nagiosgraph-1.3.1

Create the directory structure and copy files

# mkdir /usr/local/nagios/instances/nagios_prod/nagiosgraph
# mkdir /usr/local/nagios/instances/nagios_prod/nagiosgraph/rrd
# cp -r . /usr/local/nagios/instances/nagios_prod/nagiosgraph/
# chmod go+rX /usr/local/nagios/instances/nagios_prod/nagiosgraph
# chown nagios.apache /usr/local/nagios/instances/nagios_prod/nagiosgraph/rrd

# cd /usr/local/nagios/instances/nagios_prod/nagiosgraph/lib
# cp insert.* /usr/local/nagios/instances/nagios_prod/sbin/
# touch /usr/local/nagios/instances/nagios_prod/var/nagiosgraph.log /usr/local/nagios/instances/nagios_prod/var/spool/perfdata.log
# chown nagios.apache /usr/local/nagios/instances/nagios_prod/var/nagiosgraph.log /usr/local/nagios/instances/nagios_prod/var/spool/perfdata.log
# chmod 664 /usr/local/nagios/nagios_prod/var/nagiosgraph.log
# chmod 644 /usr/local/nagios/nagios_prod/var/spool/perfdata.log

# cd /usr/local/nagios/instances/nagios_prod/nagiosgraph/share
# cp nagiosgraph.js /usr/local/nagios/instances/nagios_prod/share/
# cp nagiosgraph.css /usr/local/nagios/instances/nagios_prod/share/stylesheets/
# chown nagios.nagios /usr/local/nagios/instances/nagios_prod/share/nagiosgraph.js
# chown nagios.nagios /usr/local/nagios/instances/nagios_prod/share/stylesheets/nagiosgraph.css

# cd /usr/local/nagios/instances/nagios_prod/nagiosgraph/cgi
# cp * /usr/local/nagios/instances/nagios_prod/sbin/
# cd /usr/local/nagios/instances/nagios_prod/nagiosgraph/etc/
# mv ngshared.pm  /usr/local/nagios/instances/nagios_prod/sbin/
# mv nagiosgraph.conf  /usr/local/nagios/instances/nagios_prod/sbin/
# mv servdb.conf /usr/local/nagios/instances/nagios_prod/etc/
# mv hostdb.conf /usr/local/nagios/instances/nagios_prod/etc/
# mv map  /usr/local/nagios/instances/nagios_prod/etc/
# chown nagios.nagios  /usr/local/nagios/instances/nagios_prod/sbin/*

# cp /usr/local/nagios/instances/nagios_prod/nagiosgraph/etc/*  /usr/local/nagios/instances/nagios_prod/etc/
# mv /usr/local/nagios/instances/nagios_prod/etc/nagiosgraph.conf  /usr/local/nagios/instances/nagios_prod/sbin/
# mv /usr/local/nagios/instances/nagios_prod/etc/ngshared.pm  /usr/local/nagios/instances/nagios_prod/sbin/

Configuration file edits
Insert.pl & insert.sh file
Edit insert.pl & insert.sh and modify the lib path.

# The configuration file and ngshared.pm must be in this directory.

use lib '/usr/local/nagios/instances/nagios_prod/sbin';

Various CGI files
Edit the perl scripts in the cgi and lib directories, modifying the “use lib” for the following files in the /usr/local/nagios/instances/nagios_prod/sbin directory.

showhost.cgi
show.cgi
showhost.cgi
showservice.cgi
testcolor.cgi

Change the following line.

use lib '/usr/local/nagios/instances/nagios_prod/sbin';

Nagiosgraph.conf file
Edit nagiosgraph.conf change the following lines.

log_file=/usr/local/nagios/instances/nagios_prod/var/nagiosgraph.log
rrddir = /usr/local/nagios/instances/nagios_prod/nagiosgraph/rrd
mapfile = /usr/local/nagios/instances/nagios_prod/etc/map

# Location of performance data log file. Comment out if not used.
perflog = /usr/local/nagios/instances/nagios_prod/var/spool/perfdata.log

# Location of hosts/database control file
hostdb = /usr/local/nagios/instances/nagios_prod/etc/hostdb.conf

# Location of services/database control file
servdb = /usr/local/nagios/instances/nagios_prod/etc/servdb.conf

# Nagios CGI URL
nagioscgiurl = https://cvglp9999/nagios_prod/cgi-bin

# JavaScript configuration
# The URL for the nagiosgraph.js file
javascript = /nagios_prod/nagiosgraph.js

# Stylesheet - added to head of show.cgi, etc. Comment out if not used
stylesheet = /nagios_prod/stylesheets/nagiosgraph.css

nagios.cfg file
Edit nagios.cfg and change /add.
Add the line

# NagiosGraph
cfg_file=/usr/local/nagios/instances/nagios_prod/etc/serviceextinfo.cfg

Modify the following lines.

process_performance_data=1
service_perfdata_file_processing_command=process-service-perfdata-prod
service_perfdata_file=/usr/local/nagios/instances/nagios_prod/var/spool/perfdata.log
service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=30

checkcommands.cfg file
Add to your checkcommands.cfg file in the shared Nagios libexec directory

define command {
       command_name  process-service-perfdata-prod
       command_line  /usr/bin/perl /usr/local/nagios/instances/nagios_prod/sbin/insert.pl
     }

serviceextinfo.cfg file
Create serviceextinfo.cfg in /usr/local/nagios/instances/nagios_prod/etc/ and add the entries in for the performance data to graph. Refer to Nagios documentation on the syntax for the serviceextinfo.cfg file. This file will tell which service monitors you would like to collect performance data on.
Example entry:

define serviceextinfo {
  service_description  Memory Usage
  hostgroup_name       windows-prod-servers
  notes_url       /nagios_prod/cgi-bin/show.cgi?host=$HOSTNAME$&service=$SERVICEDESC$&db=ntmem,memused,memwarn,memcrit,memmmax
  icon_image     extinfo.gif
}

define serviceextinfo {
  service_description  CPU_Load
  hostgroup_name    windows-prod-servers
  notes_url       /nagios_prod/cgi-bin/show.cgi?host=$HOSTNAME$&service=$SERVICEDESC$&db=ntload,avg05min,avg30min
  icon_image     extinfo.gif
}

CGI.CFG File
Edit cgi.cfg and add line under main_config_file entry.

xedtemplate_config_file=/usr/local/nagios/instances/nagios_prod/etc/serviceextinfo.cfg

Map file
Edit the map file in usr/bin/perl /usr/local/nagios/instances/nagios_prod/etc directory. The default Nagiosgraph supplied one doesn’t have any Windows service map files. A good source for examples can be found at http://nerhood.wordpress.com/2004/09/22/nagiosgraph-with-windows-support/ and has the basic map files needed for a good start.

Example:

# Service type: ntload
# check command: check_nt -H Address -v CPULOAD -l5,80,90,30,70,80
# output: CPU Load 9% (5 min average) 11% (10 min average) 3% (30 min average)
# perfdata: .5 min avg Load.=9%;80;90;0;100 .15 min avg Load.=11%;70;80;0;100 .30 min average Load.=3%;80;90;0;100
/output:.*?Load (\d+)% .*?(\d+)% /
and push @s, [ ntload,
[ avg05min, GAUGE, $1 ],
[ avg30min, GAUGE, $2 ] ];

# Service type: ntmem
#   check command: check_nt -H Address -v MEMUSE -w 50 -c 90
#   output: Memory usage: total:2467.75 Mb - used: 510.38 Mb (21%) - free: 1957.37 Mb (79%)
#   perfdata: Memory usage=510.38Mb;1233.88;2220.98;0.00;2467.75
/perfdata:.*usage=([.0-9]+)Mb;([.0-9]+);([.0-9]+);([.0-9]+);([.0-9]+)/
and push @s, [ ntmem,
       [ memused, GAUGE, $1*1024**2 ],
       [ memwarn, GAUGE, $2*1024**2 ],
       [ memcrit, GAUGE, $3*1024**2 ],
       [ memmmax, GAUGE, $5*1024**2 ] ];

Complete Install
Now that NagiosGraph is installed all that remains is to restart your Nagios instance and Apache and check the results.

# service nagios_prod restart
# service httpd restart

I know this is a long and detailed article, but this can be done and has great benefit. If you would like a downloadable pdf of this post it is available Multiple_Nagios_instances_single_server.

Monitoring CPU temp on Fedora

recently found the need to monitor CPU temp on a Fedora 16 install.

$ sudo yum install lm_sensors

Once installed we need to configure lm_sensors. Run the command

$ sudo sensors-detect

You will be prompted with questions, accepting the defaults will work

$ sudo sensors-detect
# sensors-detect revision 5946 (2011-03-23 11:54:44 +0100)
# System: Gigabyte Technology Co., Ltd. EP45C-DS3R

This program will help you determine which kernel modules you need
to load to use lm_sensors most effectively. It is generally safe
and recommended to accept the default answers to all questions,
unless you know what you're doing.

Some south bridges, CPUs or memory controllers contain embedded sensors.
Do you want to scan for them? This is totally safe. (YES/no):
Silicon Integrated Systems SIS5595...                       No
VIA VT82C686 Integrated Sensors...                          No
VIA VT8231 Integrated Sensors...                            No
AMD K8 thermal sensors...                                   No
AMD Family 10h thermal sensors...                           No
AMD Family 11h thermal sensors...                           No
AMD Family 12h and 14h thermal sensors...                   No
Intel digital thermal sensor...                             Success!
(driver `coretemp')
Intel AMB FB-DIMM thermal sensor...                         No
VIA C7 thermal sensor...                                    No
VIA Nano thermal sensor...                                  No

Some Super I/O chips contain embedded sensors. We have to write to
standard I/O ports to probe them. This is usually safe.
Do you want to scan for Super I/O sensors? (YES/no):
Probing for Super-I/O at 0x2e/0x2f
Trying family `National Semiconductor'...                   No
Trying family `SMSC'...                                     No
Trying family `VIA/Winbond/Nuvoton/Fintek'...               No
Trying family `ITE'...                                      Yes
Found `ITE IT8718F Super IO Sensors'                        Success!
(address 0x290, driver `it87')
Probing for Super-I/O at 0x4e/0x4f
Trying family `National Semiconductor'...                   No
Trying family `SMSC'...                                     Yes
Found unknown chip with ID 0x0b00

Some systems (mainly servers) implement IPMI, a set of common interfaces
through which system health data may be retrieved, amongst other things.
We first try to get the information from SMBIOS. If we don't find it
there, we have to read from arbitrary I/O ports to probe for such
interfaces. This is normally safe. Do you want to scan for IPMI
interfaces? (YES/no):
Probing for `IPMI BMC KCS' at 0xca0...                      No
Probing for `IPMI BMC SMIC' at 0xca8...                     No

Some hardware monitoring chips are accessible through the ISA I/O ports.
We have to write to arbitrary I/O ports to probe them. This is usually
safe though. Yes, you do have ISA I/O ports even if you do not have any
ISA slots! Do you want to scan the ISA I/O ports? (yes/NO):
Lastly, we can probe the I2C/SMBus adapters for connected hardware
monitoring devices. This is the most risky part, and while it works
reasonably well on most systems, it has been reported to cause trouble
on some systems.
Do you want to probe the I2C/SMBus adapters now? (YES/no):
Using driver `i2c-i801' for device 0000:00:1f.3: Intel ICH10
Module i2c-dev loaded successfully.
Next adapter: nouveau-0000:01:00.0-0 (i2c-0)
Do you want to scan it? (YES/no/selectively):
Client found at address 0x49
Probing for `National Semiconductor LM75'...                No
Probing for `National Semiconductor LM75A'...               No
Probing for `Dallas Semiconductor DS75'...                  No
Probing for `National Semiconductor LM77'...                No
Probing for `Analog Devices ADT7410'...                     No
Probing for `Dallas Semiconductor DS1621/DS1631'...         No
Probing for `National Semiconductor LM73'...                No
Probing for `National Semiconductor LM92'...                No
Probing for `National Semiconductor LM76'...                No
Probing for `Maxim MAX6633/MAX6634/MAX6635'...              No
Probing for `SMSC EMC1023'...                               No
Probing for `SMSC EMC1043'...                               No
Probing for `SMSC EMC1053'...                               No
Probing for `SMSC EMC1063'...                               No
Client found at address 0x50
Probing for `Analog Devices ADM1033'...                     No
Probing for `Analog Devices ADM1034'...                     No
Probing for `SPD EEPROM'...                                 No
Probing for `EDID EEPROM'...                                Yes
(confidence 8, not a hardware monitoring chip)

Next adapter: nouveau-0000:01:00.0-1 (i2c-1)
Do you want to scan it? (YES/no/selectively):
Next adapter: ivtv i2c driver #0 (i2c-3)
Do you want to scan it? (yes/NO/selectively):

Now follows a summary of the probes I have just done.
Just press ENTER to continue:

Driver `it87':
* ISA bus, address 0x290
Chip `ITE IT8718F Super IO Sensors' (confidence: 9)

Driver `coretemp':
* Chip `Intel digital thermal sensor' (confidence: 9)

Do you want to overwrite /etc/sysconfig/lm_sensors? (YES/no):

Answer yes to the last question and all the support modules are loaded.

How do I read sensors chip data such as temperature?
Type the following command at shell prompt:

$ sensors

Sample Output
coretemp-isa-0000
Adapter: ISA adapter
Core 0:       +34.0°C  (high = +80.0°C, crit = +100.0°C)

coretemp-isa-0001
Adapter: ISA adapter
Core 1:       +34.0°C  (high = +80.0°C, crit = +100.0°C)

it8718-isa-0290
Adapter: ISA adapter
in0:          +1.06 V  (min =  +0.00 V, max =  +4.08 V)
in1:          +2.00 V  (min =  +0.00 V, max =  +4.08 V)
in2:          +3.26 V  (min =  +0.00 V, max =  +4.08 V)
in3:          +2.98 V  (min =  +0.00 V, max =  +4.08 V)
in4:          +0.24 V  (min =  +0.00 V, max =  +4.08 V)
in5:          +0.02 V  (min =  +0.00 V, max =  +4.08 V)
in6:          +0.10 V  (min =  +0.00 V, max =  +4.08 V)
in7:          +3.17 V  (min =  +0.00 V, max =  +4.08 V)
Vbat:         +3.22 V
fan1:        1418 RPM  (min =    0 RPM)
fan2:         672 RPM  (min =    0 RPM)
fan3:        1019 RPM  (min =    0 RPM)
fan4:        1056 RPM  (min =    0 RPM)
temp1:        +40.0°C  (low  = +127.0°C, high = +127.0°C)  sensor = thermistor
temp2:        +26.0°C  (low  = +127.0°C, high = +127.0°C)  sensor = thermal diode
temp3:         -2.0°C  (low  = +127.0°C, high = +127.0°C)  sensor = thermistor

If you want to watch realtime output using the watch command, type

$ watch sensors

Installing Nagios Part 1

For the next series we will be discussing installing and configuring Nagios for monitoring. The series will include installing Nagios, plugins, agents and various custom monitors. Also, adding other features including graphing and performance charting. Nagios is a powerful monitoring tools used by many IT departments and has the support of the Open Source community. So let’s get started!

Our choice for OS is Scientific Linux 6.0 which is based on RHEL 6.0, a good stable and enterprise level OS. the current version of Nagios is 3.3.1. There are several pieces to Nagios that we will need to install, the core and plugins. This will give us the base installation we need to get running with Nagios. Once they are installed and configured we will install the agents on the systems we want to monitor.

Logon to your Linux installation and download the Nagios Core and the plugins packages from here.

Prerequisites
During portions of the installation you’ll need to have root access to your machine.
Lets download the following packages on your Fedora / RHEL installation before continuing.

Apache
PHP
GCC compiler
GD development libraries

Change to root (sudo su -)

# yum install httpd php
# yum install gcc glibc glibc-common
# yum install gd gd-devel

Create Nagios Account and Group
Create a new nagios user account and give it a password.

# /usr/sbin/useradd -m nagios
# passwd nagios
Create a new nagcmd group for allowing external commands to be submitted through the web interface. Add both the nagios user and the apache user to the group.

# /usr/sbin/groupadd nagcmd
# /usr/sbin/usermod -a -G nagcmd nagios
# /usr/sbin/usermod -a -G nagcmd apache

Change into a directory or make one for downloading the Nagios Core and plugins then download them using the following commands.

Download the source code tarballs of both Nagios and the Nagios plugins (visit http://www.nagios.org/download/ for links to the latest versions). These directions were tested with Nagios 3.3.1 and Nagios Plugins 1.4.15.

# wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.3.1.tar.gz
# wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz

Now we can extract and compile the code.

# tar xzf nagios-3.3.1.tar.gz
# cd nagios

Run the Nagios configure script, passing the name of the group you created earlier like so:

# ./configure --with-command-group=nagcmd
When the configure completes you will receive a summary.

*** Configuration summary for nagios 3.3.1 07-25-2011 ***:

General Options:
————————-
Nagios executable: nagios
Nagios user/group: nagios,nagios
Command user/group: nagios,nagcmd
Embedded Perl: no
Event Broker: yes
Install ${prefix}: /usr/local/nagios
Lock file: ${prefix}/var/nagios.lock
Check result directory: ${prefix}/var/spool/checkresults
Init directory: /etc/rc.d/init.d
Apache conf.d directory: /etc/httpd/conf.d
Mail program: /bin/mail
Host OS: linux-gnu

Web Interface Options:
————————
HTML URL: http://localhost/nagios/
CGI URL: http://localhost/nagios/cgi-bin/
Traceroute (used by WAP): /bin/traceroute
Now let’s start to compile the Nagios Core source code.
# make all

Install binaries, init script, sample config files and set permissions on the external command directory.

# make install
# make install-init
# make install-config
# make install-commandmode
# make install-webconf

Now we have Nagios installed but not started, at this point we need to do a bit of customization.

Customize Configuration
Sample configuration files have now been installed in the /usr/local/nagios/etc directory. These sample files should work fine for getting started with Nagios.

Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you’d like to use for receiving alerts.

# vi /usr/local/nagios/etc/objects/contacts.cfg

Create a nagiosadmin account for logging into the Nagios web interface. Remember the password you assign to this account – you’ll need it later.
# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
# New password:
# Re-type new password:
# Adding password for user nagiosadmin

Restart Apache to make the new settings take effect.
# service httpd restart

Compile and Install the Nagios Plugins

Extract the Nagios plugins source code tarball.

# tar xzf nagios-plugins-1.4.15.tar.gz
# cd nagios-plugins-1.4.15

Compile and install the plugins.
# ./configure --with-nagios-user=nagios --with-nagios-group=nagios
# make
# make install

Start Nagios
Add Nagios to the list of system services and have it automatically start when the system boots.

# chkconfig --add nagios
# chkconfig nagios on

Verify the sample Nagios configuration files. This is very important to find the errors and correct them before starting.

# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Running pre-flight check on configuration data…
Checking services…
Checked 8 services.
Checking hosts…
Checked 1 hosts.
Checking host groups…
Checked 1 host groups.
Checking service groups…
Checked 0 service groups.
Checking contacts…
Checked 1 contacts.
Checking contact groups…
Checked 1 contact groups.
Checking service escalations…
Checked 0 service escalations.
Checking service dependencies…
Checked 0 service dependencies.
Checking host escalations…
Checked 0 host escalations.
Checking host dependencies…
Checked 0 host dependencies.
Checking commands…
Checked 24 commands.
Checking time periods…
Checked 5 time periods.
Checking for circular paths between hosts…
Checking for circular host and service dependencies…
Checking global event handlers…
Checking obsessive compulsive processor commands…
Checking misc settings…

Total Warnings: 0
Total Errors: 0

Things look okay – No serious problems were detected during the pre-flight check
If there are no errors, start Nagios.
# service nagios start

Modify SELinux Settings

Fedora / RHEL ships with SELinux (Security Enhanced Linux) installed and in Enforcing mode by default. This can result in “Internal Server Error” messages when you attempt to access the Nagios CGIs.

See if SELinux is in Enforcing mode.
# getenforce

Put SELinux into Permissive mode.
# setenforce 0

To make this change permanent, you’ll have to modify the settings in /etc/selinux/config and reboot.

Instead of disabling SELinux or setting it to permissive mode, you can use the following command to run the CGIs under SELinux enforcing/targeted mode:

# chcon -R -t httpd_sys_content_t /usr/local/nagios/sbin/
# chcon -R -t httpd_sys_content_t /usr/local/nagios/share/

For information on running the Nagios CGIs under Enforcing mode with a targeted policy, visit the Nagios Support Portal or Nagios Community Wiki.

Login to the Web Interface

You should now be able to access the Nagios web interface at the URL below. You’ll be prompted for the username (nagiosadmin) and password you specified earlier.

http://localhost/nagios/

Click on the “Service Detail” navbar link to see details of what’s being monitored on your local machine. It will take a few minutes for Nagios to check all the services associated with your machine, as the checks are spread out over time.

Now we have the base Nagios installation running. Look for the next part in the series where we will cover installing the agents!

Note: If you have a fresh install of CentOS 6.2 you will run into an issue with iptables blocking incoming port 80 so you will not be able o bring up Nagios console. Thanks to G Hamilton for reminding me of this feature.

To add the exception perform the following steps.

# iptables -I INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
# service iptables save
# service iptables restart

Configure SNMP for RHEL 6

SNMP is a very powerful and useful tool to add to your Linux system. It is great for sending alerts to different monitoring systems like Nagios and many other great products. But first it needs to be installed and configured before you can put this under rated tool to work.

Install SNMP using yum, you will need net-snmp-utils, net-snmp, net-snmp-libs.

$ sudo yum install –y net-snmp-utils net-snmp net-snmp-libs

Now that we have snmp it is time to configure SNMP.

Edit file /etc/snmp/snmpd.conf using your favorite text editor, mine just happens to be vi.

$ sudo vi /etc/snmp/snmpd.conf

Change/Modify line(s) as follows:

Find following Line:

com2sec notConfigUser default public

Replace with (make sure you replace 192.168.0.0/24 replace with your network IPs) following lines:

com2sec local localhost public
com2sec mynet 192.168.1.0/24 public

Find and change the lines.

group notConfigGroup v1 notConfigUser
group notConfigGroup v2c notConfigUser

Replace with:

group MyRWGroup v1         local
group MyRWGroup v2c        local
group MyRWGroup usm        local
group MyROGroup v1         mynet
group MyROGroup v2c        mynet
group MyROGroup usm        mynet

Locate and change the following line.

view systemview included system

Replace with.

view all included .1 80

Locate and change the following line.

access notConfigGroup “” any noauth exact systemview none none

Replace with.

access MyROGroup “” any noauth exact all none none
access MyRWGroup “” any noauth exact all all none

Locate and change the following lines.

syslocation Unknown (edit /etc/snmp/snmpd.conf)
syscontact Root (configure /etc/snmp/snmp.local.conf)

Replace with your credentials.

syslocation RHEL 6, Linux Server.
syscontact sysadmin <root@localhost>

Now let’s start your snmp server and test it.

Set the snmpd service to start on reboot or system start.

$ sudo  chkconfig snmpd on

Fire it up !!

$ sudo  service snmpd start

Test your snmp server

# snmpwalk -v 1 -c public localhost IP-MIB::ipAdEntIfIndex

Life of a Geek Admin

The Daily adventures of a true geek administrator

Tag Archives: monitoring

Windows 2008 R2 Cluster Report with PowerShell

Installing Nagios Part 2 – The Agents

Multiple Nagios Instances on a Single Server

Monitoring CPU temp on Fedora

Installing Nagios Part 1

Configure SNMP for RHEL 6