New Custom Alert

Updated 21 January 2025

If we want to implement a custom alert on the Nagios, then we can follow this guide.

Client Side:

First of all you have to write the script.

In the script below, we are actually running a SQL query in which we are checking if the value on the column “version” has reached to the value of “1073741824”. We are implementing this alert because the maximum that can be reached is “2147483648”. So, we are making the alert for half of this value.

The script is:

shell

#!/bin/bash

# Variables
critical=1073741824
init11_max_limit=2147483648

##AccountID and Max Version
result=$(mysql -urefill -prefill -h localhost accounts -N -e "select accountId, version from accounts order by version desc limit 1;")
#####Extracting the accountId and Version from the query output
accountId=$(echo "$result" | awk '{print $1}')
version=$(echo "$result" | awk '{print $2}')

if [ "$version" -ge "$critical" ]; then
  echo "CRITICAL: accounts.accounts current version is $version for $accountId reaches to the $critical threshold"
  exit 2
else
  echo "OK: accounts.accounts current version $version for $accountId is safe"
  exit 0
fi

Now we have a condition. This alert is to be implemented on node “YEDB01B”. On this node the NRPE client should be installed. If it is not, then you have to install as follows:

shell

yum install nrpe nagios-plugins
systemctl enable nrpe
systemctl start nrpe

Now update the file “/etc/nagios/nrpe.cfg” with the following at end of file:

bash

command[check_accounts_version]=/path/to/your/script.sh

The path is usually for the script is “/usr/lib64/nagios/plugins/”

Now reload the nrpe:

bash

systemctl reload nrpe

Server Side:

Now as the NRPE client has been installed. Now we will need a Nagios Server. We are assuming that it is already installed.

Now we will go towards the default directory of the Nagios Server. The path is:

bash

/usr/local/nagios/etc/objects/

Here we will be using the following two important files:

commands.cfg
services.cfg

In the first file, we will define the command through which we will be checking our alert. As mentioned above, we chose the command name to be “check_accounts_version”. We can define it as:

bash

define command {
    command_name    check_accounts_version
    command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -c check_accounts_version
}

Now we will update the services.cfg file.

Basically it is the file in which we define our alerts and its settings like what should be the recipients of the alert, after how much time of occurrence it should appear and etc.

The basic stanza for the alert is:

bash

define service {
    use                 generic-service
    host_name           YEDB01B
    service_description Accounts Version Check
    check_command       check_accounts_version
}

But we will be using the following as we have multiple departments in our organization like (L1 and L2) and need to send the alerts accordingly as per their policy.

bash

# Template for accounts.accounts Version Check
define service {
        name                            check_accounts_version
        use                             basic-service
        check_interval                  3
        retry_interval                  1
        notification_options            w,c,r,u
        check_command                   check_nrpe!check_accounts_version
        register                        0
}

# Service for YEDB01B host
define service {
        use                             check_accounts_version
        service_description             Count of Version in accounts.accounts
        contact_groups                  seamless
        host_name                       YEDB01B
        register                        1
}

# Escalation for Level One
define serviceescalation {
        host_name                       YEDB01B
        service_description             Count of Version in accounts.accounts
        contact_groups                  levelone
        first_notification              1
        last_notification               5
        notification_interval           10
        escalation_options              w,u,c,r
}

# Escalation for Level Two
define serviceescalation {
        host_name                       YEDB01B
        service_description             Count of Version in accounts.accounts
        contact_groups                  leveltwo
        first_notification              6
        last_notification               10
        notification_interval           10
        escalation_options              w,u,c,r
}

After we have done the above additions, then we should check that if our changes are correct and validate them. For this, we can use the following command:

bash

sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

The result shows like:

Then we will restart the service of nagios and nrpe on the server node.

bash

systemctl restart nagios
systemctl reload nrpe

After this, login to the GUI of Nagios and locate the alert.

See the highlighted part above.

Spotted a mistake or want something added? Send me a note.

📊New Custom Alert

Client Side:

Server Side:

New Custom Alert