Sensu and Mailer Alert Handler

Below is a working mailer handler configuration for Sensu. The mailer settings go to “/etc/sensu/conf.d/mailer.json” and then update the handler file “/etc/sensu/conf.d/handlers.json”.

shanker@mon1:~$ cat /etc/sensu/conf.d/mailer.json
{
 "mailer": {
   "type": "pipe",
   "command": "handler-mailer.rb",
   "admin_gui": "https://XXX/",
   "mail_from": “alerts@XXXX",
   "mail_to": [
     “alerts@XXX" ],
   "subscriptions": {
     “group:XXXX_oncall": {
       "mail_to": “sde@XXX"
     },
     "admins": {
       "mail_to": “shanker.balan@XXXX"
     }
   },
   "smtp_address": "localhost",
   "smtp_port": "25",
   "smtp_domain": “XXX"
 }
}
shanker@mon1:~$ cat /etc/sensu/conf.d/handlers.json
{
 "handlers": {
   "default": {
     "type": "set",
     "handlers": [
       "logstash",
       "debug"
     ]
   },
   "mailer": {
     "type": "pipe",
     "command": "handler-mailer.rb",
     "filters": [
       "filter_warn_hourly"
     ],
     "severities": [
       "ok",
       "warning",
       "critical",
       "unknown"
     ]
   },
   "sms": {
     "type": "pipe",
     "command": "handler-sms.rb",
     "severities": [
       "critical"
     ]
   },
   "logstash": {
     "type": "pipe",
     "command": "handler-logstash.rb",
     "severities": [
       "ok",
       "warning",
       "unknown",
       "critical"
     ]
   }
 }
}

You can test by creating a dummy event and firing the handler manually.

Sample “event.json”

shanker@mon1:~$ cat event.json
{"timestamp":"2016-09-13T07:22:45.392029+0000","level":"info","message":"processing event","event":{"client":{"name":"spam2.aws.example.com","address":"unknown","subscriptions":[],"keepalives":false,"version":"0.25.3"},"check":{"standalone":true,"interval":180,"refresh":300,"source":"spam2.aws.example.com","command":"/usr/lib/nagios/plugins/check_snmp -H spam2.aws.example.com -C gujmail-public -o 1.3.6.1.4.1.20632.2.4 -w 50 -c 60 -l deferredQueueSize -t 10","occurrences":3,"handlers":["default","mailer"],"name":"deferredQueueSize-spam2.aws.example.com","issued":1473751365,"executed":1473751365,"duration":0.005,"output":"SNMP CRITICAL - deferredQueueSize *260* | deferredQueueSize=260 \n","status":2,"type":"standard","origin":"mon1.aws.example.com","history":["2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2"],"total_state_change":0},"occurrences":1384,"action":"create","timestamp":1473751365,"id":"fafcc224-9776-43c9-aab1-1e522ffd1a9b","last_state_change":1473502425,"last_ok":null}}
shanker@mon1:~$ cat event.json | jq .event | /opt/sensu/embedded/bin/handler-mailer.rb warning: event filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin
warning: occurrence filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin
mail -- sent alert for spam2.aws.example.com/deferredQueueSize-spam2.aws.example.com to ["alerts@XXXX"]
shanker@mon1:~$

Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

More Posts - Website

Follow Me:
TwitterLinkedIn

Signing Multiple Domains with the same OpenDKIM Key

Below is a working KeyTable and SigningTable example for OpenDKIM that uses the percent character (%) to replace the domain of the sender while generating the signature.


# SigningTable
# If the first field contains only a "%" character, it
# will be replaced by the domain found in the From:
# header field. Similarly, within the optional second
# field, any "%" character will be replaced by the
# domain found in the From: header field.
* default._domainkey.mydomain.tld


# KeyTable
# If the
# first value consists solely of a percent sign ("%")
# character, it will be replaced by the apparent domain
# of the sender when generating a signature. If the
# third value starts with a slash ("/") character, or
# "./" or "../", then it is presumed to refer to a file
# from which the private key should be read, otherwise
# it is itself a PEM-encoded private key or a
# base64-encoded DER private key; a "%" in the third
# value in this case will be replaced by the apparent
# domain name of the sender.
default._domainkey.mydomain.tld %:default:/etc/opendkim/keys/default.private

This ensures that the d=domain.tld and s=default selector gets set for any domains that you are hosting on the mail server. Adding new domains does not require any changes to the SigningTable or KeyTable files.

You however still need to add the DKIM TXT record for each domain that you have.

Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

More Posts - Website

Follow Me:
TwitterLinkedIn

Site To Site IPSEC VPN Tunnel Between Google Cloud And AWS with pfSense

Been porting workloads from AWS to GCE lately and one big piece of infrastructure that I rely on is pfSense.

pfSense is a FreeBSD based appliance which does advanced routing, firewall and VPN for your cloud-based infrastructure. Using pfSense, one can establish IPSEC tunnels between the various AWS regions and the clients office network. While pfSense is available from the AWS Marketplace, it’s currently not yet available on Google Cloud.

Screen Shot 2015-02-26 at 22.42.51

StrongSwan is a viable replacement for environments where pfSense is not an option. Below is a working “site to site” StrongSwan configuration running on Ubuntu 14.04 LTS GCE instance and works with pfSense 2.2. Please note that pfSense 2.2 has moved from racoon to StrongSwan.

# apt-get install strongswan
conn myconn
  reqid = 1
  fragmentation = yes
  keyexchange = ikev1
  reauth = yes
  forceencaps = no
  rekey = yes
  installpolicy = yes
  type = tunnel
  dpdaction=restart
  dpddelay = 10s
  dpdtimeout = 60s
  auto = route
  left =  10.240.x.x # my private IP as assigned to eth0 on GCE instance 
  right = 103.x.x.x # the site I am connecting to
  leftid = 1.2.3.4 # my GCE ephemeral / static IP
  ikelifetime = 28800s
  lifetime = 3600s
  ike = aes256-sha1-modp1024!
  esp = aes256-sha1!
  leftauth = psk
  rightauth = psk
  rightid = 103.x.x.x # the site I am connecting to
  aggressive = no
  rightsubnet = 192.168.x.0/24 # my office private subnet
  leftsubnet = 10.240.0.0/16 # my GCE private network

Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

More Posts - Website

Follow Me:
TwitterLinkedIn