Sanxiago and other Daemons Software development, system administration and other daemons

Python vimrc

Posted on April 11, 2011

A nice tip if you work with python under vi, use this vimrc config file.

http://svn.python.org/projects/python/trunk/Misc/Vim/vimrc

you can just copy it to your home directory
wget http://svn.python.org/projects/python/trunk/Misc/Vim/vimrc ~/python_vimrc

you can start vi with vi -u ~/python_vimrc

Or make it your default .vimrc you will end with nice syntax highlighting

Filed under: Blog No Comments

Nagios Acknowledge for the Masses

Posted on September 26, 2010

I made this simple perl script to help with the acknowledging of multiple alerts.

When running in a large environment, and during a large maintenance alerts can flood the user and even with the use aid of servicegroups and hostgroups the alerts can overwhelm the user.

The script lists any problem unacknowledged or without unscheduled downtime.
Similar to what this link does:

/cgi-bin/status.cgi?host=all&type=detail&servicestatustypes=29&hoststatustypes=15&serviceprops=10

To setup the script, make sure you edit the paths to your nagios status.dat, and the command FIFO file.
Script should be able to write to the FIFO file.

To use the script, run without arguments, in interactive mode.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
#!/usr/bin/perl
 
#####################################################################################################
#   This script provides help to acknowledge multiple services during a large maintenance
#   Sometimes host groups and service groups do not suffice
#      Script requires the setup of the location of the status.dat and the FIFO file
#      Script should be able to write to the FIFO file
#	Command is run interactively
#      Santiago Velasco - sanxiago.com
#########################################################################################
 
	my $command_file = "/usr/local/nagios/var/rw/nagios.cmd";
	my $status_file = "/usr/local/nagios/var/status.dat";
	my $time = time();
	my %state = (1 ,'WARNING', 2,'CRITICAL', 3,'UNKNOWN');
	my $user = $ARGV[0];
	my $msg = $ARGV[1];
	my $search_string = $ARGV[3];
 
print STDERR "\n\nACKNOWLEDGE AND SCHEDULE DOWNTIME FOR MULTIPLE SERVICES\n\n";
 
while(!defined($user) or $user =~ /\;|\[|\]/  or length($user)<=1){
	print STDERR "Type in yout USER that acknowledges:\n";
	$user = <>;
	$user =~ s/\n//;
}
while (!defined($msg) or $msg =~ /\;|\[|\]/ or length($msg)<=1 ){
	print STDERR "Type in the MESSAGE that will be used for all acknowledges:\n";
	$msg = <>;
	$msg =~ s/\n//;
}	
print STDERR "Type in a string that matches the service_description of the services you want to ack.\n Leave it blank to list all alerts):\n"; $search_string = <>; $search_string =~ s/\n//; if(length($search_string)<=1){
	$search_string='.*';
}
 
if (-r $status_file){	
	open (STATUS, $status_file);
}
else {
	print STDERR "FAILED TO READ NAGIOS STATUS FILE\n";
	exit 1;
}
while(<STATUS>){
	if($_ =~ /service \{/){
	$is_service = 1;
	}
	if($_ =~ /\}/ and $service_description=~/$search_string/){
	$is_service =0;
		if(defined($current_state) and $current_state and $acknowledged==0 and $scheduled_downtime==0 ){
		# Command Format:
		# [time] ACKNOWLEDGE_SVC_PROBLEM;<host_name>;<service_description>;<sticky>;<notify>;<persistent>;<author>;<comment>
		# [time] SCHEDULE_SVC_DOWNTIME;<host_name>;<service_desription><start_time>;<end_time>;<fixed>;<trigger_id>;<duration>;<author>;<comment>
 		undef($ack_true);
		print STDERR "\n---------------------------------------------------------\n";
		print STDERR "Acknowledge $service_description @ $host_name $current_state".$state{$current_state}."?\n$plugin_output\n[y/n/s] (s followed by the number of minutes of scheduled downtime) (Enter to skip)\n";
		$ack_true=<>;
		# if acknowledge yes
		if($ack_true=~/^y/){
			if (!(-w $command_file)){ print STDERR "FAILED TO OPEN FIFO FILE"; exit 1; }
			open (CMD, '>>'.$command_file);
			print CMD "[$time] ACKNOWLEDGE_SVC_PROBLEM;$host_name;$service_description;1;0;1;$user;$msg\n";
			close (CMD);
		# if schedule downtime 
		}elsif($ack_true=~/^s(.*)/){
			my $duration = $1;
			if($duration=~/[^\d]*([0-9]+).*/){
				#expect duration in minutes convert to seconds
				$duration=int($1)*60;
			}else{
				$duration=3600;
			}
                        my $end_time = $time + $duration;
 
                        if (!(-w $command_file)){ print STDERR "FAILED TO OPEN FIFO FILE"; exit 1; }
                        open (CMD, '>>'.$command_file);
			print CMD "[$time] SCHEDULE_SVC_DOWNTIME;$host_name;$service_description;$time;$end_time;1;0;$duration;$user;$msg\n";
                        close (CMD);
		}
		}
	undef($current_state);
	undef($host_name);
	}
	if($is_service){
		if($_=~/host_name\=(.*)/){
		$host_name=$1;
		}
		if($_=~/service_description\=(.*)/){
		$service_description=$1;
		}
                if($_=~/current_state\=([0-9]*)/){
                $current_state=$1;
                }
                if($_=~/problem_has_been_acknowledged\=([0-9]*)/){
                $acknowledged=$1;
                }
                if($_=~/plugin_output\=(.*)/){
                $plugin_output=$1;
                }
		if($_=~/scheduled_downtime_depth\=([0-9]*)/){
		$scheduled_downtime=$1;
		}
	}
}
close(STATUS);

Python object dump

Posted on September 26, 2010

When working with objects and arrays there are times you need to debug a certain object or array and list all its contents.
Most languages provide a functionality to do so, in the case of Perl you have Data::Dumper and in PHP you have print_r

I was looking for something like that and found:

Recipe

I truly recommend it.
It can be easily added as an extra module.

And Now for Something Completely Different

Posted on April 30, 2010

Filed under: Blog, Python No Comments

Bank decreases security in attempt to increase password strength

Posted on April 30, 2010

I was asked to set my phone password by my bank, following these rules:
1. Password must have 7 digits
2. No 2 digits can repeat in a password
3. Consecutive digits are not allowed

Some security expert thought the best way to protect the "stupid" users from choosing easy passwords.
Was to enforce rules 2 and 3.
Lets keep in mind that without the rules 2 and 3 we had 9'999'999 possible passwords.

Rule 2 means you must pick 7 numbers out of the 9 digits without repeating any digit.
Using simple math we have
nPr = n! / (n-r)!
Were n is 9 as there are 9 digits in a phone, as r is 7 as that is the digits we must pick out.
We have: 9! / 2 = 181,440
As a result we have only 181,440 Valid passwords.

*Rule 3, sequences of numbers are not allowed.
NCm - ( N – m + 1 )Cm
We have: 6435 - 84 = 6351
Thats 6351 passwords we are unable to use.
181,440 - 6,351 = 175,089

Now the evil hacker who wants to access your account only has 175,089 passwords to guess from.
Look at the common phone digits layout:
1 2 3
4 5 6
7 8 9
    0

My guess is probably most users when unable to pick the code they wanted because it was not compliant with rules 2 and 3 picked a password based on the phone layout.

Here is my guess on the top 4 passwords.
1-4-7 3-6-9 - 0
1-4-7 3-6-9 - 5
2-5-8-0 1-4-7
2-5-8-0 3-6-9

* http://www.albaiges.com/matematicas/combinatoria/combinacionesordenadas.htm

Filed under: Blog No Comments

Python Logging Module

Posted on April 30, 2010

I started using Python's logging module in my Python scripts.

Below is an example on how I created a python logger object.

First I created this constructor method to ease the logger object creation.

The create_logger function receives the following parameters:

  • logName
  • This is the only parameter that is mandatory and does not have a default value.

  • logLevel='debug'
  • This is the logging level in lower case

  • logFile='test.log'
  • This is the log path/file were log will be written

  • stream=0
  • 0 no STDOUT, 1 print to STDOUT

Hardcoded you will see the max size of the log before it rotates. Log rotation is done using logging.handlers.RotatingFileHandler if you do not want to use log rotation you would probably need to use logging.handlers.FileHandler instead.

#my_log.py
import logging
import logging.handlers
 
def create_logger(logName, logLevel='debug', logFile='test.log', stream=0):
	LOG_FILENAME=logFile
	# Files will rotate at 200MB
	MAX_BYTES = 209715200
	# Files versions to keep
	BACKUP_COUNT = 10
	LEVELS = {'debug': logging.DEBUG,
        	  'info': logging.INFO,
	          'warning': logging.WARNING,
	          'error': logging.ERROR,
	          'critical': logging.CRITICAL}
        logLevel = LEVELS.get(logLevel, logging.NOTSET)
 
	# Set up a specific logger with our desired output level
	my_logger = logging.getLogger(logName)
	my_logger.setLevel(logLevel)
        if(logFile != 0  and logFile != ''):
		# Add the log message handler to the logger
		handler = logging.handlers.RotatingFileHandler(LOG_FILENAME, maxBytes=MAX_BYTES, backupCount=BACKUP_COUNT)
	        # create formatter
	        formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
	        # add formatter to ch
	        handler.setFormatter(formatter)
		my_logger.addHandler(handler)
	# If required we print out stream, or if no logFile was specified
	if(stream != 0 or logFile == 0 or logFile == ''):
	        handler = logging.StreamHandler()
        	my_logger.addHandler(handler)
	return my_logger

This is a simple example on how the logging module is used:

#test_log.py
import sys
import my_log.py
# We create the logger, using the script name sys.arv[0] and that is the logName, can be an arbitrary string.
# Named arguments are optional, and all have default values if left undeclared
my_logger=cacti_util.create_logger(sys.argv[0],stream=1,logFile='testing.log', logLevel='debug')
 
# We log the program start
my_logger.info(sys.argv[0] + ' started')
 
i=0
try:
	while(1<2):
		i = i + 1
		my_logger.warn('i = %d' % i) # this is an example warning in the log
except:
    my_logger.exception(sys.exc_info()[0]) # logger.exception is used to print full details on python exceptions
    my_logger.info(sys.argv[0] + ' died')

This is an example of what would be written in the log.


2010-04-30 16:47:33,200 - ping.py - INFO - ping.py started
2010-04-30 16:47:33,201 - ping.py - WARNING - i = 1
2010-04-30 16:47:33,201 - ping.py - WARNING - i = 2
2010-04-30 16:47:33,201 - ping.py - WARNING - i = 3
# ... AFTER SEVERAL LINES LATER I PRESS CTRL + C TO BREAK THE LOOP
2010-04-30 16:47:33,254 - ping.py - WARNING - i = 407
2010-04-30 16:47:33,254 - ping.py - ERROR -
Traceback (most recent call last):
File "ping.py", line 14, in
my_logger.warn('i = %d' % i) # this is an example warning in the log
File "/usr/lib/python2.6/logging/__init__.py", line 1060, in warning
self._log(WARNING, msg, args, **kwargs)
File "/usr/lib/python2.6/logging/__init__.py", line 1165, in _log
self.handle(record)
File "/usr/lib/python2.6/logging/__init__.py", line 1175, in handle
self.callHandlers(record)
File "/usr/lib/python2.6/logging/__init__.py", line 1212, in callHandlers
hdlr.handle(record)
File "/usr/lib/python2.6/logging/__init__.py", line 673, in handle
self.emit(record)
File "/usr/lib/python2.6/logging/handlers.py", line 73, in emit
logging.FileHandler.emit(self, record)
File "/usr/lib/python2.6/logging/__init__.py", line 852, in emit
StreamHandler.emit(self, record)
File "/usr/lib/python2.6/logging/__init__.py", line 792, in emit
self.flush()
File "/usr/lib/python2.6/logging/__init__.py", line 754, in flush
self.stream.flush()
KeyboardInterrupt
2010-04-30 16:47:33,271 - ping.py - INFO - ping.py died

The logging module offers other features such as email logging.
Here is a nice simple example.

Mysql Innodb monitoring with nagios

Posted on February 13, 2010

I am building a full innodb nrpe plugin.

We ran into a problem, the status gets trunctated and we are unable to plot all the correct values.
A colleague pointed me to this link.
http://www.mysqlperformanceblog.com/2008/10/31/full-innodb-status/

Below is a small script to get the full innodb status file using the hack discussed in the link above.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
mysql_port=3306;
 
#Get PID of mysql_d running on $mysql_port
mysql_pid=`pgrep -f "mysqld.*--port=$mysql_port "`;
 
#If no PID was found, no mysqld instance is running on $mysql_port
if  `test -z $mysql_pid`
then
        echo "ERROR: No Mysqld instance running on port $mysql_port"
        exit 2;
fi
# We look for the full INNODB status file as described in:
# http://www.mysqlperformanceblog.com/2008/10/31/full-innodb-status/
for i in `ls -l /proc/$mysql_pid/fd | grep deleted | grep tmp | awk '{print $9}'`;
        do
        if [ ${#innodb_status_file} == 0 ]
        then
        innodb_status_file=`grep -l "INNODB MONITOR OUTPUT" /proc/$mysql_pid/fd/$i` ;
        else
        break;
        fi
done;
# If we did not find the file
if [ ${#innodb_status_file} == 0 ]
then
        echo "ERROR: unable to find INNODB MONITOR file"
        exit 2;
fi
 
cat $innodb_status_file;

Today I found this project, I will test this out.
http://www.xaprb.com/blog/2006/07/02/innotop-mysql-innodb-monitor/

Thinking outside the box

Posted on February 12, 2010

Filed under: Blog No Comments

Check IIS servers that require user authenticaction

Posted on February 12, 2010

We needed to monitor a couple of IIS servers that required user authentication.
We currently use nagios and cacti to monitor our servers.

I cooked this simple script, that provides a method to check a IIS webserver page that require NTLM Authentication.
Horse work is done entirely by curl, I tested 7.12.1 with libcurl/7.12.1

To test if your current curl binary does the trick call curl like this

curl -u $user:$pass --ntlm  --stderr /dev/null $uri  -i

Examine the results, you should first see a page with a 401 unauthorized response, then you should see the authorization being sent over to the server, If the user and pass are correct and curl ntlm worked then you should see the end page with status code 200 OK or 302 Page Moved if its a redirect.

The script receives a URL as a parameter, logins to the IIS server using the curl binary, then it parses the output of the command and after it sees the authentication was sent, captures the response code.

Timeout pass and user values are hard-coded in the below example, the script currently only has handlers for some response codes, but a switch was used to add more in an easy way.
Response code is found with regexp /HTTP/1.1 ([0-9]{3}) .*/
if your server returns a different status code you might need to change that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
#!/usr/bin/perl
#       02/Feb/10                       nagios@sanxiago.com
#  check_http page for IIS servers with ntlm authentication
#
# this check receives a URL as a parameter, logins to the IIS server
# using the curl binary, then it parses the output of the command
# and captures the response code. Timeout pass and user values are currently hardcoded
# script currently only has handlers for some response codes, but a switch was used to 
# add more in an easy way. Response code is found with regexp /HTTP\/1\.1 ([0-9]{3}) .*/
 
use Switch;
use Time::HiRes;
use Getopt::Long;
 
sub print_usage (){
print "Usage: $0 --uri="http://somepage" --user=DJohn --pass=p4ssw0rd\n" ;
}
 
GetOptions( "U|uri=s" => \$uri, "u|user=s" => \$user,"p|pass=s"=> \$pass);
 
if(!defined($uri) and !defined $user and !defined $pass){
print_usage();
}
 
$timeout=30;            # Timeout in seconds
 
$start = Time::HiRes::time();
run_command("curl -u $user:$pass --ntlm  --stderr /dev/null $uri  -i ");
$time = sprintf("%.2f",Time::HiRes::time()-$start);
 
switch ($http_code){
case 200 {print $time."s OK"; exit(0);}
case 302 {print $time."s PAGE MOVED"; exit(1);}
case 404 {print $time."s PAGE NOT FOUND"; exit(2);}
case 500 {print $time."s SERVER ERROR"; exit(2);}
case 401 {print $time."s UNAUTHORIZED"; exit(2);}
else     {print $time."s ERROR $output"; exit(-1);}
}
 
sub run_command {
$command=shift;
$pid = open(PIPE, "$command  |") or die $!;
eval {
       $output="";
       local $SIG{ALRM} = sub { die "TIMEDOUT" };
       alarm($timeout);
        while (<PIPE>) {
                if($_=~/HTTP\/1\.1 ([0-9]{3}) .*/ && $authentication_sent){
                        $http_code=$1;
                }
                if($_=~/WWW-Authenticate/){
                        $authentication_sent=1;
                }
                $output=$output.$_;
        }
        close(PIPE);
};
if ($@) {
    die $@ unless $@ =~ /TIMEDOUT/;
    print "TIMEOUT";
    kill 9, $pid;
    $? ||= 9;
    exit(2);
}
}

As you will see this script can easily be edited to serve as a data input in cacti or other monitoring app.