10 Sep 2009
监控服务器日志滚动

需求

完成一个 java日志监控脚本, 发现10分钟日志不滚动就发送报警邮件和报警短

信.

日志地址: web1-4 的

/home/test/software/java/Jtest/log/*/M.log

邮件报警标题

[监控][JAVA]未滚动日志列表 [20090907_125901] 发给xxx

思路

tail 日志的最后3-10行. 算出md5sum值. 对比前后两次是否一致, 如果一致就说明有问题.

#!/bin/bash
#转载请注明出处，谢谢合作
#writen by jeantoe
#jeantoe@gmail.com
#2009/09/08

#Information
sendtime=`date +%Y%m%d_%H%M%S`
MAIL='jeantoe@gmail.com'
MAILFILE='/usr/local/nagios/toJAVAmail'
#MBS='your cell phone number'

HOSTLIST='web1 web2 web3 web4'
FILES='
test.osgi.common
test.osgi.core.buy
test.osgi.core.follow
test.osgi.service.buy
test.osgi.core.lottery
test.osgi.core.lottery.analyzer
test.osgi.core.notify
test.osgi.ice.buy.server
test.osgi.core.s2m
test.osgi.core.schedule
test.osgi.core.tds
'
FILESNUMBER=11
COMMANDLOG='tail -n 3'
SLEEP_SEC=600

#collect A
for i in $HOSTLIST
do
 /usr/bin/ssh $i "
   echo "start $i " > /logs/javalog/$i.a.all
 "
 for b in $FILES
  do
   /usr/bin/ssh $i "
    $COMMANDLOG /home/test/software/java/Jtest/log/${b}/M.log > /logs/javalog/$i.a.${b}
    /usr/bin/md5sum /logs/javalog/$i.a.${b}| sed "s@$i.a.${b}@$i.${b}@" >> /logs/javalog/$i.a.all
    scp /logs/javalog/$i.a.all test00:/logs/javalog
  "
  done
done

#sleep $SLEEP_SEC

#collect B
for i in $HOSTLIST
do
  /usr/bin/ssh $i "
   echo "start $i " > /logs/javalog/$i.b.all
 "
 for b in $FILES
  do
   /usr/bin/ssh $i "
    $COMMANDLOG /home/test/software/java/Jtest/log/${b}/M.log > /logs/javalog/$i.b.${b}
    echo "start $i " > $i.b.all
    /usr/bin/md5sum /logs/javalog/$i.b.${b}| sed "s@$i.b.${b}@$i.${b}@" >> /logs/javalog/$i.b.all
    scp /logs/javalog/$i.b.all test00:/logs/javalog
  "
  done
done

#send mail

echo "To: $MAIL" >$MAILFILE
echo "From: autopost@gmail.com" >>$MAILFILE
echo "Subject: [Monitor][JAVA]未滚动日志列表[$sendtime] " >> $MAILFILE
for i in $HOSTLIST
do
   a_ALL=`cat  /logs/javalog/$i.a.all | wc -l`
   b_ALL=`cat  /logs/javalog/$i.b.all | wc -l`
   DIFF=`diff -y --suppress-common-lines /logs/javalog/$i.a.all /logs/javalog/$i.b.all | wc -l`
   if (( $a_ALL != $b_ALL ));then
    echo "${i}日志文件错误"
   elif (( $DIFF <= $FILESNUMBER ));then
    echo "THIS IS ${i}" >> $MAILFILE
    echo "JAVA日志十分钟没滚了,赶紧踢他一脚" >> $MAILFILE
    for x in `awk '$1 ~ /[^start]/{print $1}' /logs/javalog/$i.a.all`
      do
       grep $x /logs/javalog/$i.b.all | sed 's@\(.*\)\(/logs/javalog/web...\)\(.*\)@/home/test/software/java/Jtest/log/\3/M.log \1@'>> $MAILFILE
      done
    echo "             ">> $MAILFILE
#   此处可插入手机短信通知
   else
     exit 0
   fi
done
cat $MAILFILE | /usr/lib/sendmail -t

脚本写的很一般，偷懒了，变量不太规范可能除了我没人能对它修改了。

10 Sep 2009 监控服务器日志滚动

10 Sep 2009
监控服务器日志滚动