Posts Tagged bash
Script to rotate the MySQL General Query Log
Posted by Jon Zobrist in Bash, Linux, MySQL Server Support, Programming on October 24, 2011
You want general query logging in, but don’t want to keep those pesky query log files around?
I generally setup the MySQL user and run this from cron.
Here’s my script to do so, here’s a gzip’d version, and here is a zipped version
#!/bin/bash
#setup a user in mysql with the RELOAD priviledge
#grant RELOAD on *.* to reloader@'localhost' identified by 'PASSWORD';
#flush privileges;
user="reloader"
password="PASSWORD"
LOG="${HOME}/mysqld/mysqld.log"
LOGARCHIVE="${HOME}/logarchive"
NEW_LOG="${LOGARCHIVE}/mysqld-`date +%F-%s`.log"
KEEP_MIN=15
MAX_LOAD="5.0"
RUNFILE="${HOME}/rotate-general-log.pid"
if [ -f ${RUNFILE} ]
then
echo "Runfile ${RUNFILE} exists, exiting at `date`"
${HOME}/bin/page.sh "Runfile ${RUNFILE} exists, exiting at `date`"
exit 1
else
echo "$$" > ${RUNFILE}
fi
#if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
then
echo "System load less than ${MAX_LOAD}, proceeding [DEBUG]"
else
echo "log rotate delayed, due to system load > ${MAX_LOAD}"
${HOME}/bin/page.sh "log rotate delayed, due to system load > ${MAX_LOAD}"
/bin/rm ${RUNFILE}
exit 1
fi
MYSQLD_PID=`pgrep mysqld`
if [ ! "${MYSQLD_PID}" ]
then
echo "Mysqld is NOT running, paging and exiting at `date`"
${HOME}/bin/page.sh "NO MySQLD on `hostname` at `date`"
/bin/rm ${RUNFILE}
exit 1
else
echo "Mysqld running at `date`, PID ${MYSQLD_PID}, continuing"
fi
if [ -d "${LOGARCHIVE}" ]
then
echo "Moving general log at `date`"
/bin/mv ${LOG} ${NEW_LOG}
touch ${LOG}
chmod og-rwx ${LOG}
echo "Flushing general log at `date`"
mysqladmin -u ${user} -p --password="${password}" flush-logs
echo "Done flushing general log at `date`"
echo "Gzipping ${NEW_LOG} log at `date`"
gzip ${NEW_LOG}
echo "Done gzipping ${NEW_LOG} log at `date`"
echo "Size is `du -sh ${LOGARCHIVE}` pre-clean"
echo "Cleaning ${LOGARCHIVE}, deleting files older than ${KEEP_MIN} minutes at `date`"
find ${LOGARCHIVE} -iname "*.gz" -mmin +${KEEP_MIN} -print -exec /bin/rm {} \;
echo "Done cleaning ${LOGARCHIVE}, at `date`"
echo "Size is `du -sh ${LOGARCHIVE}` post-clean"
else
echo "Missing logarchive dir ${LOGARCHIVE}"
${HOME}/bin/page.sh "Missing logarchive dir ${LOGARCHIVE}"
/bin/rm ${RUNFILE}
exit 1
fi
/bin/rm ${RUNFILE}
echo "Done at `date`"
MCOD – 9.28.2011 – Simple OS X Bash Changes
Posted by Jon Zobrist in Mac Command of the Day, Macintosh on September 28, 2011
Macintosh Command of the Day Sept 28, 2011
I have no idea why I did not do this sooner!
On my Macs I always had that md5sum the command doesn’t exist, and I hate that the output of the md5 command that does exist is not the same as the GNU tools. Sure it’s BSD (And I’ll always love my BSD), but I think compatibility to the GNU tools buys more for BSD and OS X than incompatibility.
Add this to your ~/.bashrc (and source it or restart your terminal)
alias md5sum='md5 -r '
Now your md5sum <filename> works the same as on the 1,000,000,000 Linux boxes out there.
Also in my .bashrc on my Macbook Air is
alias ll='ls -Falh' alias lg='ls -Falh|grep -i ' declare -x EDITOR="/usr/bin/vim" declare -x JAVA_HOME="/usr/"
-Jon
Delete lots of EC2 EBS volumes
Posted by Jon Zobrist in AWS, Bash, EBS, Programming on May 23, 2011
Here is a quick script to delete lots of EBS volumes on Amazon EC2.
The output from something like
ec2-describe-instances i-6fc30f01 | grep ‘^BLOCKDEVICE’ | awk ‘{ print $3 }’ > delete-these-volumes.txt
Then run
./delete-volumes.sh delete-these-volumes.txt
Can be fed into this script (you may want to edit the file first, I accept no responsibility for you deleting your volumes, with or without my script.
Download the gzipped script here.
Example output from me deleting 60 real volumes:
My EC2 EBS RAID-0 creation script
Posted by Jon Zobrist in AWS, Bash, EBS, EC2, Linux, Linux Support - Servers, Programming on May 4, 2011
I want to write up a full howto/tutorial, but have not had the time.
Here is the script that I referred to in my post at http://jonzobrist.com/2011/04/16/ec2-raid0-on-linux-with-ebs-volumes-notes/
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License HERE for more details.
http://jonzobrist.com/files/setup-ec2-raid-0.zip
http://jonzobrist.com/files/setup-ec2-raid-0.sh.gz
This script needs an AWS command line setup, and may need some minor tweaking if you’re not running a Ubuntu server.
Ubuntu’s latest AMI’s are available for 10.04 here.
Please feel free to submit patches, comments, or questions.
Thanks to everyone whose helpful posts online and in the AWS forums helped me with this script. I could not have done it with out your generous sharing of work.
Enjoy!
S3-du.sh script to get bucket size on Amazon AWS S3
Posted by Jon Zobrist in AWS, Bash, FreeBSD, Linux, Programming, S3 on March 19, 2011
Here is my script s3-du.sh that I wrote to determine how big a bucket is on Amazon S3. I’m sure there are other ways to skin this cat, but I wrote this and figured I’d share it. Feel free to use it under any free license.
Requires you to have a working s3ls and your Amazon AWS credentials setup already. I use Tim Kays AWS tools.
Links to the file as text, as gzip, as zip.
#!/bin/bash
if [ "${1}" ]
then
NUM=0
COUNT=0
for N in `s3ls ${1} | awk ‘{print $11}’ | grep [0-9]`
do
NUM=`expr $NUM + $N`
((COUNT++))
done
KB=`expr ${NUM} / 1024`
MB=`expr ${NUM} / 1048576`
GB=`expr ${NUM} / 1073741824`
echo “${COUNT} files in bucket ${1}”
echo “${NUM} B”
echo “${KB} KB”
echo “${MB} MB”
echo “${GB} GB”
else
echo “Usage : ${0} s3-bucket”
exit 1
fi
LCOD – 11.17.09 – Simple page/alert email script
Posted by Jon Zobrist in Bash, Linux, Linux Command of the Day, Programming on November 17, 2009
Here’s a quick page script for *nix boxes with mail.
I grabbed the mail / file redirect from
http://theos.in/shell-scripting/send-mail-bash-script/
#!/bin/bash
#Author : dougnaka@gmail.comThis e-mail address is being protected from spam bots, you need JavaScript enabled to view it
#Description : Quick alert/page script, edit EMAIL="" to include a comma separated list of emails to send to
#Usage : page.sh "Short or long message"
if [ "${1}" ]
then
SUBJECT="[ALERT] ${1}"
EMAIL=" someone@example.com"
EMAILMESSAGE="/tmp/`date +%s`-message"
echo "ALERT: at `date`" > ${EMAILMESSAGE}
echo "${1}" >> ${EMAILMESSAGE}
echo "From `hostname`" >> ${EMAILMESSAGE}
mail -s "${SUBJECT}" "${EMAIL}" < ${EMAILMESSAGE}
/bin/rm ${EMAILMESSAGE}
else
echo "Usage: ${0} MESSAGE"
exit 1
fi
LCOD 11.14.07 – Quick RAID check script for cron
Posted by Jon Zobrist in Bash, Linux, Linux Command of the Day, Programming on November 14, 2007
This is a quick script that will let you run the new raid checks, or repair, or pause a running raid check. It doesn’t do e-mails or anything, and from what I read on this thread it’s something mdadm needs to be set to watch for. I ran the check on a RAID and received no output, so I guess it was ok?
I will update the script as I find out more. I’d like to add a simple escalation from check to rebuild? (I think it does this anyways), and an e-mail status report, or even a syslog call saying I ran all things look good.
#!/bin/bash
###############
# Quick script to use raid functionality
# Author : Jon Zobrist < jon@bluesun.netThis e-mail address is being protected from spam bots, you need JavaScript enabled to view it >
# http://www.bluesun.net/
# License : LGPL
# Version : 1.0
# Date : 11.14.07
# install/notes:
# put it somewhere, /root/bin/ is where I prefer
# put this in your root crontab to run from cron monthly
# @monthly /root/bin/checkraids check > /root/logs/cron.log 2>&1
###############
case "$1" in
check)
for raid in /sys/block/md*/md/sync_action
do
echo check > $raid
done
;;
repair)
for raid in /sys/block/md*/md/sync_action
do
echo repair > $raid
done
;;
pause)
for raid in /sys/block/md*/md/sync_action
do
echo idle > $raid
done
;;
status)
for raid in /sys/block/md*/md/sync_action
do
echo "******************************"
echo $raid `cat $raid`
echo ""
done
;;
*)
echo "Usage : $0 check|repair|pause|status"
;;
esac
LCOD – 6.6.07 – Simple SSH remote command execution
Posted by Jon Zobrist in Linux, Linux Command of the Day on June 6, 2007
ssh -X -C -f HOSTNAME PROGRAM
Of course you need to be running an X server (Linux/*BSD Desktop) on your client, and you need to have
ForwardX11 yes
or
ForwardX11Trusted yes
in your CLIENT’s /etc/ssh/ssh_config file
and
X11Forwarding yes
in /etc/ssh/sshd_config
on the SERVER.
LCOD – 12.6.05 – finding what programs are on what ports
Posted by Jon Zobrist in Linux, Linux Command of the Day on December 6, 2005
So, you’ve run netstat -an and found something listening on a port you’re not sure what it is?
Not to worry, you can always run this command to find out;
Substitue $PORT with your actual port.
lsof -i | grep $PORT
So, say you’d run
netstat -an | grep LIST |grep tcp
and found this process
tcp 0 0 0.0.0.0:32803 0.0.0.0:* LISTEN
Not sure what 32803 was you checked /etc/services
grep 32803 /etc/services
and got nothing!
So, now you run
lsof -i | grep 32803
and you see
skype 18160 mdxdoug 22u IPv4 101807497 TCP *:32803 (LISTEN)
skype 18160 mdxdoug 23u IPv4 101807498 UDP *:32803
So it’s SKYPE! and not backdoor.pl or something. Good to know.
Enjoy!
NOTE:
If you’re using this to find a service that IS in /etc/services, you should grep for the service by name instead of port.
grep 143 /etc/services
imap 143/tcp imap2 imap4 #Interim Mail Access Protocol v2
lsof -L | grep imap
