When an Elastic Load Balancer handles a connection it sends it’s own (internal/private/10.x) address instead of the clients. It sends the clients along with the request as X-Forwarded-For. To log this you need to log X-Forwarded-For instead of the source IP.

Here are 2 links discussing the problem, the first covers a basic Apache & Tomcat setup, but the second one has a point about direct access getting not logged and has an Apache httpd specific solution.

http://blog.kenweiner.com/2009/09/amazon-elb-capturing-client-ip-address.html

http://blog.grahampoulter.com/2011/10/how-to-log-client-ip-from-apache-behind.html

Thanks @grahampoulter and @kweiner

Related posts:

1. Reading : Peecho Minimizing Downtime on Amazon AWS   What the guys over at Peecho have done is...

Related posts brought to you by Yet Another Related Posts Plugin.

You want general query logging in, but don’t want to keep those pesky query log files around?

I generally setup the MySQL user and run this from cron.

Here’s my script to do so, here’s a gzip’d version, and here is a zipped version

#!/bin/bash
#setup a user in mysql with the RELOAD priviledge
#grant RELOAD on *.* to reloader@'localhost' identified by 'PASSWORD';
#flush privileges;

user="reloader"
password="PASSWORD"
LOG="${HOME}/mysqld/mysqld.log"
LOGARCHIVE="${HOME}/logarchive"
NEW_LOG="${LOGARCHIVE}/mysqld-`date +%F-%s`.log"
KEEP_MIN=15
MAX_LOAD="5.0"
RUNFILE="${HOME}/rotate-general-log.pid"

if [ -f ${RUNFILE} ]
 then
  echo "Runfile ${RUNFILE} exists, exiting at `date`"
  ${HOME}/bin/page.sh "Runfile ${RUNFILE} exists, exiting at `date`"
  exit 1
 else
  echo "$$" > ${RUNFILE}
fi

#if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
 then
  echo "System load less than ${MAX_LOAD}, proceeding [DEBUG]"
else
  echo "log rotate delayed, due to system load > ${MAX_LOAD}"
  ${HOME}/bin/page.sh "log rotate delayed, due to system load > ${MAX_LOAD}"
  /bin/rm ${RUNFILE}
  exit 1
fi

MYSQLD_PID=`pgrep mysqld`

if [ ! "${MYSQLD_PID}" ]
 then
  echo "Mysqld is NOT running, paging and exiting at `date`"
  ${HOME}/bin/page.sh "NO MySQLD on `hostname` at `date`"
  /bin/rm ${RUNFILE}
  exit 1
else
  echo "Mysqld running at `date`, PID ${MYSQLD_PID}, continuing"
fi

if [ -d "${LOGARCHIVE}" ]
 then
  echo "Moving general log at `date`"
  /bin/mv ${LOG} ${NEW_LOG}
  touch ${LOG}
  chmod og-rwx ${LOG}
  echo "Flushing general log at `date`"
  mysqladmin -u ${user} -p --password="${password}" flush-logs
  echo "Done flushing general log at `date`"
  echo "Gzipping ${NEW_LOG} log at `date`"
  gzip ${NEW_LOG}
  echo "Done gzipping ${NEW_LOG} log at `date`"
  echo "Size is `du -sh ${LOGARCHIVE}` pre-clean"
  echo "Cleaning ${LOGARCHIVE}, deleting files older than ${KEEP_MIN} minutes at `date`"
  find ${LOGARCHIVE} -iname "*.gz" -mmin +${KEEP_MIN} -print -exec /bin/rm {} \;
  echo "Done cleaning ${LOGARCHIVE}, at `date`"
  echo "Size is `du -sh ${LOGARCHIVE}` post-clean"
else
 echo "Missing logarchive dir ${LOGARCHIVE}"
  ${HOME}/bin/page.sh "Missing logarchive dir ${LOGARCHIVE}"
 /bin/rm ${RUNFILE}
 exit 1
fi

/bin/rm ${RUNFILE}
echo "Done at `date`"

Related posts:

1. I am always cautious with this on my clipboard.. su mysql cd rm -Rf data/* innodb/* tmp/* logs/* mysqld/*...
2. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
3. My EC2 EBS RAID-0 creation script I want to write up a full howto/tutorial, but have...

Related posts brought to you by Yet Another Related Posts Plugin.

There are a ton of cool things to do on the Internet. New doors are open to everyone. I’m surprised how often we take it for granted that everyone is in on the latest trend in tech. Here is my humble addition to a list of things that I think people should be using online.

1. Twitter – News *stream*, or should I say FLOOD. Follow smart people, get smart (filtered) news and info. Want to blow your news mind? Get tweetdeck and put in a search for any hot topic. (Don’t follow #earthquake unless you want to feel constant fear).

2. Facebook – Connect with your family and friends. Be benign on Facebook! The Internet is public, immortal, and Facebook does hate your privacy.

3. Amazon AWS/EC2 - What you don’t need a virtual server? You sure about that? Not for your blog? Not even if it scales infinitely? Not even if it’s free?

4. WordPress – Joomla and Drupal are cool, but WordPress is the king of the web page CMS.

5. Gmail – Seriously, stop deleting your email, get a gmail account. Use your own domains (Google Apps is still free for < 10 users).

6. Google Docs – If you haven’t had 10 people all editing the same spreadsheet at the same time you have not Cloud’d it up.

7. Cloud Music (Google Music, Amazon Music Locker, iCloud, Soundcloud, Spotify) – This is new, try them all out, find new music, sync your own.

8. Google – Search done right. Everyone has been playing catchup for a while now, and I’m sure that one day they will, but until then, google.com

9. Snopes – The Internet means rapid access to information sharing, but many people share false information. Sites like snopes.com

10. Shopping - Deal sites like slickdeals.net, fatwallet.com, woot and more track deals as they happen, often with good comments on how to maximize them. The people on some of these sites are mad geniuses when it comes to getting the most for your buck.

11. Skype – Everyone has it, get on and video chat your friends in other countries for free. Ride this one until Microsoft torpedos it, and we all move to Google Chat, which you should be on already via your gmail account.

12. Linux – If you are even slightly technically inclined, Linux opens the door to you (for free) to everything from high end movie effects to  computer forensics. Get started with a Live CD from Ubuntu (Your computer is probably 64-bit, and you probably want the desktop version – You can boot the CD and use Linux without doing anything to your computer), and NO it does not run Office or any Windows program, but it does run thousands of cool programs.

13. Photo sites (Picasa, Flickr, Smugmug) There is no reason you should be burning a photo CD to send to your friends and family. Get an upload utility, and start putting your photos on the ‘net. You don’t have to share them, and I would highly recommend NOT sharing them publicly unless they are very public information. I do not post pictures with faces in them without permission from the person owning the face, and, in general, don’t do this.

14. Education (Khanadademy, Alison.com, MIT Open Courseware, Instructables, k12) – There are too many to name, and pretty much access to infinite information is it’s own education. Don’t think that just because a skill isn’t directly computer related that you can’t learn howto do it online, and for maybe for free.

15. Wikipedia - What is a wikipedia? Well, a wiki is a website that anyone can edit the pages of, so, Wikipedia is an encyclopedia that anyone can edit. Not always right, but rarely uninformative.

Well, I hope this helps. Please send me your lists or additions (comment below, or email to jon@jonzobrist.com).

Related posts:

1. Finally On EC2 I can’t believe it took me this long to nut-up...

Related posts brought to you by Yet Another Related Posts Plugin.

So, I have a script that can run from init. You configure it from a config file, set it to start on boot via normal init, and then you can run it on to snapshot your data to Amazon S3 anytime.

It supports directories/users/permissions as well as full MySQL dumps, only specific database MySQL dumps, and only specific table per database dump.

It goes both ways, run it with /etc/init.d/bluesun-setup.sh start and it deletes the local directories, and downloads all the updated ones from Amazon S3.

If you run it with /etc/init.d/bluesun-setup.sh updateS3 # it will push all the same configured files to Tar/GZ files on S3.

I need to polish it a bit more, but it’ll be free/open source here soon.

Any feedback is appreciated.

Related posts:

1. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
2. My EC2 EBS RAID-0 creation script I want to write up a full howto/tutorial, but have...
3. Thinking about hosting a WordPress site on S3 I recently moved my Joomla backed consulting website completely to...

Related posts brought to you by Yet Another Related Posts Plugin.

su mysql

cd

rm -Rf data/* innodb/* tmp/* logs/* mysqld/*

mysql_install_db –defaults-file=${HOME}/etc/my.cnf

${HOME}/libexec/mysqld –defaults-file=${HOME}/etc/my.cnf > ${HOME}/logs/init.log 2>&1 &

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

My kids have dove into the game Minecraft and one of the first things they wanted was their own servers that they could customize and create lots of world.

We’re now running 5 servers on my Linux box. I don’t notice a performance drop in using it for anything, and each one is set to use 1GB of RAM, fortunately I have 8 GB…

This should be a quick howto that will work on Linux and Macintosh as long as you have screen and java.

I run the Minecraft servers under screen, so I can disconnect and reconnect to them, and they’ll keep running if my connection stops.

Quick setup :

(I use -R instead of -S to start and name the screen, since it will resume the same screen if it’s already running)

mkdir minecraft

cd minecraft

Edit a server.properties file, here’s a sample one :

Go to http://www.minecraft.net/download.jsp and copy the link that says “minecraft_server.jar” (it will update based on version, so copy it each time)

wget -O minecraft_server.jar http://www.minecraft.net/download/minecraft_server.jar?v=1307312614783

(that link will change with each version)

Now start your server

screen -R minecraft

java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui

Now type CTRL+a d

This will disconnect you from your  screen.

To reconnect run

screen -R minecraft

Now connect to your computer from minecraft, you may need to connect by IP address and port.

Running multiple servers on the same host is as easy as changing the port and setting up another directory, and the screen instances.

This is just a simple setup assuming you have a Mac or a Linux box already running.

I hope to cover using a Dynamic DNS service like DynDNS.com , opening the ports in your router so your friends can connect from outside.

It may also help to find out your public IP address, you can go to http://jonzobrist.com/ip.php

I wonder if you could run a minecraft server with only 512MB RAM, would be perfect for the free tier of Amazon AWS..

Related posts:

1. Quick remote SMTP page script in Python Disclaimer : You own what you do with this script,...
2. LCOD 5.29.2011 – 2 quick TCP port check scripts in Python Disclaimer : These scripts could be considered malicious and should...

Related posts brought to you by Yet Another Related Posts Plugin.

I love Ubuntu.

I use it everywhere. I love the Debian based dpkg, the default installs, the Amazon AWS compatible EUC cloud.

I’m also a big fan of cool names, I like having printers and domains and whatever on some big, geeky, internal naming schema, like everything relates to Norse mythology or Star Trek.

But, I CANNOT STAND the code names for Ubuntu projects being used to refer to the version. I’m running Ubuntu 9.04, or 8.04 LTS, or 10.04 LTS, etc. I am not running Ubuntu Honey Badger, or whatever code name they developed under.

This is only annoying when I’m trying to find instructions about how to do something and people say things like  ’Well, I tried this and it worked on Hoary’. WTF is a Hoary? Please, don’t tell me, I don’t care.

So, for my own sanity I am posting this rant, and linking to the page with a list of Development Codes for Ubuntu, as I’m sure, going forward, I’m going to need it.

https://wiki.ubuntu.com/DevelopmentCodeNames

And for my quick use, the versions I use most:

8.04 Hardy Heron

9.04 Jaunty Jackalope

10.04 Lucid lynx

11.04 Natty Narwhal

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Disclaimer : You own what you do with this script, and are responsible for it. This script could cause problems with SMTP / mail server administrators, and you should be sure to get any use approved. I make no claims about the scripts fitness for any specific use.

I have several servers that are not allowed to communicate with the outside world. Often times this helps a lot, but it can be a pain when trying to send e-mail alerts. Combine that sometimes I’m not root on these boxes so I can’t simply change the postfix/exim/qmail/sendmail default relay server to something.

The work around is this script, which uses a hard coded SMTP server, so you’ll need to edit the script to change the from, to, and the smtp servers IP.

I haven’t set it to take arguments, as I rarely re-use it for different things. Let me know if you’d benefit from that and I’ll throw them in.

Here is the script in gzipped format

Here is the code of the script :

#!/usr/bin/python
#Author : jon@jonzobrist.com
#License : BSD/public/freeware

import smtplib
import sys

def prompt(prompt):
return raw_input(prompt).strip()

fromaddr = “noreply@example.com”
#toaddrs = ['userA@example.com','userB@example.com','Phone1@txt.att.net','Phone2@txt.att.net','userC@example.com']
toaddrs = ['userA@example.com']
subject = “[ALERT] Alert from localhost”

msg = (“From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n”
% (fromaddr,toaddrs,subject))
msg = msg + sys.argv[1]
server = smtplib.SMTP(‘server.ip.or.hostname’)
#server.set_debuglevel(1)
server.sendmail(fromaddr, toaddrs, msg)
server.quit()

Enjoy!

Related posts:

1. LCOD 5.29.2011 – 2 quick TCP port check scripts in Python Disclaimer : These scripts could be considered malicious and should...
2. My EC2 EBS RAID-0 creation script I want to write up a full howto/tutorial, but have...

Related posts brought to you by Yet Another Related Posts Plugin.

I want to write up a full howto/tutorial, but have not had the time.

Here is the script that I referred to in my post at http://jonzobrist.com/2011/04/16/ec2-raid0-on-linux-with-ebs-volumes-notes/

This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License HERE for more details.

http://jonzobrist.com/files/setup-ec2-raid-0.zip

http://jonzobrist.com/files/setup-ec2-raid-0.sh.gz

This script needs an AWS command line setup, and may need some minor tweaking if you’re not running a Ubuntu server.

Ubuntu’s latest AMI’s are available for 10.04 here.

Please feel free to submit patches, comments, or questions.

Thanks to everyone whose helpful posts online and in the AWS forums helped me with this script. I could not have done it with out your generous sharing of work.

Enjoy!

Related posts:

1. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
2. EC2 RAID0 on Linux with EBS volumes notes   I’m almost done with my publish / post of...
3. Getting started with Chef. I write a lot of scripts, configure a lot of...

Related posts brought to you by Yet Another Related Posts Plugin.

I write a lot of scripts, configure a lot of systems, and do a lot of snapshot related backup/restore/sync stuff on servers in house, and in the cloud.
Chef is a Ruby-based configuration management engine. There is a good intro here.

I set up a Chef Server. There’s a good overview here.

I followed the instructions to setup a Chef server from this Wiki on Opscode.

I also referred to the information from Grig Gheorghiu’s blog here.

Now that my server is up, I’m hoping to port my server configurations to Chef, with heavy reliance on EC2 and hopefully UEC.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.