When an Elastic Load Balancer handles a connection it sends it’s own (internal/private/10.x) address instead of the clients. It sends the clients along with the request as X-Forwarded-For. To log this you need to log X-Forwarded-For instead of the source IP.

Here are 2 links discussing the problem, the first covers a basic Apache & Tomcat setup, but the second one has a point about direct access getting not logged and has an Apache httpd specific solution.

http://blog.kenweiner.com/2009/09/amazon-elb-capturing-client-ip-address.html

http://blog.grahampoulter.com/2011/10/how-to-log-client-ip-from-apache-behind.html

Thanks @grahampoulter and @kweiner

Related posts:

1. Reading : Peecho Minimizing Downtime on Amazon AWS   What the guys over at Peecho have done is...

Related posts brought to you by Yet Another Related Posts Plugin.

You want general query logging in, but don’t want to keep those pesky query log files around?

I generally setup the MySQL user and run this from cron.

Here’s my script to do so, here’s a gzip’d version, and here is a zipped version

#!/bin/bash
#setup a user in mysql with the RELOAD priviledge
#grant RELOAD on *.* to reloader@'localhost' identified by 'PASSWORD';
#flush privileges;

user="reloader"
password="PASSWORD"
LOG="${HOME}/mysqld/mysqld.log"
LOGARCHIVE="${HOME}/logarchive"
NEW_LOG="${LOGARCHIVE}/mysqld-`date +%F-%s`.log"
KEEP_MIN=15
MAX_LOAD="5.0"
RUNFILE="${HOME}/rotate-general-log.pid"

if [ -f ${RUNFILE} ]
 then
  echo "Runfile ${RUNFILE} exists, exiting at `date`"
  ${HOME}/bin/page.sh "Runfile ${RUNFILE} exists, exiting at `date`"
  exit 1
 else
  echo "$$" > ${RUNFILE}
fi

#if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
if [ $(echo "`cut -f1 -d ' ' /proc/loadavg` < ${MAX_LOAD}" | bc) -eq 1 ];
 then
  echo "System load less than ${MAX_LOAD}, proceeding [DEBUG]"
else
  echo "log rotate delayed, due to system load > ${MAX_LOAD}"
  ${HOME}/bin/page.sh "log rotate delayed, due to system load > ${MAX_LOAD}"
  /bin/rm ${RUNFILE}
  exit 1
fi

MYSQLD_PID=`pgrep mysqld`

if [ ! "${MYSQLD_PID}" ]
 then
  echo "Mysqld is NOT running, paging and exiting at `date`"
  ${HOME}/bin/page.sh "NO MySQLD on `hostname` at `date`"
  /bin/rm ${RUNFILE}
  exit 1
else
  echo "Mysqld running at `date`, PID ${MYSQLD_PID}, continuing"
fi

if [ -d "${LOGARCHIVE}" ]
 then
  echo "Moving general log at `date`"
  /bin/mv ${LOG} ${NEW_LOG}
  touch ${LOG}
  chmod og-rwx ${LOG}
  echo "Flushing general log at `date`"
  mysqladmin -u ${user} -p --password="${password}" flush-logs
  echo "Done flushing general log at `date`"
  echo "Gzipping ${NEW_LOG} log at `date`"
  gzip ${NEW_LOG}
  echo "Done gzipping ${NEW_LOG} log at `date`"
  echo "Size is `du -sh ${LOGARCHIVE}` pre-clean"
  echo "Cleaning ${LOGARCHIVE}, deleting files older than ${KEEP_MIN} minutes at `date`"
  find ${LOGARCHIVE} -iname "*.gz" -mmin +${KEEP_MIN} -print -exec /bin/rm {} \;
  echo "Done cleaning ${LOGARCHIVE}, at `date`"
  echo "Size is `du -sh ${LOGARCHIVE}` post-clean"
else
 echo "Missing logarchive dir ${LOGARCHIVE}"
  ${HOME}/bin/page.sh "Missing logarchive dir ${LOGARCHIVE}"
 /bin/rm ${RUNFILE}
 exit 1
fi

/bin/rm ${RUNFILE}
echo "Done at `date`"

Related posts:

1. I am always cautious with this on my clipboard.. su mysql cd rm -Rf data/* innodb/* tmp/* logs/* mysqld/*...
2. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
3. My EC2 EBS RAID-0 creation script I want to write up a full howto/tutorial, but have...

Related posts brought to you by Yet Another Related Posts Plugin.

My Comments on the post

“Cheap Hosting Case Study: You Get What You Pay For (And Sometimes Not At All)”

These are all interesting points that anyone using AWS, and especially Spot Instances should know. The service provider (Amazon AWS) should not be blamed for the user mis-understanding how their service works.

There are 3 things about Spot instances that everyone should know, that Amazon is less than clear about on their product pages, and I found out at the AWS Summit in SF.
1. You have your instance only for as long as your price is higher than the current spot price, and this is measured much more frequently than the bid interval of 1 hour. So you could launch an instance for up to $.15/hr it could run for 30 minutes, and then be terminated unexpectedly when the spot price is over $.15/hr
2. People can, and do, bid HIGHER than the on-demand price for spot instances to get priority machine allocation. This means they get priority OVER all other spot prices, AND over “normal” on demand users.
3. Spot instances are terminated without a safety net, all data/jobs/processes on them need to be designed to be re-entrant, and fail-able. The mentality for typical spot instance usage is, there is a need, a spot instance is launched, it checks out a job from a queue (like SQS, but frequently a custom setup), it runs, *IF* it finishes, it returns success, *IF* it does not finish, another instance picks up the job.
The better your application and plan meet the functionality of spot instances the more you can work the system and save a lot.
Design and plan for failure, and you will love the cloud.
Design and plan like you have a physical server, and you will complain that the cloud let you down.

Related posts:

1. Thinking about hosting a WordPress site on S3 I recently moved my Joomla backed consulting website completely to...
2. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
3. Reading : Peecho Minimizing Downtime on Amazon AWS   What the guys over at Peecho have done is...

Related posts brought to you by Yet Another Related Posts Plugin.

I’ve always thought of myself as an entrepreneur. My first business was setup at a family reunion when I was a boy, and was a candy distribution network through my cousins. It ended in all of them getting some free candy, and me being out all of my $20 birthday money. I enjoy thinking up business ideas, and planning them out, and sharing them with friends. I have not launched many businesses, and my only “real” business venture was to start my own Internet Service Provider (ISP) in 1996. I learned a lot, and ended up selling it to a larger ISP.
Today the barrier to starting a business is lower than ever. There is lots of names for the advantages that modern technology bring new startups, and ‘The Lean Startup’ is probably the most commonly referred to.

I was browsing makeuseof.com and found these articles that have interesting links for those starting up new businesses.

3 Free Tools to Plan and Visualise Your Start-Up Business
10 Awesome & Inspiring Blogs for Entrepreneurs and Business Owners
10 Best Websites That Show You How to Start a Business

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Macintosh Command of the Day Sept 28, 2011

I have no idea why I did not do this sooner!

On my Macs I always had that md5sum the command doesn’t exist, and I hate that the output of the md5 command that does exist is not the same as the GNU tools. Sure it’s BSD (And I’ll always love my BSD), but I think compatibility to the GNU tools buys more for BSD and OS X than incompatibility.

Add this to your ~/.bashrc (and source it or restart your terminal)

alias md5sum='md5 -r '

Now your md5sum <filename> works the same as on the 1,000,000,000 Linux boxes out there.

Also in my .bashrc on my Macbook Air is

alias ll='ls -Falh'
alias lg='ls -Falh|grep -i '
declare -x EDITOR="/usr/bin/vim"
declare -x JAVA_HOME="/usr/"

-Jon

Related posts:

1. MCOD – 8.31.2011 – Lock your screen in OS X from the menubar     1. Launch the “Keychain Access” application located in...
2. LCOD 5.29.2011 – 2 quick TCP port check scripts in Python Disclaimer : These scripts could be considered malicious and should...

Related posts brought to you by Yet Another Related Posts Plugin.

What the guys over at Peecho have done is cool. Similar to the path I’ve taken with auto-scaling and machine updating from S3.

I am going to add some of their ideas, and will incorporate that in my upcoming posts with scripts to do it.

Check it out on their blog – http://www.peecho.com/blog/minimizing-downtime-on-amazon-aws.html

Related posts:

1. Moved my consulting website to Amazon S3 It’s a Joomla site, but I rarely have updated it,...
2. S3-du.sh script to get bucket size on Amazon AWS S3 Here is my script s3-du.sh that I wrote to determine...
3. Thinking about hosting a WordPress site on S3 I recently moved my Joomla backed consulting website completely to...

Related posts brought to you by Yet Another Related Posts Plugin.

The short version is that you are at my site because someone hijacked your new or private tab in Internet Explorer.

Please do not blame me, my web servers have been slammed with lots of traffic from this and it will cost me money, and I did not cause it.

Here is how to fix it. Below is a detailed description of what I think happened.

You will need to use the windows registry editor to fix it, or you can download this registry file, and double click on it.

Open Regedit and go to : HKLM\Software\Microsoft\Internet Explorer\AboutURLs
On the right pane double click on the tabs value and change it to : res://ieframe.dll/tabswelcome.htm.

I use Amazon Web Services to host this blog. Part of those services is a load balancer called an Enterprise Load Balancer (ELB).

I point my site at an amazon name (www-jonzobrist-com-954435911.us-east-1.elb.amazonaws.com) and Amazon handles the IP addresses and networking. The upside is I get great scalability for very low cost. Sometimes people set their load balancer incorrectly, and point a hostname (in this case gg.blogpear.com) directly at one of the IP addresses in their load balancer pool. This is wrong because Amazon can change at any time which IP address gets assigned to which load balancer, and they do not guarantee you will ever get that IP back. Someone, who is probably a malicious hacker type, hijacked your browser tab for either new tab or private tab in your Internet Explorer browser. They pointed it at a DNS name gg.blogpear.com, and that DNS name at an IP address on Amazon’s ELB. Somehow Amazon gave me that IP for my www.jonzobrist.com pool, so I got all the traffic. This killed my web servers quickly, and took me most of the day to recover from. I did so initially by setting up rules to return a quick 403 – permission denied error to all the requests. Then as I investigated it further, I figured out (I think) what happened. So now, you get redirected to this page, and hopefully you will get your computer cleaned up and we can all move on without too much trouble.

I recommend you also get a virus scanner, something like Avast, which is free for the non pro version. Download it from Avast.

I would also recommend you download and use Google Chrome and Mozilla Firefox, as they are both more secure (and generally better) web browsers.

I hope this helps!

-Jon Zobrist <jon@jonzobrist.com> http://www.jonzobrist.com/

Related posts:

1. Thinking about hosting a WordPress site on S3 I recently moved my Joomla backed consulting website completely to...
2. Amazon keeps doing it again and again! AWS route53 Amazon almost makes me laugh whenever I sign up my...
3. Have my auto update script mostly ready, this is a test! Hopefully the new server will start automatically behind the ELB,...

Related posts brought to you by Yet Another Related Posts Plugin.

There are a ton of cool things to do on the Internet. New doors are open to everyone. I’m surprised how often we take it for granted that everyone is in on the latest trend in tech. Here is my humble addition to a list of things that I think people should be using online.

1. Twitter – News *stream*, or should I say FLOOD. Follow smart people, get smart (filtered) news and info. Want to blow your news mind? Get tweetdeck and put in a search for any hot topic. (Don’t follow #earthquake unless you want to feel constant fear).

2. Facebook – Connect with your family and friends. Be benign on Facebook! The Internet is public, immortal, and Facebook does hate your privacy.

3. Amazon AWS/EC2 - What you don’t need a virtual server? You sure about that? Not for your blog? Not even if it scales infinitely? Not even if it’s free?

4. WordPress – Joomla and Drupal are cool, but WordPress is the king of the web page CMS.

5. Gmail – Seriously, stop deleting your email, get a gmail account. Use your own domains (Google Apps is still free for < 10 users).

6. Google Docs – If you haven’t had 10 people all editing the same spreadsheet at the same time you have not Cloud’d it up.

7. Cloud Music (Google Music, Amazon Music Locker, iCloud, Soundcloud, Spotify) – This is new, try them all out, find new music, sync your own.

8. Google – Search done right. Everyone has been playing catchup for a while now, and I’m sure that one day they will, but until then, google.com

9. Snopes – The Internet means rapid access to information sharing, but many people share false information. Sites like snopes.com

10. Shopping - Deal sites like slickdeals.net, fatwallet.com, woot and more track deals as they happen, often with good comments on how to maximize them. The people on some of these sites are mad geniuses when it comes to getting the most for your buck.

11. Skype – Everyone has it, get on and video chat your friends in other countries for free. Ride this one until Microsoft torpedos it, and we all move to Google Chat, which you should be on already via your gmail account.

12. Linux – If you are even slightly technically inclined, Linux opens the door to you (for free) to everything from high end movie effects to  computer forensics. Get started with a Live CD from Ubuntu (Your computer is probably 64-bit, and you probably want the desktop version – You can boot the CD and use Linux without doing anything to your computer), and NO it does not run Office or any Windows program, but it does run thousands of cool programs.

13. Photo sites (Picasa, Flickr, Smugmug) There is no reason you should be burning a photo CD to send to your friends and family. Get an upload utility, and start putting your photos on the ‘net. You don’t have to share them, and I would highly recommend NOT sharing them publicly unless they are very public information. I do not post pictures with faces in them without permission from the person owning the face, and, in general, don’t do this.

14. Education (Khanadademy, Alison.com, MIT Open Courseware, Instructables, k12) – There are too many to name, and pretty much access to infinite information is it’s own education. Don’t think that just because a skill isn’t directly computer related that you can’t learn howto do it online, and for maybe for free.

15. Wikipedia - What is a wikipedia? Well, a wiki is a website that anyone can edit the pages of, so, Wikipedia is an encyclopedia that anyone can edit. Not always right, but rarely uninformative.

Well, I hope this helps. Please send me your lists or additions (comment below, or email to jon@jonzobrist.com).

Related posts:

1. Finally On EC2 I can’t believe it took me this long to nut-up...

Related posts brought to you by Yet Another Related Posts Plugin.

1. Launch the “Keychain Access” application located in /Applications/Utilities.
2. Choose “Show Status in Menu Bar” from the View menu.
3. Now you have a lock in the menu bar. Use this menu to “Lock Screen” when you leave your computer

Thanks to Mac Daily News for this howto, from here -

http://macdailynews.com/2003/03/13/lock_your_mac_os_x_screen_from_the_menu_bar_by_adding_lock_icon/

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

http://www.groubal.com/barnes-noble-screwed-me-out-of-my-touchpad/

I could have got one 15 other places, but why order it twice?

Now it may be too late. Oh well, I complained online, and don’t really care if HP Touchpad dies more silently to me than it would have if I had got some at a discount.

I think the key lesson is $99/$149 is WAY under what most people (2M) are willing to pay for a nice tablet. VS. $500/800 that they are being sold for by major players.

Economics of scale seem very game-able at this point. That is, it seems a safe bet for someone to order a few million tablets of whatever nice OS that they can sell for $99/149 or $199/249 and work those economies of scale.

Related posts:

1. HP WebOS TouchPad discontinued – FIRE SALE Short answer : Want a good tablet for CHEAP? Get...

Related posts brought to you by Yet Another Related Posts Plugin.