Sunday, August 10, 2014

Configuring Tomcat for memcached session persistence on Ubuntu

Get the files


wget http://repo1.maven.org/maven2/de/javakaffee/msm/memcached-session-manager/1.8.2/memcached-session-manager-1.8.2.jar
wget http://repo1.maven.org/maven2/de/javakaffee/msm/memcached-session-manager-tc7/1.8.2/memcached-session-manager-tc7-1.8.2.jar
wget http://repo1.maven.org/maven2/net/spy/spymemcached/2.11.1/spymemcached-2.11.1.jar

Copy them to Tomcat lib folder

sudo cp memcached-session-manager-* /usr/share/tomcat7/lib
sudo cp spymemcached-* /usr/share/tomcat7/lib

Configure Tomcat context.xml in /etc/tomcat7





   
    WEB-INF/web.xml

                 memcachedNodes="tomcat-sessions.ark24w.cfg.euw1.cache.amazonaws.com:11211"
             requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" />

   
   

   
   

Thursday, October 08, 2009

Adding Hudson and SVN revision number into Grails application properties

Wow, it's been absolute ages since I last posted an entry here. Oh well, better late than never. In looking for an excuse to post something, I thought I'd add the strategy I came up with to incorporate both the Hudson build number and our SVN revision number into our Grails application.properties. From that line you can tell, obviously, we use Hudson for our build automation.

This build number insertion approach enables us to easily identify from both a test deployment or a customer deployment which Hudson build was used, and which SVN revision number was used simply by looking at the app.version that I show in the footer.gsp of our app.

To do this is fairly simple. It requires 3 simple steps including 2 changes to your build.xml file that Grails generates for you.
  1. In my project files, my application.properties looks as follows:

    app.version=3.3

    ... or whatever major minor version your app is on. Ours is currently at 3.3.

  2. In my build.xml file I added a task:

    <target name="buildnumber">
    <property environment="env"/>
    <replaceregexp file="${local.dir}/application.properties"
    match="app.version=(.*)"
    replace="app.version=\1.${env.BUILD_NUMBER}.${env.SVN_REVISION}"
    byline="true"/>
    </target>

    This task is run as a depends of prod.war. ${local.dir} is actually mapped to '.' in my build file properties, so you can always just make it "./application.properties" in the task above.

  3. prod.war task is changed as follows:

    <target name="prod.war" depends="buildnumber" ...
So what happens is that when Hudson triggers the build, the 'buildnumber' task in 2 above appends both the Hudson build number and the SVN revision number to the app.version number before Grails builds the WAR file.

This way, we know which Hudson build is being used, and which SVN revision was used in that build. This helps us in both QM and out in the wild when a customer reports a problem. Also, with SVN we never have to tag a build because the revision number is the tag. If we need to make bug fix changes we can branch from a revision and we're good to go.

Tuesday, March 10, 2009

Localhost suddenly broken and I'm blaming Windows Update

Wow, it's been absolute ages since I last posted. SO much has happened since my last post. CentraStage is going very well. We're growing from strength to strength but easy does it nicely.

Anyway, today I just wanted to quickly note a frustration I experienced today. After months of never having any trouble using 'localhost' as a reference to my Tomcat instance, or my local database, 'localhost' suddenly broke. Kaput! Dood! Every time I tried to use localhost the connection was refused. I Googled this and found a variety of potential reasons, none of them all that encouraging since they had very little to do with anything I used on my machine.

A number of posts referred to entries in the hosts file, but these referred to the /etc/hosts file in Unix/Linux-land. Nevertheless, there is a Windows hosts file buried in %windir%\system32\drivers\etc\hosts. So I looked at that file and there was a single entry:

::1 localhost

Many Google posts had another line as a translation of localhost to 127.0.0.1. What was peculiar is that if I used 127.0.0.1 in place of localhost in my apps, it would work fine. So something seemed wrong with my translation. Yet I had done nothing that should have affected this, and the hosts file itself was last changed in September 2006.

I conferred with a colleague who indicated that he had the two entries as I'd seen in some of the Google posts, but both of his entries were commented out. After trying numerous things, including attempting to roll back to a system restore point prior to my most recent Windows Update (which as far as I'm concerned is when things started misbehaving) - an exercise that was futile since all my restore points are benign (Windows says there was a problem rolling back and that no changes were made) - I decided to have a go at changing the hosts file.

Look no further than: http://support.microsoft.com/default.aspx?scid=kb;en-us;923947 for how to do this. So I commented out the one line that referred to localhost, restarted my app, and baddabing, baddaboom - it all just worked.

So I have no idea what caused this problem, but needless to say it disabled me for the day. I still strongly suspect the Windows Update that ran this morning, but I'll never really know. But it's fixed now thank goodness for that!

Wednesday, March 05, 2008

The Gist of Spring Framework

I've avoided the Spring Framework for some time now. Mostly because I found the XML configuration a little daunting. I could see how nicely the little Hello World examples they gave could work, but the thought of using that on any real-world application seemed impractical. I don't know about you, but I *hate*, wait, let me rephrase that - *HATE* - loads of XML configuration. A few years back it became the trend-du-jour to slap all your configuration into XML files, away from properties files. There was some benefit, since XML files are multi-dimensional in their logical structure.

Anyway, years later, and I hate XML configuration. And Spring seemed to me to be the abomination of all XML configuration tabernacles. I've always kept my eye on it, since everybody and his grandmother seemed to be using it and raving about it. So right now I'm working on a web application that incorporates a few services, including web services, and I figured that since I'd noticed Springs configuration has improved a lot with the use of annotations and autowiring, I'd give Spring a whirl.

I'll probably comment more about my experience with Spring as time goes by, but for now I wanted to make one comment to those fellow Spring newbies out there who might be puzzling with a thing or two. What I call, the 'gist' of Spring. And this is it:

The application context replaces the use of the 'new' keyword.

So, what does that mean? Obviously you still use the 'new' keyword, but the one thing that I couldn't quite get, was when they gave examples of classes that were so wonderfully injected and aspected by Spring, they never showed exactly how to instantiate instances of that class, the one being worked over by Spring. And objects (or 'beans', the word I wish had never entered Java vocabulary) created out of Spring all seemed to be geared toward a Singleton approach. In other words you created an EmailService that all objects used, that needed it of course.

Well this couldn't be further from the truth. Truth be told, you can create any form of object from your Spring application context. And that is the key- the 'gist'. When you need an object, that you want to be setup for you by Spring, get a handle to the ApplicationContext, and use that to create your bean, and it will do all the rest for you. Any class, any object, so long as you've set it up correctly in your application context XML file(s), you're good to go. It's pretty sweet actually.

The next challenge I'm finding is how to get hold of the ApplicationContext. In my case it was fairly simple, in that I can get it from the servlet context for my web application. But you need to then keep a reference to it so you can actually use it in your code. I'm still figuring out the best way to do this. Obviously all the samples use a very simple local application context in the form of some or other test, which also doesn't really help me, but I suppose my frustrations are mostly due to my ignorance. Again though Spring developers, it wouldn't hurt to also show the call in your examples that shows how the actual bean we're after gets created. In other words "ctx.getBean("myBean");". Look at your examples and you'll see. You show the class annotated up to the wazoo, but you don't show how that class is instantiated. Probably because it's trivial to you, but not to us newbies.

Tuesday, October 16, 2007

Money does not float!

Money Doesn't Grow On Trees
Somebody told me the other day, since money is made from paper, and paper comes from trees, actually, money *does* grow on trees! Anyway, I digress.

There's also a software adage that says "money does not float". What this means is that for currency calculations you should never use a 'float', or floating point arithmetic. I bet we've all done it. We automatically think that a double or float will be fine for working with amounts, like money. If the amounts are all whole numbers then a long or int would suffice, but for working with amounts, floating point arithmetic is probably the worst option you can take. But I bet there's probably some of your code, out there, in production, happily purring along - using floats. Well be afraid, be very afraid.

An Example of the Problem
For starters, consider the following. What would the output of the following line of code be do you think?
System.out.println(1.20f - 1.00f == 0.20f);
Go on, take a guess. 'true', or 'false'? That's right, of course it's a trick question, and of course it outputs a big, fat, ugly:
false
That's the problem with floating point arithmetic. It's not precise. If you've got financial code using floats, and you're making decisions in your code based on the type of logic above, you're screwed. :-)

Therein Lies The Rub

So if you're anything like me, and you're not a mathematician, you're thinking "but why?" Glad you asked. It turns out that floating point arithmetic is quirky in computers. It stems from the fact that computers store numbers in binary format. Makes sense I guess. Now on the face of it, this doesn't seem problematic, but take for instance the number "1/3". How do we write that in decimal? To be totally accurate, we would have to write 0.3333333333333 and so on and so forth into eternity. But since we have limited patience and limited paper, we 'approximate' the number. We write as many 3's as we think are important to write. Anything beyond that, who cares, we'll suck it up. But, fact is, it's therefore not a precise number, it's simply an approximation, but in our estimation, an acceptable approximation. If I want 1/3 + 1/3, I'm happy with 0.66, even though the correct answer is 0.6666666 into eternity. In computers, with converting decimal numbers to their binary representation, there are similar problems. For example, there's no exact binary representation for the decimal '0.1'. So what you get is a repeating 0110 0110 0110 0110 up to a point, depending on the bit size. Most computers use a standard called IEEE 754 for representing floating point numbers in binary. With this decimal to binary conversion there are often rounding errors that creep in and depending on the circumstances, this can come back to bite you. I won't go much further into this. Joel On Software gives a great explanation of it with his article on the Excel bug which was influenced by this decimal to binary oddity. Read it here. The point I wanted to bring home, is that using floating point numbers for currency calculations can cause rounding errors depending on what the calculation is and what that translates to in binary.

Let's look at an example:

So why use the darn thing then? Why floating point arithmetic?
Well for one things it's fast. Using floating point arithmetic on computers is apparently a big deal. When you hear computer speed benchmarks referred to as flops, or megaflops, or teraflops, guess what? It's a measure of how many floating point operations can be done per second. Somehow that's a big deal to us geeks. :-)

Big Brother, I mean, Decimal To The Rescue

You've read this far, and you're about to throw your hands up in despair. "What the heck do I do then?" you retort. Indeed. What do we use? Well it seems the best thing to do when working with something like currency, is to work in the lowest unit of that currency and work the conversion magic in the presentation, be that in a user interface or in a paper report. So you do all your calculations in cents, but display the result in $'s. And apparently, a class in Java (and I believe in some other languages) is BigDecimal. BigDecimal doesn't use floating point arithmetic, therefore it is accurate. You may be wondering why we don't just always use this. Well it's a lot slower than floating point arithmetic. So it's a trade-off.

Tuesday, September 11, 2007

Is your VM lazy or eager?

My dad would turn in his grave if I said that sometimes it is actually better to be lazy.

Introduction
We had a strange bug appear on our Suse and Fedora 4 builds today. Our main client product threw a NoClassDefFoundError on a part of the code where it shouldn't have needed to throw that error about the class it was complaining it couldn't find. Basically I used jacob.jar, a very slick little Java-COM bridging library, to do some Windows service interrogation for a health check feature we implemented on our client product. Actually, in this case our client talks to a service but in the context of our entire product suite, the service and the GUI are collectively "the client". Anyway, moving on.

This feature is obviously only necessary on Windows builds, so we don't package the jacob.jar in our non-Windows builds, which was fine because we don't call the potentially offending methods on the non-Windows builds either. Even though there were import statements for these classes, we felt sure that it wouldn't be a problem because those classes are never needed, so naturally they would never be loaded. Or so I ass-u-me'd!

What Tha?!
When our QA guys reported the bug, I felt sure there was another place in our code that tried to load those classes. But after searching the code, I found nothing. Then I assumed that maybe it was the import statements at the top of the class that were causing the problem. So I removed those and fully qualified the class names in the method that was never called. Kaboom! Still complained of NoClassDefFoundError.

Test The Assumption
Okay, so in times like this, it's best to isolate the problem and see whether you can reproduce it with a simple case. So I created 2 classes, MainClass and SomeOtherClass as follows.

Here's MainClass:
package com.test.classloading;

import com.acme.clientlib.SomeOtherClass;

public class MainClass {

static {
System.out.println("Loading MainClass");
}

public void doYourThing() {
System.out.println("Enter doYourThing()");

SomeOtherClass soc = new SomeOtherClass()
soc.doStuff();

System.out.println("Exit doYourThing()");
}

private void doYourOtherThing() {
System.out.println("Enter doYourOtherThing()");

System.out.println("Doing other thing");

System.out.println("Exit doYourOtherThing()");
}

public static void main(String[] args) {
MainClass mc = new MainClass();
mc.doYourThing();
mc.doYourOtherThing();
}
}

And here's SomeOtherClass:
package com.acme.clientlib;

public class SomeOtherClass {

static {
System.out.println("Loading SomeOtherClass");
}

public void doStuff() {
System.out.println("doStuff called");
}
}

As you can see, I have static initializer blocks that indicate when the classes are loaded and initialized.

When I run this test I get the output I expect:
Loading MainClass
Enter doYourThing()
Loading SomeOtherClass
doStuff called
Exit doYourThing()
Enter doYourOtherThing()
Doing other thing
Exit doYourOtherThing()

Ok, so next I removed the call to SomeOtherClass's doStuff() method as follows:
        MainClass mc = new MainClass();
// mc.doYourThing();
mc.doYourOtherThing();

When I run that on Windows I get:
Loading MainClass
Enter doYourOtherThing()
Doing other thing
Exit doYourOtherThing()

So it would appear that SomeOtherClass never gets loaded. Cool! We're all set!

Not so fast
Unfortunately when I ran that same code on the Fedora box (in my case) with Sun JDK 1.5, I got the dreaded NoClassDefFoundError again! What gives?! How can it work on Windows, but not on Linux?! At first we thought it might be compiler inlining optimizations from some static final's we use in our code, but that didn't make sense either, as it would then only be inlined on Windows builds, not Linux builds.

Well as I'm sure you know, when a class is loaded, it must be initialized but it must first be linked before it can be initialized. To link a class it must be prepared, verified, and (optionally) resolved. It seems that in this case, the Windows VM is doing a lazy job of that last step, resolving (i.e. only doing it when it absolutely has to) whereas the Linux VM is being a bit overeager. Okay, so an easy fix then would be to simply move all of that code into a separate class. Then when the general class is loaded, the Windows specific stuff will be elsewhere. No problem. So I changed my code and moved all the jacob.jar dependent code into a separate class. I tested it on Fedora. Worked like a charm. I checked that into CVS, and sat back in the comfort of knowing I'd fixed the bug once and for all... or so I ass-u-me'd.

An annoyingly knowledgeable mate of mine ("YOU KNOW WHO YOU ARE!") pointed out a small catch as per TheOracle(tm), TheSourceOfAllKnowledge(tm)... also known as TheSpec(tm). In particular, he pointed me to the VM spec, section 2.17.1:

"The resolution step is optional at the time of initial linkage. An implementation may resolve a symbolic reference from a class or interface that is being linked very early, even to the point of resolving all symbolic references from the classes and interfaces that are further referenced, recursively."

Yikes! So my fix was really just a band aid! There's every possibility that even though I'd moved the code into another class, some VM implementation might actually try and resolve those classes as well. Ok, well to cut a long story short (er, too late ;-) the best solution for this problem was to simply call the other class I'd created via reflection.

Conclusion
So remember that then. Never assume. Did I say that clearly enough? No? Let me say it again then... NEVER ... ASSUME! When in doubt, check the spec!

There's a reason why there's an acronym called "RTFM".

Wednesday, August 29, 2007

Hoorah for Hudson

Introduction

If you haven't heard yet, there's a neat little build tool (server) called Hudson that is available for free to a good home. I read about Hudson on Glen Smith's blog, and when competent people rave about something I usually sit up and take notice. I believe Hudson was started by Kohsuke Kawaguchi though there are others now involved in development with him. You can read about Hudson here, and download the latest version here.

At my company, we'd gotten by doing manual builds at the appropriate time for whatever the purpose. Sounds horrid, but the fact of the matter was that everything was checked into CVS, so if you needed a build, it was there to be checked out and built. However, as our product suite grew and our developer numbers grew, it was never going to be an ideal situation. Thing is, what to do about it? Cue Hudson.

I installed Hudson and played around with it and found it to be a very competent build server, so I talked my boss into letting me stick it on a dedicated machine, so that we could use it as an interim tool. The idea at that time was to get something more robust, like TeamCity, from Jetbrains. Well that was a few months ago, and we currently have 29 jobs setup on Hudson.

What's so great about Hudson?

Glad you asked. Well a number of things actually. I could go on and on so I'm just going to mention some of the main things:
  1. It's 1 WAR file.
    In case you missed what I just said, I said it's ... one... WAR file. So you drop that into your (in our case) Tomcat webapps root, and you're done. I like that Hudson defaults to creating a .hudson folder in your user home folder. This makes it easy-peasy to upgrade Hudson: simply download a new WAR, nuke the old one and its folder and restart Tomcat. Done. Hudson even comes with a 'Shutdown' feature that will gracefully end any currently executing builds, and prevent new builds from starting until after the restart.

  2. AJAXification.
    Other than the pointless fact that it has buzzword gravity, the AJAX behaviour of Hudson is quite useful. If you're on the home page, builds that are scheduled or busy executing will automatically show up in the sidebar, without the whole page reloading every few minutes. There's AJAX behaviour when you fill in textfields etc. Hudson will prompt you with values for that textfields from existing data in Hudson. That kind of thing. Very useful.

  3. Smart touches.
    You can see that the people who are developing Hudson, also use Hudson, and have an understanding for the needs of a build server. An example is that Hudson exports an environment variable for the build number, while executing a job. So you can tap into that value and use it in marking your deliverables. That way you can track bugs back to a particular build, and do better forensic detective work. We use that build number when closing a bug in Bugzilla.

    Hudson also does its CVS checkout using a datetime limiter i.e. check out this module as at this time. At first this was a bit frustrating. If I checked something into CVS and triggered a Hudson build manually, and the change wasn't in the resulting build because the build servers time was off by a few minutes, so I wondered why on earth Hudson didn't simply take the latest HEAD snapshot. Well, as it turns out, there's a good reason. Thing is, most build servers operate in a team environment and it is quite possible that while Hudson is busy checking stuff out of CVS, somebody else is busy checking stuff in. So using a specific time ensures that it checks out everything as it was at that moment, even if it takes Hudson 5 minutes to check it all out. Of course somebody could also be doing a long check-in that crosses that time moment, but that's a lot less likely since CVS commits are usually quick use cases. Ok, truth be told, Hudsons main purpose is, I think, to run scheduled build jobs, at a time when it is unlikely that developers will be actively committing to CVS.

  4. Master slave functionality.
    Our dedicated build server is actually a bit on the puny side. So I thought to myself "Self, wouldn't it be useful if we could use some of our steroid-taking servers to do some of the build work for us?" And lo and behold, Hudson has the ability to create slave nodes, so your master Hudson build server can farm work off to your slave nodes. We haven't actually set this up yet, mainly because of time constraints, but pretty soon I see that as being a very useful feature for us.

  5. Email notification.
    Hudson lets you know when a build fails. It did bother me that it doesn't let you know when a build succeeds, but then it occurred to me that the idea behind Hudson is to schedule builds, or have them trigger from CVS commits, so really, you only want to know when something goes wrong. Also, if a build has failed, then succeeds, it emails you once to let you know that all is back to normal.

  6. RSS Feeds.
    You can pull RSS feeds for all the projects, so to check the latest build and result of a build you simply use your RSS browser. Click on that link and you go straight there. Neat!
So far the only real downside that has impacted us is that you have to checkout modules on a branch, not a tag. For some reason it can't handle tags. After some research I found that this is actually a bug, and that prior to build 1.65 it could handle tags. It's already logged as a bug here though there's no fix estimate at the time of writing this post.

Conclusion
There are many other useful touches to Hudson that make it quite an impressive tool for something that is free, as in beer. We have all of our products now configured to build on Hudson and we have dependencies defined between projects, so that if we make a change to a core component, dependent components build immediately after the core component builds, and that way we detect API breakages very quickly. It should be very easy to hookup Ant test scripts to also run, since they'll be part of the overall build, so hopefully in the near future we have that configured too. I certainly know that our QA guys are much happier having a website they can go to for the latest stuff, and one that has us all on the same page when cross-referencing bugs with bug fixes, and their purported fix build number.

Give it a try and drop me a comment with your comments, good or bad.