Starting in Maven?

It surprises me again and again when I meet developers who are using Maven and they don't have a local repository. For anyone using Maven, once the decision has been made to actually use it, the next thing you do is get a local Maven repository. Really, its at the top of my list for things to do.

If you are working on projects at home, open source or not, at work with a single developer, whatever, you should install a Repository Manager. If you have more than one developer using Maven, you are simply insane not to use a one. So why would you install a repository manager even if you are only a single developer? Surely its hard?

Well, no, it is incredibly simple, Sonatype's Nexus takes 10 minutes to set up for example. Here are a few if you don't already have one:

• Sonatype's Nexus - this comes in a Pro and Free version. I use Free in three different places and it has served me very well. 
• Apache Archiva - I used this in my last job, and I didn't like it as much as I like Nexus. It seemed a little more complex to set up. 
• JFrog's Artifactory - I haven't used this, but they do a lot with the Gradle community and are going to do some work around Grails plugins. 

So back to why? A few reasons

• Unfortunately, not everything is in Maven Central. This means you have to edit your settings.xml to add each repo in as you need it. Don't, put them in your Repository Manager and just point to it - let it sort them out. Add a new repo? Just add it in the Repository Manager. 
• If you are doing Maven releases properly, you are using version ranges. Version ranges cause Maven to scan all repos in your settings.xml for updates on a regular basis - and you don't want this traffic going far if you can help it, it slows you right down.
• If you don't have a Repository Manager, you are either (a) not releasing you artifact (a cardinal sin) or (b) only releasing to central (which is feasible for some projects, e.g. easyb). 
• It allows you to blow away your local repository when it gets untidy from snapshot builds which people use periodically. Note if you do everything using snapshot builds, you are using Maven in a  broken fashion.
• You can put a Repository Manager on your own machine as a proxy as well, if you want to keep as much network traffic local for example, but you have the main Repository Manager hosted elsewhere - on a slow VPN or in another country.

Releasing

I've known people working on projects who have never released - they just work with snapshots! Thats nuts! A release in maven is an individual artifact (your entire war would be one, as well as an individual api) - it is something that has been cleaned, started from scratch, compiled, verified, all tests run, tagged in the remote repository, checked out again to a different location and all  that done again and then uploaded to your Repository Manager. You should be releasing very regularly. And be fine grained about your artifacts! As soon as you have done a significant lot of work, release it. 

Version Ranges

Version ranges are important for two main areas of Maven usage. 

• You have chosen an open source project based on its functionality, license, code quality, whatever. But you choose a specific version. For example, in a recent project we chose SOLR - 3.3.0. Our use of the 3.3.0 means we use <version>[3.3.0]</version> as our version. This means I want 3.3.0 and nothing else. Not 3.3.1, not 3.3.5, not 3.4.0. Why is that important? SOLR when it released 3.4.0 caused the meta data in (Apache) Maven Central to get corrupted - this means the only version available was 3.4.0. If I had instead specified <version>3.3.0</version> - if Maven didn't have it locally, it would go pick up 3.4.0 and happily use it. If the corruption had caused only 2.3.0 to be available, Maven would have used that - it would have been a mess! Always specify exactly what ranges you are  happy to include artifacts for - if stuff breaks, lodge JIRA tickets on Sonatype's JIRA, they are pretty quick about it and will be ensured actually using what you have chosen. 
• Generally projects decide on how they issue a new version number - some just use two (major.minor), where the major changes on a breaking api change, some use three levels. You should specify a range when you are happy so new releases turn up - and typically you will do this of your own artifacts. You start at version 1.1 (because thats how ranges work, don't start at 1.0 or 0.x, thats just nonsense) and keep releasing until you break. Release early and often. If someone then who is using your artifact says [1,2) or say [1.7,2) (if 1.7 is the start of your api having what they want) then they will just get new releases as they come out, ensuring they keep current. These kinds of version ranges make Maven use really nice and easy to use when it comes to dependencies (along with composites, another post). 

Composites are just pom's that contain a collection of dependencies - we have composite-cxf, composite-spring, composite-jackson, composite-logging-api, composite-logging-api, etc. Composites keep your pom clean, they take out all of the stuff that isn't really specifically to do with your App, but the kind of App you are building. They become accepted building blocks - and they rev. like normal artifacts. Changing from Spring 2.5 to Spring 3.1? rev the major version of your pom - existing artifacts won't be broken, you can fix them as necessary. Composites can be great for public facing releases as well - if Wicket 1.6 was just a pom containing fine grained artifacts that were hard version ranged, then we they released bug fixes (early and often), you could override specific artifacts and bring in the new releases to get the ones you want. 

And another cardinal sin - it happens, but rarely, do not put dependencies in parent poms. Except for the plugins which often need them (and try and use composites for them), but your parent poms should not have modules or dependencies defined. Put them in the composites. 

(It sometimes happens when you are using projects that are so terribly broken you can't help it, but its rare)

Do yourself a favour, if you are using Maven, install a Repository Manager. 

Initializing your Grails Services

In one of my current Grails projects I need to do what I would normally use @PostConstruct for, but am unable to as usually GORM has not completed its additional methods by the time it comes to use them. So the general recommendation is to do it in the bootstrap - a technique I generally do not like as it means you cannot make it a best practice easily - just include a dependency and its always done.

Ideally I think I'll need to turn this into a plugin, but I hate source code plugins enough to just include the Bootstrap at the moment.

So this goes in your Grails conf, and its a slight modification of the one you find on Google - it checks the scope of the bean, singleton ones only are initialized.

import org.codehaus.groovy.grails.commons.GrailsClass
import org.codehaus.groovy.grails.commons.GrailsClassUtils

class BootStrap {
  def grailsApplication

  def init = { servletContext ->
    grailsApplication.serviceClasses.each { GrailsClass gClass ->

      String scope = GrailsClassUtils.getStaticPropertyValue(gClass.clazz, "scope")

      if (!scope || scope == "singleton") {
        def serviceBean = grailsApplication.mainContext.getBean(gClass.propertyName)

        if (serviceBean.metaClass.respondsTo(serviceBean, 'grailsInit')) {
          serviceBean.grailsInit()
        }
      }
    }
  }

  def destroy = {
  }
}

Grails and Maven - blue grails

One of the things that has kept me away from Grails for a long time - even after spending a couple of years knee-deep in it has been the extremely poor support for a clean build and release.

The combination of Gant scripts and source-code plugin dependencies, the use of Ivy as a dependency manager, endless outstanding bugs and a tendency to just not understand why this was so important (I'm sure everyone had their list) led me to abandon using Grails or pushing Grails in any real way in my work life while working at Sky Television (NZ). I still used it in a side project, but not in any anger and long since left the developers discussion list.

As I prepared to take on a new job doing Grails development again, going back from an environment that focused on neat, tidy granular artifacts that were tested and released, wired cleanly with Spring and assembled with clarity and speed was something I didn't want to loose. I learnt a lot from the two guys I worked with in terms of the use of Spring, Maven and API design. I only hoped that Grails would be able to live up to it now it had moved on.

On jumping back into Grails I saw that essentially little had changed (even if it was about to with 2.0 and binary plugins). Application design still seemed to encourage big single projects where all controllers, domain objects, services and so forth were all in the one project, making it harder and harder to test, longer and longer to build and deploy. Knowledge of Spring by users was limited, consequences of Hibernate ignored, discipline around tagging and releasing artifacts almost non-existent. EDIT: I mean this in the context of the Grails community as a whole:

• Spring encourages the use of interfaces. Grails encourages you to inject the service's actual class by its name (e.g. userService for com.bob.UserService) but by using a def you can mock it. Its Groovy, but its nasty.
• Grail's finders ignore the consequences of the SQL generates and most developers appear to not look at the SQL Hibernate generates. Hibernate does a bad job with anything thats not basic, and createCriteria is a demon. Whether this is better or worse than any other usage of it I'm not sure.
• Grails doesn't come with a "release" script - crazy in my opinion. The concept of clean, build, tag, checkout from tag in a different place, build is critical to knowing you have a reproducible clean build. Not having it is crazy and is an artifact of people who don't use Maven, or one of the many who use Maven who don't use properly.

So for the past few months I have been working with the University of Auckland and TEIQ to build a set of best practices around modular Grails development, using Maven in the sanest way possible (and it is sane, and nice, and clean). Sorting out the way we deal with the inevitable buggy plugins (including the Grails Maven Plugin), building composites that bundle together packages of artifacts that relate, standardising web services structures and looking always for opportunities to make things better and cleaner, faster and clearer, better tested and more robust. Its a great opportunity and I thank TEIQ and particularly the University for giving me the chance to make a difference. And great people - did I mention great people?

As part of this work, we are taking some of what we have learnt and pushing it out into open source - or probably more open documentation. To make the best use of what we have done, we really need to have the Grails plugins that we use available as Maven artifacts - and I can't see the Grails guys happy about us doing that. I'm going to document it here on my blog - to make sure I do it (document it that is) - I'd recommend taking what I write if it works for your team and putting it in your Wiki and augmenting it over time.

The first thing I'm putting out there which may be of some worth to people who come to Grails and consider using Maven is my set of scripts I use. Its growing - but it provides me with a great set of shortcuts to the commands I regularly use. Enjoy :-)

War against disinformation about Google

Quyn Do posted an article entitled "War on the Google Giant: Engineers Defect, Then Rise Against".  The incredible absurdity of this post so astounded me that when @gianouts re-tweeted @darrylgman's tweet that this post was actually *interesting* I was flabbergasted.

To begin with, this person didn't seem to realize, with all evidence to the contrary, that using the free Gmail with targeted adverts would in fact target adverts based on their email.

And then they make the assertion that "our information gets sold to advertisers". Really? Where can I buy the service? It  targets your demographic, what you search for, what you are interested in and I buy those words - Google matches them up and serves ads based on them. Even the remotest amount of research would tell you this, so clearly not a clue here either.

And then Quyn talks about Disconnect - the tool for Google Chrome that blocks out cookies from various sites. Does Quyn not know that Disconnect was written to allow people to disconnect from Facebook peer sites? A little bit of searching (again) about Brian Kennish would have turned up Facebook Disconnect. It was so popular he left Google in a blaze of publicity and created Disconnect, a general purpose plugin for Chrome (when I last looked). Again, outstanding lack of insight or research on the topic.

And Adblock - adblock has existed for *eons*, it was a Firefox plugin for ever - this has *nothing whatsoever* to do with Google, what the hell is it even in the article for?

And finally about people leaving for Facebook - why, a completely different topic thrown in just so the heading vaguely matches? Seriously? People have been leaving for Facebook because they are willing to put something out and tweak it until it works, which is a different strategy than Google. They've had their pet projects cancelled and with their noses out of joint, have gone to Facebook. I'm sure there is more to it than that, people want a change of scenery, but what has it got to do with privacy at Google? Facebook if anything has a far, far worse track record than Google.

This post has to have been written by either (a) a person with an anti-Google agenda willing to bet people will be too stupid when reading it to realize what a appallingly badly written opinion piece it is or (b) someone who really is that stupid.

I'm betting on the first, there are a lot of anti-Googlers around at the moment.

For my own sanity

Oracle XE and EC2/EBS instances was not a fun time. Oracle being at fault here.

So I needed an XE instance and rather than downloading and installing a VM specifically for this particular test, I figured an easier way would just be to choose an Ubuntu VM AMI image and set it up there.

So some things to remember:

• Make sure you use a 32bit i386 AMI (or it won't install)
• When it starts up, you won't have enough swap to install XE, so add more
• Make sure bc is installed (sudo apt-get install bc)
• Follow the instructions for installing Oracle XE here and here

So the last bit which is really the difficult thing to find is that the listeners will kill you. When Oracle XE installs, it hard codes the machine name somewhere. So if you shut down and start up again, you get it failing the listener.

 

So you need to go into $ORACLE_HOME/network/admin and edit tnsnames.ora AND listener.ora and change the hostname from ip-whatever to localhost. But that isn't enough - Oracle itself will still screw up as it registers with the original name as installed each time - so you won't be able to connect to the listener, even though diagnostics imply it worked. So you need to look into the log (which is in $ORACLE_HOME/network/log) and you'll see something like this:

22-SEP-2010 20:05:55 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=ip-10-203-55-158)(USER=oracle))(COMMAND=status)(ARGUMENTS=64)(SERVICE=LISTENER)(VERSION=169869568)) * status * 0

That HOST=ip-10-203-55-158 - thats the domain name originally registered against your machine. Take that name and edit /etc/hosts (sudo vi /etc/hosts) and add it into localhost's config:

 

127.0.0.1 localhost ip-10-203-55-158

Then restart your Oracle XE

sudo /etc/init.d/oracle-xe restart

By the way, I'm using a script to tunnel to the box (so Oracle is never exposed to the internet)

 

#!/bin/sh

ssh -i KEYNAME.pem -L 8080:localhost:8080 -L 1521:localhost:1521   ubuntu@$1

 

This shows up as

21-SEP-2010 10:05:20 * (CONNECT_DATA=(SID=XE)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=47502)) * establish * XE * 12505

TNS-12505: TNS:listener does not currently know of SID given in connect descriptor

failure when trying to use jdbc to connect using the typical url: jdbc:oracle:thin:@localhost:1521:XE when the IP address keeps changing.

(I added all that in for anyone using a search to find this page)

 

 

When tests go brown

So I'm at Citcon 2010 in Wellington - and its after a morning session around BDD and Fit style testing that I'm writing this. People in the discussion use all sorts of frameworks - Cucumber, WebRat, Concordion and of course myself using easyb.

One of the interesting discussions I had last night was with Nigel Charman - part of the impetus behind the recent changes in easyb that I have done around turning it into a parse/run (instead of just run/evaluate). Nigel posited that they needed precondition checks - a third state basically that said that this test doesn't actually meet the qualifications required to run. Now you could just fail here - but that doesn't actually tell you anything, normally a failing test implies incorrect code, but in this case the code may be correct, but the data is wrong - so this mornings discussion further clarified that typically preconditions are around data being wrong or systems being unavailable. And knowing that separate from a failing test is a useful state.

As it turns out, the latest version of Junit (4.8.1) has an assume style clause [LINK] which supports the concept, but still only has the red/green colours. What I'm more interested in is reporting on it.

What does this have to do with easyb? Well, easyb also doesn't have this concept - and recently an issue was filed requesting shoulds be added to the given and when clauses (e.g. given "account is in arrears", { account =  getAccount(1234); account.inArrears().shouldBe true }. I accepted the request but said it should be in a plugin as I don't think it fit with the use of should and the concept of a failure. Now I understand why I am unhappy about it being there, and particularly I understand the pain this person is actually having.

So perhaps its time to revisit this particular issue - should easyb support another state - perhaps "brown" whereby preconditions or assumptions about the state is not met. This should be reported on to allow for dealing with in another manner.  I'll take it to the dev team and see what they think.

Goodbye, Java Posse

So I can pretty much say I have listened to all the episodes of the Java Posse - I came in a bit late in the peace - around 150, but I went back and downloaded them all and they entertained and informed me. But now its time to stop - and I had a lot more to say (thats what a blog is for right?), but really, Joe - you have spoiled it so badly it is no longer listenable.

Dick, Carl and Tor, thanks for the good times. Good luck to you - I hope you keep enough of your loyal fans to keep it interesting for the next five.